FAS Research Computing Status - Istoricul incidentelor

holylfs06 degraded

Fri, 19 Jun 2026 19:00:00 +0000

Type: Incident

Duration: 19 hours and 9 minutes

Affected Components:

Jun 19, 19:00:00 GMT+0
În curs de investigare - holylfs06 storage may be slow or unresponsive. We are investigating this issue..

Jun 20, 14:09:15 GMT+0
Rezolvat - This incident has been resolved..

2026 MGHPCC power downtime June 15-18, 2026

Mon, 15 Jun 2026 13:00:00 +0000

Type: Maintenance Duration: 3 days, 8 hours and 16 minutes Affected Components: Globus Data Transfer, Coldfront, Boston Data Center, GPU nodes (Holyoke), Infiniband - Holyoke/MGHPCC, FASSE Compute Cluster (Holyoke), SLURM Scheduler - FASSE, Boston Tier 2 NFS (new), Holyoke Tier 2 NFS (new), Holylabs, Holyoke Firewall, Network - Holyoke/MGHPCC, Holyoke-Boston fiber link (short path), Infiniband - Boston, HolyLFS06 (Tier 0), bosECS, Boston Specialty Storage, Software & Modules, Holyoke/MGHPCC Data Center, Cannon Compute Cluster (Holyoke), Network - Boston, Cambridge firewall and other redundancy, License Servers, seas_compute, FASSE login nodes, Starfish, Web Proxies, Authentication, Virtual Infrastructure - Holyoke, FIINE billing portal, CEPH Storage Boston (Tier 2), NESE (NorthEast Storage Exchange), Isilon Storage Holyoke (Tier 1), holECS, Holyoke Specialty Storage, FASRC Downloads Site, Citrix, HolyLFS05 (Tier 0), Virtual Infrastructure - Boston, Network - Cambridge, FASRC VPN (Cambridge) , FASRC VPN (Boston), BosLFS02 (Tier 0), Isilon Storage Boston (Tier 1), Home Directory Storage - Boston, Netscratch (Global Scratch), Tape - (Tier 3), Boston Compute Nodes, portal.rc.fas.harvard.edu, Login Nodes - Holyoke, Login Nodes - Boston, Samba Cluster, HolyLFS04 (Tier 0), Holystore01 (Tier 0), FASRC Two-Factor (OpenAuth), Holyoke-Boston fiber link (long path), Grafana Cloud (FASRC), SLURM Scheduler - Cannon, Kempner Cluster GPU, Kempner Cluster CPU, Cannon Open OnDemand/VDI, FASSE Open OnDemand/VDI Jun 15, 13:00:01 GMT+0 - Identificat - Maintenance is now in progress Jun 15, 13:00:00 GMT+0 - Identificat - The yearly power downtime at our Holyoke data center, MGHPCC, has been scheduled by the facility. This year's power downtime will take place on Tuesday June 15th - 18th, 2025\. There will be no June monthly maintenance as a result. Since the facility will be powered down for two days this year, we will not be performing the usual maintenance tasks. That said, networking and other key infrastructure will be doing maintenance. **IMPORTANT NOTE**: FASRC storage at both Holyoke and Boston **will be** affected and should not be expected to be available throughout the downtime. Please plan ahead accordingly. * **Monday June 15th** \- Power-down begins at 9AM * **Tuesday June 16th** \- Power out at MGHPCC * **Wednesday June 17th** \- Power out at MGHPCC * **Thursday June 18th** \- Expected return to full service by 5PM * **Friday June 19th** \- Please note that June 19th is a university holiday ![Monday June 15th - Power-down begins at 9AM Tuesday June 16th - Power out at MGHPCC Wednesday June 17th - Power out at MGHPCC Thursday June 18th - Expected return to full service by 5PM](https://www.rc.fas.harvard.edu/wp-content/uploads/2026/05/mghpcc_powerdown_2026.jpg) **For more detailed information and follow-up, please see:** **or this** [**Status Page**](https://status.rc.fas.harvard.edu/) Jun 18, 12:16:49 GMT+0 - Identificat - MGHPCC has completed their maintenance and restored power to the facility. FASRC will now begin the power-up process. Please be aware that this takes several hours. We will update this status once complete. NOTE: A reminder that tomorrow (Friday) is a university holiday. Jun 18, 20:45:50 GMT+0 - Identificat - Power-up is nearly complete, but a delay earlier in the day has us slightly behind. New ETA is 6PM. Jun 18, 21:15:42 GMT+0 - Finalizat - The yearly power downtime at our Holyoke data center, MGHPCC, has completed. The clusters and storage are back online and login nodes and OOD nodes are now available. If you have an issue/need help, please send a ticket to [rchelp@rc.fas.harvard.edu](mailto:rchelp@rc.fas.harvard.edu) with details. IMPORTANT NOTE: Tomorrow, June 19th is a university holiday. FASRC staff will return Monday to address any lingering issues and any new tickets.

holylfs06 degraded

Wed, 10 Jun 2026 19:41:54 +0000

Type: Incident

Duration: 25 minutes

Affected Components:

Jun 10, 20:07:18 GMT+0
Rezolvat - This incident has been resolved..

Jun 10, 19:41:54 GMT+0
În curs de investigare - Holylfs06 storage may be slow or unresponsive We are currently investigating this incident..

Portal unavailable

Tue, 9 Jun 2026 16:22:32 +0000

Type: Incident

Duration: 3 hours and 16 minutes

Affected Components:

Jun 9, 19:38:55 GMT+0
Rezolvat - The Portal website should be accessible for all now. This incident has been resolved..

Jun 9, 16:22:32 GMT+0
În curs de investigare - There is an SSL issue with the Portal which will cause an error for anyone attempting to connect. Investigating..

Authentication outage

Mon, 8 Jun 2026 20:24:54 +0000

Type: Incident

Duration: 5 minutes

Affected Components:

Jun 8, 20:24:54 GMT+0
În curs de investigare - Authentication issues with openauth/radius. This incident was created by an automated monitoring service..

Jun 8, 20:29:54 GMT+0
Rezolvat - Openauth/radius is now operational. This update was created by an automated monitoring service..

Coldfront server error

Mon, 8 Jun 2026 20:02:18 +0000

Type: Incident

Duration: 12 minutes

Affected Components:

Jun 8, 20:02:18 GMT+0
În curs de investigare - Coldfront is showing a server error. Investigating..

Jun 8, 20:14:34 GMT+0
Rezolvat - This incident has been resolved. Coldfront is now available..

lofin.rc.fas.harvard.edu refusing new connections

Mon, 8 Jun 2026 13:16:28 +0000

Type: Incident

Duration: 9 minutes

Affected Components: ,

Jun 8, 13:16:28 GMT+0
În curs de investigare - [lofin.rc.fas.harvard.edu](http://lofin.rc.fas.harvard.edu) is refusing new connections and will error out when attempting to SSH in. Investigating..

Jun 8, 13:25:32 GMT+0
Rezolvat - This incident has been resolved and SSH login to [login.rc.fas.harvard.edu](http://login.rc.fas.harvard.edu) is working again. Thanks for your patience..

FASRC VPN (Cambridge) is back up

Fri, 5 Jun 2026 04:06:54 +0000

Type: Incident

Affected Components:

Jun 5, 04:06:55 GMT+0
În curs de investigare - FASRC VPN (Cambridge) is down at the moment. This incident was automatically created by Instatus monitoring..

Jun 5, 04:16:32 GMT+0
Rezolvat - FASRC VPN (Cambridge) is back up. This incident was automatically resolved by Instatus monitoring..

Password reset

Tue, 2 Jun 2026 20:42:13 +0000

Type: Incident

Duration: 22 minutes

Affected Components: ,

Jun 2, 20:42:13 GMT+0
În curs de investigare - Password reset emails are not being sent out at this time. We are currently investigating this incident..

Jun 2, 21:04:09 GMT+0
Rezolvat - Password reset emails are now being sent out. This incident has been resolved..

Cannon cluster down

Fri, 29 May 2026 20:21:17 +0000

Type: Incident

Duration: 2 days, 18 hours and 24 minutes

Affected Components: , , , , , , , ,

Jun 1, 14:45:41 GMT+0
Rezolvat - Slurm crashed on 4:30p on Friday due to a user running a large sacct query against the Slurm database. This caused the database host to run out of memory and crash the scheduler. To prevent this from reoccurring we are reducing the time range that users are permitted to query at one time to 7 days. Thus if you need to cover a month you would need to query in four 7 day increments. We do ask users to be judicious in their querying of the Slurm. Only ask for those fields that you require. Please also ensure any AI agents you have running limit their queries appropriately..

May 29, 20:21:17 GMT+0
În curs de investigare - The Slurm scheduler is experiencing an error which is impacting jobs. The Cannon cluster will be inaccessible while we troubleshoot. We are currently investigating this incident..

May 29, 21:13:09 GMT+0
Identificat - To temporarily stabilize the situation, we have reduced the maximum query time for sacct and other Slurm commands to be 1 day. We have filed a ticket with SchedMD to further analyze the issue. The cluster is back up and the scheduler is accepting new jobs. We will continue to monitor for emergencies over the weekend, and resume in-depth troubleshooting on Monday. .

Account approvals down

Fri, 22 May 2026 15:18:39 +0000

Type: Incident

Duration: 3 days, 23 hours and 2 minutes

Affected Components:

May 22, 15:18:39 GMT+0
În curs de investigare - Account approvals through Portal are not available at this time. We are currently investigating this incident..

May 26, 14:20:10 GMT+0
Rezolvat - This incident has been resolved. Login is again available for approvals/onboarding..

FASRC VPN (Cambridge) is back up

Fri, 22 May 2026 04:50:10 +0000

Type: Incident

Affected Components:

May 22, 04:50:10 GMT+0
În curs de investigare - FASRC VPN (Cambridge) is down at the moment. This incident was automatically created by Instatus monitoring..

May 22, 05:09:48 GMT+0
Rezolvat - FASRC VPN (Cambridge) is back up. This incident was automatically resolved by Instatus monitoring..

FASRC VPN (Cambridge) is back up

Fri, 22 May 2026 03:35:48 +0000

Type: Incident

Affected Components:

May 22, 03:35:48 GMT+0
În curs de investigare - FASRC VPN (Cambridge) is down at the moment. This incident was automatically created by Instatus monitoring..

May 22, 03:55:19 GMT+0
Rezolvat - FASRC VPN (Cambridge) is back up. This incident was automatically resolved by Instatus monitoring..

FASRC VPN (Cambridge) is back up

Fri, 22 May 2026 03:05:04 +0000

Type: Incident

Affected Components:

May 22, 03:05:05 GMT+0
În curs de investigare - FASRC VPN (Cambridge) is down at the moment. This incident was automatically created by Instatus monitoring..

May 22, 03:14:35 GMT+0
Rezolvat - FASRC VPN (Cambridge) is back up. This incident was automatically resolved by Instatus monitoring..

login.rc.fas.harvard.edu is responding normally

Fri, 22 May 2026 02:40:38 +0000

Type: Incident

Affected Components: , , ,

May 22, 02:40:38 GMT+0
În curs de investigare - login.rc.fas.harvard.edu is not responding normally. This incident was automatically created..

May 22, 03:40:12 GMT+0
Rezolvat - \\\[login.rc.fas.harvard.edu\\\](http://login.rc.fas.harvard.edu) is responding normally. This incident was automatically resolved..

FASRC VPN (Cambridge) is back up

Fri, 22 May 2026 02:24:22 +0000

Type: Incident

Affected Components:

May 22, 02:24:22 GMT+0
În curs de investigare - FASRC VPN (Cambridge) is down at the moment. This incident was automatically created by Instatus monitoring..

May 22, 02:53:53 GMT+0
Rezolvat - FASRC VPN (Cambridge) is back up. This incident was automatically resolved by Instatus monitoring..

FASRC VPN (Cambridge) is back up

Fri, 22 May 2026 01:43:38 +0000

Type: Incident

Affected Components:

May 22, 02:13:11 GMT+0
Rezolvat - FASRC VPN (Cambridge) is back up. This incident was automatically resolved by Instatus monitoring..

May 22, 01:43:38 GMT+0
În curs de investigare - FASRC VPN (Cambridge) is down at the moment. This incident was automatically created by Instatus monitoring..

Fiine financial tool is down

Thu, 21 May 2026 14:18:48 +0000

Type: Incident

Duration: 45 minutes

Affected Components:

May 21, 15:04:16 GMT+0
Rezolvat - This incident has been resolved. Fiine is back online..

May 21, 14:18:48 GMT+0
În curs de investigare - The FIINE financial tool is down. This affects only a small number of people, but we are noting here out of caution..

portal.rc.fas.harvard.edu is back up

Thu, 21 May 2026 13:17:12 +0000

Type: Incident

Affected Components:

May 21, 13:17:12 GMT+0
În curs de investigare - portal.rc.fas.harvard.edu is down at the moment. This incident was created automatically..

May 21, 13:27:23 GMT+0
Rezolvat - portal.rc.fas.harvard.edu is back up. his incident was resolved automatically..

FASRC VPN (Cambridge) is back up

Thu, 21 May 2026 04:43:03 +0000

Type: Incident

Affected Components:

May 21, 04:43:03 GMT+0
În curs de investigare - FASRC VPN (Cambridge) is down at the moment. This incident was automatically created by Instatus monitoring..

May 21, 05:29:42 GMT+0
Rezolvat - FASRC VPN (Cambridge) is back up. This incident was automatically resolved by Instatus monitoring..

MGHPCC power work - Part 2 May 18

Mon, 18 May 2026 11:00:00 +0000

Type: Maintenance

Duration: 3 days, 13 hours and 15 minutes

Affected Components: , , , , , , , ,

May 18, 11:00:00 GMT+0
Identificat - Rescheduled to May 18.

May 18, 11:00:00 GMT+0
Identificat - Our Holyoke data center, MGHPCC, will be doing power work on Row 8A. This work, which is being completed over the course of 2 weeks, will bring online another power feed which will increase power capacity. In order to do this work, it will require us to idle half the nodes in 8a for the duration of the week. This means all partitions in this row will be at half capacity. Existing jobs should drain naturally and no job should need to be canceled. The impacted partitions are: ``` arguelles_delgado_h100 bigmem bigmem_intermediate blackhole_gpu dvorkin eddy enos gershman gpu gpu_h200 gpu_requeue hejazi hernquist_ice hoekstra hsph hsph_gpu huce_ice iaifi_gpu_requeue intermediate itc_cluster itc_gpu janson_sapphire joonholee jshapiro kempner kempner_priority kempner_dev kempner_eng kempner_h200_priority kempner_h100 kempner_h100_priority kempner_h100_priority2 kempner_h100_priority3 kempner_h100_priority4 kempner_interactive kovac kozinsky kozinsky_gpu kozinsky_requeue murphy_ice mweber_compute mweber_gpu olveczky_sapphire ortegahernandez_ice rivas sapphire seas_compute seas_gpu siag siag_combo test yao yao_priority zhuang ```.

May 18, 11:00:01 GMT+0
Identificat - Maintenance is now in progress.

May 22, 00:14:55 GMT+0
Finalizat - The power work has completed successfully. All nodes have been returned to normal service..

MGHPCC power work - Part 1 May 11

Mon, 11 May 2026 11:00:00 +0000

Type: Maintenance

Duration: 5 days and 12 hours

Affected Components: , , , , , , , ,

May 11, 11:00:00 GMT+0
Identificat - Rescheduled to May 11.

May 11, 11:00:00 GMT+0
Identificat - Our Holyoke data center, MGHPCC, will be doing power work on Row 8A. This work, which will occur this week and next week, will bring online another power feed which will increase power capacity. In order to do this work, it will require us to idle half the nodes in 8a for the duration of the week. This means all partitions in this row will be at half capacity. Existing jobs should drain naturally and no job should need to be canceled. The impacted partitions are: ``` arguelles_delgado_h100 bigmem bigmem_intermediate blackhole_gpu dvorkin eddy enos gershman gpu gpu_h200 gpu_requeue hejazi hernquist_ice hoekstra hsph hsph_gpu huce_ice iaifi_gpu_requeue intermediate itc_cluster itc_gpu janson_sapphire joonholee jshapiro kempner kempner_priority kempner_dev kempner_eng kempner_h200_priority kempner_h100 kempner_h100_priority kempner_h100_priority2 kempner_h100_priority3 kempner_h100_priority4 kempner_interactive kovac kozinsky kozinsky_gpu kozinsky_requeue murphy_ice mweber_compute mweber_gpu olveczky_sapphire ortegahernandez_ice rivas sapphire seas_compute siag siag_combo test yao yao_priority zhuang ```.

May 16, 23:00:00 GMT+0
Finalizat - Maintenance has completed successfully.

May 11, 11:00:01 GMT+0
Identificat - Maintenance is now in progress.

www.rc.fas.harvard.edu is back up

Wed, 6 May 2026 16:20:55 +0000

Type: Incident

Affected Components:

May 6, 16:20:55 GMT+0
În curs de investigare - www.rc.fas.harvard.edu is down at the moment. This incident was created automatically..

May 6, 16:30:56 GMT+0
Rezolvat - www.rc.fas.harvard.edu is back up. This incident was resolved automatically..

Monthly maintenance May 4th 2026 9am-1pm

Mon, 4 May 2026 13:00:00 +0000

Type: Maintenance

Duration: 4 hours

Affected Components: , , , , , , , , , , , , , , ,

May 4, 13:00:00 GMT+0
Identificat - FASRC monthly maintenance will take place on May 4th 2026\. Our maintenance tasks should be completed between 9am-1pm. **NOTICES:** * Annual data center power downtime: The annual downtime at MGHPCC will take place June 15 - June 18\. This year's downtime will be one day longer. More details will be sent to all users next month. * Training: Upcoming training from FASRC and other sources can be found on our Training Calendar. at * Status Page: You can subscribe to our status to receive notifications of maintenance, incidents, and their resolution at (click Get Updates for options). **MAINTENANCE TASKS** Cannon cluster will be paused during this maintenance?: **YES** FASSE cluster will be paused during this maintenance?: **YES** * Slurm 25.11.5 Upgrade * Audience: All cluster users * Impact: Jobs will be paused during the upgrade * Reboot remaining stuck nodes from power outage * Audience: N/A * Impact: No visible impact to user * Two-Factor/OpenAuth ([two-factor.rc.fas.harvard.edu](http://two-factor.rc.fas.harvard.edu)) replacement * Audience: All account holders * Impact: The server will be unavailable during maintenance. You will be unable to obtain a new or replacement OpenAuth token during this period. * Domain controller replacement * Audience: Internal * Impact: End users should not see any impact * OOD/Open OnDemand reboots * Audience: All OOD users, reboot of the head nodes * Impact: Running sessions will _not_ be affected * Login node reboots * Audience; All login node users * Impact: Login nodes will reboot during the maintenance window * Netscratch 90-day retention cleanup * Audience; All netscratch users * Impact: Files older than 90 days will be removed per our [scratch policy](https://docs.rc.fas.harvard.edu/kb/policy-scratch/). Please note that this cleanup can happen at any time, not just during maintenance. Thank you, FAS Research Computing .

May 4, 13:00:01 GMT+0
Identificat - Maintenance is now in progress.

May 4, 14:56:02 GMT+0
Identificat - The scheduler is re-opened and jobs un-paused. Other, non-impacting, work continues..

May 4, 17:00:00 GMT+0
Finalizat - Maintenance has completed successfully.

Starfish maintenance

Fri, 1 May 2026 20:00:00 +0000

Type: Maintenance

Duration: 1 hour

Affected Components:

May 1, 20:00:00 GMT+0
Identificat - Starfish will be upgraded to the latest version on Friday, May 1st from 4pm-5pm. The service and dashboard will be down during this time. .

May 1, 20:00:01 GMT+0
Identificat - Maintenance is now in progress.

May 1, 21:00:00 GMT+0
Finalizat - Maintenance has completed successfully.

OpenOnDemand maintenance

Thu, 30 Apr 2026 12:00:00 +0000

Type: Maintenance

Duration: 2 hours

Affected Components: ,

Apr 30, 12:00:00 GMT+0
Identificat - At 8am on Thursday April 30th we will be upgrading from Open OnDemand version 4.0.7 to 4.1.4 on both the Cannon and FASSE clusters. This is not expected to impact running jobs. This upgrade adds the Jobs->Project Manager menu item and fixes an issue that affected access to the Clusters->Shell Access menu item when using Firefox..

Apr 30, 12:00:01 GMT+0
Identificat - Maintenance is now in progress.

Apr 30, 14:00:00 GMT+0
Finalizat - Maintenance has completed successfully.

login.rc.fas.harvard.edu is responding normally

Thu, 30 Apr 2026 02:31:25 +0000

Type: Incident

Affected Components: , , ,

Apr 30, 02:31:26 GMT+0
În curs de investigare - login.rc.fas.harvard.edu is not responding normally. This incident was automatically created..

Apr 30, 15:43:53 GMT+0
Rezolvat - \\\[login.rc.fas.harvard.edu\\\](http://login.rc.fas.harvard.edu) is responding normally. This incident was automatically resolved..

Login and OOD node access restricted due to serious security issue - No ETA

Thu, 30 Apr 2026 02:11:01 +0000

Type: Incident

Duration: 16 hours and 2 minutes

Affected Components: , , , ,

Apr 30, 02:11:01 GMT+0
Identificat - **Due to a serious in-the-wild exploit which can compromise Fedora-based Linux distributions including Rocky, which is used on the cluster, we need to restrict access. All login and OOD nodes are shut down until a fix can be put in place. Jobs running on the cluster will continue running.** **No ETA, There is not fix at this time. We will update our status page in the morning once we have more information or a fix to roll out.** **This is a serious exploit and we do not take this measure lightly. Please follow this status page for updates and eventual resolution.**.

Apr 30, 14:44:58 GMT+0
Identificat - We are developing a plan of attack to mitigate this exploit. Please know that this is a very serious issue and so we are treating it as such. Thank you for your understanding. We are currently awaiting further information from the Redhat/Fedora/Rocky community but building a plan in the meantime with the information we have. More details to follow as we can share them. If you need to access storage (except scratch and home directories), [Globus ](https://docs.rc.fas.harvard.edu/kb/globus-file-transfer/)is still online and available. But again, login nodes and OOD are not available..

Apr 30, 15:40:41 GMT+0
Identificat - To mitigate this exploit we will need to restart -all nodes- on the cluster. This **will begin at 1PM** and run until all nodes have restarted (no ETA). This **will mean** any un-finished jobs will be **terminated**. There is no way to avoid this. We will then be validating the fix before re-opening the login. OOD nodes, and scheduler. Next steps and updates will be posted here..

Apr 30, 18:12:41 GMT+0
Rezolvat - The cluster has been rebooted and all nodes, including login and OOD, have been patched. The scheduler is re-opened and jobs which were preempted/requeued have priority for re-scheduling. Some non-standard, lab-owned nodes may still require patching. The owners of these machines may be contacted about this. Thank you for your patience. This is a global issue and is being addressed at centers everywhere..

holylfs06 down

Wed, 29 Apr 2026 16:00:11 +0000

Type: Incident

Duration: 33 minutes

Affected Components:

Apr 29, 16:00:11 GMT+0
Identificat - Holylfs06 storage is down. We are investigating. More details as they are known..

Apr 29, 16:32:45 GMT+0
Rezolvat - Holylfs06 is accessible again. This incident has been resolved..

Website security maintenance (www.rc and docs.rc) 4-28-26 1pm

Tue, 28 Apr 2026 17:00:00 +0000

Type: Maintenance

Duration: 17 minutes

Affected Components: ,

Apr 28, 17:00:00 GMT+0
Identificat - Security updates are required for [www.rc.fas.harvard.edu](http://www.rc.fas.harvard.edu) and [docs.rc.fas.harvard.edu](http://docs.rc.fas.harvard.edu) This work will take place today between 1pm and 2pm Both sites will be down for very short periods during the updates..

Apr 28, 17:00:01 GMT+0
Identificat - Maintenance is now in progress.

Apr 28, 17:16:58 GMT+0
Finalizat - Website maintenance has completed successfully..

Starfish - Out of date scans

Mon, 20 Apr 2026 21:03:38 +0000

Type: Incident

Duration: 7 days and 18 hours

Affected Components:

Apr 20, 21:03:38 GMT+0
Identificat - Some starfish scans restarted following earlier issue and may be up to a week out of date due to the delayed scans Two scanning agents are still down. So please bear this in mind when viewing Starfish data..

Apr 21, 13:48:40 GMT+0
În curs de monitorizare - The down agents are back in service. Scans are on-going but data will still be out of date until they catch up..

Apr 28, 15:03:28 GMT+0
Rezolvat - Scans are running normally. Any out of date stats will catch up shortly..

login.rc.fas.harvard.edu is responding normally

Sun, 19 Apr 2026 11:55:47 +0000

Type: Incident

Affected Components: , , ,

Apr 19, 11:55:47 GMT+0
În curs de investigare - login.rc.fas.harvard.edu is not responding normally. This incident was automatically created..

Apr 19, 12:55:25 GMT+0
Rezolvat - \\\[login.rc.fas.harvard.edu\\\](http://login.rc.fas.harvard.edu) is responding normally. This incident was automatically resolved..

FASRC monthly maintenance April 6th 2026 9am-1pm

Mon, 6 Apr 2026 13:00:00 +0000

Type: Maintenance

Duration: 4 hours

Affected Components: , , , , , , , , ,

Apr 6, 13:00:01 GMT+0
Identificat - Maintenance is now in progress.

Apr 6, 17:00:00 GMT+0
Finalizat - Maintenance has completed successfully.

Apr 6, 13:00:00 GMT+0
Identificat - FASRC monthly maintenance will take place on April 6th 2026\. Our maintenance tasks should be completed between 9am-1pm. **NOTICES:** * Training: Upcoming training from FASRC and other sources can be found on our Training Calendar. at * Status Page: You can subscribe to our status to receive notifications of maintenance, incidents, and their resolution at (click Get Updates for options). * We'd love to hear success stories about your or your lab's use of FASRC. Submit your story [here](https://www.rc.fas.harvard.edu/user-stories/). **MAINTENANCE TASKS** Cannon cluster will be paused during this maintenance?: **NO** FASSE cluster will be paused during this maintenance?: **NO** * [two-factor.rc.fas.harvard.edu](http://two-factor.rc.fas.harvard.edu) [OpenAuth](https://docs.rc.fas.harvard.edu/kb/openauth/) cut-over to new server * Audience: New accounts or anyone requesting an OpenAuth token * Impact: two-factor will be unavailable while moving to a new server * RStudio Server (Open OnDemand) * Audience: RStudio Server users on Cannon and FASSE * Impact: We will be decommissioning some versions of RStudio Server so we can properly maintain all production versions. Versions to be decommissioned: * R 4.1.3 (Bioconductor 3.14, RStudio 2022.02.0) * R 4.1.0 (Bioconductor 3.13, RStudio 1.4.1717) * R 4.0.3 (Bioconductor 3.12, Rstudio 1.3.1093) * R 4.0.0 (Bioconductor 3.11, Rstudio 1.3.1093) * If you use one of these versions, we recommend replacing it with the most recent version, R 4.4.2 (Bioconductor 3.20, RStudio 2024.12.0). You must reinstall previously installed libraries. * Domain controller replacement * Audience: Internal * Impact: End users should not see any impact * OOD/Open OnDemand reboots * Audience: All OOD users, reboot of the head nodes * Impact: Running sessions will _not_ be affected * Login node reboots * Audience; All login node users * Impact: Login nodes will reboot during the maintenance window * Netscratch 90-day retention cleanup * Audience; All netscratch users * Impact: Files older than 90 days will be removed per our [scratch policy](https://docs.rc.fas.harvard.edu/kb/policy-scratch/). Please note that this cleanup can happen at any time, not just during maintenance. Thank you, FAS Research Computing .

FASRC monthly maintenance Monday March 2nd, 2026 9am-1pm

Mon, 2 Mar 2026 14:00:00 +0000

Type: Maintenance

Duration: 4 hours

Affected Components: , , , , , , , , , , , , , , , , , , , ,

Mar 2, 14:00:01 GMT+0
Identificat - Maintenance is now in progress.

Mar 2, 14:00:00 GMT+0
Identificat - Monthly maintenance will take place on Monday March 2nd, 2026\. Our maintenance tasks should be completed between 9am-1pm. **NOTICES:** * Training: Upcoming training from FASRC and other sources can be found on our Training Calendar. at * Status Page: You can subscribe to our status to receive notifications of maintenance, incidents, and their resolution at (click Get Updates for options). * We'd love to hear success stories about your or your lab's use of FASRC. Submit your story [here](https://www.rc.fas.harvard.edu/user-stories/). **MAINTENANCE TASKS** Cannon cluster will be paused during this maintenance?: **YES** FASSE cluster will be paused during this maintenance?: **YES** * Slurm scheduler update * Audience: All cluster users * Impact: Jobs will be paused during maintenance * OOD node reboots * Audience; All Open OnDemand users * Impact: OOD nodes will reboot during the maintenance window * Login node reboots * Audience: All login node users * Impact: Login nodes will reboot during the maintenance window * Netscratch retention purge * Audience: All users of Netscratch * Impact: Files older than 90 days will be removed. Please note that retention cleanup can and does run at any time, not just during the maintenance window. Thank you, FAS Research Computing [https://www.rc.fas.harvard.edu/](https://www.rc.fas.harvard.edu/upcoming-training/).

Mar 2, 18:00:00 GMT+0
Finalizat - Maintenance has completed successfully.

Starfish maintenance Feb 25, 2026 all day

Wed, 25 Feb 2026 14:00:00 +0000

Type: Maintenance

Duration: 1 day

Affected Components:

Feb 25, 14:00:00 GMT+0
Identificat - Starfish will be unavailable starting Wednesday, February 25th at 9AM until Thursday, February 26th at 9AM, for routine maintenance. The online dashboard will be inaccessible during this time..

Feb 26, 14:00:00 GMT+0
Finalizat - Maintenance has completed successfully.

Feb 25, 14:00:01 GMT+0
Identificat - Maintenance is now in progress.

NESE tape maintenance Feb 19th 2026

Thu, 19 Feb 2026 13:00:00 +0000

Type: Maintenance

Duration: 9 hours

Affected Components:

Feb 19, 13:00:01 GMT+0
Identificat - Maintenance is now in progress.

Feb 19, 13:00:00 GMT+0
Identificat - From our partners at NESE. Details follow: We are installing four new tape frames, which will bring the tape system raw storage capacity to 253 petabytes. **Service Affected:** NESE Tape Service **Maintenance Window:** 8:00 AM - 5:00 PM (EST) * The tape service will be unavailable. * All upgrade activities are expected to be completed on the same day. NOTES: * Monitor the MGHPCC Slack #nese channel for status updates and announcements * Monitor for real-time updates on progress Subscribe to for updates and announcements.

Feb 19, 22:00:00 GMT+0
Finalizat - Maintenance has completed successfully.

Security updates needed for www.rc.fas.harvard.edu and docs.rc.fas.harvard.edu

Mon, 9 Feb 2026 21:10:00 +0000

Type: Maintenance

Duration: 8 minutes

Affected Components: ,

Feb 9, 21:10:00 GMT+0
Identificat - Security updates will require a brief interruption for our primary websites [www.rc.fas.harvard.edu](http://www.rc.fas.harvard.edu) and [docs.rc.fas.harvard.edu](http://docs.rc.fas.harvard.edu) We will endeavour to keep this update as short as possible. Each site may be unavailable for a few minutes..

Feb 9, 21:10:01 GMT+0
Identificat - Maintenance is now in progress.

Feb 9, 21:17:51 GMT+0
Finalizat - Maintenance has completed successfully..

NESE tape maintenance Feb 9th 2026

Mon, 9 Feb 2026 13:00:00 +0000

Type: Maintenance

Duration: 9 hours

Affected Components:

Feb 9, 13:00:00 GMT+0
Identificat - From our partners at NESE. Details follow: In the process of the tape front-end file caching system upgrade, we will be installing a new IBM Storage Scale System 6000\. We will provide an additional update for when the software integration and data transfer from the current IBM Elastic Storage System 5000 will be performed. **Service Affected:** NESE Tape Service **Maintenance Window: No Downtime expected** NOTES: * Monitor the MGHPCC Slack #nese channel for status updates and announcements * Monitor for real-time updates on progress * Subscribe to for updates and announcements.

Feb 9, 13:00:01 GMT+0
Identificat - Maintenance is now in progress.

Feb 9, 22:00:00 GMT+0
Finalizat - Maintenance has completed successfully.

FASRC monthly maintenance Monday February 2nd, 2026 9am-1pm

Mon, 2 Feb 2026 14:00:00 +0000

Type: Maintenance

Duration: 4 hours

Affected Components: , , , , , , , , , , , , , , , , , , , ,

Feb 2, 14:00:00 GMT+0
Identificat - Monthly maintenance will take place on Monday February 2nd, 2026\. Our maintenance tasks should be completed between 9am-1pm. **NOTICES:** * Training: Upcoming training from FASRC and other sources can be found on our Training Calendar. at * Status Page: You can subscribe to our status to receive notifications of maintenance, incidents, and their resolution at (click Get Updates for options). * We'd love to hear success stories about your or your lab's use of FASRC. Submit your story [here](https://www.rc.fas.harvard.edu/user-stories/). **MAINTENANCE TASKS** Cannon cluster will be paused during this maintenance?: **YES** FASSE cluster will be paused during this maintenance?: **YES** * MaxTime change * Audience: Cluster users * Impact: In order to improve scheduling efficiency and stability, we will be setting a maximum run time on all partitions that have MaxTime set to UNLIMITED to a MaxTime of 3 days. The unrestricted partition will be set to 365 days. Partitions that already have MaxTime set will retain their current setting. Partition owners wishing to set a different MaxTime for their partition should contact FASRC. Note that we do no guarantee uptime and so users should utilize checkpointing to save state in case of node failure. * Slurm upgrade to 25.11.2 * Audience: All cluster users * Impact: Jobs will be paused during maintenance * OOD node reboots * Audience; All Open OnDemand users * Impact: OOD nodes will reboot during the maintenance window * Login node reboots * Audience; All login node users * Impact: Login nodes will reboot during the maintenance window Thank you, FAS Research Computing [https://www.rc.fas.harvard.edu/](https://www.rc.fas.harvard.edu/upcoming-training/).

Feb 2, 14:00:01 GMT+0
Identificat - Maintenance is now in progress.

Feb 2, 18:00:00 GMT+0
Finalizat - Maintenance has completed successfully.

FASRC monthly maintenance Monday January 12th, 2026 9am-1pm

Mon, 12 Jan 2026 14:00:00 +0000

Type: Maintenance

Duration: 4 hours

Affected Components: , , , , , , , , , , , , , , , , , , ,

Jan 12, 14:00:00 GMT+0
Identificat - Monthly maintenance will take place on January 12th, 2026\. Our maintenance tasks should be completed between 9am-1pm. **NOTICES:** * Changes to SEAS partitions, please see tasks below. * Changes to job age priority weighting, please see tasks below. * Status Page: You can subscribe to our status to receive notifications of maintenance, incidents, and their resolution at (click Get Updates for options). * We'd love to hear success stories about your or your lab's use of FASRC. Submit your story [here](https://www.rc.fas.harvard.edu/user-stories/). **MAINTENANCE TASKS** Cannon cluster will be paused during this maintenance?: **YES** FASSE cluster will be paused during this maintenance?:**YES** * Slurm upgrade to 25.11.1 * Audience: All cluster users (Cannon and FASSE) * Impact: Jobs will be paused during maintenance * In conjunction with SEAS we will modify seas\_gpu and seas\_compute time limits * Audience: SEAS users * Impact: seas\_gpu: will be set to 2 days maximum seas\_compute: will be set to 3 days maximum Existing pending jobs longer than these limits will be set to 2 day and 3 day run times depending on partition. * Job Age Priority Weight Change * Audience: Cluster users * Impact: We will be adjusting the weight applied to the priority earned by jobs by virtue of their age. Currently job priority is made up of two factors, Fairshare and Job Age. The Job Age factor is currently set such that jobs gain priority over 3 days with a maximum priority equivalent to jobs with Fairshare of 0.5\. This keeps low fairshare jobs from languishing at the bottom of the queue. With the current settings though, users with low fairshare can gain significant advantage over users with higher relative fairshare. To remedy this we will be adjusting the Job Age weight to cap out at an equivalent Fairshare of 0.1\. This will still allow jobs with 0 fairshare to gain priority and thus not languish while letting fairshare govern a wider range of higher priority jobs. * Login node reboots * Audience; All login node users * Impact: Login nodes will reboot during the maintenance window * Open OnDemand (OOD) node reboots * Audienc:; All OOD users * Impact: OOD nodes will reboot during the maintenance window * Netscratch retention will run * Audience: All cluster netscratch users * Impact: Files older than 90 days will be removed. Please note that retention cleanup can and does run at any time, not just during the maintenance window. Thank you, FAS Research Computing [https://www.rc.fas.harvard.edu/](https://www.rc.fas.harvard.edu/upcoming-training/).

Jan 12, 14:00:01 GMT+0
Identificat - Maintenance is now in progress.

Jan 12, 18:00:00 GMT+0
Finalizat - Maintenance has completed successfully.

Monthly Maintenance and MGHPCC Power Work - Dec. 8, 2025 6am-6pm

Mon, 8 Dec 2025 11:00:00 +0000

Type: Maintenance

Duration: 12 hours

Affected Components: , , , , , , , , , , , , , , , , , , , , , , , ,

Dec 8, 11:00:00 GMT+0
Identificat - Monthly maintenance will take place on December 8th. Our maintenance tasks should be completed between 9am-1pm. However: _Additionally_, MGHPCC will be performing power upgrades on the odd side of Row 8A where much of our computer resides. This is the final upgrade for this row. Current estimate for this work is a 12 hour window 6am-6pm. A list of the affected partitions is provided at the bottom of this notice. The nodes in those partitions will be drained prior to the work and will be powered down. Once the work is completed, those nodes will be returned to service. **Notices:** * New FASSE partition `fasse_gpu_h200`. This partitions has 2 H200 nodes and a 3day limit. It is available now. * 11/26 - 11/28 are university holidays (Thanksgiving). No on-site support, FASRC staff will return on 12/1. * Training: Upcoming training from FASRC and other sources can be found on our Training Calendar. at * Status Page: You can subscribe to our status to receive notifications of maintenance, incidents, and their resolution at (click Get Updates for options). * We'd love to hear success stories about your or your lab's use of FASRC. Submit your story [here](https://www.rc.fas.harvard.edu/user-stories/). **MAINTENANCE TASKS** Cannon cluster will be paused during this maintenance?: **PARTIAL OUTAGE/YES** FASSE cluster will be paused during this maintenance?: **PARTIAL OUTAGE/YES** * Power work on Row 8A odd * Audience: Users of the partitions listed below * Impact: These nodes and partitions will be fully or partially down all day * OneFS (Isilon) upgrade * Audience: All Isilon (Tier 1) shares * Impact: Some VMs will be impacted including Cannon OOD, CBScentral, MCZapps/MCZbase, Portal, and Rclic1 (license server) * Slurm upgrade to 25.05.5 * Audience: All cluster users * Impact: Jobs will be paused during maintenance * Login node reboots * Audience: All login node users * Impact: Login nodes will reboot during the maintenance window **Impacted Cannon Partitions (Full or Partial Outage):** * arguelles\_delgado\_gpu\_a100 * arguelles\_delgado\_gpu\_mixed * bigmem\_intermediate * blackhole\_gpu * eddy * gershman * gpu\_requeue * hejazi * hernquist\_ice * hoekstra * huce\_ice * iaifi\_gpu * iaifi\_gpu\_priority * iaifi\_gpu\_requeue * itc\_gpu * jshapiro * kempner * kempner\_dev * kempner\_priority * kempner\_h100 * kempner\_h100\_priority * kempner\_h100\_priority2 * kempner\_h100\_priority3 * kempner\_interactive * kempner\_requeue * kovac * kozinsky * kozinsky\_gpu * kozinsky\_priority * kozinsky\_requeue * murphy\_ice * ortegahernandez\_ice * rivas * seas\_compute * seas\_gpu * serial\_requeue * siag\_combo * siag\_gpu * sur * zhuang.

Dec 8, 11:00:01 GMT+0
Identificat - Maintenance is now in progress.

Dec 8, 23:00:00 GMT+0
Finalizat - Maintenance has completed successfully.

holylfs04 migrations

Fri, 5 Dec 2025 14:00:00 +0000

Type: Maintenance

Duration: 4 days, 1 hour and 10 minutes

Affected Components:

Dec 5, 14:00:00 GMT+0
Identificat - The holylfs04 migration to holylfs06 has begun. All holylfs04 folders will be **read-only** for the duration of the migration, from **Friday, December 5th at 9AM until end of day on Monday, December 8th.** All labs with holylfs04 have been informed via email; please email [rdm@rc.fas.harvard.edu](mailto:rdm@rc.fas.harvard.edu) if you have any questions..

Dec 9, 15:09:30 GMT+0
Finalizat - Maintenance has completed successfully..

NESE tape system maintenance 12/1/25-12/5/25

Mon, 1 Dec 2025 11:00:00 +0000

Type: Maintenance

Duration: 5 days

Affected Components: ,

Dec 1, 11:00:00 GMT+0
Identificat - NESE, the Northeast Storage Exchange at MGHPCC which supplies the Tier3 tape service used by FASRC, will be offline for maintenance on the system Dec 1st - 5th. There will be ongoing performance-affecting maintenance until Dec 12th. Please see below for details. WHO: Any lab who has or is moving data to tape. IMPACT: No access 12/1/25 - 12/5/25\. Reduced performance 12/5/25 - 12/12/25. > NESE tape system maintenance and major software upgrade is scheduled to begin on December 1, 2025\. As a result, the NESE Tape service will be offline from December 1 to December 5. > > Starting December 8 through December 12, the service will be back online with reduced performance. All maintenance activities are planned to conclude on December 12, 2025. > > * Monitor: for real-time updates on progress > * Subscribe to for updates and announcements.

Dec 1, 11:00:01 GMT+0
Identificat - Maintenance is now in progress.

Dec 6, 11:00:00 GMT+0
Finalizat - Maintenance has completed successfully.

Starfish dashboard maintenance Nov. 14th 5-6PM

Fri, 14 Nov 2025 22:00:00 +0000

Type: Maintenance

Duration: 1 hour

Affected Components:

Nov 14, 22:00:01 GMT+0
Identificat - Maintenance is now in progress.

Nov 14, 23:00:00 GMT+0
Finalizat - Maintenance has completed successfully.

Nov 14, 22:00:00 GMT+0
Identificat - There is a planned upgrade of the Starfish dashboard scheduled for Friday November 14th starting at 5PM. The dashboard will be down for an hour while the upgrade is performed..

Monthly Maintenance and MGHPCC Power Work - Nov. 3, 2025 6am-6pm

Mon, 3 Nov 2025 11:00:00 +0000

Type: Maintenance

Duration: 9 hours and 4 minutes

Affected Components: , , , , , , , , , , , , , , , , , , ,

Nov 3, 11:00:00 GMT+0
Identificat - Monthly maintenance will take place on November 3rd. Additionally, MGHPCC will be performing power upgrades on the even side of Row 8A where much of our computer resides. A further upgrade will take place Dec. 8th on the odd side. A list of the affected partitions is provided at the bottom of this notice. The nodes in those partitions will be drained prior to the work and will be powered down. Once the work is completed, those nodes will be returned to service. Current estimate is a 12 hour window. We will adjust as we know more. **MAINTENANCE TASKS** Cannon cluster will be paused during this maintenance?: **PARTIAL OUTAGE/YES** FASSE cluster will be paused during this maintenance?: **PARTIAL OUTAGE/YES** * Power work on Row 8A Even * Audience: Users of the partitions listed below * Impact: These nodes and partitions will be fully or partially down all day * Slurm upgrade to 25.05.4 * Audience: All cluster users * Impact: Jobs will be paused during maintenance * Block repo.anaconda.com cluster wide * Audience: Anyone attempting to use repo.anaconda.com * Impact: This change should not impact your Python workflow on the cluster. But if it does, consider using the open-source channel, `conda-forge`, through Miniforge distribution to install Python packages. This can be done by following our instructions on * Change Slurm User to Local User * Audience: All cluster users * Impact: Behind the scenes. No impact to users * Login node reboots (morning) * Audience: Anyone logged into a FASRC Cannon or FASSE login node * Impact: All login nodes will rebooted during this maintenance window * Netscratch cleanup ( ) * Audience: Cluster users * Impact: Files older than 90 days will be removed. Please note that retention cleanup can and does run at any time, not just during the maintenance window. **AFFECTED PARTITIONS** Nov. 3, 2025 - All Day Power Work Partial or Full Outage Apples to: arguelles\_delgado\_h100 bigmem dvorkin eddy enos gpu gpu\_h200 gpu\_requeue hsph hsph\_gpu intermediate itc\_cluster joonholee jshapiro kempner\_dev kemkpner\_eng kempner\_requeue mweber\_compute mweber\_gpu olveczky\_sapphire sapphire seas\_compute seas\_gpu serial\_requeue yao yao\_gpu yao\_priority test.

Nov 3, 11:00:01 GMT+0
Identificat - Maintenance is now in progress.

Nov 3, 20:04:13 GMT+0
Finalizat - Maintenance has completed successfully including power work at MGHPCC. A reminder that additional all-day power work will take place on Dec 8th, along with our maintenance from 9am-1pm.

FASRC monthly maintenance Monday October 6th, 2025 9am-1pm

Mon, 6 Oct 2025 13:00:00 +0000

Type: Maintenance

Duration: 4 hours

Affected Components: , , , , , , ,

Oct 6, 13:00:01 GMT+0
Identificat - Maintenance is now in progress.

Oct 6, 13:00:00 GMT+0
Identificat - FASRC monthly maintenance will take place Monday October 6th, 2025 from 9am-1pm **NOTICES** * Training: Upcoming training from FASRC and other sources can be found on our Training Calendar. at * Status Page: You can subscribe to our status to receive notifications of maintenance, incidents, and their resolution at (click Get Updates for options). * Upcoming holidays: Columbus / Indigenous Peoples’ Day - October 13 **MAINTENANCE TASKS** Cannon cluster will be paused during this maintenance?: **NO** FASSE cluster will be paused during this maintenance?: **NO** * DNS server reboots * Audience: All FASRC services * Impact: Rolling reboot should have no impact * Login node reboots * Audience: Anyone logged into a FASRC Cannon or FASSE login node * Impact: All login nodes will rebooted during this maintenance window * Netscratch cleanup ( ) * Audience: Cluster users * Impact: Files older than 90 days will be removed. Please note that retention cleanup can and does run at any time, not just during the maintenance window. Thank you, FAS Research Computing .

Oct 6, 17:00:00 GMT+0
Finalizat - Maintenance is now in progress.

VPN concentrator rolling updates overnight

Tue, 30 Sep 2025 06:00:00 +0000

Type: Maintenance

Sep 30, 07:00:00 GMT+0
Finalizat - Maintenance has completed successfully.

Sep 30, 06:00:00 GMT+0
Finalizat - Networking will be patching the VPN concentrators overnight. This will be done in a rolling order so that one ore more are always online. This may cause active VPN connections to drop, but they can be re-connected shortly after. ETA is one hour total..

Sep 30, 06:00:00 GMT+0
Identificat - Networking will be patching the VPN concentrators overnight. This will be done in a rolling order so that one ore more are always online. This may cause active VPN connections to drop, but they can be re-connected shortly after. ETA is one hour total..

FASRC monthly maintenance Monday September 8th, 2025 9am-1pm

Mon, 8 Sep 2025 13:00:00 +0000

Type: Maintenance

Duration: 4 hours

Affected Components: , , , , , , , , , , , , , , , , , , ,

Sep 8, 13:00:01 GMT+0
Identificat - Maintenance is now in progress.

Sep 8, 17:00:00 GMT+0
Finalizat - Maintenance has completed successfully.

Sep 8, 13:00:00 GMT+0
Identificat - FASRC monthly maintenance will take place Monday September 8th, 2025 from 9am-1pm **NOTICES** * Training: Upcoming training from FASRC and other sources can be found on our Training Calendar. at * Status Page: You can subscribe to our status to receive notifications of maintenance, incidents, and their resolution at (click Get Updates for options). * Upcoming holidays: Labor Day, Monday September 1st **MAINTENANCE TASKS** Cannon cluster will be paused during this maintenance?: **YES** FASSE cluster will be paused during this maintenance?: **YES** * Slurm Upgrade to 25.05.2 * Audience: All cluster users * Impact: Jobs and the scheduler will be paused during this upgrade * Domain controller work * Audience: Internal network * Impact: No impact expected * Login node reboots * Audience: Anyone logged into a FASRC Cannon or FASSE login node * Impact: All login nodes will rebooted during this maintenance window * Netscratch cleanup ( ) * Audience: Cluster users * Impact: Files older than 90 days will be removed. Please note that retention cleanup can and does run at any time, not just during the maintenance window. Thank you, FAS Research Computing .

Starfish upgrade Wednesday, August 20th 5PM-7PM

Wed, 20 Aug 2025 21:00:00 +0000

Type: Maintenance

Duration: 2 hours

Affected Components:

Aug 20, 21:00:01 GMT+0
Identificat - Maintenance is now in progress.

Aug 20, 21:00:00 GMT+0
Identificat - Starfish will be performing an upgrade on Wednesday, August 20th from 5PM-7PM. The web interface will be unavailable during that timeframe..

Aug 20, 23:00:00 GMT+0
Finalizat - Maintenance has completed successfully.

SEAS: seas_gpu partition GPU upgrades 8/11 - 8/14

Mon, 11 Aug 2025 10:00:00 +0000

Type: Maintenance

Duration: 4 days, 9 hours and 39 minutes

Affected Components:

Aug 11, 10:00:00 GMT+0
Identificat - Between 8/11/25 6AM - 8/14/25 5PM FASRC will be upgrading 14 of the H100 GPU nodes in the `seas_gpu `partition to H200 GPUs. This sill also affect `mweber_gpu` A reservation has been set which will drain the nodes of jobs prior to the maintenance. The SEAS GPU partition will be running at 75% capacity during these updates. FASRC has hundreds of GPU, so users should feel free to utilize `gpu_requeue` if needed for their jobs. Affected nodes: `mweber_gpu` nodes (13): ``` holygpu8a[18204,18301-18304,18401-18404,18501-18502,18601-18602] ``` seas\_gpu nodes (14): ``` holygpu8a[16101-16104,16201-16204,16301-16304,16401-16402] ``` Please reach out to [rchelp@rc.fas.harvard.edu](mailto:rchelp@rc.fas.harvard.edu) if you have any questions or concerns..

Aug 15, 14:13:24 GMT+0
Identificat - Maintenance is still in progress - imaging of the new H200 nodes is ongoing. Current ETA is end of day Friday. For further questions, please contact rchelp@rc.fas.harvard.edu.

Aug 15, 19:39:27 GMT+0
Finalizat - Maintenance has completed successfully. SEAS H200 nodes have been imaged and are back in service. .

Aug 11, 10:00:01 GMT+0
Identificat - Maintenance is now in progress.