FAS Research Computing - A cooling incident at the MGHPCC Datacenter has effected a number of systems – घटना विवरण

All cooling, including the water cooling for water-cooled compute nodes, is back online. All partitions are open for jobs. Some compute nodes in various partitions may still require individual attention, so not every compute node is back online, but we will work to bring them all online in the coming hours.

Most storage in Holyoke is back up.

The Slurm scheduler is back up and accepting jobs. However, most public partitions are down as the water cooling systems for those compute racks require in-person attention. RC staff are already en route to the datacenter to address this.

The Academic Cluster is back up.

A cooling failure caused temperatures in the MGHPCC datacenter to exceed the safe range of operation for many systems, causing them to power down to prevent permanent damage.

The cooling issue has been resolved and we are beginning to power systems back on. Expect outage on various systems until the issue is resolved.