Cooling Distribution Unit (CDU) work 9/22 - 10/2 See details for affected partitions and dates
تأثیر میگذارد بر
- تکمیل شد · اکتبر 02, 2026 در 3:30 ب.ظ.UTC
CDU maintenance has completed and partition reservations have been removed.
Thank you for your patience for this important infrastructure upkeep.
- بهروزرسانی · سپتامبر 24, 2026 در 5:32 ب.ظ.UTC
CDU 5 work is complete.
CDU 6 work is in progress.
- بهروزرسانی · سپتامبر 22, 2026 در 11:00 ق.ظ.UTCMaintenance is now in progress
- در حال انجام · سپتامبر 09, 2026 در 7:32 ب.ظ.UTC
Over the past year we have noticed that water circulating through two of the Cooling Distribution Units (CDU) in Row 8a and their compute nodes has become significantly discoloured. This impurity causes cooling problems which as a result increases chances of node failure. To remedy FASRC has scheduled a full flush of these CDUs and their attached nodes. Unfortunately to do the full flush we have to fully power down all the nodes and drain the water, which is a process that takes several days to complete.
To minimize disruption we have staggered this work over two weeks. The work on CDU5 will take place from 9/22 - 9/25, while CDU6 will take place from 9/29 - 10/2. Lists of impacted partitions are below. For those impacted we recommend using other resources on the cluster during that time such as shared and gpu_h200 or any of the requeue partitions. Note this work impacts both Cannon and FASSE.
No jobs will be cancelled, rather blocking reservations are in place to naturally drain the nodes prior to the work.
Thank you for your patience as we work to improve cluster stability and hardware longevity.
CDU 5 (9/22 - 9/25) -Complete
Cannon:arguelles_delgado_gpu_a100arguelles_delgado_gpu_mixedbigmem_intermediateblackhole_gpueddygershmanhejazihernquist_icehoekstrahuce_iceiaifi_gpuitc_gpujshapirokovackozinskykozinsky_gpumurphy_icortegahernandez_icerivasseas_computeseas_gpusiagsiag_gpusiag_combosurzhuangFASSE:fasse_ultramemCDU 6 (9/29 - 10/2)
Cannon:
arguelles_delgado_h100
bigmem
dvorkin
eddy
enos
gpu
hsph
hsph_gpu
intermediate
itc_cluster
janson_sapphire
joonholee
jshapiro
olveczky_sapphire
sapphire
test
yao
yao_alphatns
yao_gpu
FASSE:
cnl

