FAS Research Computing - Hinweisverlauf

Our maintenance tasks should be completed between 9am-1pm.

- CDU work continues until October 2nd 8am-5pm but will not affect running jobs or new jobs.

Phase 1 is complete. See status page for timeline and affected partitions.

​- Power work on row 7c October 4th 5pm - October 5th 5pm. See status page for affected partitions.

NOTICES:

• VPN: The VPN will be cut over to new hardware on Sunday Sept. 27th. connections will drop and you will need to reconnect.

• Training: Upcoming training from FASRC and other sources can be found on our Training Calendar. at https://www.rc.fas.harvard.edu/upcoming-training/

• Status Page: You can subscribe to our status to receive notifications of maintenance, incidents, and their resolution at https://status.rc.fas.harvard.edu/ (click Get Updates for options).

MAINTENANCE TASKS

Cannon cluster will be paused during this maintenance?: YES
FASSE cluster will be paused during this maintenance?: YES

• Network maintenance 

  • Audience: Clusters (Cannon and FASSE)

  • Impact: The cluster will be paused during this work.
    Some network disruption may occur.

• Login node reboots 

  • Audience: All login nodes

  • Impact: Login nodes will be unavailable until after maintenance

• OOD/Open OnDemand reboots

  • Audience: All OOD users

  • Impact: OOD will be unavailable until after maintenance

• Netscratch 90-day retention cleanup

  • Audience; All netscratch users

  • Update: Due to low space on netscratch, please note that retention cleanup will be run more frequently. We are reviewing our 90 day policy with a possible age reduction.

  • Impact: Files older than 90 days will be removed per our scratch policy. Please note that this cleanup can happen at any time, not just during maintenance.

Thank you,
FAS Research Computing
https://docs.rc.fas.harvard.edu/
https://www.rc.fas.harvard.edu/upcoming-training/

FASRC will be doing OS upgrades to the latest version of Rocky 8.10 from August 24-27th. These rolling upgrades will improve cluster security and install the latest Cuda 13.2 driver.

Upgrades will happen in stages with each stage occurring between 8a-5p each day. During that period the nodes being upgraded will be unavailable. No jobs will be canceled but jobs will stay pending if all nodes in the partition are down.

Note that during this week the cluster will be in a partially upgraded state, thus jobs that span multiple nodes may become unstable due to mismatching libraries. Users concerned about this should wait until August 28th to restart runs.

Users should plan their work accordingly.

The upgrade schedule with impacted partitions is as follows:

August 24th:

FASSE
boslogin
private login nodes

August 25th: Cannon Part 1

holylogin
arguelles_delgado
blackhole
davies_gpu
davies
desai
eddy
holy-cow
holy-smokes
huce_bigmem
huce_cascade
huttenhower
jacobsen2
janson_bigmem
janson_cascade
janson
ke
lukin
nguyen
olveczky_gpu
remoteviz
seas_compute
shared
sompolinsky_gpu
tambe
vishwanath
whipple
xlin
xlin_ice
zhuang_gpu
zhuang

August 26th: Cannon Part 2
arguelles_delgado
conroy
davies
doshi-velez
dsouza
edwards
geophysics
giribet
gpu_test
hernquist
huce_cascadeimasc
janson
kempner_h200
kempner_rtx
murphy
ni_lab
olveczky
ortegahernandez
pehlevan
seas_compute
shakhnovich
shared
unrestricted
xlin
yin
zon

August 27th: Cannon Part 3

arguelles_delgado_gpu_a100
arguelles_delgado_gpu_mixed
arguelles_delgado_h100
bigmem_intermediate
bigmem
blackhole_gpu
dvorkin
eddy
enos
gershman
gpu
gpu_h200
hejazi
hernquist_ice
hoekstra
hsph_gpu
hsph
huce_ice
iaifi_gpu
intermediate
itc_cluster
itc_gpu
janson_sapphire
joonholee
jshapiro
kempner_h100
kempner_h200
kempner
kempner_interactive
kovac
kozinsky_gpu
kozinsky
murphy_ice
mweber_compute
mweber_gpu
olveczky_sapphire
ortegahernandez_ice
rivas
sapphire
seas_compute
seas_gpu
seas_gpu_perf
siag_gpu
siag_combo
siag
sur
test
yao_alphatns
yao_gpu
yao
zhuang