FAS Research Computing - h-nfs15-p inaccessible – Dettagli dell'incidente

Lieve interruzione in corso

Status page for the Harvard FAS Research Computing cluster and other resources.

Cluster Utilization (VPN and FASRC login required): Cannon | FASSE


Please scroll down to see details on any Incidents or maintenance notices.
Monthly maintenance occurs on the first Monday of the month (except holidays).

GETTING HELP
Documentation: https://docs.rc.fas.harvard.edu | Account Portal https://portal.rc.fas.harvard.edu
Email: rchelp@rc.fas.harvard.edu | Support Hours

The colors shown in the bars below were chosen to increase visibility for color-blind visitors.
For higher contrast, switch to light mode at the bottom of this page if the background is dark and colors are muted.

h-nfs15-p inaccessible

Risolto
Parzialmente fuori servizio
Avviato il 2 mesi faDurato 20 ore 5 minuti

Componenti interessati

Storage

Parzialmente fuori servizio da 7:00 PM a 3:05 PM

Boston Tier 2 NFS

Parzialmente fuori servizio da 7:00 PM a 3:05 PM

Aggiornamenti
  • Risolto
    UTC
    Risolto

    XFS journal recovery has completed, and kou_lab is remounted. The lab should verify the filesystem and any data that may have been written after 11:00 AM on 8/11/26 (Tuesday).

    In simple terms, too many open files caused the server to become unresponsive, which triggered an automatic reset. After the reboot, XFS required an extended recovery period before the filesystem could be mounted again.

    This does not mean the lab caused the issue. The file descriptor limit is a system-wide kernel resource shared by all filesystems and workloads on the server, so any share or process running a large job could have contributed. kou_lab was essentially collateral damage from machine-wide resource exhaustion, a “noisy neighbor” effect on a shared system.

  • Individuato
    UTC
    Individuato

    Most shares on h-nfs15-p are back up and accessible at this time.

    kou_lab storage is still down.

    We are continuing to work on a fix for this incident. Updates to come.

  • Indagine in corso
    UTC
    Indagine in corso

    Storage shares on h-nfs15-p may be inaccessible at this time. Affected labs include:

    • bellono_lab

    • debivort_lab

    • shakhnovich_lab

    • doyle_lab

    • holbrook_lab

    • kou_lab

    • koutrakis_lab

    • brennan_lab

    We are currently investigating this incident. We will continue to update as we know more

For issues not shown here, please contact FASRC via
https://portal.rc.fas.harvard.edu or email rchelp@rc.fas.harvard.edu