FAS Research Computing - h-nfs15-p inaccessible – Incidentdetaljer

Alla system fungerar

Status page for the Harvard FAS Research Computing cluster and other resources.

Cluster Utilization (VPN and FASRC login required): Cannon | FASSE


Please scroll down to see details on any Incidents or maintenance notices.
Monthly maintenance occurs on the first Monday of the month (except holidays).

GETTING HELP
Documentation: https://docs.rc.fas.harvard.edu | Account Portal https://portal.rc.fas.harvard.edu
Email: rchelp@rc.fas.harvard.edu | Support Hours

The colors shown in the bars below were chosen to increase visibility for color-blind visitors.
For higher contrast, switch to light mode at the bottom of this page if the background is dark and colors are muted.

h-nfs15-p inaccessible

Löst
Partiellt avbrott
Började sex dagar sedanVarade ungefär 20 timmar

Påverkade

Storage

Partiellt avbrott från 7:00 PM till 3:05 PM

Boston Tier 2 NFS (new)

Partiellt avbrott från 7:00 PM till 3:05 PM

Uppdateringar
  • Löst
    UTC
    Löst

    XFS journal recovery has completed, and kou_lab is remounted. The lab should verify the filesystem and any data that may have been written after 11:00 AM on 8/11/26 (Tuesday).

    In simple terms, too many open files caused the server to become unresponsive, which triggered an automatic reset. After the reboot, XFS required an extended recovery period before the filesystem could be mounted again.

    This does not mean the lab caused the issue. The file descriptor limit is a system-wide kernel resource shared by all filesystems and workloads on the server, so any share or process running a large job could have contributed. kou_lab was essentially collateral damage from machine-wide resource exhaustion, a “noisy neighbor” effect on a shared system.

  • Identifierat
    UTC
    Identifierat

    Most shares on h-nfs15-p are back up and accessible at this time.

    kou_lab storage is still down.

    We are continuing to work on a fix for this incident. Updates to come.

  • Undersöker
    UTC
    Undersöker

    Storage shares on h-nfs15-p may be inaccessible at this time. Affected labs include:

    • bellono_lab

    • debivort_lab

    • shakhnovich_lab

    • doyle_lab

    • holbrook_lab

    • kou_lab

    • koutrakis_lab

    • brennan_lab

    We are currently investigating this incident. We will continue to update as we know more

For issues not shown here, please contact FASRC via
https://portal.rc.fas.harvard.edu or email rchelp@rc.fas.harvard.edu