FAS Research Computing - h-nfs15-p inaccessible – Деталі інциденту

Всі системи працюють

Status page for the Harvard FAS Research Computing cluster and other resources.

Cluster Utilization (VPN and FASRC login required): Cannon | FASSE


Please scroll down to see details on any Incidents or maintenance notices.
Monthly maintenance occurs on the first Monday of the month (except holidays).

GETTING HELP
Documentation: https://docs.rc.fas.harvard.edu | Account Portal https://portal.rc.fas.harvard.edu
Email: rchelp@rc.fas.harvard.edu | Support Hours


The colors shown in the bars below were chosen to increase visibility for color-blind visitors.
For higher contrast, switch to light mode at the bottom of this page if the background is dark and colors are muted.

h-nfs15-p inaccessible

Вирішено
Часткова недоступність
Почалося 6 днів томуТривала близько 20 годин

Постраждалі

Storage

Часткова недоступність від 7:00 PM до 3:05 PM

Boston Tier 2 NFS (new)

Часткова недоступність від 7:00 PM до 3:05 PM

Оновлення
  • Вирішено
    UTC
    Вирішено

    XFS journal recovery has completed, and kou_lab is remounted. The lab should verify the filesystem and any data that may have been written after 11:00 AM on 8/11/26 (Tuesday).

    In simple terms, too many open files caused the server to become unresponsive, which triggered an automatic reset. After the reboot, XFS required an extended recovery period before the filesystem could be mounted again.

    This does not mean the lab caused the issue. The file descriptor limit is a system-wide kernel resource shared by all filesystems and workloads on the server, so any share or process running a large job could have contributed. kou_lab was essentially collateral damage from machine-wide resource exhaustion, a “noisy neighbor” effect on a shared system.

  • Визначено
    UTC
    Визначено

    Most shares on h-nfs15-p are back up and accessible at this time.

    kou_lab storage is still down.

    We are continuing to work on a fix for this incident. Updates to come.

  • Вивчається
    UTC
    Вивчається

    Storage shares on h-nfs15-p may be inaccessible at this time. Affected labs include:

    • bellono_lab

    • debivort_lab

    • shakhnovich_lab

    • doyle_lab

    • holbrook_lab

    • kou_lab

    • koutrakis_lab

    • brennan_lab

    We are currently investigating this incident. We will continue to update as we know more