facet3d - Notice history

Resolved — Authentication outage
1 Aug 2026, 04:47–10:58 UTC (6h 11m)

Signed-in users were unable to load the editor or dashboard. Session refresh requests failed, which signed people out and prevented projects from opening or saving.

Cause. A scheduled monthly verification job which restores our backups into an isolated environment to prove they work — required more temporary disk space than the server had available. It exhausted the disk, and our primary database stopped accepting writes as a result.

Resolution. The database was restarted and completed automatic recovery. Service was restored at 10:58 UTC.

Data. No data was lost or corrupted. The database refused new writes rather than writing bad data, which is the safe failure mode. Backups were unaffected and remain intact.

Prevention. The verification job is being changed to stream data rather than stage it on local disk, and our monitoring is being extended to detect database unavailability directly. It did not alert during this incident.


Thanks again for your patience and Happy Building !
Facet™ Team

We implemented a fix and are currently monitoring the result.

This incident has been resolved and all systems are operating normally. Thanks for bearing with us.