Around 11:30 PM, one of the core GCFI breakers tripped resulting in the UPS array running until around just after midnight before dropping entirely.

We had not set up monitoring on the APC UPS as we hadn’t anticipated that as a failure point due to the reliability of the Colorado electrical grid, and didn’t anticipate the breaker itself flipping after months of continued use.

All services should be online again and working to catch up on any dropped content overnight, and we apologize again for the inconvenience.

We’ll be trying to identify the root cause throughout the day, working on the electrical connections, and setting up alerting through the APC UPS to detect and persistent A/C loss.

  • @DragoReborn7
    link
    English
    11 year ago

    Thanks a ton for concise update and for all of your hard work!