Nodes showing as disconnected

Incident Report for Trustgrid Public

Resolved

At this time we believe all nodes have corrected their control plane connection status. We have a fix for the issue that is causing the backend service to crash and it will be deployed to production in a cloud release early next week.
Posted Feb 21, 2025 - 14:21 CST

Update

We are still seeing a large number of nodes showing disconnected after the control plane service stablized. We are taking steps to attempt to resolve this.
Posted Feb 21, 2025 - 13:44 CST

Monitoring

The issue appears to have resolved. We have confirmed the same control plane service had an issue as on February 14 and we are confirming the same root cause.

For clarification, there is no indication that this impacting data plane traffic. Only connectivity to the Trustgrid control plane.
Posted Feb 21, 2025 - 13:27 CST

Investigating

We are seeing a large number of disconnected nodes which seems similar to the event on February 14th. We are actively investigating the issue.
Posted Feb 21, 2025 - 13:20 CST
This incident affected: Node (Cloud Controller).