Goldsky - Query Timeouts for Subgraph APIs – Incident details

Query Timeouts for Subgraph APIs

Resolved
Operational
Started 10 months agoLasted 25 minutes

Affected

Subgraphs

Operational from 6:47 PM to 7:12 PM

API

Operational from 6:47 PM to 7:12 PM

Updates
  • Resolved
    Resolved

    This is now resolved.

    An AWS EFS disk that was powering our IPFS provider was corrupted, and all file requests were timing out. Recovery was slowed due to a deadlock in the EFS.

    To future proof, our reliability team is going to be working on providing more redundancy to this stack as a p0 priority.

    No data was lost.

  • Investigating
    Investigating

    We are seeing elevated timeouts and are working on resolving this issue.