Resolved
Query Timeouts for Subgraph APIs

Started
December 27, 2023 at 6:47 PM
Status
Resolved after 25 minutes

Impact

Operational
Affected
Subgraphs
API
  • Resolved
    Resolved

    This is now resolved.

    An AWS EFS disk that was powering our IPFS provider was corrupted, and all file requests were timing out. Recovery was slowed due to a deadlock in the EFS.

    To future proof, our reliability team is going to be working on providing more redundancy to this stack as a p0 priority.

    No data was lost.

  • Investigating
    Investigating

    We are seeing elevated timeouts and are working on resolving this issue.