All systems operational

API
100.0% uptime
Nov 2023 · 100.0%Dec · 100.0%Jan 2024 · 100.0%
Nov 2023100.0% uptime
Dec 2023100.0% uptime
Jan 2024100.0% uptime
Dashboard
100.0% uptime
Nov 2023 · 100.0%Dec · 100.0%Jan 2024 · 100.0%
Nov 2023100.0% uptime
Dec 2023100.0% uptime
Jan 2024100.0% uptime

Mirror

100.0% uptime

Mirror Replication

100.0% uptime
Nov 2023 · 100.0%Dec · 100.0%Jan 2024 · 100.0%
Nov 2023100.0% uptime
Dec 2023100.0% uptime
Jan 2024100.0% uptime

Chain Ingestion

100.0% uptime
Nov 2023 · 100.0%Dec · 100.0%Jan 2024 · 100.0%
Nov 2023100.0% uptime
Dec 2023100.0% uptime
Jan 2024100.0% uptime

Subgraph Sources

100.0% uptime
Nov 2023 · 100.0%Dec · 100.0%Jan 2024 · 100.0%
Nov 2023100.0% uptime
Dec 2023100.0% uptime
Jan 2024100.0% uptime

Subgraphs

100.0% uptime

API

Operational

Chain Ingestion

100.0% uptime
Nov 2023 · 100.0%Dec · 100.0%Jan 2024 · 100.0%
Nov 2023100.0% uptime
Dec 2023100.0% uptime
Jan 2024100.0% uptime

Arweave Search Gateway

99.59% uptime
Nov 2023 · 100.0%Dec · 98.81%Jan 2024 · 100.0%
Nov 2023100.0% uptime
Dec 202398.81% uptime
Jan 2024100.0% uptime

Notice history

Jan 2024

No notices reported this month

Dec 2023

Subgraph deploy API timeouts
  • Resolved
    Resolved

    This incident has been resolved. Our IPFS backend would fail to automatically recover when it encountered a race condition. We made changes to the backend so that recovery should be automatic and should not cause intermittent issues with subgraph deploys.

  • Investigating
    Investigating

    When deploying a subgraph, you may see a Deployment Failed error.

    While we work to resolve this issue, please try redeploying the subgraph after a few minutes.

Query Timeouts for Subgraph APIs
  • Resolved
    Resolved

    This is now resolved.

    An AWS EFS disk that was powering our IPFS provider was corrupted, and all file requests were timing out. Recovery was slowed due to a deadlock in the EFS.

    To future proof, our reliability team is going to be working on providing more redundancy to this stack as a p0 priority.

    No data was lost.

  • Investigating
    Investigating

    We are seeing elevated timeouts and are working on resolving this issue.

API Timeouts for some subgraphs
  • Resolved
    Resolved

    There were some routing issues that caused timeouts for some subgraphs. This was resolved, but the team will continue monitoring.

  • Identified
    Identified

    We are seeing elevated timeouts and are working on resolving this issue.

Arweave Search Gateway will be undergoing maintenance
  • Resolved
    Resolved

    This incident has been resolved.

  • Identified
    Identified

    We will be halting new indexing to the arweave search gateway as we update the underlying infrastructure for faster queries.

Nov 2023

SQL Validation temporarly disabled
  • Resolved
    Resolved

    Our pre-run SQL Validation is temporarily disabled due to it failing validation for valid pipeline SQL. All pipelines will work as expected, but error messages won't surface until the pipeline is run.

Mirror pipelines may be unstable
  • Resolved
    Resolved

    This incident has been resolved.

  • Monitoring
    Monitoring

    All pipelines are now stable and we are monitoring in case of regression.

  • Identified
    Identified

    Our upstream cloud provider is having network connection issues. We are working with them on mitigating effects to customers, but there will be lag on when data is delivered to sinks.

Nov 2023 to Jan 2024