Mirrornode testnet GCS recordstream halted
Incident Report for Hedera
Resolved
Backfill of the GCP bucket has been completed. Both the GCP and AWS buckets for testnet mirror nodes are at parity.
Posted Mar 28, 2023 - 20:43 UTC
Monitoring
A fix has been implemented and we are monitoring the results.
Posted Mar 23, 2023 - 23:00 UTC
Investigating
This update is meant for testnet mirrornode operators using the GCP bucket. In an effort to address technical debt from the last testnet reset, we cycled a credential on GCP. Unfortunately, we inadvertently deactivated a key, which led to the temporary disabling of uploaders.

We have since re-enabled the uploaders. However, backfilling the record stream is taking longer than expected, and estimate it will be fully resolved within the next 4-5 days. During this period testnet mirrornode operators relying solely on the GCP bucket may experience halted operations.

As a temporary workaround, we recommend using the AWS bucket in the meantime. Please note that this issue does not impact the Hedera-managed testnet mirrornode; it only affects operators running their own mirrornodes.

We apologize for any inconvenience this may cause and appreciate your patience as we work to resolve this issue. We will provide updates on the status page as the situation progresses.
Posted Mar 23, 2023 - 21:57 UTC