Streaming Logs Elevated Errors

Incident
09 November 2023, 16:29 UTC

Streaming Logs Elevated Errors

Status: closed
Start: 09 November 2023, 15:37 UTC
End: 09 November 2023, 16:29 UTC
Duration: 52 minutes
Affected Components:
Observability Real-time Log Streaming
Affected Groups:
All Public Users
Investigating

09 November 2023, 15:37 UTC

09 November 2023, 15:37 UTC

We're investigating elevated errors in Streaming Logs. 

Identified

09 November 2023, 15:41 UTC

09 November 2023, 15:41 UTC

This issue has been identified and a fix is being implemented. 

Monitoring

09 November 2023, 15:55 UTC

09 November 2023, 15:55 UTC

A fix has been implemented and we are monitoring the results.

Resolved

09 November 2023, 16:29 UTC

09 November 2023, 16:29 UTC

Engineering has confirmed that Streaming Logs has been fully restored. Customers sending log messages from the affected POPs would experience a similar proportion of log messages discarded from 15:20 to 15:53 UTC.

This incident is resolved.

Affected customers may have experienced impact to varying degrees and to a shorter duration than as set forth above.