Streaming Logs Elevated Errors

Incident

08 November 2023, 19:45 UTC

Streaming Logs Elevated Errors

Status: closed

Start: 08 November 2023, 17:30 UTC

End: 08 November 2023, 19:45 UTC

Duration: 2 hours 15 minutes

Affected Components:

Observability Real-time Log Streaming

Affected Groups:

All Public Users

Investigating

08 November 2023, 17:30 UTC

We're investigating elevated errors in Streaming Logs.

All other locations and services are unaffected

Identified

08 November 2023, 17:33 UTC

This issue has been identified and a fix is being implemented.

Monitoring

08 November 2023, 17:55 UTC

A fix has been implemented and we are monitoring the results.

Monitoring

08 November 2023, 19:05 UTC

A fix has been implemented and we are still monitoring the results. We have seen partial improvement for the Streaming Logs. Our Dublin (DUB) POP is still experiencing elevated errors at this time.

Resolved

08 November 2023, 19:45 UTC

Engineering has confirmed that Streaming Logs has been fully restored. Customers sending log messages from the affected POPs would experience a similar proportion of log messages discarded from 16:50 to 19:35 UTC.

This incident is resolved.

Affected customers may have experienced impact to varying degrees and to a shorter duration than as set forth above.