Impact to Signal Sciences Data Collection

Incident
27 February 2024, 22:26 UTC

Impact to Signal Sciences Data Collection

Status: closed
Start: 27 February 2024, 19:43 UTC
End: 27 February 2024, 22:26 UTC
Duration: 2 hours 43 minutes
Affected Components:
Security Agents and Module Agent Downloads
Affected Groups:
All Public Users
Investigating

27 February 2024, 19:43 UTC

27 February 2024, 19:43 UTC

Our teams have detected an issue that is currently impacting Signal Sciences data collectors. New events from agents may not be processed by our back end services, agents may erroneously appear to be offline, and agents may not be receiving updated rules sets.

Customers may experience also delays in data available in the Signal Sciences console. 

Identified

27 February 2024, 20:17 UTC

27 February 2024, 20:17 UTC

Our engineers have identified the contributing factor and are applying a fix to our Agent Downloads service.


Monitoring

27 February 2024, 21:28 UTC

27 February 2024, 21:28 UTC

Engineering has deployed a fix and have confirmed a gradual recovery to our Agent Downloads service. We will continue to monitor until we’ve confirmed that customer experience has been fully restored.

Resolved

27 February 2024, 22:26 UTC

27 February 2024, 22:26 UTC

Engineering has confirmed that our Agents Downloads service has been fully restored. Customers may have experienced intermittent issues viewing data in their Signal Sciences Dashboard. In addition, customers would also see a delay on request blocking or no blocking decisions made by Signal Sciences agents from as early as 16:40 UTC and an increase in performance degradation at 19:20 UTC to 22:03 UTC.

This incident is resolved.

Affected customers may have experienced impact to varying degrees and to a longer or shorter duration than as set forth above.