No. 004 · Wing of the Global Push
One Regular Expression
· 27 minutes
Impact 3.6/10
- 1New managed WAF rule is deployed globally at once
- 2Its regular expression backtracks catastrophically on ordinary traffic
- 3CPU on every edge machine hits 100%
- 4Sites behind the network return 502s worldwide
- 5Global kill switch for the WAF rules restores traffic ✕
The fastest deploy pipeline is also the fastest outage pipeline.
Source: Cloudflare blog, July 2019
Impact
Five dimensions, 0 to 10 each. The overall 3.6/10 is their average. How impact is scored
| Dimension | Score | Basis |
|---|
| Reach | 8 | Sites behind Cloudflare, worldwide |
|---|
| Duration | 2 | 27 minutes |
|---|
| Loss | 0 | None stated in the source |
|---|
| Recovery | 2 | A global kill switch for the WAF rules |
|---|
| Cascade | 6 | Customer sites returned 502 errors |
|---|
Cloudflare regex outage, July 2019. Duration: 27 minutes. 5 hops from the first change to the failure.
See this exhibit in the 3D hall