P0 INCIDENTRedis maxmemory Evicting Hot Keys Mid-Traffic#ENG-WAR-025AmazonNetflix
Mid~20 min15:00
API CPU Usage99.2%↑ 42%
P99 Latency2450 ms↑ 400%
5xx Error Rate12.4%↑ 12%
DB Connections14,492↑ 800%
bastion-prod-1.internal — bash
[SYSTEM] War-Room terminal initialised. Bastion host connection established.
[SYSTEM] Active incident: Redis maxmemory Evicting Hot Keys Mid-Traffic
[SYSTEM] Type "help" for a list of investigation commands.
user@bastion:~$
Execute Remediation⚠ PROD
Users are being randomly logged out. Customer support tickets are flooding in. Redis memory usage shows 100% utilization. Redis INFO stats show evicted_keys increasing rapidly. The Redis instance stores both session tokens (must never be evicted) and product catalog cache (fine to evict). The Redis maxmemory-policy is set to allkeys-lru, which evicts any key regardless of type.

What is your first action?