P0 INCIDENTRace Condition in Distributed Payment Processing#ENG-WAR-040StripePayPal
Senior~30 min15:00
API CPU Usage99.2%↑ 42%
P99 Latency2450 ms↑ 400%
5xx Error Rate12.4%↑ 12%
DB Connections14,492↑ 800%
bastion-prod-1.internal — bash
[SYSTEM] War-Room terminal initialised. Bastion host connection established.
[SYSTEM] Active incident: Race Condition in Distributed Payment Processing
[SYSTEM] Type "help" for a list of investigation commands.
user@bastion:~$
Execute Remediation⚠ PROD
Your loyalty points redemption system has a bug: some customers have negative point balances (e.g., -500 points). Customer support confirms these customers didn't cheat — they appear to have made two rapid simultaneous purchases that both used the same points. The application code checks: if user.points >= purchase_cost: user.points -= purchase_cost but doesn't lock the row during the check-then-act.

What is your first action?