Production Incident Checklist
First 10 minutes
- Confirm the incident and affected service.
- Identify customer impact.
- Assign an incident owner.
- Check dashboards, alerts, logs, and recent changes.
- Determine whether the issue is isolated or widespread.
- Stop unsafe changes and consider rollback or failover.
During mitigation
- Keep one clear hypothesis at a time.
- Record important observations and actions.
- Prefer reversible changes.
- Communicate impact and status clearly.
- Do not confuse a workaround with root cause.
After recovery
- Verify the service from the user's perspective.
- Preserve relevant logs and evidence.
- Identify root cause and contributing factors.
- Create prevention actions with owners.
- Update the runbook so the next incident is faster to resolve.