systemdesign.comDefining Reliability Thresholds
Set concrete, measurable reliability targets based on user impact and system context.
Reliability thresholds are not arbitrary numbers—they must reflect user expectations and business impact. Start by identifying what failures users notice and how those failures affect their workflow. Define acceptable downtime, error rates, or latency increases per time window, tying each to observable outcomes like 'checkout failure rate must stay below 0.1% of sessions'.
Break thresholds into layers: user-facing, service-level, and component-level. User-facing thresholds might demand 99.9% availability over 30 days. Service-level could set max retry latency of 200 ms for 99th percentile. Component-level might require automatic failover within 10 seconds. Each layer informs testing and monitoring goals.
Thresholds must be revisited when requirements change or scale shifts significantly. A 99.5% availability target may suffice for internal tools but not for core user-facing services. Record decisions, assumptions, and failure cost estimates. This enables informed investment: spending more on a component only up to the point where reliability gains no longer justify complexity or cost.
Key points
- Base thresholds on user impact, not conventions.
- Define layered targets for user, service, and component levels.
- Revisit thresholds as requirements or scale evolve.