ORIGINAL COMPANY-CONTEXT PRACTICE
Recover a synthetic telecom activation workflow
Original practice scenario: A fictional service activation spans billing, subscriber records and network provisioning. One system succeeds while another times out. Design recovery behavior.
Written for independent practice using company context; this is not a verified question from a company interview.
A REASONED APPROACH
Model answer.
Model activation as a durable workflow with observable partial states. A timeout means the outcome is uncertain, so blindly repeating every operation can create inconsistent duplicate changes.
Work through it
- Give the activation and each external action stable idempotency identities, persisting intended state and acknowledgements before advancing stages.
- Query uncertain outcomes where supported, retry transient failures within a budget and define compensations for reversible steps rather than assuming atomic rollback.
- Test crashes between actions and acknowledgements, stale callbacks and compensation failures; expose pending recovery and manual review when automation cannot resolve state.
Tradeoffs to discuss
- A saga avoids distributed locking but requires carefully defined compensations and user-visible intermediate states.
Common pitfalls
- Do not mark the service active before required systems confirm.
- Compensation is a new business operation, not a guarantee that history can be erased.