Tested disaster recovery for a critical transactional system
Our approach to designing and — crucially — regularly testing disaster recovery for systems that cannot lose data.
This is a reference architecture describing how Shivacha approaches this class of system. It does not describe a specific client engagement and contains no client names or results.
Challenge
A transactional platform has backups but untested recovery procedures, unclear recovery objectives and no protection against ransomware scenarios.
Context
Relevant to fintech, healthcare, e-commerce and any organisation where downtime or data loss is costly.
Approach
How we approach it
- 1
Objectives per system
Recovery time and point objectives agreed with the business.
- 2
Layered protection
Multi-AZ high availability, cross-region replication and immutable backups.
- 3
Automated recovery
Scripted, infrastructure-as-code recovery environments.
- 4
Scheduled drills
Regular recovery tests including ransomware scenarios.
Implementation
Key implementation elements
Backup isolation
Backups in a separate, locked-down account.
Restore automation
One-command restore into a clean environment.
Failover runbooks
Step-by-step, rehearsed procedures.
Drill reports
Documented results and improvements after every test.
Outcome
The intended result is recovery that is proven rather than assumed: objectives defined, procedures automated and drills showing the system can actually be restored.
Lessons
- An untested backup is a hope, not a plan.
- Ransomware scenarios require backups the attacker cannot reach.
- Recovery objectives should be set by business impact, not technical preference.
Services
Services involved
Disaster Recovery
Disaster recovery design and testing — RTO/RPO-driven backup, replication and failover across regions and clouds.
Learn moreHigh Availability Architecture
High-availability architectures — redundancy, failover, load balancing and graceful degradation for critical systems.
Learn moreCloud Security
Cloud security architecture and posture management — IAM, network controls, encryption, logging and threat detection.
Learn moreMore work
Other reference architectures
Internal developer platform on Kubernetes with GitOps
How we build paved roads so product teams can create, deploy and operate services without infrastructure tickets.
Learn morePermission-aware enterprise knowledge assistant
How we design a RAG assistant that answers from thousands of internal documents while respecting every user's access rights.
Learn moreAgentic claims intake with human approval
A reference design for an AI workflow that reads claim submissions, extracts and validates data, and prepares cases for adjusters.
Learn moreNext step
Discuss a similar project.
Building something like "Tested disaster recovery for a critical transactional system"? Tell us about your goals and constraints and we will reply with an approach and next steps.
- Senior engineer reads every enquiry
- Reply within one business day
- NDA on request
Your details are used only to reply to this enquiry.