Moving a production system serving 200,000 users from on-premise servers to AWS without any downtime sounds like the kind of thing that only happens in vendor case studies. We know, because we've read those case studies and they always omit the parts that were actually hard.
This is the unedited version. Here's exactly what we did for Apex Corp — and more importantly, what almost went wrong.
The Starting Point
Apex Corp was running their entire platform on three physical servers in a data centre in Kuala Lumpur. The setup had been running for eight years and was characterised by:
- A 16-year-old PostgreSQL database with no documented schema and undocumented stored procedures
- Application servers running Ubuntu 14.04 (end-of-life since 2019)
- No monitoring beyond "call the DBA when something breaks"
- Manual deployment processes documented in a Word document last updated in 2018
- 200,000 active users who could not tolerate downtime during business hours
Phase 1: Discovery and Documentation (Weeks 1–3)
Before we touched any infrastructure, we spent three weeks understanding everything about the existing system. This included documenting every network dependency, mapping every cron job and scheduled task, profiling the database to understand query patterns, and identifying all hardcoded IP addresses in application code.
The hardcoded IP addresses alone took a full week to find and replace. They were everywhere — application configuration files, database stored procedures, and even in a table in the production database itself.
Phase 2: Parallel Infrastructure (Weeks 4–8)
We built the entire AWS environment in parallel with the existing production system, with no traffic running to it initially. This included RDS PostgreSQL with read replicas, ECS Fargate clusters for the application tier, CloudFront for static asset delivery, ElastiCache for session storage, and Terraform for all infrastructure as code.
The database migration was the riskiest part. We set up AWS Database Migration Service to continuously replicate data from the on-premise PostgreSQL to RDS, with a lag that stayed under 2 seconds. This meant that when we cut over, we'd have an up-to-date database ready to go.
Phase 3: Dry Runs and Load Testing (Weeks 9–11)
We ran four complete migration dress rehearsals. Each simulated the full cutover process, measured how long each step took, and identified things that didn't work as planned.
Rehearsal two revealed that our application deployment to ECS took 12 minutes — longer than planned for in the maintenance window. We optimised the container image and got it down to 4 minutes. Rehearsal three revealed that one stored procedure referenced a server-specific file path for PDF generation. That would have been a production incident.
We load-tested the AWS environment at 3x peak production traffic. It handled it without issue. The on-premise environment struggled at 1.5x peak.
Phase 4: The Cutover (Saturday 2am)
We chose Saturday 2am AEST (the lowest traffic window) for the actual cutover:
- 2:00am — Enabled maintenance mode on the existing load balancer
- 2:03am — Stopped all write traffic to the old database
- 2:04am — Verified DMS replication had caught up (0 seconds lag)
- 2:05am — Promoted RDS replica to primary
- 2:09am — Deployed application containers to ECS
- 2:13am — Ran smoke tests against the new environment
- 2:18am — Updated DNS (60-second TTL set 24 hours earlier)
- 2:20am — Disabled maintenance mode; monitored dashboards for 2 hours
Total downtime: 20 minutes planned maintenance. Zero user-visible errors after the window lifted.
What We Set Up for the Future
After cutover, Apex Corp had: DataDog monitoring with alerting on all critical metrics, automated database backups with 30-day retention and tested restore procedures, auto-scaling configured for the application tier, a proper DR environment in a second AWS region, and CI/CD pipelines that deploy in 6 minutes, replacing the 2-hour manual process.
Infrastructure cost is down 43% from the previous hardware bill.
The Lesson
Zero-downtime migrations aren't complicated — they're disciplined. The work is in the discovery, the dress rehearsals, and the preparation. The actual cutover night should be boring if you've done the work correctly.
If you're considering a cloud migration, start with a conversation. Our cloud team has run migrations for systems of every size.
