moyosapp_beta.0.0.3.3_beta1/docs/deployment-runbook.md

Deployment Runbook

Date: January 2025
Status: Complete
Purpose: Step-by-step deployment procedures for Moyos Wedding App

---

Overview

This runbook provides detailed procedures for deploying the Moyos Wedding App to production, including pre-deployment checks, deployment steps, post-deployment verification, and rollback procedures.

---

Pre-Deployment Checklist

Code Quality Checks

• All tests pass (npm test)
• TypeScript compiles (npm run typecheck)
• Linter passes (npm run lint)
• Build succeeds (npm run build)
• Security audit passed (npm run security:audit)

Environment Verification

• Environment variables configured (.env.production)
• Database migrations tested
• Database connection verified
• Redis connection verified (if used)
• Supabase configured (if used)

Infrastructure Checks

• Server resources available (CPU, RAM, disk)
• Backup system operational
• Monitoring configured
• SSL certificates valid
• DNS configured correctly

Documentation

• Changelog updated
• Release notes prepared
• Deployment plan reviewed
• Rollback plan prepared

---

Deployment Process

Step 1: Pre-Deployment Backup

# Create database backup
./scripts/backup-db.sh

# Create storage backup (if needed)
./scripts/backup-storage.sh

# Verify backups
ls -lh /opt/backups/db/ | head -5
ls -lh /opt/backups/storage/ | head -5

Time: ~5-10 minutes

---

Step 2: Deploy Application

Option A: Automated Deployment (Recommended)

# Production deployment
./scripts/deploy.sh --production

# Staging deployment
./scripts/deploy.sh --staging

Option B: Manual Deployment

# Navigate to app directory
cd /opt/moyos-wedding-app/app

# Pull latest code
git pull origin main

# Install dependencies
npm ci --production=false

# Generate Prisma client
npx prisma generate

# Run migrations
npx prisma migrate deploy

# Build application
npm run build

# Restart with PM2
pm2 reload moyos-wedding-app

Time: ~10-15 minutes

---

Step 3: Verify Deployment

# Check PM2 status
pm2 status

# Check application logs
pm2 logs moyos-wedding-app --lines 50

# Health check
curl -f https://your-domain.com/api/health

# Check metrics
curl -f https://your-domain.com/api/metrics

Expected Results:

• PM2 shows app as "online"
• Health check returns 200 OK
• No critical errors in logs

Time: ~5 minutes

---

Step 4: Smoke Tests

# Run smoke tests (if available)
./scripts/smoke-tests.sh

# Or manually test:
# - Landing page loads
# - RSVP form accessible
# - Admin login works
# - API endpoints respond

Time: ~5-10 minutes

---

Step 5: Monitor

# Monitor application
pm2 monit

# Watch logs
pm2 logs moyos-wedding-app --lines 100

# Check system resources
htop
df -h

Duration: 15-30 minutes post-deployment

---

Post-Deployment Verification

Application Health

• Health endpoint returns 200 OK
• Metrics endpoint accessible
• No critical errors in logs
• Response times acceptable (< 500ms p95)

Functional Tests

• Landing page loads correctly
• RSVP form submission works
• Admin dashboard accessible
• Gallery displays images
• Music requests functional
• Guestbook entries visible

Performance Checks

• Page load times < 2s
• API response times < 500ms
• Database query times < 100ms
• Memory usage stable
• CPU usage normal

Security Checks

• TLS/SSL working (HTTPS)
• Security headers present
• Rate limiting active
• CSRF protection working
• No exposed secrets

---

Rollback Procedure

When to Rollback

• Critical errors in logs
• Health checks failing
• Performance degradation
• Security issues discovered
• User-reported critical bugs

Rollback Steps

Step 1: Stop Application

pm2 stop moyos-wedding-app

Step 2: Restore Previous Version

# Option A: Git revert
cd /opt/moyos-wedding-app/app
git revert HEAD
git pull

# Option B: Checkout previous commit
git checkout <previous-commit-hash>

Step 3: Restore Database (if needed)

# Only if database changes were made
./scripts/restore-db.sh /opt/backups/db/wedding_app_backup_<timestamp>.sql.gz

Step 4: Rebuild and Restart

# Install dependencies
npm ci --production=false

# Generate Prisma client
npx prisma generate

# Build application
npm run build

# Restart application
pm2 restart moyos-wedding-app

Step 5: Verify Rollback

# Health check
curl -f https://your-domain.com/api/health

# Check logs
pm2 logs moyos-wedding-app --lines 50

Time: ~15-20 minutes

---

Zero-Downtime Deployment

Blue-Green Deployment (Advanced)

For zero-downtime deployments:

1. Deploy to Secondary Port

   PORT=3001 npm start
   pm2 start ecosystem.config.js --env production -- --port 3001
   

2. Update Nginx Upstream

   upstream app {
       least_conn;
       server 127.0.0.1:3000;  # Old instance
       server 127.0.0.1:3001;  # New instance
   }
   

3. Reload Nginx

   nginx -t
   systemctl reload nginx
   

4. Stop Old Instance

   pm2 stop moyos-wedding-app --instance 0
   

5. Update PM2 Config

   # Update ecosystem.config.js to use port 3000
   pm2 reload ecosystem.config.js
   

---

Deployment Troubleshooting

Build Fails

Symptoms:

• npm run build exits with error
• TypeScript compilation errors

Solutions:

• Check TypeScript errors: npm run typecheck
• Verify dependencies: npm ci
• Check Node.js version: node --version (should be 20.x)
• Review build logs for specific errors

Migration Fails

Symptoms:

• prisma migrate deploy fails
• Database connection errors

Solutions:

• Verify DATABASE_URL is correct
• Check database connectivity: psql $DATABASE_URL -c "SELECT 1;"
• Review migration files for errors
• Check database permissions

Application Won't Start

Symptoms:

• PM2 shows app as "errored" or "stopped"
• Health check fails

Solutions:

• Check logs: pm2 logs moyos-wedding-app --err
• Verify environment variables
• Check port availability: netstat -tulpn | grep 3000
• Verify .next directory exists
• Check disk space: df -h

Performance Issues

Symptoms:

• Slow response times
• High memory usage
• High CPU usage

Solutions:

• Check PM2 cluster mode: pm2 status
• Review application logs for slow queries
• Check database performance
• Monitor system resources: htop
• Review Nginx configuration

---

Deployment Schedule

Recommended Times

• Staging: Anytime (low traffic)
• Production: Off-peak hours (2-4 AM local time)
• Hotfixes: As needed (with approval)

Communication

• Notify team before deployment
• Post-deployment status update
• Document any issues encountered

---

Deployment Checklist Summary

Pre-Deployment

• All tests pass
• Build succeeds
• Environment configured
• Backups created
• Team notified

Deployment

• Code deployed
• Migrations run
• Application restarted
• Health checks pass

Post-Deployment

• Smoke tests pass
• Monitoring active
• No critical errors
• Performance acceptable
• Team notified of success

---

Emergency Contacts

• On-Call Engineer: [Contact Info]
• Database Admin: [Contact Info]
• Infrastructure Team: [Contact Info]

---

Deployment Log Template

Date: [Date]
Time: [Time]
Deployed By: [Name]
Environment: [staging/production]
Version: [Git commit hash]
Changes: [Brief description]

Pre-Deployment:
- [ ] Checklist completed

Deployment:
- [ ] Backup created
- [ ] Code deployed
- [ ] Migrations run
- [ ] Application restarted

Post-Deployment:
- [ ] Health checks passed
- [ ] Smoke tests passed
- [ ] Monitoring verified

Issues:
[Any issues encountered]

Rollback:
[If rollback was needed, document reason and steps]

---

Status: Deployment runbook complete and ready for use. ✅