Building Bulletproof Video Data Persistence
A routine deployment almost wiped 2,215 video timestamps across 76 meetings. I built triple-layer protection so it can never happen again.
A routine deployment almost wiped out months of work. A database migration command in the deploy script cleared all video data. I recovered from backups, but it was a wake-up call: 2,215 video timestamps across 76 archived meetings needed better protection.
Triple-Layer Protection
Layer 1: Scraper protection. The agenda scraper now preserves existing video URLs during updates:
if (!empty($existingItem->video_url)) {
$itemData['video_url'] = $existingItem->video_url;
}
Even a full re-scrape of all meetings leaves video timestamps intact.
Layer 2: JSON backups. An export command dumps all video data to a portable JSON file:
php artisan videos:export-data
# Creates: video_data_export_2025-08-11.json (76 meetings, 2,215 items)
php artisan videos:import-data
# Full restoration in ~25 seconds
Layer 3: Database seeders. A seeder runs during every deployment, automatically restoring any missing video data as a safety net.
Architecture Simplification
I also migrated from individual video clips to timestamp-based seeking:
Before: .../clips/45_c2a_resolution_3f0a13.mp4 (individual file)
After: .../full_video.mp4#t=3187 (timestamp in full video)
This eliminated FFmpeg clip extraction entirely -- 50% storage reduction, faster processing, simpler codebase, and standard HTML5 video seeking. The conversion touched 501 existing clip URLs, with previous URLs preserved in metadata for rollback.
The Lesson
Archived government meetings are historical records that can't be regenerated. They need stronger protection than regular application data. Deploy scripts that modify data should have safeguards: comment out dangerous commands by default, require explicit flags for destructive operations, create automatic backups before migrations.
Built while working on Democracy.cc.
Back to danielships ->