Skip to main content

Storage Migration

After importing database records, you need to copy the actual media files — video files, HLS streaming playlists, thumbnails, previews, avatars, and captions — from PeerTube's storage to Vidra's storage backend.

PeerTube Storage Layout​

PeerTube stores media in a well-defined directory structure under its configured storage root (default: /var/www/peertube/storage/):

/var/www/peertube/storage/
├── videos/                          # Original video files (web-video format)
│   ├── {uuid}.mp4
│   ├── {uuid}-480.mp4
│   └── {uuid}-720.mp4
├── streaming-playlists/
│   └── hls/
│       └── {uuid}/                  # Per-video HLS directory
│           ├── master.m3u8
│           ├── {resolution}.m3u8
│           └── {uuid}-{resolution}-fragmented.mp4
├── thumbnails/                      # Video thumbnails
│   └── {uuid}.jpg
├── previews/                        # Video preview images
│   └── {uuid}.jpg
├── avatars/                         # User and channel avatars
│   └── {uuid}.png
├── captions/                        # Subtitle files
│   └── {uuid}-{language}.vtt
└── torrents/                        # Torrent files (optional)
    └── {uuid}-{resolution}.torrent

Vidra Storage Layout​

Vidra serves static files from compatible paths, so PeerTube clients continue to work:

/data/vidra/storage/
├── web-videos/                      # Served at /static/web-videos/
│   └── {uuid}.mp4
├── streaming-playlists/
│   └── hls/                         # Served at /static/streaming-playlists/hls/
│       └── {uuid}/
│           ├── master.m3u8
│           └── *.m3u8 / *.mp4
├── thumbnails/
│   └── {uuid}.jpg
├── previews/
│   └── {uuid}.jpg
├── avatars/
│   └── {uuid}.png
└── captions/
    └── {uuid}-{language}.vtt

Migration by Storage Type​

PeerTube Local → Vidra Local​

The simplest path: use rsync to copy files directly.

# Copy all media directories
rsync -avz --progress \
  user@peertube-server:/var/www/peertube/storage/videos/ \
  /data/vidra/storage/web-videos/

rsync -avz --progress \
  user@peertube-server:/var/www/peertube/storage/streaming-playlists/ \
  /data/vidra/storage/streaming-playlists/

rsync -avz --progress \
  user@peertube-server:/var/www/peertube/storage/thumbnails/ \
  /data/vidra/storage/thumbnails/

rsync -avz --progress \
  user@peertube-server:/var/www/peertube/storage/previews/ \
  /data/vidra/storage/previews/

rsync -avz --progress \
  user@peertube-server:/var/www/peertube/storage/avatars/ \
  /data/vidra/storage/avatars/

rsync -avz --progress \
  user@peertube-server:/var/www/peertube/storage/captions/ \
  /data/vidra/storage/captions/
Incremental Sync

Run rsync multiple times before cutover. Only the final run (with PeerTube in maintenance mode) needs to transfer the delta. This minimizes downtime.

PeerTube Local → Vidra S3​

Upload PeerTube's local files to your S3-compatible bucket:

# Install AWS CLI (or compatible tool for Backblaze B2, DigitalOcean Spaces, etc.)
# Configure credentials in ~/.aws/credentials or environment variables

# Upload web videos
aws s3 sync /var/www/peertube/storage/videos/ \
  s3://vidra-storage/web-videos/ \
  --endpoint-url $S3_ENDPOINT

# Upload HLS playlists
aws s3 sync /var/www/peertube/storage/streaming-playlists/ \
  s3://vidra-storage/streaming-playlists/ \
  --endpoint-url $S3_ENDPOINT

# Upload thumbnails, previews, avatars, captions
for dir in thumbnails previews avatars captions; do
  aws s3 sync /var/www/peertube/storage/$dir/ \
    s3://vidra-storage/$dir/ \
    --endpoint-url $S3_ENDPOINT
done

PeerTube S3 → Vidra S3​

If both PeerTube and Vidra use S3-compatible storage:

# Cross-bucket sync (same provider)
aws s3 sync s3://peertube-videos/ s3://vidra-storage/web-videos/ \
  --endpoint-url $S3_ENDPOINT

aws s3 sync s3://peertube-streaming/ s3://vidra-storage/streaming-playlists/ \
  --endpoint-url $S3_ENDPOINT

For different providers, download locally first, then upload:

# Download from PeerTube's S3
aws s3 sync s3://peertube-videos/ /tmp/migration/videos/ \
  --endpoint-url $PEERTUBE_S3_ENDPOINT

# Upload to Vidra's S3
aws s3 sync /tmp/migration/videos/ s3://vidra-storage/web-videos/ \
  --endpoint-url $VIDRA_S3_ENDPOINT

PeerTube Local → Vidra IPFS​

If Vidra is configured with IPFS storage, files need to be added to IPFS and their CIDs recorded:

# Add video files to IPFS
for file in /var/www/peertube/storage/videos/*.mp4; do
  CID=$(ipfs add -Q "$file")
  echo "$file -> $CID" >> migration-cids.log
done

# Pin all added content
ipfs pin add $(cat migration-cids.log | awk '{print $3}')

After adding files to IPFS, update the ipfs_cid field in Vidra's video records:

-- Update video records with IPFS CIDs
-- (Use the mapping from migration-cids.log)
UPDATE videos SET ipfs_cid = 'QmXyz...' WHERE id = 'video-uuid';

Verification​

After copying media, verify file integrity:

# Count files
echo "PeerTube videos: $(find /var/www/peertube/storage/videos -name '*.mp4' | wc -l)"
echo "Vidra videos: $(find /data/vidra/storage/web-videos -name '*.mp4' | wc -l)"

# Spot-check with checksums
md5sum /var/www/peertube/storage/videos/some-uuid.mp4
md5sum /data/vidra/storage/web-videos/some-uuid.mp4

# Verify HLS playlists are valid
for m3u8 in /data/vidra/storage/streaming-playlists/hls/*/master.m3u8; do
  if ffprobe "$m3u8" 2>/dev/null; then
    echo "OK: $m3u8"
  else
    echo "BROKEN: $m3u8"
  fi
done

Test Playback​

Before cutover, test that Vidra can serve the migrated files:

# Check that static file serving works
curl -I http://your-vidra-host:8080/static/web-videos/some-uuid.mp4
# Should return 200 OK with Content-Type: video/mp4

curl -I http://your-vidra-host:8080/static/streaming-playlists/hls/some-uuid/master.m3u8
# Should return 200 OK with Content-Type: application/vnd.apple.mpegurl

Estimating Transfer Time​

Storage SizeMethodEstimated Time
10 GBrsync over LAN~5 min
10 GBrsync over WAN (100 Mbps)~15 min
100 GBrsync over LAN~45 min
100 GBS3 sync (same region)~30 min
1 TBrsync over WAN~8+ hours
1 TBS3 sync (cross-region)~4+ hours

For large instances (1 TB+), start the initial rsync days before the cutover window. Only the final incremental sync needs to happen during downtime.

Next Steps​