skyview/docs/MIGRATION_GUIDE.md

# SkyView Database Migration Guide

This guide covers the transition from SkyView's in-memory data storage to persistent SQLite database storage, introduced in version 0.1.0.

## Overview of Changes

### What's New
- **Persistent Storage**: Aircraft position history survives restarts
- **Callsign Enhancement**: Enriched aircraft information with airline/airport data
- **Embedded Aviation Database**: OpenFlights airline and airport data included
- **Configurable Retention**: Control how long historical data is kept
- **Privacy Controls**: Comprehensive privacy mode for sensitive environments

### What's Changed
- **Memory Usage**: Reduced memory footprint for aircraft tracking
- **Startup Time**: Slightly longer initial startup due to database initialization
- **Configuration**: New database and callsign configuration sections
- **File Structure**: New database file created in system or user directories

## Pre-Migration Checklist

### System Requirements
- **Disk Space**: Minimum 100MB available for database and backups
- **Permissions**: Write access to `/var/lib/skyview/` (system) or `~/.local/share/skyview/` (user)
- **SQLite**: No additional installation required (embedded in SkyView)

### Current Data
⚠️ **Important**: In-memory aircraft data from previous versions cannot be preserved during migration. Historical tracking will start fresh after the upgrade.

## Migration Process

### Automatic Migration (Recommended)

#### For Debian Package Users
```bash
# Update to new version
sudo apt update && sudo apt upgrade skyview-adsb

# Service will automatically restart and initialize database
sudo systemctl status skyview
```

#### For Manual Installation Users
```bash
# Stop current SkyView instance
sudo systemctl stop skyview  # or kill existing process

# Backup current configuration
cp /etc/skyview/config.json /etc/skyview/config.json.backup

# Start new version (database will be created automatically)
sudo systemctl start skyview
```

### Manual Database Setup

If automatic initialization fails, you can manually initialize the database:

```bash
# Create database directory
sudo mkdir -p /var/lib/skyview
sudo chown skyview:skyview /var/lib/skyview

# Run SkyView with explicit database initialization
sudo -u skyview /usr/bin/skyview --init-database --config /etc/skyview/config.json
```

## Configuration Updates

### New Configuration Sections

Add these sections to your existing `config.json`:

```json
{
  "database": {
    "path": "",
    "max_history_days": 7,
    "backup_on_upgrade": true
  },
  "callsign": {
    "enabled": true,
    "cache_hours": 24,
    "external_apis": true,
    "privacy_mode": false
  }
}
```

### Configuration Migration

#### Default System Configuration
The new default configuration will be created at `/etc/skyview/config.json` if it doesn't exist:

```json
{
  "server": {
    "host": "",
    "port": 8080
  },
  "sources": [
    {
      "id": "local",
      "name": "Local Receiver",
      "host": "localhost",
      "port": 30005,
      "format": "beast",
      "latitude": 0,
      "longitude": 0,
      "altitude": 0,
      "enabled": true
    }
  ],
  "settings": {
    "history_limit": 500,
    "stale_timeout": 60,
    "update_rate": 1
  },
  "origin": {
    "latitude": 51.4700,
    "longitude": -0.4600,
    "name": "Default Origin"
  },
  "database": {
    "path": "",
    "max_history_days": 7,
    "backup_on_upgrade": true
  },
  "callsign": {
    "enabled": true,
    "cache_hours": 24,
    "external_apis": true,
    "privacy_mode": false
  }
}
```

#### Preserving Custom Settings

If you have customized your configuration, merge your existing settings with the new sections:

```bash
# Backup original
cp /etc/skyview/config.json /etc/skyview/config.json.original

# Edit configuration to add new sections
sudo nano /etc/skyview/config.json
```

## Privacy Configuration

### Enabling Privacy Mode

For sensitive environments that require no external data transmission:

```json
{
  "callsign": {
    "enabled": true,
    "external_apis": false,
    "privacy_mode": true,
    "cache_hours": 168
  }
}
```

### Privacy Mode Features
- **No External API Calls**: Completely disables OpenSky Network and other external APIs
- **Local-Only Enhancement**: Uses embedded OpenFlights data for callsign lookup
- **Offline Operation**: Full functionality without internet connectivity
- **Compliance Ready**: Suitable for air-gapped or restricted networks

## Post-Migration Verification

### Database Verification
```bash
# Check database file exists and has correct permissions
ls -la /var/lib/skyview/skyview.db
# Should show: -rw-r--r-- 1 skyview skyview [size] [date] skyview.db

# Verify database schema
sqlite3 /var/lib/skyview/skyview.db "SELECT version FROM schema_info ORDER BY version DESC LIMIT 1;"
# Should return current schema version number
```

### Service Health Check
```bash
# Check service status
sudo systemctl status skyview

# Check logs for any errors
sudo journalctl -u skyview -f

# Verify web interface accessibility
curl -I http://localhost:8080/
# Should return: HTTP/1.1 200 OK
```

### Feature Testing
1. **Historical Data**: Verify aircraft positions persist after restart
2. **Callsign Enhancement**: Check that airline names appear for aircraft with callsigns
3. **Performance**: Monitor memory and CPU usage compared to previous version

## Troubleshooting

### Common Migration Issues

#### Database Permission Errors
```
Error: unable to open database file
```

**Solution**:
```bash
sudo chown -R skyview:skyview /var/lib/skyview/
sudo chmod 755 /var/lib/skyview/
sudo chmod 644 /var/lib/skyview/skyview.db
```

#### Migration Failures
```
Error: migration failed at version X
```

**Solution**:
```bash
# Stop service
sudo systemctl stop skyview

# Remove corrupted database
sudo rm /var/lib/skyview/skyview.db

# Restart service (will recreate database)
sudo systemctl start skyview
```

#### Configuration Conflicts
```
Error: unknown configuration field 'database'
```

**Solution**: Update configuration file with new sections, or reset to default configuration.

### Rolling Back

If you need to revert to the previous version:

```bash
# Stop current service
sudo systemctl stop skyview

# Install previous package version
sudo apt install skyview-adsb=[previous-version]

# Remove database directory (optional)
sudo rm -rf /var/lib/skyview/

# Restore original configuration
sudo cp /etc/skyview/config.json.backup /etc/skyview/config.json

# Start service
sudo systemctl start skyview
```

⚠️ **Note**: Rolling back will lose all historical aircraft data stored in the database.

## Performance Impact

### Expected Changes

#### Memory Usage
- **Before**: ~50-100MB RAM for aircraft tracking
- **After**: ~30-60MB RAM (reduced due to database storage)

#### Disk Usage
- **Database**: ~10-50MB depending on retention settings and traffic
- **Backups**: Additional ~10-50MB for migration backups

#### Startup Time
- **Before**: 1-2 seconds
- **After**: 2-5 seconds (database initialization)

### Optimization Recommendations

#### For High-Traffic Environments
```json
{
  "database": {
    "max_history_days": 3,
    "backup_on_upgrade": false
  },
  "settings": {
    "history_limit": 100
  }
}
```

#### For Resource-Constrained Systems
```json
{
  "callsign": {
    "enabled": false
  },
  "database": {
    "max_history_days": 1
  }
}
```

## Benefits After Migration

### Enhanced Features
- **Persistent History**: Aircraft tracks survive system restarts
- **Rich Callsign Data**: Airline names, routes, and aircraft types
- **Better Analytics**: Historical data enables trend analysis
- **Improved Performance**: Reduced memory usage for long-running instances

### Operational Improvements
- **Service Reliability**: Database recovery after crashes
- **Maintenance Windows**: Graceful restart without data loss
- **Monitoring**: Historical data for performance analysis
- **Compliance**: Privacy controls for regulatory requirements

## Support

### Getting Help
- **Documentation**: Check `/usr/share/doc/skyview-adsb/` for additional guides
- **Logs**: Service logs available via `journalctl -u skyview`
- **Configuration**: Example configs in `/usr/share/skyview/examples/`
- **Community**: Report issues at project repository

### Reporting Issues
When reporting migration issues, include:
- SkyView version (before and after)
- Operating system and version
- Configuration file content
- Error messages from logs
- Database file permissions (`ls -la /var/lib/skyview/`)

This migration enables SkyView's evolution toward more sophisticated aircraft tracking while maintaining the simplicity and reliability of the existing system.