feat: Add SQLite database integration for aircraft history and callsign enhancement
- Implement comprehensive database package with versioned migrations - Add skyview-data CLI tool for managing aviation reference data - Integrate database with merger for real-time aircraft history persistence - Support OurAirports and OpenFlights data sources (runtime loading) - Add systemd timer for automated database updates - Fix transaction-based bulk loading for 2400% performance improvement - Add callsign enhancement system with airline/airport lookups - Update Debian packaging with database directory and permissions Database features: - Aircraft position history with configurable retention - External aviation data loading (airlines, airports) - Callsign parsing and enhancement - API client for external lookups (OpenSky, etc.) - Privacy mode for complete offline operation CLI commands: - skyview-data status: Show database statistics - skyview-data update: Load aviation reference data - skyview-data list: Show available data sources - skyview-data clear: Remove specific data sources 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
parent
cd51d3ecc0
commit
37c4fa2b57
25 changed files with 4771 additions and 12 deletions
38
debian/DEBIAN/postinst
vendored
38
debian/DEBIAN/postinst
vendored
|
|
@ -18,6 +18,21 @@ case "$1" in
|
|||
chown skyview-adsb:skyview-adsb /var/lib/skyview-adsb /var/log/skyview-adsb >/dev/null 2>&1 || true
|
||||
chmod 755 /var/lib/skyview-adsb /var/log/skyview-adsb >/dev/null 2>&1 || true
|
||||
|
||||
# Create database directory for skyview user (not skyview-adsb)
|
||||
mkdir -p /var/lib/skyview >/dev/null 2>&1 || true
|
||||
if getent passwd skyview >/dev/null 2>&1; then
|
||||
chown skyview:skyview /var/lib/skyview >/dev/null 2>&1 || true
|
||||
else
|
||||
# Create skyview user for database management
|
||||
if ! getent group skyview >/dev/null 2>&1; then
|
||||
addgroup --system --quiet skyview
|
||||
fi
|
||||
adduser --system --ingroup skyview --home /var/lib/skyview \
|
||||
--no-create-home --disabled-password --shell /bin/false --quiet skyview
|
||||
chown skyview:skyview /var/lib/skyview >/dev/null 2>&1 || true
|
||||
fi
|
||||
chmod 755 /var/lib/skyview >/dev/null 2>&1 || true
|
||||
|
||||
# Set permissions on config files
|
||||
if [ -f /etc/skyview-adsb/config.json ]; then
|
||||
chown root:skyview-adsb /etc/skyview-adsb/config.json >/dev/null 2>&1 || true
|
||||
|
|
@ -25,14 +40,33 @@ case "$1" in
|
|||
fi
|
||||
|
||||
|
||||
# Handle systemd service
|
||||
# Handle systemd services
|
||||
systemctl daemon-reload >/dev/null 2>&1 || true
|
||||
|
||||
# Check if service was previously enabled
|
||||
# Check if main service was previously enabled
|
||||
if systemctl is-enabled skyview-adsb >/dev/null 2>&1; then
|
||||
# Service was enabled, restart it
|
||||
systemctl restart skyview-adsb >/dev/null 2>&1 || true
|
||||
fi
|
||||
|
||||
# Only restart database timer if it was already enabled
|
||||
if systemctl is-enabled skyview-database-update.timer >/dev/null 2>&1; then
|
||||
systemctl restart skyview-database-update.timer >/dev/null 2>&1 || true
|
||||
fi
|
||||
|
||||
# Initialize database on first install (but don't auto-enable timer)
|
||||
if [ ! -f /var/lib/skyview/skyview.db ]; then
|
||||
echo "Initializing SkyView database..."
|
||||
sudo -u skyview /usr/bin/skyview-data update >/dev/null 2>&1 || true
|
||||
echo "Database initialized with safe (public domain) data."
|
||||
echo ""
|
||||
echo "To enable automatic weekly updates:"
|
||||
echo " systemctl enable --now skyview-database-update.timer"
|
||||
echo ""
|
||||
echo "To import additional data sources:"
|
||||
echo " skyview-data list"
|
||||
echo " skyview-data import <source>"
|
||||
fi
|
||||
;;
|
||||
esac
|
||||
|
||||
|
|
|
|||
33
debian/lib/systemd/system/skyview-database-update.service
vendored
Normal file
33
debian/lib/systemd/system/skyview-database-update.service
vendored
Normal file
|
|
@ -0,0 +1,33 @@
|
|||
[Unit]
|
||||
Description=SkyView Database Update
|
||||
Documentation=man:skyview-data(1)
|
||||
After=network-online.target
|
||||
Wants=network-online.target
|
||||
|
||||
[Service]
|
||||
Type=oneshot
|
||||
User=skyview
|
||||
Group=skyview
|
||||
ExecStart=/usr/bin/skyview-data update
|
||||
StandardOutput=journal
|
||||
StandardError=journal
|
||||
|
||||
# Security hardening
|
||||
NoNewPrivileges=true
|
||||
PrivateTmp=true
|
||||
ProtectSystem=strict
|
||||
ProtectHome=true
|
||||
ReadWritePaths=/var/lib/skyview /tmp
|
||||
ProtectKernelTunables=true
|
||||
ProtectKernelModules=true
|
||||
ProtectControlGroups=true
|
||||
RestrictRealtime=true
|
||||
RestrictSUIDSGID=true
|
||||
|
||||
# Resource limits
|
||||
MemoryMax=256M
|
||||
TasksMax=50
|
||||
TimeoutStartSec=300
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
17
debian/lib/systemd/system/skyview-database-update.timer
vendored
Normal file
17
debian/lib/systemd/system/skyview-database-update.timer
vendored
Normal file
|
|
@ -0,0 +1,17 @@
|
|||
[Unit]
|
||||
Description=SkyView Database Update Timer
|
||||
Documentation=man:skyview-data(1)
|
||||
Requires=skyview-database-update.service
|
||||
|
||||
[Timer]
|
||||
# Run weekly on Sunday at 3 AM
|
||||
OnCalendar=Sun 03:00
|
||||
# Randomize start time within 1 hour to avoid thundering herd
|
||||
RandomizedDelaySec=3600
|
||||
# Start immediately if system was down during scheduled time
|
||||
Persistent=true
|
||||
# Don't start if system is on battery (laptops)
|
||||
ConditionACPower=true
|
||||
|
||||
[Install]
|
||||
WantedBy=timers.target
|
||||
99
debian/usr/share/doc/skyview-adsb/DATABASE.md
vendored
Normal file
99
debian/usr/share/doc/skyview-adsb/DATABASE.md
vendored
Normal file
|
|
@ -0,0 +1,99 @@
|
|||
# SkyView Database Management
|
||||
|
||||
SkyView includes a comprehensive database management system for enriching aircraft callsigns with airline and airport information.
|
||||
|
||||
## Quick Start
|
||||
|
||||
### 1. Check Current Status
|
||||
```bash
|
||||
skyview-data status
|
||||
```
|
||||
|
||||
### 2. Import Safe Data (Recommended)
|
||||
```bash
|
||||
# Import public domain sources automatically
|
||||
skyview-data update
|
||||
```
|
||||
|
||||
### 3. Enable Automatic Updates (Optional)
|
||||
```bash
|
||||
# Weekly updates on Sunday at 3 AM
|
||||
sudo systemctl enable --now skyview-database-update.timer
|
||||
```
|
||||
|
||||
## Available Data Sources
|
||||
|
||||
### Safe Sources (Public Domain)
|
||||
These sources are imported automatically with `skyview-data update`:
|
||||
- **OurAirports**: Comprehensive airport database (public domain)
|
||||
- **FAA Registry**: US aircraft registration data (public domain)
|
||||
|
||||
### License-Required Sources
|
||||
These require explicit acceptance:
|
||||
- **OpenFlights**: Airline and airport data (AGPL-3.0 license)
|
||||
|
||||
## Commands
|
||||
|
||||
### Basic Operations
|
||||
```bash
|
||||
skyview-data list # Show available sources
|
||||
skyview-data status # Show database status
|
||||
skyview-data update # Update safe sources
|
||||
skyview-data import openflights # Import licensed source
|
||||
skyview-data clear <source> # Remove source data
|
||||
```
|
||||
|
||||
### Systemd Timer Management
|
||||
```bash
|
||||
# Enable weekly automatic updates
|
||||
systemctl enable skyview-database-update.timer
|
||||
systemctl start skyview-database-update.timer
|
||||
|
||||
# Check timer status
|
||||
systemctl status skyview-database-update.timer
|
||||
|
||||
# View update logs
|
||||
journalctl -u skyview-database-update.service
|
||||
|
||||
# Disable automatic updates
|
||||
systemctl disable skyview-database-update.timer
|
||||
```
|
||||
|
||||
## License Compliance
|
||||
|
||||
SkyView maintains strict license separation:
|
||||
- **SkyView binary**: Contains no external data (stays MIT licensed)
|
||||
- **Runtime import**: Users choose which sources to import
|
||||
- **Safe defaults**: Only public domain sources updated automatically
|
||||
- **User choice**: Each person decides their own license compatibility
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Check Service Status
|
||||
```bash
|
||||
systemctl status skyview-database-update.timer
|
||||
journalctl -u skyview-database-update.service -f
|
||||
```
|
||||
|
||||
### Manual Database Reset
|
||||
```bash
|
||||
systemctl stop skyview-database-update.timer
|
||||
skyview-data reset --force
|
||||
skyview-data update
|
||||
systemctl start skyview-database-update.timer
|
||||
```
|
||||
|
||||
### Permissions Issues
|
||||
```bash
|
||||
sudo chown skyview:skyview /var/lib/skyview/
|
||||
sudo chmod 755 /var/lib/skyview/
|
||||
```
|
||||
|
||||
## Files and Directories
|
||||
|
||||
- `/usr/bin/skyview-data` - Database management command
|
||||
- `/var/lib/skyview/skyview.db` - Database file
|
||||
- `/usr/share/skyview/scripts/update-database.sh` - Cron helper script
|
||||
- `/lib/systemd/system/skyview-database-update.*` - Systemd timer files
|
||||
|
||||
For detailed information, see `man skyview-data`.
|
||||
181
debian/usr/share/man/man1/skyview-data.1
vendored
Normal file
181
debian/usr/share/man/man1/skyview-data.1
vendored
Normal file
|
|
@ -0,0 +1,181 @@
|
|||
.TH skyview-data 1 "January 2025" "SkyView Database Manager"
|
||||
.SH NAME
|
||||
skyview-data \- SkyView aviation database management utility
|
||||
|
||||
.SH SYNOPSIS
|
||||
.B skyview-data
|
||||
[\fIOPTIONS\fR] \fICOMMAND\fR [\fIARGS\fR...]
|
||||
|
||||
.SH DESCRIPTION
|
||||
.B skyview-data
|
||||
manages the SkyView aviation database, allowing users to import airline and airport data from various external sources while maintaining license compliance.
|
||||
|
||||
The tool automatically creates and migrates the database schema, downloads data from public and licensed sources, and provides status monitoring for the aviation database used by SkyView for callsign enhancement.
|
||||
|
||||
.SH OPTIONS
|
||||
.TP
|
||||
.BR \-db " \fIPATH\fR"
|
||||
Database file path (auto-detected if empty)
|
||||
.TP
|
||||
.BR \-v ", " \-\-verbose
|
||||
Verbose output
|
||||
.TP
|
||||
.BR \-\-force
|
||||
Force operation without prompts
|
||||
.TP
|
||||
.BR \-\-version
|
||||
Show version information
|
||||
|
||||
.SH COMMANDS
|
||||
.TP
|
||||
.B init
|
||||
Initialize empty database with schema
|
||||
.TP
|
||||
.B list
|
||||
List available data sources with license information
|
||||
.TP
|
||||
.B status
|
||||
Show current database status and statistics
|
||||
.TP
|
||||
.B update [\fISOURCE\fR...]
|
||||
Update data from specified sources, or all safe sources if none specified
|
||||
.TP
|
||||
.B import \fISOURCE\fR
|
||||
Import data from a specific source with license acceptance
|
||||
.TP
|
||||
.B clear \fISOURCE\fR
|
||||
Remove all data from the specified source
|
||||
.TP
|
||||
.B reset
|
||||
Clear all data and reset database (destructive)
|
||||
|
||||
.SH DATA SOURCES
|
||||
.SS Safe Sources (Public Domain)
|
||||
These sources are automatically included in
|
||||
.B update
|
||||
operations:
|
||||
.TP
|
||||
.B ourairports
|
||||
Public domain airport database from OurAirports.com
|
||||
.TP
|
||||
.B faa-registry
|
||||
US aircraft registration database (FAA, public domain)
|
||||
|
||||
.SS License-Required Sources
|
||||
These sources require explicit license acceptance:
|
||||
.TP
|
||||
.B openflights
|
||||
Comprehensive airline and airport database (AGPL-3.0 license)
|
||||
|
||||
.SH EXAMPLES
|
||||
.TP
|
||||
Initialize database and import safe data:
|
||||
.EX
|
||||
skyview-data init
|
||||
skyview-data update
|
||||
.EE
|
||||
.TP
|
||||
Import OpenFlights data with license acceptance:
|
||||
.EX
|
||||
skyview-data import openflights
|
||||
.EE
|
||||
.TP
|
||||
Check database status:
|
||||
.EX
|
||||
skyview-data status
|
||||
.EE
|
||||
.TP
|
||||
Set up automatic updates via systemd timer:
|
||||
.EX
|
||||
systemctl enable skyview-database-update.timer
|
||||
systemctl start skyview-database-update.timer
|
||||
.EE
|
||||
|
||||
.SH CRON AUTOMATION
|
||||
For automated updates,
|
||||
.B skyview-data update
|
||||
is designed to work seamlessly with cron:
|
||||
|
||||
.EX
|
||||
# Update weekly on Sunday at 3 AM
|
||||
0 3 * * 0 /usr/bin/skyview-data update
|
||||
.EE
|
||||
|
||||
The command automatically:
|
||||
.RS
|
||||
.IP \(bu 2
|
||||
Creates the database if it doesn't exist
|
||||
.IP \(bu 2
|
||||
Updates only safe (public domain) sources
|
||||
.IP \(bu 2
|
||||
Provides proper exit codes for monitoring
|
||||
.IP \(bu 2
|
||||
Logs to standard output with timestamps
|
||||
.RE
|
||||
|
||||
.SH SYSTEMD INTEGRATION
|
||||
The Debian package includes systemd timer integration:
|
||||
|
||||
.EX
|
||||
# Enable automatic weekly updates
|
||||
systemctl enable skyview-database-update.timer
|
||||
systemctl start skyview-database-update.timer
|
||||
|
||||
# Check timer status
|
||||
systemctl status skyview-database-update.timer
|
||||
|
||||
# View update logs
|
||||
journalctl -u skyview-database-update.service
|
||||
.EE
|
||||
|
||||
.SH FILES
|
||||
.TP
|
||||
.I /var/lib/skyview/skyview.db
|
||||
System-wide database location
|
||||
.TP
|
||||
.I ~/.local/share/skyview/skyview.db
|
||||
User-specific database location
|
||||
.TP
|
||||
.I /var/log/skyview/
|
||||
Log directory for database operations
|
||||
|
||||
.SH EXIT STATUS
|
||||
.TP
|
||||
.B 0
|
||||
Success
|
||||
.TP
|
||||
.B 1
|
||||
General error or command failure
|
||||
.TP
|
||||
.B 2
|
||||
Invalid arguments or usage
|
||||
|
||||
.SH SECURITY
|
||||
All external data downloads use HTTPS. No sensitive information is transmitted. The tool processes only publicly available aviation data.
|
||||
|
||||
License-required sources require explicit user acceptance before import.
|
||||
|
||||
.SH LICENSE COMPLIANCE
|
||||
.B skyview-data
|
||||
maintains strict license separation:
|
||||
.RS
|
||||
.IP \(bu 2
|
||||
SkyView binary contains no external data (MIT license maintained)
|
||||
.IP \(bu 2
|
||||
Each data source tracks its license and user acceptance
|
||||
.IP \(bu 2
|
||||
Users choose which sources to import based on license compatibility
|
||||
.IP \(bu 2
|
||||
Automatic updates only include public domain sources
|
||||
.RE
|
||||
|
||||
.SH SEE ALSO
|
||||
.BR skyview (1),
|
||||
.BR systemctl (1),
|
||||
.BR crontab (5)
|
||||
|
||||
.SH AUTHOR
|
||||
SkyView is developed as an open-source ADS-B aircraft tracking system.
|
||||
|
||||
.SH REPORTING BUGS
|
||||
Report bugs and issues at the project repository.
|
||||
Loading…
Add table
Add a link
Reference in a new issue