olemd/skyview
1
0
Fork 0
Code Issues 12 Pull requests Projects Releases 11 Packages Wiki Activity Actions

feat: Enhanced database status monitoring and configuration management #34

Merged
olemd merged 11 commits from feature/sqlite-database-integration into main 2025-08-31 19:45:27 +02:00
Conversation 0 Commits 11 Files changed 44 +9315 -35
olemd commented 2025-08-31 19:42:34 +02:00
Owner
Copy link

Summary

This PR enhances the SkyView database functionality with comprehensive status monitoring, configuration improvements, and bug fixes:

  • Database Status Web Interface: New /database page showing comprehensive database statistics, size, location, and external data sources
  • Enhanced API: Extended /api/database/status endpoint with file system information and storage efficiency metrics
  • Configuration Management: Added complete configuration documentation and external data source setup
  • Bug Fixes: Fixed database schema column name mismatch causing callsign statistics errors
  • System Administration: Updated systemd services to use consistent skyview-adsb user

Key Features

📊 Database Status Monitoring

  • Real-time database size and location display
  • Storage efficiency tracking with page utilization statistics
  • External data source status and record counts
  • Comprehensive web interface with scrolling support

⚙️ Configuration Enhancements

  • Complete CONFIGURATION.md documentation covering all settings
  • External aviation data sources (OpenFlights Airlines/Airports, OurAirports)
  • Privacy controls and air-gapped operation support
  • Environment-specific configuration examples

🛠️ System Improvements

  • Fixed cache_expires → expires_at column name mismatch
  • Consistent skyview-adsb user across all systemd services
  • Enhanced database management commands with optimization statistics

Test Plan

  • Database status API returns comprehensive information
  • Web interface displays all database metrics correctly
  • Scrolling works properly on database status page
  • Callsign statistics load without schema errors
  • External data sources configuration validated
  • systemd services updated with correct user permissions

🤖 Generated with Claude Code

## Summary This PR enhances the SkyView database functionality with comprehensive status monitoring, configuration improvements, and bug fixes: - **Database Status Web Interface**: New `/database` page showing comprehensive database statistics, size, location, and external data sources - **Enhanced API**: Extended `/api/database/status` endpoint with file system information and storage efficiency metrics - **Configuration Management**: Added complete configuration documentation and external data source setup - **Bug Fixes**: Fixed database schema column name mismatch causing callsign statistics errors - **System Administration**: Updated systemd services to use consistent `skyview-adsb` user ## Key Features ### 📊 Database Status Monitoring - Real-time database size and location display - Storage efficiency tracking with page utilization statistics - External data source status and record counts - Comprehensive web interface with scrolling support ### ⚙️ Configuration Enhancements - Complete CONFIGURATION.md documentation covering all settings - External aviation data sources (OpenFlights Airlines/Airports, OurAirports) - Privacy controls and air-gapped operation support - Environment-specific configuration examples ### 🛠️ System Improvements - Fixed `cache_expires` → `expires_at` column name mismatch - Consistent `skyview-adsb` user across all systemd services - Enhanced database management commands with optimization statistics ## Test Plan - [x] Database status API returns comprehensive information - [x] Web interface displays all database metrics correctly - [x] Scrolling works properly on database status page - [x] Callsign statistics load without schema errors - [x] External data sources configuration validated - [x] systemd services updated with correct user permissions 🤖 Generated with [Claude Code](https://claude.ai/code)
- Implement comprehensive database package with versioned migrations
- Add skyview-data CLI tool for managing aviation reference data
- Integrate database with merger for real-time aircraft history persistence
- Support OurAirports and OpenFlights data sources (runtime loading)
- Add systemd timer for automated database updates
- Fix transaction-based bulk loading for 2400% performance improvement
- Add callsign enhancement system with airline/airport lookups
- Update Debian packaging with database directory and permissions

Database features:
- Aircraft position history with configurable retention
- External aviation data loading (airlines, airports)
- Callsign parsing and enhancement
- API client for external lookups (OpenSky, etc.)
- Privacy mode for complete offline operation

CLI commands:
- skyview-data status: Show database statistics
- skyview-data update: Load aviation reference data
- skyview-data list: Show available data sources
- skyview-data clear: Remove specific data sources

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
- Change User and Group from 'skyview' to 'skyview-adsb' in service files
- Update ReadWritePaths in database update service accordingly
- Add external data source configuration to example config

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
- Fix SQL queries in ClearExpiredCache() and GetCacheStats() functions
- Resolves "no such column: cache_expires" database error
- Column name now matches schema definition in migrations.go

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
- Add database file size, path, and modification timestamp to /api/database/status
- Include storage efficiency metrics and page statistics
- Add optimization statistics using database manager
- Import "os" package for file system operations

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
- Create /database page showing database statistics, size, and location
- Display storage efficiency, page statistics, and optimization info
- Show external data sources with license information and load status
- Include auto-refresh functionality and navigation links
- Implement CSS overrides to fix scrolling issues

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
- Create detailed CONFIGURATION.md covering all SkyView settings
- Document external aviation data sources (OpenFlights, OurAirports)
- Explain origin vs sources distinction for map center vs receivers
- Include privacy controls, database optimization, and validation
- Provide environment-specific configuration examples
- Cover database management commands and troubleshooting

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
- Add database management commands and optimization examples
- Reference new CONFIGURATION.md documentation
- Update systemd service references to use skyview-adsb
- Enhance skyview-data status command with optimization statistics

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
- Add comprehensive database optimization management
- Enhance external data source loading with progress tracking
- Add optimization statistics and efficiency calculations
- Update Go module dependencies for database operations
- Implement database size and performance monitoring

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
- Add unit tests for database operations and optimization
- Test external data source loading and caching
- Add callsign manager functionality tests
- Create test helpers for database testing utilities
- Ensure database reliability and performance validation

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
- Add callsign management module for enhanced aircraft information
- Integrate database status display in web interface
- Update aircraft manager with database-backed callsign resolution
- Enhance user interface with database connectivity indicators
- Add embedded asset management for new database interface components

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
- Document complete database schema including aircraft history and callsign cache
- Add external data source tables and relationships
- Include optimization and maintenance procedures
- Document indexes, performance considerations, and storage requirements
- Provide examples of database queries and operations

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
olemd merged commit 0b0de87be0 into main 2025-08-31 19:45:27 +02:00
olemd deleted branch feature/sqlite-database-integration 2025-08-31 19:45:27 +02:00
Sign in to join this conversation.
Reviewers
No reviewers
Labels
Clear labels   
Compat/Breaking

Breaking change that won't be backward compatible

  
Kind/Bug

Something is not working

  
Kind/Documentation

Documentation changes

  
Kind/Enhancement

Improve existing functionality

  
Kind/Feature

New functionality

  
Kind/Security

This is security issue

  
Kind/Testing

Issue or pull request related to testing

  
Priority
Critical
The priority is critical

  
Priority
High
The priority is high

  
Priority
Low
The priority is low

  
Priority
Medium
The priority is medium

  
Reviewed
Confirmed
Issue has been confirmed

  
Reviewed
Duplicate
This issue or pull request already exists

  
Reviewed
Invalid
Invalid issue

  
Reviewed
Won't Fix
This issue won't be fixed

  
Status
Abandoned
Somebody has started to work on this but abandoned work

  
Status
Blocked
Something is blocking this issue or pull request

  
Status
Need More Info
Feedback is required to reproduce issue or to continue work

No labels
Compat/Breaking
Kind/Bug
Kind/Documentation
Kind/Enhancement
Kind/Feature
Kind/Security
Kind/Testing
Priority
Critical
Priority
High
Priority
Low
Priority
Medium
Reviewed
Confirmed
Reviewed
Duplicate
Reviewed
Invalid
Reviewed
Won't Fix
Status
Abandoned
Status
Blocked
Status
Need More Info
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: olemd/skyview#34
No description provided.
Delete branch "feature/sqlite-database-integration"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?