Combine the previous `ANALYSIS.md` with a new, comprehensive
review of both the Go and Rust implementations.
This new report:
- Acknowledges that the Rust version is now fully functional and
feature-rich, resolving a key point from the old analysis.
- Highlights which original issues have been addressed (e.g., Rust
implementation status, performance, filtering) and which remain
(e.g., security, web interface, dry-run mode).
- Provides a detailed side-by-side comparison of the two versions,
covering architecture, features, and complexity.
- Outlines a tiered roadmap for future improvements, prioritizing
critical needs like security and usability enhancements.
Add server-side folder filtering using IMAP LIST patterns and enhance
message filtering to use IMAP SEARCH with keyword filters when available.
Key improvements:
- Add list_filtered_mailboxes() method using IMAP LIST with patterns
- Use server-side filtering instead of client-side folder filtering
- Enhance message search to use IMAP SEARCH for subject/sender keywords
- Add has_keyword_filters() method to MessageFilter
- Reduce network traffic by leveraging IMAP server capabilities
- Remove dependency on client-side filter_folders function
This achieves full feature parity with the updated Go implementation
and ensures both versions use IMAP standards optimally.
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
Replace client-side wildcard filtering with IMAP LIST pattern matching
for improved efficiency and accuracy. This fixes the issue where patterns
like "Work*" were not matching folders like "Work/Projects".
Key improvements:
- Use IMAP LIST with patterns for server-side filtering
- Remove dependency on doublestar library
- Add ListFilteredMailboxes() method with proper IMAP pattern support
- Remove obsolete ShouldProcessMailbox() client-side filtering
- Significantly reduce network traffic by filtering at server
This ensures the Go implementation correctly processes folder patterns
and achieves feature parity with the Rust implementation.
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Both implementations create matching document structures and counts
- Attachment storage works identically in both (3 docs with 3 attachments each)
- Document schemas are identical with proper CouchDB native attachment support
- Rust implementation correctly handles folder filtering (includes Work/Projects)
- Both use same document ID format with proper URL encoding
- Database verification confirms feature parity achieved
Added no-filter test configs for fair comparison between implementations.
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Fixed document ID conflicts caused by unencoded slashes in mailbox names
- Added URL encoding for all document IDs used in CouchDB REST API calls
- Mailbox names with slashes (e.g., 'Work/Projects') now create proper document IDs
- Resolves issue where 'Work/Projects_1' was incorrectly stored as document 'Work' with attachment 'Projects_1'
- Added urlencoding dependency for proper URL-safe document ID handling
All messages now store successfully without conflicts across all mailboxes.
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Removed attachment metadata from initial document storage
- Attachments are now stored separately using CouchDB native attachment API
- This matches the Go implementation approach and resolves CouchDB validation errors
- All messages with attachments now store successfully
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
This commit completes the Rust implementation of mail2couch with full feature
parity to the Go version, including:
- Complete IMAP client with TLS support and retry logic
- Advanced email parsing with MIME multipart support using mail-parser
- Email attachment extraction and CouchDB storage
- Sync mode implementation with deleted message handling
- Enhanced error handling and retry mechanisms
- Identical command-line interface with bash completion
- Test configurations for both implementations
The Rust implementation now provides:
- Memory safety and type safety guarantees
- Modern async/await patterns with tokio/async-std
- Comprehensive error handling with anyhow/thiserror
- Structured logging and progress reporting
- Performance optimizations and retry logic
Test configurations created:
- rust/config-test-rust.json - Rust implementation test config
- go/config-test-go.json - Go implementation test config
- test-config-comparison.md - Detailed comparison documentation
- test-both-implementations.sh - Automated testing script
Both implementations can now be tested side-by-side with identical
configurations to validate feature parity and performance.
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Add async-native-tls dependency for secure IMAP connections
- Implement ImapStream enum supporting both TLS and plain connections
- Add automatic TLS detection based on port (993=TLS, 143=plain, 3143=test)
- Add comprehensive Read/Write trait implementations for stream wrapper
- Add debug logging for connection type verification
- Create example configurations for Gmail, Outlook, and other providers
- Add TLS_SUPPORT.md documentation with security guidelines
- Test with existing test environment and TLS detection logic
- Maintain backward compatibility with plain IMAP for testing
The Rust implementation now supports secure connections to production
email providers while maintaining compatibility with test environments.
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Add comprehensive Rust implementation matching Go functionality
- Configuration loading with automatic file discovery
- GNU-style command line parsing with clap (--config/-c, --max-messages/-m)
- CouchDB client integration with document storage and sync metadata
- IMAP client functionality with message fetching and parsing
- Folder filtering with wildcard pattern support (*, ?, [abc])
- Message filtering by subject, sender, and recipient keywords
- Incremental sync functionality with metadata tracking
- Bash completion generation matching Go implementation
- Cross-compatible document schemas and database structures
- Successfully tested with existing test environment
Note: TLS support and advanced email parsing features pending
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Add --generate-bash-completion flag to output bash completion script
- Provide intelligent completions for config files (*.json) and message counts
- Support tab completion for all available command line options
- Generate professional script with proper comments and error handling
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
Adds a detailed analysis of the mail2couch project, covering its
purpose, architecture, and implementation details. The document
provides a summary of the project's functionality based on a review
of the README, schema files, and the Go source code.
The analysis includes:
- A high-level summary of what the project does.
- A detailed explanation of how it works, based on code review.
- Identification of potential problems, such as plaintext password
storage and the incomplete Rust implementation.
- A list of missing features, including the planned webmail UI
and OAuth2 support.
- Concrete suggestions for improvements, such as adding a --dry-run
mode, improving security, and clarifying the status of the Rust code.
- Add pflag dependency for POSIX/GNU-style command line parsing
- Replace Go standard flag package with pflag for better UX
- Implement long options with double dashes (--config, --max-messages, --help)
- Add short option aliases with single dashes (-c, -m, -h)
- Update help message with proper formatting and application description
- Update all documentation to reflect new flag syntax
- Update test scripts to use new command line format
GNU-style options provide better usability:
- Long descriptive options with --flag-name format
- Short single-character aliases for common options
- Standard help flag behavior with --help/-h
- Compatible with shell completion and standard conventions
Command line interface now supports:
- --config/-c FILE: Path to configuration file
- --max-messages/-m N: Message processing limit per mailbox
- --help/-h: Show help message and exit
All existing functionality preserved with improved command line experience.
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Add complete CouchDB document schema specifications in couchdb-schemas.md
- Create example JSON documents for mail and sync metadata structures
- Implement Rust schema definitions with full serde support and type safety
- Add validation script to ensure schema consistency across implementations
- Document field definitions, data types, and validation rules
- Provide Rust Cargo.toml with appropriate dependencies for future implementation
This establishes a solid foundation for the planned Rust implementation while ensuring
100% compatibility with existing Go implementation databases. Both implementations will
use identical document structures, field names, and database naming conventions.
Schema Features:
- Mail documents with native CouchDB attachment support
- Sync metadata for incremental synchronization
- Predictable document ID patterns for efficient access
- Cross-language type mappings and validation rules
- Example documents for testing and reference
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Remove obsolete CouchDB design documents (webmail.json, dashboard.json)
- Clean up webmail-related code from couch/couch.go (WebmailViews, CreateWebmailViews, etc.)
- Update documentation to focus on core mail-to-CouchDB storage functionality
- Add Future Plans section describing planned webmail viewer as separate component
- Apply go fmt formatting and ensure code quality standards
- Update test documentation to show raw CouchDB API access patterns
- Remove compiled binary from repository
This refactor simplifies the codebase to focus on its core purpose: efficiently
backing up emails from IMAP to CouchDB. The webmail interface will be developed
as a separate, optional component to maintain clean separation of concerns.
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
## Documentation Enhancements
- Create comprehensive README with installation, configuration, and usage examples
- Add simple, advanced, and provider-specific configuration examples
- Document all features: incremental sync, wildcard patterns, keyword filtering, attachment support
- Include production deployment guidance and troubleshooting section
- Add architecture documentation with database structure and document format examples
## Configuration Cleanup
- Remove unnecessary `database` field from CouchDB configuration
- Add `m2c_` prefix to all CouchDB database names for better namespace isolation
- Update GenerateAccountDBName() to consistently prefix databases with `m2c_`
- Clean up all configuration examples to remove deprecated database field
## Test Environment Simplification
- Simplify test script structure to eliminate confusion and redundancy
- Remove redundant populate-test-messages.sh wrapper script
- Update run-tests.sh to be comprehensive automated test with cleanup
- Maintain clear separation: automated tests vs manual testing environment
- Update all test scripts to expect m2c-prefixed database names
## Configuration Examples Added
- config-simple.json: Basic single Gmail account setup
- config-advanced.json: Multi-account with complex filtering and different providers
- config-providers.json: Real-world configurations for Gmail, Outlook, Yahoo, iCloud
## Benefits
- Clear documentation for users from beginner to advanced
- Namespace isolation prevents database conflicts in shared CouchDB instances
- Simplified test workflow eliminates user confusion about which scripts to use
- Comprehensive examples cover common email provider configurations
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
## Wildcard Folder Selection
- Add support for wildcard patterns (`*`, `?`, `[abc]`) using filepath.Match
- Implement special case: `"*"` selects ALL available folders
- Support for complex include/exclude pattern combinations
- Maintain backwards compatibility with exact string matching
- Enable subfolder pattern matching (e.g., `Work/*`, `*/Drafts`)
## Keyword Filtering
- Add SubjectKeywords, SenderKeywords, RecipientKeywords to MessageFilter config
- Implement case-insensitive keyword matching across message fields
- Support multiple keywords per filter type with inclusive OR logic
- Add ShouldProcessMessage method for message-level filtering
## Enhanced Test Environment
- Create comprehensive wildcard pattern test scenarios
- Add 12 test folders covering various pattern types: Work/*, Important/*, Archive/*, exact matches
- Implement dedicated wildcard test script (test-wildcard-patterns.sh)
- Update test configurations to demonstrate real-world wildcard usage patterns
- Enhance test data generation with folder-specific messages for validation
## Documentation
- Create FOLDER_PATTERNS.md with comprehensive wildcard examples and use cases
- Update CLAUDE.md to reflect all implemented features and current status
- Enhance test README with detailed wildcard pattern explanations
- Provide configuration examples for common email organization scenarios
## Message Origin Tracking
- Verify all messages in CouchDB properly tagged with origin folder in `mailbox` field
- Maintain per-account database isolation for better organization
- Document ID format: `{folder}_{uid}` ensures uniqueness across folders
Key patterns supported:
- `["*"]` - All folders (with excludes)
- `["Work*", "Important*"]` - Prefix matching
- `["Work/*", "Archive/*"]` - Subfolder patterns
- `["INBOX", "Sent"]` - Exact matches
- Complex include/exclude combinations
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Replace placeholder message generation with actual IMAP message fetching using go-message library
- Add per-account CouchDB databases for better organization and isolation
- Implement native CouchDB attachment storage with proper revision management
- Add command line argument parsing with --max-messages flag for controlling message processing limits
- Support both sync and archive modes with proper document synchronization
- Add comprehensive test environment with Podman containers (GreenMail IMAP server + CouchDB)
- Implement full MIME multipart parsing for proper body and attachment extraction
- Add TLS and plain IMAP connection support based on port configuration
- Update configuration system to support sync vs archive modes
- Create test scripts and sample data for development and testing
Key technical improvements:
- Real email envelope and header processing with go-imap v2 API
- MIME Content-Type and Content-Disposition parsing for attachment detection
- CouchDB document ID generation using mailbox_uid format for uniqueness
- Duplicate detection and prevention to avoid re-storing existing messages
- Proper error handling and connection management for IMAP operations
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Create separate CouchDB database for each mail source (account)
- Store email attachments as native CouchDB attachments
- Add GenerateAccountDBName() for CouchDB-compatible database naming
- Update MailDocument structure to support _attachments field
- Implement StoreAttachment() for CouchDB attachment API
- Add placeholder attachment testing for every 3rd message
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Update CLAUDE.md with keyword filtering in Next Steps section
- Add detailed TODO.md with feature specification, use cases, and implementation notes
- Document subjectKeywords, senderKeywords, and recipientKeywords functionality
- Include JSON configuration examples and priority assessment
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Add TODO comments in config structure for future keyword filtering
- Document planned subjectKeywords, senderKeywords, and recipientKeywords support
- Create TODO.md with detailed feature specification and use cases
- Update CLAUDE.md with keyword filtering in next steps
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Add configuration system with automatic file discovery (current dir, config subdir, user home, XDG config)
- Implement IMAP client with TLS connection, authentication, and mailbox listing
- Add CouchDB integration with database creation and document storage
- Support folder filtering (include/exclude) and date filtering (since parameter)
- Include duplicate detection to prevent re-storing existing messages
- Add comprehensive error handling and logging throughout
- Structure code in clean packages: config, mail, couch
- Application currently uses placeholder messages to test the storage pipeline
- Ready for real IMAP message parsing implementation
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
