mail2couch/CLAUDE.md

# CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

## Project Overview

mail2couch is a utility for backing up mail from various sources (primarily IMAP) to CouchDB. The project supports two implementations:
- **Go implementation**: Located in `/go/` directory (currently the active implementation)
- **Rust implementation**: Planned but not yet implemented

## Development Commands

### Go Implementation (Primary)

```bash
# Build the application
cd go && go build -o mail2couch .

# Run the application with automatic config discovery
cd go && ./mail2couch

# Run with specific config file
cd go && ./mail2couch -config /path/to/config.json

# Run with message limit (useful for large mailboxes)
cd go && ./mail2couch -max-messages 100

# Run with both config and message limit
cd go && ./mail2couch -config /path/to/config.json -max-messages 50

# Run linting/static analysis
cd go && go vet ./...

# Run integration tests with Podman containers
cd test && ./run-tests.sh

# Run specialized tests
cd test && ./test-wildcard-patterns.sh
cd test && ./test-incremental-sync.sh

# Run unit tests (none currently implemented)
cd go && go test ./...

# Check dependencies
cd go && go mod tidy
```

## Architecture

### Core Components

1. **Configuration (`config/`)**: JSON-based configuration system
   - Supports multiple mail sources with filtering options
   - CouchDB connection settings
   - Each source can have folder and message filters

2. **Mail Handling (`mail/`)**: IMAP client implementation
   - Uses `github.com/emersion/go-imap/v2` for IMAP operations
   - Supports TLS connections
   - Fetches and processes email messages from IMAP mailboxes

3. **CouchDB Integration (`couch/`)**: Database operations
   - Uses `github.com/go-kivik/kivik/v4` as CouchDB driver
   - Handles database creation and document management
   - Defines `MailDocument` structure for email storage

### Configuration Structure

The application uses `config.json` for configuration with the following structure:
- `couchDb`: Database connection settings (URL, credentials)
- `mailSources`: Array of mail sources with individual settings:
  - Protocol support (currently only IMAP)
  - Connection details (host, port, credentials)
  - `mode`: Either "sync" or "archive" (defaults to "archive" if not specified)
    - **sync**: 1-to-1 relationship - CouchDB documents match exactly what's in the mail account (may remove documents from CouchDB)
    - **archive**: Archive mode - CouchDB keeps all messages ever seen, even if deleted from mail account (never removes documents)
  - Filtering options for folders and messages with wildcard support
  - Enable/disable per source

### Configuration File Discovery

The application automatically searches for configuration files in the following order:
1. Path specified by `-config` command line flag
2. `./config.json` (current working directory)
3. `./config/config.json` (config subdirectory)
4. `~/.config/mail2couch/config.json` (user XDG config directory)
5. `~/.mail2couch.json` (user home directory)

This design ensures the same `config.json` format will work for both Go and Rust implementations.

### Current Implementation Status

- ✅ Configuration loading with automatic file discovery
- ✅ Command line flag support for config file path
- ✅ Per-account CouchDB database creation and management
- ✅ IMAP connection and mailbox listing
- ✅ Build error fixes
- ✅ Real IMAP message retrieval and parsing
- ✅ Email storage to CouchDB framework with native attachments
- ✅ Folder filtering logic with wildcard support (`*`, `?`, `[abc]` patterns)
- ✅ Date filtering support
- ✅ Keyword filtering support (subject, sender, recipient keywords)
- ✅ Duplicate detection and prevention
- ✅ Sync vs Archive mode implementation
- ✅ CouchDB attachment storage for email attachments
- ✅ Full message body and attachment handling with MIME multipart support
- ✅ Command line argument support (--max-messages flag)
- ✅ Per-account CouchDB databases for better organization
- ✅ Incremental sync functionality with IMAP SEARCH and sync metadata tracking
- ❌ Rust implementation

### Key Dependencies

- `github.com/emersion/go-imap/v2`: IMAP client library
- `github.com/go-kivik/kivik/v4`: CouchDB client library

### Incremental Sync Implementation

The application implements intelligent incremental synchronization to avoid re-processing messages:

- **Sync Metadata Storage**: Each mailbox sync operation stores metadata including last sync timestamp and highest UID processed
- **IMAP SEARCH Integration**: Uses IMAP SEARCH with SINCE criteria for efficient server-side filtering of new messages
- **Per-Mailbox Tracking**: Sync state is tracked independently for each mailbox in each account
- **Fallback Behavior**: Gracefully falls back to fetching recent messages if IMAP SEARCH fails
- **First Sync Handling**: Initial sync can use config `since` date or perform full sync

Sync metadata documents are stored in CouchDB with ID format: `sync_metadata_{mailbox}` and include:
- `lastSyncTime`: When this mailbox was last successfully synced
- `lastMessageUID`: Highest UID processed in the last sync
- `messageCount`: Number of messages processed in the last sync

### Development Notes

- The main entry point is `main.go` which orchestrates the configuration loading, CouchDB setup, and mail source processing
- Each mail source gets its own CouchDB database named using `GenerateAccountDBName()` function with `m2c_` prefix
- Each mail source is processed sequentially with proper error handling
- The application uses real IMAP message parsing with go-message library for full email processing
- Message filtering by folder (wildcard patterns), date (since), and keywords is implemented
- Duplicate detection prevents re-storing existing messages
- Sync vs Archive mode determines whether to remove documents from CouchDB when they're no longer in the mail account
- Email attachments are stored as native CouchDB attachments linked to the email document
- Comprehensive test environment with Podman containers and automated test scripts
- The application uses automatic config file discovery as documented above


### Next Steps

The following enhancements could further improve the implementation:

1. **Error Recovery**: Add retry logic for network failures and partial sync recovery
2. **Performance Optimization**: Add batch operations for better CouchDB insertion performance
3. **Unit Testing**: Add comprehensive unit tests for all major components
4. **Advanced Filtering**: Add support for more complex filter expressions and regex patterns
5. **Monitoring**: Add metrics and logging for production deployment
6. **Configuration Validation**: Enhanced validation for configuration files
7. **Multi-threading**: Parallel processing of multiple mailboxes or accounts

## Development Guidelines

### Code Quality and Standards
- All code requires perfect linting and tool-formatting, exceptions are allowed only if documented properly
- We always want linting and formatting of our code to be perfect