mail2couch/test
Ole-Morten Duesund ea6235b674 feat: implement real IMAP message parsing with native CouchDB attachments
- Replace placeholder message generation with actual IMAP message fetching using go-message library
- Add per-account CouchDB databases for better organization and isolation
- Implement native CouchDB attachment storage with proper revision management
- Add command line argument parsing with --max-messages flag for controlling message processing limits
- Support both sync and archive modes with proper document synchronization
- Add comprehensive test environment with Podman containers (GreenMail IMAP server + CouchDB)
- Implement full MIME multipart parsing for proper body and attachment extraction
- Add TLS and plain IMAP connection support based on port configuration
- Update configuration system to support sync vs archive modes
- Create test scripts and sample data for development and testing

Key technical improvements:
- Real email envelope and header processing with go-imap v2 API
- MIME Content-Type and Content-Disposition parsing for attachment detection
- CouchDB document ID generation using mailbox_uid format for uniqueness
- Duplicate detection and prevention to avoid re-storing existing messages
- Proper error handling and connection management for IMAP operations

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-08-01 17:04:10 +02:00
..
dovecot feat: implement real IMAP message parsing with native CouchDB attachments 2025-08-01 17:04:10 +02:00
config-test.json feat: implement real IMAP message parsing with native CouchDB attachments 2025-08-01 17:04:10 +02:00
generate-ssl.sh feat: implement real IMAP message parsing with native CouchDB attachments 2025-08-01 17:04:10 +02:00
podman-compose.yml feat: implement real IMAP message parsing with native CouchDB attachments 2025-08-01 17:04:10 +02:00
populate-greenmail.py feat: implement real IMAP message parsing with native CouchDB attachments 2025-08-01 17:04:10 +02:00
populate-test-messages.sh feat: implement real IMAP message parsing with native CouchDB attachments 2025-08-01 17:04:10 +02:00
README.md feat: implement real IMAP message parsing with native CouchDB attachments 2025-08-01 17:04:10 +02:00
run-tests.sh feat: implement real IMAP message parsing with native CouchDB attachments 2025-08-01 17:04:10 +02:00
start-test-env.sh feat: implement real IMAP message parsing with native CouchDB attachments 2025-08-01 17:04:10 +02:00
stop-test-env.sh feat: implement real IMAP message parsing with native CouchDB attachments 2025-08-01 17:04:10 +02:00

mail2couch Test Environment

This directory contains a complete test environment for mail2couch using Podman containers.

Overview

The test environment provides:

  • CouchDB: Database for storing email messages
  • GreenMail IMAP Server: Java-based mail server designed for testing with pre-populated test accounts and messages
  • Test Configuration: Ready-to-use config for testing both sync and archive modes

Quick Start

Run Full Integration Tests

./run-tests.sh

This will:

  1. Start all containers
  2. Populate test data
  3. Run mail2couch
  4. Verify results
  5. Clean up

Manual Testing

# Start test environment
./start-test-env.sh

# Run mail2couch manually
cd ../go
./mail2couch -config ../test/config-test.json

# Stop test environment when done
cd ../test
./stop-test-env.sh

Test Accounts

The test environment includes these IMAP accounts:

Username Password Mode Purpose
testuser1 password123 archive General archive testing
testuser2 password456 - Additional test user
syncuser syncpass sync Testing sync mode (1-to-1)
archiveuser archivepass archive Testing archive mode

Each account contains:

  • 10 messages in INBOX (every 3rd has an attachment)
  • 3 messages in Sent folder
  • Various message types for comprehensive testing

Services

CouchDB

GreenMail Server

  • Host: localhost
  • IMAP Port: 3143 (plain)
  • IMAPS Port: 3993 (SSL)
  • SMTP Port: 3025
  • Server: GreenMail (Java-based test server)

Database Structure

mail2couch will create separate databases for each mail source:

  • test_user_1 - Test User 1 (archive mode)
  • test_sync_user - Test Sync User (sync mode)
  • test_archive_user - Test Archive User (archive mode)

Testing Sync vs Archive Modes

Sync Mode (syncuser)

  • Database exactly matches mail account
  • If messages are deleted from IMAP, they're removed from CouchDB
  • 1-to-1 relationship

Archive Mode (archiveuser, testuser1)

  • Database preserves all messages ever seen
  • Messages deleted from IMAP remain in CouchDB
  • Archive/backup behavior

File Structure

test/
├── podman-compose.yml          # Container orchestration
├── config-test.json            # Test configuration
├── run-tests.sh               # Full integration test
├── start-test-env.sh          # Start environment
├── stop-test-env.sh           # Stop environment
├── generate-ssl.sh            # Generate SSL certificates
├── populate-test-messages.sh  # Create test messages
├── dovecot/
│   ├── dovecot.conf          # Dovecot configuration
│   ├── users                 # User database
│   ├── passwd                # Password database
│   └── ssl/                  # SSL certificates
└── README.md                 # This file

Prerequisites

  • Podman and podman-compose
  • OpenSSL (for certificate generation)
  • curl and nc (for connectivity checks)
  • Go (for building mail2couch)

Troubleshooting

Containers won't start

# Check podman status
podman ps -a

# View logs
podman logs mail2couch_test_couchdb
podman logs mail2couch_test_imap

CouchDB connection issues

  • Verify CouchDB is running: curl http://localhost:5984
  • Check admin credentials: admin/password

IMAP connection issues

  • Test IMAP connection: nc -z localhost 143
  • Check Dovecot logs: podman logs mail2couch_test_imap

Permission issues

  • Ensure scripts are executable: chmod +x *.sh
  • Check file permissions in dovecot directory

Advanced Usage

Add custom test messages

Edit populate-test-messages.sh to create additional test scenarios.

Modify IMAP configuration

Edit dovecot/dovecot.conf and restart containers.

Test with SSL

Update config-test.json to use port 993 and enable SSL.

Custom test scenarios

Create additional configuration files for specific test cases.