mail2couch/test

mail2couch Test Environment

This directory contains a complete test environment for mail2couch using Podman containers.

Overview

The test environment provides:

• CouchDB: Database for storing email messages
• GreenMail IMAP Server: Java-based mail server designed for testing with pre-populated test accounts and messages
• Test Configuration: Ready-to-use config for testing both sync and archive modes
• HTML Webmail Interface: Beautiful, responsive web interface for viewing archived emails

Quick Start

Run Basic Integration Tests

./run-tests.sh

This comprehensive test will:

1. Start all containers with cleanup
2. Populate test data
3. Build and run mail2couch
4. Verify database creation and document storage
5. Test incremental sync behavior
6. Clean up automatically

Run Wildcard Pattern Tests

./test-wildcard-patterns.sh

This will test various wildcard folder patterns including:

• * (all folders)
• Work* (prefix patterns)
• */Drafts (subfolder patterns)
• Complex include/exclude combinations

Run Incremental Sync Tests

./test-incremental-sync.sh

This will test incremental synchronization functionality:

• First sync establishes baseline
• New messages are added to test accounts
• Second sync should only fetch new messages
• Sync metadata tracking and IMAP SEARCH with SINCE

Manual Testing Environment

# Start persistent test environment (for manual experimentation)
./start-test-env.sh

# Run mail2couch manually with different configurations
cd ../go
./mail2couch -config ../test/config-test.json
./mail2couch -config ../test/config-wildcard-examples.json

# Stop test environment when done
cd ../test
./stop-test-env.sh

Test Scripts Overview

Automated Testing (Recommended)

• ./run-tests.sh: Complete integration test with automatic cleanup
  • Starts containers, populates data, runs mail2couch, verifies results
  • Tests basic functionality, database creation, and incremental sync
  • Cleans up automatically - perfect for CI/CD or quick validation

Specialized Feature Testing

• ./test-wildcard-patterns.sh: Comprehensive folder pattern testing
  • Tests *, Work*, */Drafts, and complex include/exclude patterns
  • Self-contained with own setup/teardown
• ./test-incremental-sync.sh: Incremental synchronization testing
  • Tests sync metadata tracking and IMAP SEARCH with SINCE
  • Multi-step validation: baseline sync → add messages → incremental sync
  • Self-contained with own setup/teardown

Manual Testing Environment

• ./start-test-env.sh: Start persistent test containers
  • Keeps environment running for manual experimentation
  • Populates test data once
  • Use with different configurations for development
• ./stop-test-env.sh: Clean up manual test environment
  • Only needed after using start-test-env.sh

Test Accounts

The test environment includes these IMAP accounts:

Username	Password	Mode	Folder Pattern	Purpose
testuser1	password123	archive	* (exclude Drafts, Trash)	Wildcard all folders test
syncuser	syncpass	sync	Work*, Important*, INBOX	Work pattern test
archiveuser	archivepass	archive	INBOX, Sent, Personal	Specific folders test
testuser2	password456	archive	Work/*, Archive/*	Subfolder pattern test

Each account contains:

• 10 messages in INBOX (every 3rd has an attachment)
• 3 messages in each additional folder
• Test folders: Sent, Work/Projects, Work/Archive, Work/Temp, Personal, Important/Urgent, Important/Meetings, Archive/2024, Archive/Projects, Archive/Drafts, Drafts, Trash
• Various message types for comprehensive wildcard testing

Services

CouchDB

• URL: http://localhost:5984
• Admin: admin / password
• Web UI: http://localhost:5984/_utils

GreenMail Server

• Host: localhost
• IMAP Port: 3143 (plain)
• IMAPS Port: 3993 (SSL)
• SMTP Port: 3025
• Server: GreenMail (Java-based test server)

Accessing Test Data

After running mail2couch, you can access the stored emails via CouchDB's REST API:

📋 Database Access:

• All databases: http://localhost:5984/_all_dbs
• Specific database: http://localhost:5984/m2c_specific_folders_only
• All documents: http://localhost:5984/m2c_specific_folders_only/_all_docs
• Individual message: http://localhost:5984/m2c_specific_folders_only/INBOX_12

🔍 Raw Data Examples:

• Database info: http://localhost:5984/m2c_specific_folders_only
• Document content: http://localhost:5984/m2c_specific_folders_only/INBOX_1
• Email attachments: http://localhost:5984/m2c_specific_folders_only/INBOX_1/{attachment_name}

Database Structure

mail2couch will create separate databases for each mail source (with m2c_ prefix):

• m2c_wildcard_all_folders_test - Wildcard All Folders Test (archive mode)
• m2c_work_pattern_test - Work Pattern Test (sync mode)
• m2c_specific_folders_only - Specific Folders Only (archive mode)
• m2c_subfolder_pattern_test - Subfolder Pattern Test (archive mode)

Each database contains documents with:

• mailbox field indicating the origin folder
• Native CouchDB attachments for email attachments
• Full message headers and body content
• JSON documents accessible via CouchDB REST API

Testing Sync vs Archive Modes

Sync Mode (syncuser)

• Database exactly matches mail account
• If messages are deleted from IMAP, they're removed from CouchDB
• 1-to-1 relationship

Archive Mode (archiveuser, testuser1)

• Database preserves all messages ever seen
• Messages deleted from IMAP remain in CouchDB
• Archive/backup behavior

Wildcard Pattern Examples

The test environment demonstrates these wildcard patterns:

All Folders Pattern (*)

{
  "folderFilter": {
    "include": ["*"],
    "exclude": ["Drafts", "Trash"]
  }
}

Processes all folders except Drafts and Trash.

Work Pattern (Work*)

{
  "folderFilter": {
    "include": ["Work*", "Important*", "INBOX"],
    "exclude": ["*Temp*"]
  }
}

Includes Work/Projects, Work/Archive, Important/Urgent, Important/Meetings, and INBOX. Excludes Work/Temp.

Specific Folders

{
  "folderFilter": {
    "include": ["INBOX", "Sent", "Personal"],
    "exclude": []
  }
}

Only processes the exact named folders.

Subfolder Pattern (Work/*)

{
  "folderFilter": {
    "include": ["Work/*", "Archive/*"],
    "exclude": ["*/Drafts"]
  }
}

Includes all subfolders under Work and Archive, but excludes any Drafts subfolder.

File Structure

test/
├── podman-compose.yml            # Container orchestration (GreenMail + CouchDB)
├── config-test.json              # Main test configuration with wildcard examples
├── config-wildcard-examples.json # Advanced wildcard patterns
├── run-tests.sh                  # Automated integration test (recommended)
├── test-wildcard-patterns.sh     # Specialized wildcard pattern testing
├── test-incremental-sync.sh      # Specialized incremental sync testing
├── start-test-env.sh             # Start persistent test environment
├── stop-test-env.sh              # Stop test environment
├── populate-greenmail.py         # Create test messages across multiple folders
├── dovecot/                      # Dovecot configuration (legacy, unused)
└── README.md                     # This file

Prerequisites

• Podman and podman-compose
• OpenSSL (for certificate generation)
• curl and nc (for connectivity checks)
• Go (for building mail2couch)

Troubleshooting

Containers won't start

# Check podman status
podman ps -a

# View logs
podman logs mail2couch_test_couchdb
podman logs mail2couch_test_imap

CouchDB connection issues

• Verify CouchDB is running: curl http://localhost:5984
• Check admin credentials: admin/password

IMAP connection issues

• Test IMAP connection: nc -z localhost 143
• Check Dovecot logs: podman logs mail2couch_test_imap

Permission issues

• Ensure scripts are executable: chmod +x *.sh
• Check file permissions in dovecot directory

Advanced Usage

Add custom test messages

Edit populate-test-messages.sh to create additional test scenarios.

Modify IMAP configuration

Edit dovecot/dovecot.conf and restart containers.

Test with SSL

Update config-test.json to use port 993 and enable SSL.

Custom test scenarios

Create additional configuration files for specific test cases.