feat: implement real IMAP message parsing with native CouchDB attachments
- Replace placeholder message generation with actual IMAP message fetching using go-message library - Add per-account CouchDB databases for better organization and isolation - Implement native CouchDB attachment storage with proper revision management - Add command line argument parsing with --max-messages flag for controlling message processing limits - Support both sync and archive modes with proper document synchronization - Add comprehensive test environment with Podman containers (GreenMail IMAP server + CouchDB) - Implement full MIME multipart parsing for proper body and attachment extraction - Add TLS and plain IMAP connection support based on port configuration - Update configuration system to support sync vs archive modes - Create test scripts and sample data for development and testing Key technical improvements: - Real email envelope and header processing with go-imap v2 API - MIME Content-Type and Content-Disposition parsing for attachment detection - CouchDB document ID generation using mailbox_uid format for uniqueness - Duplicate detection and prevention to avoid re-storing existing messages - Proper error handling and connection management for IMAP operations 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
parent
79f19a8877
commit
ea6235b674
22 changed files with 1262 additions and 66 deletions
150
test/README.md
Normal file
150
test/README.md
Normal file
|
|
@ -0,0 +1,150 @@
|
|||
# mail2couch Test Environment
|
||||
|
||||
This directory contains a complete test environment for mail2couch using Podman containers.
|
||||
|
||||
## Overview
|
||||
|
||||
The test environment provides:
|
||||
- **CouchDB**: Database for storing email messages
|
||||
- **GreenMail IMAP Server**: Java-based mail server designed for testing with pre-populated test accounts and messages
|
||||
- **Test Configuration**: Ready-to-use config for testing both sync and archive modes
|
||||
|
||||
## Quick Start
|
||||
|
||||
### Run Full Integration Tests
|
||||
```bash
|
||||
./run-tests.sh
|
||||
```
|
||||
This will:
|
||||
1. Start all containers
|
||||
2. Populate test data
|
||||
3. Run mail2couch
|
||||
4. Verify results
|
||||
5. Clean up
|
||||
|
||||
### Manual Testing
|
||||
```bash
|
||||
# Start test environment
|
||||
./start-test-env.sh
|
||||
|
||||
# Run mail2couch manually
|
||||
cd ../go
|
||||
./mail2couch -config ../test/config-test.json
|
||||
|
||||
# Stop test environment when done
|
||||
cd ../test
|
||||
./stop-test-env.sh
|
||||
```
|
||||
|
||||
## Test Accounts
|
||||
|
||||
The test environment includes these IMAP accounts:
|
||||
|
||||
| Username | Password | Mode | Purpose |
|
||||
|----------|----------|------|---------|
|
||||
| `testuser1` | `password123` | archive | General archive testing |
|
||||
| `testuser2` | `password456` | - | Additional test user |
|
||||
| `syncuser` | `syncpass` | sync | Testing sync mode (1-to-1) |
|
||||
| `archiveuser` | `archivepass` | archive | Testing archive mode |
|
||||
|
||||
Each account contains:
|
||||
- 10 messages in INBOX (every 3rd has an attachment)
|
||||
- 3 messages in Sent folder
|
||||
- Various message types for comprehensive testing
|
||||
|
||||
## Services
|
||||
|
||||
### CouchDB
|
||||
- **URL**: http://localhost:5984
|
||||
- **Admin**: `admin` / `password`
|
||||
- **Web UI**: http://localhost:5984/_utils
|
||||
|
||||
### GreenMail Server
|
||||
- **Host**: localhost
|
||||
- **IMAP Port**: 3143 (plain)
|
||||
- **IMAPS Port**: 3993 (SSL)
|
||||
- **SMTP Port**: 3025
|
||||
- **Server**: GreenMail (Java-based test server)
|
||||
|
||||
## Database Structure
|
||||
|
||||
mail2couch will create separate databases for each mail source:
|
||||
- `test_user_1` - Test User 1 (archive mode)
|
||||
- `test_sync_user` - Test Sync User (sync mode)
|
||||
- `test_archive_user` - Test Archive User (archive mode)
|
||||
|
||||
## Testing Sync vs Archive Modes
|
||||
|
||||
### Sync Mode (`syncuser`)
|
||||
- Database exactly matches mail account
|
||||
- If messages are deleted from IMAP, they're removed from CouchDB
|
||||
- 1-to-1 relationship
|
||||
|
||||
### Archive Mode (`archiveuser`, `testuser1`)
|
||||
- Database preserves all messages ever seen
|
||||
- Messages deleted from IMAP remain in CouchDB
|
||||
- Archive/backup behavior
|
||||
|
||||
## File Structure
|
||||
|
||||
```
|
||||
test/
|
||||
├── podman-compose.yml # Container orchestration
|
||||
├── config-test.json # Test configuration
|
||||
├── run-tests.sh # Full integration test
|
||||
├── start-test-env.sh # Start environment
|
||||
├── stop-test-env.sh # Stop environment
|
||||
├── generate-ssl.sh # Generate SSL certificates
|
||||
├── populate-test-messages.sh # Create test messages
|
||||
├── dovecot/
|
||||
│ ├── dovecot.conf # Dovecot configuration
|
||||
│ ├── users # User database
|
||||
│ ├── passwd # Password database
|
||||
│ └── ssl/ # SSL certificates
|
||||
└── README.md # This file
|
||||
```
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- Podman and podman-compose
|
||||
- OpenSSL (for certificate generation)
|
||||
- curl and nc (for connectivity checks)
|
||||
- Go (for building mail2couch)
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Containers won't start
|
||||
```bash
|
||||
# Check podman status
|
||||
podman ps -a
|
||||
|
||||
# View logs
|
||||
podman logs mail2couch_test_couchdb
|
||||
podman logs mail2couch_test_imap
|
||||
```
|
||||
|
||||
### CouchDB connection issues
|
||||
- Verify CouchDB is running: `curl http://localhost:5984`
|
||||
- Check admin credentials: `admin/password`
|
||||
|
||||
### IMAP connection issues
|
||||
- Test IMAP connection: `nc -z localhost 143`
|
||||
- Check Dovecot logs: `podman logs mail2couch_test_imap`
|
||||
|
||||
### Permission issues
|
||||
- Ensure scripts are executable: `chmod +x *.sh`
|
||||
- Check file permissions in dovecot directory
|
||||
|
||||
## Advanced Usage
|
||||
|
||||
### Add custom test messages
|
||||
Edit `populate-test-messages.sh` to create additional test scenarios.
|
||||
|
||||
### Modify IMAP configuration
|
||||
Edit `dovecot/dovecot.conf` and restart containers.
|
||||
|
||||
### Test with SSL
|
||||
Update `config-test.json` to use port 993 and enable SSL.
|
||||
|
||||
### Custom test scenarios
|
||||
Create additional configuration files for specific test cases.
|
||||
Loading…
Add table
Add a link
Reference in a new issue