feat: implement comprehensive wildcard folder selection and keyword filtering
## Wildcard Folder Selection
- Add support for wildcard patterns (`*`, `?`, `[abc]`) using filepath.Match
- Implement special case: `"*"` selects ALL available folders
- Support for complex include/exclude pattern combinations
- Maintain backwards compatibility with exact string matching
- Enable subfolder pattern matching (e.g., `Work/*`, `*/Drafts`)
## Keyword Filtering
- Add SubjectKeywords, SenderKeywords, RecipientKeywords to MessageFilter config
- Implement case-insensitive keyword matching across message fields
- Support multiple keywords per filter type with inclusive OR logic
- Add ShouldProcessMessage method for message-level filtering
## Enhanced Test Environment
- Create comprehensive wildcard pattern test scenarios
- Add 12 test folders covering various pattern types: Work/*, Important/*, Archive/*, exact matches
- Implement dedicated wildcard test script (test-wildcard-patterns.sh)
- Update test configurations to demonstrate real-world wildcard usage patterns
- Enhance test data generation with folder-specific messages for validation
## Documentation
- Create FOLDER_PATTERNS.md with comprehensive wildcard examples and use cases
- Update CLAUDE.md to reflect all implemented features and current status
- Enhance test README with detailed wildcard pattern explanations
- Provide configuration examples for common email organization scenarios
## Message Origin Tracking
- Verify all messages in CouchDB properly tagged with origin folder in `mailbox` field
- Maintain per-account database isolation for better organization
- Document ID format: `{folder}_{uid}` ensures uniqueness across folders
Key patterns supported:
- `["*"]` - All folders (with excludes)
- `["Work*", "Important*"]` - Prefix matching
- `["Work/*", "Archive/*"]` - Subfolder patterns
- `["INBOX", "Sent"]` - Exact matches
- Complex include/exclude combinations
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
parent
ea6235b674
commit
357cd06264
10 changed files with 602 additions and 84 deletions
101
test/README.md
101
test/README.md
|
|
@ -22,6 +22,16 @@ This will:
|
|||
4. Verify results
|
||||
5. Clean up
|
||||
|
||||
### Run Wildcard Pattern Tests
|
||||
```bash
|
||||
./test-wildcard-patterns.sh
|
||||
```
|
||||
This will test various wildcard folder patterns including:
|
||||
- `*` (all folders)
|
||||
- `Work*` (prefix patterns)
|
||||
- `*/Drafts` (subfolder patterns)
|
||||
- Complex include/exclude combinations
|
||||
|
||||
### Manual Testing
|
||||
```bash
|
||||
# Start test environment
|
||||
|
|
@ -40,17 +50,18 @@ cd ../test
|
|||
|
||||
The test environment includes these IMAP accounts:
|
||||
|
||||
| Username | Password | Mode | Purpose |
|
||||
|----------|----------|------|---------|
|
||||
| `testuser1` | `password123` | archive | General archive testing |
|
||||
| `testuser2` | `password456` | - | Additional test user |
|
||||
| `syncuser` | `syncpass` | sync | Testing sync mode (1-to-1) |
|
||||
| `archiveuser` | `archivepass` | archive | Testing archive mode |
|
||||
| Username | Password | Mode | Folder Pattern | Purpose |
|
||||
|----------|----------|------|---------------|---------|
|
||||
| `testuser1` | `password123` | archive | `*` (exclude Drafts, Trash) | Wildcard all folders test |
|
||||
| `syncuser` | `syncpass` | sync | `Work*`, `Important*`, `INBOX` | Work pattern test |
|
||||
| `archiveuser` | `archivepass` | archive | `INBOX`, `Sent`, `Personal` | Specific folders test |
|
||||
| `testuser2` | `password456` | archive | `Work/*`, `Archive/*` | Subfolder pattern test |
|
||||
|
||||
Each account contains:
|
||||
- 10 messages in INBOX (every 3rd has an attachment)
|
||||
- 3 messages in Sent folder
|
||||
- Various message types for comprehensive testing
|
||||
- 3 messages in each additional folder
|
||||
- Test folders: `Sent`, `Work/Projects`, `Work/Archive`, `Work/Temp`, `Personal`, `Important/Urgent`, `Important/Meetings`, `Archive/2024`, `Archive/Projects`, `Archive/Drafts`, `Drafts`, `Trash`
|
||||
- Various message types for comprehensive wildcard testing
|
||||
|
||||
## Services
|
||||
|
||||
|
|
@ -69,9 +80,15 @@ Each account contains:
|
|||
## Database Structure
|
||||
|
||||
mail2couch will create separate databases for each mail source:
|
||||
- `test_user_1` - Test User 1 (archive mode)
|
||||
- `test_sync_user` - Test Sync User (sync mode)
|
||||
- `test_archive_user` - Test Archive User (archive mode)
|
||||
- `wildcard_all_folders_test` - Wildcard All Folders Test (archive mode)
|
||||
- `work_pattern_test` - Work Pattern Test (sync mode)
|
||||
- `specific_folders_only` - Specific Folders Only (archive mode)
|
||||
- `subfolder_pattern_test` - Subfolder Pattern Test (archive mode)
|
||||
|
||||
Each database contains documents with:
|
||||
- `mailbox` field indicating the origin folder
|
||||
- Native CouchDB attachments for email attachments
|
||||
- Full message headers and body content
|
||||
|
||||
## Testing Sync vs Archive Modes
|
||||
|
||||
|
|
@ -85,22 +102,68 @@ mail2couch will create separate databases for each mail source:
|
|||
- Messages deleted from IMAP remain in CouchDB
|
||||
- Archive/backup behavior
|
||||
|
||||
## Wildcard Pattern Examples
|
||||
|
||||
The test environment demonstrates these wildcard patterns:
|
||||
|
||||
### All Folders Pattern (`*`)
|
||||
```json
|
||||
{
|
||||
"folderFilter": {
|
||||
"include": ["*"],
|
||||
"exclude": ["Drafts", "Trash"]
|
||||
}
|
||||
}
|
||||
```
|
||||
Processes all folders except Drafts and Trash.
|
||||
|
||||
### Work Pattern (`Work*`)
|
||||
```json
|
||||
{
|
||||
"folderFilter": {
|
||||
"include": ["Work*", "Important*", "INBOX"],
|
||||
"exclude": ["*Temp*"]
|
||||
}
|
||||
}
|
||||
```
|
||||
Includes Work/Projects, Work/Archive, Important/Urgent, Important/Meetings, and INBOX. Excludes Work/Temp.
|
||||
|
||||
### Specific Folders
|
||||
```json
|
||||
{
|
||||
"folderFilter": {
|
||||
"include": ["INBOX", "Sent", "Personal"],
|
||||
"exclude": []
|
||||
}
|
||||
}
|
||||
```
|
||||
Only processes the exact named folders.
|
||||
|
||||
### Subfolder Pattern (`Work/*`)
|
||||
```json
|
||||
{
|
||||
"folderFilter": {
|
||||
"include": ["Work/*", "Archive/*"],
|
||||
"exclude": ["*/Drafts"]
|
||||
}
|
||||
}
|
||||
```
|
||||
Includes all subfolders under Work and Archive, but excludes any Drafts subfolder.
|
||||
|
||||
## File Structure
|
||||
|
||||
```
|
||||
test/
|
||||
├── podman-compose.yml # Container orchestration
|
||||
├── config-test.json # Test configuration
|
||||
├── config-test.json # Main test configuration with wildcard examples
|
||||
├── config-wildcard-examples.json # Advanced wildcard patterns
|
||||
├── test-wildcard-patterns.sh # Wildcard pattern testing script
|
||||
├── run-tests.sh # Full integration test
|
||||
├── start-test-env.sh # Start environment
|
||||
├── stop-test-env.sh # Stop environment
|
||||
├── generate-ssl.sh # Generate SSL certificates
|
||||
├── populate-test-messages.sh # Create test messages
|
||||
├── dovecot/
|
||||
│ ├── dovecot.conf # Dovecot configuration
|
||||
│ ├── users # User database
|
||||
│ ├── passwd # Password database
|
||||
│ └── ssl/ # SSL certificates
|
||||
├── populate-greenmail.py # Create test messages with folders
|
||||
├── populate-test-messages.sh # Wrapper script
|
||||
├── dovecot/ # Dovecot configuration (legacy)
|
||||
└── README.md # This file
|
||||
```
|
||||
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue