From 031dd86b0d2fc0f38db53889966a6bb9f4c74798 Mon Sep 17 00:00:00 2001 From: Ole-Morten Duesund Date: Sat, 2 Aug 2025 15:17:04 +0200 Subject: [PATCH 1/3] feat: implement GNU-style command line options with pflag MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - Add pflag dependency for POSIX/GNU-style command line parsing - Replace Go standard flag package with pflag for better UX - Implement long options with double dashes (--config, --max-messages, --help) - Add short option aliases with single dashes (-c, -m, -h) - Update help message with proper formatting and application description - Update all documentation to reflect new flag syntax - Update test scripts to use new command line format GNU-style options provide better usability: - Long descriptive options with --flag-name format - Short single-character aliases for common options - Standard help flag behavior with --help/-h - Compatible with shell completion and standard conventions Command line interface now supports: - --config/-c FILE: Path to configuration file - --max-messages/-m N: Message processing limit per mailbox - --help/-h: Show help message and exit All existing functionality preserved with improved command line experience. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude --- CLAUDE.md | 10 ++++----- README.md | 13 ++++++------ go/config/config.go | 37 ++++++++++++++++++++++++---------- go/go.mod | 1 + go/go.sum | 2 ++ test/run-tests.sh | 4 ++-- test/test-incremental-sync.sh | 4 ++-- test/test-wildcard-patterns.sh | 2 +- 8 files changed, 46 insertions(+), 27 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index 4a2cb2b..7575003 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -20,13 +20,13 @@ cd go && go build -o mail2couch . cd go && ./mail2couch # Run with specific config file -cd go && ./mail2couch -config /path/to/config.json +cd go && ./mail2couch --config /path/to/config.json # Run with message limit (useful for large mailboxes) -cd go && ./mail2couch -max-messages 100 +cd go && ./mail2couch --max-messages 100 # Run with both config and message limit -cd go && ./mail2couch -config /path/to/config.json -max-messages 50 +cd go && ./mail2couch --config /path/to/config.json --max-messages 50 # Run linting/static analysis cd go && go vet ./... @@ -80,7 +80,7 @@ The application uses `config.json` for configuration with the following structur ### Configuration File Discovery The application automatically searches for configuration files in the following order: -1. Path specified by `-config` command line flag +1. Path specified by `--config`/`-c` command line flag 2. `./config.json` (current working directory) 3. `./config/config.json` (config subdirectory) 4. `~/.config/mail2couch/config.json` (user XDG config directory) @@ -104,7 +104,7 @@ This design ensures the same `config.json` format will work for both Go and Rust - ✅ Sync vs Archive mode implementation - ✅ CouchDB attachment storage for email attachments - ✅ Full message body and attachment handling with MIME multipart support -- ✅ Command line argument support (--max-messages flag) +- ✅ Command line argument support (GNU-style --max-messages/-m and --config/-c flags) - ✅ Per-account CouchDB databases for better organization - ✅ Incremental sync functionality with IMAP SEARCH and sync metadata tracking - ❌ Rust implementation diff --git a/README.md b/README.md index 7c0fc5e..425b9c7 100644 --- a/README.md +++ b/README.md @@ -39,7 +39,7 @@ A powerful email backup utility that synchronizes mail from IMAP accounts to Cou ### Operational Features - **Automatic Config Discovery**: Finds configuration files in standard locations -- **Command Line Control**: Override settings with `--max-messages` and `--config` flags +- **Command Line Control**: GNU-style options with `--max-messages`/`-m` and `--config`/`-c` flags - **Comprehensive Logging**: Detailed output for monitoring and troubleshooting - **Error Resilience**: Graceful handling of network issues and server problems @@ -105,7 +105,7 @@ The application will: ### Configuration File Discovery mail2couch automatically searches for configuration files in this order: -1. Path specified by `--config` flag +1. Path specified by `--config`/`-c` flag 2. `./config.json` (current directory) 3. `./config/config.json` (config subdirectory) 4. `~/.config/mail2couch/config.json` (user config directory) @@ -117,8 +117,9 @@ mail2couch automatically searches for configuration files in this order: ./mail2couch [options] Options: - --config PATH Specify configuration file path - --max-messages N Limit messages processed per mailbox per run (0 = unlimited) + -c, --config FILE Path to configuration file + -m, --max-messages N Limit messages processed per mailbox per run (0 = unlimited) + -h, --help Show help message ``` ### Folder Pattern Examples @@ -382,7 +383,7 @@ Complex setup with multiple accounts, filtering, and different sync modes: - Schedule regular backups of CouchDB databases ### Performance Tuning -- Use `--max-messages` to limit processing load +- Use `--max-messages`/`-m` to limit processing load - Run during off-peak hours for large initial syncs - Monitor IMAP server rate limits and connection limits - Consider running multiple instances for different accounts @@ -408,7 +409,7 @@ Complex setup with multiple accounts, filtering, and different sync modes: **Performance Problems**: - Use date filtering (`since`) for large mailboxes -- Implement `--max-messages` limits for initial syncs +- Implement `--max-messages`/`-m` limits for initial syncs - Monitor server-side rate limiting For detailed troubleshooting, see the [test environment documentation](test/README.md). diff --git a/go/config/config.go b/go/config/config.go index dc1808e..8d2527d 100644 --- a/go/config/config.go +++ b/go/config/config.go @@ -2,10 +2,11 @@ package config import ( "encoding/json" - "flag" "fmt" "os" "path/filepath" + + "github.com/spf13/pflag" ) type Config struct { @@ -87,23 +88,37 @@ type CommandLineArgs struct { MaxMessages int } -// ParseCommandLine parses command line arguments +// ParseCommandLine parses command line arguments using GNU-style options func ParseCommandLine() *CommandLineArgs { - configFlag := flag.String("config", "", "Path to configuration file") - maxMessagesFlag := flag.Int("max-messages", 0, "Maximum number of messages to process per mailbox per run (0 = no limit)") - flag.Parse() + var args CommandLineArgs - return &CommandLineArgs{ - ConfigPath: *configFlag, - MaxMessages: *maxMessagesFlag, + // Define long options with -- and short options with - + pflag.StringVarP(&args.ConfigPath, "config", "c", "", "Path to configuration file") + pflag.IntVarP(&args.MaxMessages, "max-messages", "m", 0, "Maximum number of messages to process per mailbox per run (0 = no limit)") + + // Add help option + pflag.BoolP("help", "h", false, "Show help message") + + pflag.Parse() + + // Handle help flag + if help, _ := pflag.CommandLine.GetBool("help"); help { + fmt.Fprintf(os.Stderr, "mail2couch - Email backup utility for CouchDB\n\n") + fmt.Fprintf(os.Stderr, "Usage: %s [OPTIONS]\n\n", os.Args[0]) + fmt.Fprintf(os.Stderr, "Options:\n") + pflag.PrintDefaults() + os.Exit(0) } + + return &args } // FindConfigFile searches for config.json in the following order: -// 1. Path specified by -config flag +// 1. Path specified by --config/-c flag // 2. ./config.json (current directory) -// 3. ~/.config/mail2couch/config.json (user config directory) -// 4. ~/.mail2couch.json (user home directory) +// 3. ./config/config.json (config subdirectory) +// 4. ~/.config/mail2couch/config.json (user config directory) +// 5. ~/.mail2couch.json (user home directory) func FindConfigFile(args *CommandLineArgs) (string, error) { if args.ConfigPath != "" { if _, err := os.Stat(args.ConfigPath); err == nil { diff --git a/go/go.mod b/go/go.mod index 1f6d85c..ce41ba3 100644 --- a/go/go.mod +++ b/go/go.mod @@ -6,6 +6,7 @@ require ( github.com/emersion/go-imap/v2 v2.0.0-beta.5 github.com/emersion/go-message v0.18.1 github.com/go-kivik/kivik/v4 v4.4.0 + github.com/spf13/pflag v1.0.7 ) require ( diff --git a/go/go.sum b/go/go.sum index 7851761..9138894 100644 --- a/go/go.sum +++ b/go/go.sum @@ -24,6 +24,8 @@ github.com/pkg/errors v0.9.1 h1:FEBLx1zS214owpjy7qsBeixbURkuhQAwrK5UwLGTwt4= github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0= github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 h1:Jamvg5psRIccs7FGNTlIRMkT8wgtp5eCXdBlqhYGL6U= github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= +github.com/spf13/pflag v1.0.7 h1:vN6T9TfwStFPFM5XzjsvmzZkLuaLX+HS+0SeFLRgU6M= +github.com/spf13/pflag v1.0.7/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg= github.com/yuin/goldmark v1.4.13/go.mod h1:6yULJ656Px+3vBD8DxQVa3kxgyrAnzto9xy5taEt/CY= gitlab.com/flimzy/testy v0.14.0 h1:2nZV4Wa1OSJb3rOKHh0GJqvvhtE03zT+sKnPCI0owfQ= gitlab.com/flimzy/testy v0.14.0/go.mod h1:m3aGuwdXc+N3QgnH+2Ar2zf1yg0UxNdIaXKvC5SlfMk= diff --git a/test/run-tests.sh b/test/run-tests.sh index 79d5a75..b5bdf53 100755 --- a/test/run-tests.sh +++ b/test/run-tests.sh @@ -83,7 +83,7 @@ cd ../test # Run mail2couch with test configuration print_status "Running mail2couch with test configuration..." -../go/mail2couch -config config-test.json -max-messages 3 +../go/mail2couch --config config-test.json --max-messages 3 # Verify results print_status "Verifying test results..." @@ -112,7 +112,7 @@ done # Test sync mode by running again (should show incremental behavior) print_status "Running mail2couch again to test incremental sync..." -../go/mail2couch -config config-test.json -max-messages 3 +../go/mail2couch --config config-test.json --max-messages 3 print_status "🎉 Basic integration tests completed successfully!" diff --git a/test/test-incremental-sync.sh b/test/test-incremental-sync.sh index 45e0005..09b5269 100755 --- a/test/test-incremental-sync.sh +++ b/test/test-incremental-sync.sh @@ -67,7 +67,7 @@ build_app() { run_first_sync() { echo -e "\n${BLUE}Running first sync...${NC}" cd go - ./mail2couch -config ../test/config-test.json -max-messages 5 + ./mail2couch --config ../test/config-test.json --max-messages 5 cd .. } @@ -138,7 +138,7 @@ EOF run_incremental_sync() { echo -e "\n${BLUE}Running incremental sync...${NC}" cd go - ./mail2couch -config ../test/config-test.json -max-messages 10 + ./mail2couch --config ../test/config-test.json --max-messages 10 cd .. } diff --git a/test/test-wildcard-patterns.sh b/test/test-wildcard-patterns.sh index e277ea3..439b1bd 100755 --- a/test/test-wildcard-patterns.sh +++ b/test/test-wildcard-patterns.sh @@ -31,7 +31,7 @@ run_test() { # Run mail2couch and capture output cd go - if ./mail2couch -config "../test/$config_file" -max-messages "$max_messages" 2>&1; then + if ./mail2couch --config "../test/$config_file" --max-messages "$max_messages" 2>&1; then echo -e "${GREEN}✅ Test completed successfully${NC}" else echo -e "${RED}❌ Test failed${NC}" From 18e13500062b2f1f5e1e4a54a47078b05cc27380 Mon Sep 17 00:00:00 2001 From: Ole-Morten Duesund Date: Sat, 2 Aug 2025 15:25:18 +0200 Subject: [PATCH 2/3] docs: add comprehensive project analysis Adds a detailed analysis of the mail2couch project, covering its purpose, architecture, and implementation details. The document provides a summary of the project's functionality based on a review of the README, schema files, and the Go source code. The analysis includes: - A high-level summary of what the project does. - A detailed explanation of how it works, based on code review. - Identification of potential problems, such as plaintext password storage and the incomplete Rust implementation. - A list of missing features, including the planned webmail UI and OAuth2 support. - Concrete suggestions for improvements, such as adding a --dry-run mode, improving security, and clarifying the status of the Rust code. --- ANALYSIS.md | 30 ++++++++++++++++++++++++++++++ 1 file changed, 30 insertions(+) create mode 100644 ANALYSIS.md diff --git a/ANALYSIS.md b/ANALYSIS.md new file mode 100644 index 0000000..7bb5855 --- /dev/null +++ b/ANALYSIS.md @@ -0,0 +1,30 @@ +### Final Project Analysis + +**What it does:** +`mail2couch` is a Go-based command-line tool that archives emails from IMAP servers into a CouchDB database. It performs efficient incremental syncs, allows for flexible filtering of folders and messages, and stores attachments natively within CouchDB. Each mail account is stored in a separate database for clear organization. + +**How it does it:** +The application is written in Go and uses a `config.json` file to manage CouchDB and IMAP credentials. It leverages the `go-imap` library for IMAP communication and `kivik` for interacting with CouchDB. It maintains a `sync_metadata` document in each CouchDB database to track the last sync time, enabling it to only fetch new messages on subsequent runs. A `sync` mode is available to keep the archive as a 1-to-1 mirror of the server, but the default `archive` mode preserves all fetched mail. The project also includes a comprehensive test suite using Podman to validate its core features. + +**Problems, Missing Features, and Suggested Improvements:** + +* **Primary Weakness: Security:** The application requires storing IMAP and CouchDB passwords in plain text within the `config.json` file. This is a significant security risk. + * **Suggestion:** Prioritize adding support for reading secrets from environment variables (e.g., `M2C_COUCH_PASSWORD`) or integrating with a secrets management tool. For services like Gmail, implementing OAuth2 would be a more secure and modern authentication method than app passwords. + +* **Incomplete Rust Implementation:** The `rust/` directory contains a non-functional placeholder for a Rust version of the tool. This could be confusing for contributors. + * **Suggestion:** The `README.md` should explicitly state that the Rust implementation is aspirational and not functional. Alternatively, if there are no plans to develop it, it could be removed to avoid confusion. + +* **Missing Core Feature: Web Interface:** The `README.md` heavily promotes a future web interface for viewing the archived emails, which is a key feature for making the archive useful. This feature is not yet implemented. + * **Suggestion:** This should be the highest priority for new feature development. The existing CouchDB schema is well-suited for this, and implementing CouchDB design documents with list and show functions would be the next logical step. + +* **Performance for Large-Scale Use:** The application processes accounts and mailboxes sequentially. + * **Suggestion:** For users with many accounts or mailboxes, performance could be significantly improved by introducing concurrency. Using Go's concurrency features (goroutines and channels) to process multiple mailboxes or even multiple accounts in parallel would be a valuable enhancement. + +* **Inefficient Keyword Filtering:** Message filtering by keywords (subject, sender, etc.) is done client-side *after* downloading the messages. + * **Suggestion:** Modify the IMAP fetching logic to use server-side `IMAP SEARCH` with keyword criteria. This would reduce bandwidth and processing time, especially for users who only need to archive a small subset of messages from a large mailbox. + +* **Lack of a License:** The project is missing a formal license, which creates legal ambiguity and may deter adoption and contribution. + * **Suggestion:** Add a standard open-source license (e.g., MIT, Apache 2.0) to the `LICENSE` file. + +* **Usability Enhancement: Dry-Run Mode:** Users have no way to test their configuration (especially folder and message filters) without performing a live sync. + * **Suggestion:** Implement a `--dry-run` flag that would log which mailboxes and messages *would* be processed and stored, without actually writing any data to CouchDB. From 5a125ba410707345c852ba42fa89ac08e1bf335e Mon Sep 17 00:00:00 2001 From: Ole-Morten Duesund Date: Sat, 2 Aug 2025 15:32:47 +0200 Subject: [PATCH 3/3] feat: add MIT license - Add the MIT license to the project. - Update the README to reference the new license. - Remove the license issue from the ANALYSIS.md document. --- ANALYSIS.md | 3 +-- LICENSE | 20 ++++++++++++++++---- README.md | 2 +- 3 files changed, 18 insertions(+), 7 deletions(-) diff --git a/ANALYSIS.md b/ANALYSIS.md index 7bb5855..8fa2d77 100644 --- a/ANALYSIS.md +++ b/ANALYSIS.md @@ -23,8 +23,7 @@ The application is written in Go and uses a `config.json` file to manage CouchDB * **Inefficient Keyword Filtering:** Message filtering by keywords (subject, sender, etc.) is done client-side *after* downloading the messages. * **Suggestion:** Modify the IMAP fetching logic to use server-side `IMAP SEARCH` with keyword criteria. This would reduce bandwidth and processing time, especially for users who only need to archive a small subset of messages from a large mailbox. -* **Lack of a License:** The project is missing a formal license, which creates legal ambiguity and may deter adoption and contribution. - * **Suggestion:** Add a standard open-source license (e.g., MIT, Apache 2.0) to the `LICENSE` file. + * **Usability Enhancement: Dry-Run Mode:** Users have no way to test their configuration (especially folder and message filters) without performing a live sync. * **Suggestion:** Implement a `--dry-run` flag that would log which mailboxes and messages *would* be processed and stored, without actually writing any data to CouchDB. diff --git a/LICENSE b/LICENSE index 353b5e6..79c42ed 100644 --- a/LICENSE +++ b/LICENSE @@ -1,9 +1,21 @@ MIT License -Copyright (c) 2025 olemd +Copyright (c) 2025 Ole-Morten Duesund -Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: -The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. -THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. \ No newline at end of file diff --git a/README.md b/README.md index 425b9c7..f6eb104 100644 --- a/README.md +++ b/README.md @@ -444,4 +444,4 @@ This project welcomes contributions! Please see [CLAUDE.md](CLAUDE.md) for devel ## License -[License information to be added] +This project is licensed under the MIT License. See the [LICENSE](LICENSE) file for details.