diff --git a/.gitignore b/.gitignore index cb02d3b..21809b1 100644 --- a/.gitignore +++ b/.gitignore @@ -2,3 +2,4 @@ __pycache__/ .pytest_cache/ .ruff_cache/ *.html +dist/ diff --git a/CLAUDE.md b/CLAUDE.md new file mode 100644 index 0000000..21b8c6e --- /dev/null +++ b/CLAUDE.md @@ -0,0 +1,78 @@ +# CLAUDE.md + +Guidance for working on this repository. It records decisions made when the +project was started (August 2026) that are not obvious from the code. + +## What this is + +Small single-file Python utilities for **Conan 1.x, targeting 1.66**. Not +Conan 2. The first (and so far only) tool is `conandeps.py`; see `README.md` +for what it does and how it is used. + +## Hard constraints + +- **Conan 1.66 only.** Use the `conans.client.conan_api.ConanAPIV1` API and + Conan 1 graph internals (`Node.binary`, `Node.dependencies`, + `conanfile.requires`). Do not introduce Conan 2 APIs or a Conan 2 code path. +- **Python 3.8–3.11.** Conan 1.x imports the `imp` module, which is gone in + 3.12. `requires-python` in `pyproject.toml` and the PEP 723 header in + `conandeps.py` both say `<3.12`; keep them in sync. +- **All development runs inside the container.** Use `./dev.sh ` for + every python/pytest/ruff/uv invocation. Never run `conan` against the + host's `~/.conan`; never `pip install` on the host. Rebuild the image with + `./dev.sh build` after touching `Containerfile`. +- **Never depend on the private remote.** The real remote is called `knor` + and only exists on the machine where the tool is used. Tests must use the + local `conan_server` fixture in `tests/test_conandeps.py`. + +## Design decisions (and why) + +- **Binary availability is computed via Conan's graph, not `conan search`.** + The graph is built once per build type through `ConanAPIV1.info()` and each + node's `binary` status is read. This honours `package_id()` overrides, + options and `default_package_id_mode` exactly like `conan install`. Do not + replace this with settings-dict matching against `conan search` output. +- **"Missing" means missing for the exact profile.** Not "no binary with + that build_type at all". +- **Overrides are parsed from Conan's WARN output.** Conan 1 does not record + overrides on the graph; `Requirements.update()` mutates `req.ref` in place + and only emits `": requirement overridden by to "`. + `conandeps` captures the output stream with a non-coloured `ConanOutput` + and matches `_OVERRIDE_RE`. `test_overrides` pins this format – if a Conan + patch release changes the wording, that test is the alarm. +- **Both explicit and implicit overrides are reported.** `override=True` + requirements are collected from `conanfile.requires` to label an override + as explicit; everything else parsed from the WARN lines is implicit. +- **`range_ref` is not a version range after an override.** Conan reuses + `Requirement.range_ref` to hold the pre-override reference, so only treat + it as a range when `req.version_range` is truthy. +- **Single file, stdlib + Conan only.** No third-party deps beyond Conan. + Packaging is hatchling with `only-include = ["conandeps.py"]`; the PEP 723 + header makes the file usable via `uv run conandeps.py` without a checkout. +- **Exit codes:** `0` fine, `1` missing binaries, `2` Conan error. CI relies + on this. + +## Workflow + +- Lint/format: `./dev.sh ruff check .` and `./dev.sh ruff format .` + (config in `pyproject.toml`, target py38, line length 100). Run + `shellcheck dev.sh` after editing the wrapper. +- Tests: `./dev.sh python -m pytest`. The fixture spins up `conan_server` + on a free port with a pre-written `server.conf` (the port cannot be given + on the command line), creates fake packages whose `build()` does nothing, + uploads them, then wipes the cache so binaries can only come from the + remote. Fake recipes need `--build=missing` at create time because + dependencies' binaries are deliberately incomplete. +- Dependencies: `uv.lock` is committed; regenerate with `./dev.sh uv lock` + after changing `pyproject.toml`. +- Commits: atomic, no `--amend`; run ruff + pytest + shellcheck before + committing. Add a memory of anything a future session could not derive + from the repository. + +## Adding another tool + +Follow the `conandeps.py` pattern: one executable file with a module +docstring explaining *how* and *why*, a PEP 723 header, a `main(argv)` that +returns an exit code, a `[project.scripts]` entry, an `only-include` entry in +`pyproject.toml`, tests under `tests/` using the shared `conan_env` fixture, +and a section in `README.md`. diff --git a/LICENSE b/LICENSE new file mode 100644 index 0000000..18cfc0d --- /dev/null +++ b/LICENSE @@ -0,0 +1,21 @@ +MIT License + +Copyright (c) 2026 Ole-Morten Duesund + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. diff --git a/README.md b/README.md index 45a733a..e1f0a75 100644 --- a/README.md +++ b/README.md @@ -1,74 +1,164 @@ # conan-utils -Small utilities for **Conan 1.x** (targets 1.66). All development happens inside -a pinned Podman container so nothing from the host environment leaks in. +Small, single-file utilities for **Conan 1.x** (targeting 1.66) that answer +questions the stock `conan info` / `conan search` commands make hard to answer. +Development and testing happen inside a pinned Podman container so nothing from +the host environment leaks in. -## conandeps.py +Licensed under the [MIT License](LICENSE). -Given a `conanfile.py`, `conandeps` reports: +## Tools -* direct and indirect dependencies (host and build context), -* whether a binary exists on the remote for **Release, Debug and RelWithDebInfo** - (configurable) for *your exact profile*, -* every requirement **override** – explicit `override=True` and implicit ones - (a downstream consumer asking for a newer version than a transitive - dependency declared) – shown both as a list and on the edges of the - dependency tree. +| Tool | What it does | +|------|--------------| +| [`conandeps`](#conandeps) | Lists direct/indirect dependencies, checks which build types have binaries on a remote for your exact profile, and shows requirement overrides. | -### Installing with uv +## conandeps -``` -uv tool install git+https:///conan-utils # or: uv tool install . from a checkout +Given a `conanfile.py`, `conandeps` tells you three things in one report: + +1. **What you depend on** – direct and indirect requirements, host and build + context, as a table and as a tree. +2. **Which binaries are missing** – for every dependency, whether a package + exists on the remote for **Release, Debug and RelWithDebInfo** (or any list + you choose), evaluated for *your exact profile* (compiler, version, libcxx, + arch, options, …), not "some binary with that build type". +3. **Which requirements were overridden** – both explicit + `self.requires("x/2.0", override=True)` and implicit ones, where a + downstream consumer simply asks for a newer version than a transitive + dependency declared. Each override is listed with who wanted what and who + forced the change, and annotated on the exact edge of the dependency tree + where it happened. + +### Installation + +Requires Python 3.8–3.11 (Conan 1.x does not run on 3.12+). With +[uv](https://docs.astral.sh/uv/): + +```bash +# As a command on your PATH, in its own venv with Conan 1.66: +uv tool install git+https:///conan-utils # or `uv tool install .` from a checkout conandeps path/to/conanfile.py -r knor -``` -This puts a `conandeps` command on your PATH in its own virtualenv with -Conan 1.66 — it does not touch or depend on the Conan you use for builds -(but it does read the same `~/.conan` config, profiles and remotes). - -The script also carries PEP 723 inline metadata, so the single file works on -its own without a checkout: - -``` +# Or run the single file directly, no checkout needed (PEP 723 inline metadata): uv run conandeps.py path/to/conanfile.py -r knor ``` -Python 3.8–3.11 is required; Conan 1.x does not run on 3.12+. +The tool's venv carries its own Conan 1.66 and does not touch the Conan you +build with, but it reads the same `~/.conan` configuration: profiles, +`remotes.json` and stored remote credentials. Log in to your remote once with +`conan user -r -p` (or set `CONAN_LOGIN_USERNAME` / `CONAN_PASSWORD`) +if it requires authentication. ### Usage ``` -./conandeps.py path/to/conanfile.py -r knor # text report -./conandeps.py path/to/conanfile.py -r knor -pr myprofile # explicit profile -./conandeps.py path/to/conanfile.py -r knor --html report.html -./conandeps.py path/to/conanfile.py -r knor --build-types Release,Debug +conandeps CONANFILE [-r REMOTE] [-pr PROFILE] [-s KEY=VALUE] [-o PKG:KEY=VALUE] + [--build-types Release,Debug,RelWithDebInfo] [-u] [--html FILE] [-v] ``` -Options mirror `conan info`: `-pr/--profile`, `-s/--settings`, `-o/--options` -(all repeatable) and `-u/--update`. Exit status is `1` when any binary is -missing, so it can gate a CI job. +| Option | Meaning | +|--------|---------| +| `CONANFILE` | Path to `conanfile.py` (or a directory containing one). | +| `-r`, `--remote` | Remote to look for binaries in, e.g. `-r knor`. Default: all configured remotes. | +| `-pr`, `--profile` | Profile to evaluate with (repeatable, like `conan -pr`). Default: your default profile. | +| `-s`, `-o` | Extra settings / options, repeatable, same syntax as `conan install`. | +| `--build-types` | Comma-separated build types to check. Default: `Release,Debug,RelWithDebInfo`. | +| `-u`, `--update` | Ask the remote for newer recipes/binaries (`conan -u`). | +| `--html FILE` | Also write a self-contained HTML report (no scripts, no external assets). | +| `-v` | Echo Conan's own output to stderr. | + +Exit status: `0` all binaries present, `1` at least one is missing, `2` Conan +failed to build the graph (for example a version conflict). This makes it +usable as a CI gate. + +### Example + +``` +$ conandeps MyProject/conanfile.py -r knor +conandeps report for MyProject/conanfile.py +remote : knor +profile: arch=x86_64, compiler=gcc, compiler.libcxx=libstdc++11, compiler.version=14, os=Linux + +Dependencies (4) and binary availability + +package kind ctx Release Debug RelWithDebInfo +------------------------------------------------------------------------- +MyDepA/1.0 direct host ok ok ok +boost/1.8 indir. host ok ok ok +MyDepB/1.0 direct host ok ok ok +Zigma/1.0 indir. host ok ok MISSING + +MISSING BINARIES (1 packages) + Zigma/1.0: missing RelWithDebInfo + RelWithDebInfo package_id 3f9c...e21a + +Overrides (1) + Zigma/1.0 wanted boost/1.7 + -> forced to boost/1.8 by your conanfile [implicit (newer direct requirement)] + +Dependency hierarchy +MyProject/conanfile.py +|-- MyDepA/1.0 +| `-- boost/1.8 +`-- MyDepB/1.0 + `-- Zigma/1.0 [MISSING: RelWithDebInfo] + `-- boost/1.8 [overrides boost/1.7 (implicit by your conanfile)] +``` + +Reading the table: + +* `ok` – a binary for your profile exists (in the remote, or already in your + cache). +* `MISSING` – no binary for the package_id your profile produces. Hover the + cell in the HTML report to see the package_id. +* `-` – not applicable (Conan marked the node `Skip` or `Editable`). +* Header-only packages show `ok` everywhere: their package_id ignores + `build_type`. + +Things worth knowing: + +* Two sibling dependencies requiring different versions of the same package + with no decision from your conanfile is a **conflict** in Conan 1.x, not an + override. Conan refuses to build the graph, and `conandeps` prints Conan's + conflict message and exits with `2`. +* An override can change the package_id of the package whose requirement was + rewritten (with the default `semver_direct_mode`, a direct requirement's + version is part of the id). If that makes *all* build types of a package go + `MISSING`, the report is telling the truth: no binary on the remote was built + against the overridden version. ### How it works -Rather than matching `conan search` output by hand, the dependency graph is -built once per build type through Conan's own `info` code path and each node's -binary status (`Cache`/`Download`/`Missing`, …) is read back. That means +Rather than hand-matching `conan search` output against your profile, the +dependency graph is built once per build type through Conan's own `info` API – +the same code path `conan install` uses – and each node's binary status +(`Cache`, `Download`, `Update`, `Missing`, …) is read back. That is why `package_id()` customisations, options and `default_package_id_mode` are -honoured exactly as a real `conan install` would. +honoured exactly as a real install would. -Conan 1.x does not keep override information on the graph – the only trace is a -`WARN: … requirement A overridden by B to C` line. The tool captures Conan's -output while building the graph and parses those lines. +Conan 1.x does not keep override information on the graph object; the only +trace is a `WARN: : requirement A overridden by B to C` line written while +the graph is resolved. The tool captures Conan's output stream during graph +construction and parses those lines. A test pins the message format. ## Development -``` -./dev.sh build # build the conan-utils-dev image (Conan 1.66, Python 3.11) -./dev.sh python -m pytest # run the tests +Everything runs inside the `conan-utils-dev` container (Conan 1.66, Python +3.11, ruff, pytest, uv) via the `dev.sh` wrapper, which mounts the checkout at +`/work`: + +```bash +./dev.sh build # build the image +./dev.sh python -m pytest # run the tests ./dev.sh ruff check . && ./dev.sh ruff format . -./dev.sh # interactive shell +./dev.sh # interactive shell ``` The tests start a throw-away `conan_server` inside the container, upload a -small graph with deliberately missing binaries and overrides, and run the tool -against it with an isolated `CONAN_USER_HOME`. +small dependency graph with deliberately missing binaries and both kinds of +override, wipe the local cache, and run the tool against the server using an +isolated `CONAN_USER_HOME`. Your own `~/.conan` is never touched. + +See [CLAUDE.md](CLAUDE.md) for the conventions that apply when working on this +repository. diff --git a/pyproject.toml b/pyproject.toml index 44fa7ef..e239e76 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -2,6 +2,10 @@ name = "conan-utils" version = "0.1.0" description = "Small utilities for Conan 1.x (targets 1.66)" +readme = "README.md" +license = "MIT" +license-files = ["LICENSE"] +authors = [{ name = "Ole-Morten Duesund" }] # Conan 1.x imports the `imp` module, which was removed in Python 3.12. requires-python = ">=3.8,<3.12" dependencies = ["conan>=1.66,<2"]