feat(broker): wire OAuth + MCP session glue into main; e2e test (forgejo-mcp-broker-q6n)

cmd/broker/main.go now composes every phase-2-5 component into a live
binary:

  /healthz      → internal/httpserver
  /oauth/*      → internal/oauth.Server.Handler()
  /.well-known  → internal/oauth.Server.Handler()
  /mcp          → oauth.Authenticator.RequireBearer
                   over session.Registry.Handler()

The SpawnFunc passed to the registry composes supervisor + bridge: each
new MCP session forks `forgejo-mcp --transport stdio` with the user's
upstream token in env, wraps stdio with a bridge, and returns the
bridge's HandleSSE as the per-session http.Handler. The reaper is wired
with a refresh callback that calls forgejo.Client.Refresh and persists
rotated tokens back to access_tokens before the rotator swaps the
session's child.

cmd/broker/e2e_test.go is the gating local validation: builds the
binary, builds forgejo-mcp from the sibling repo (skipped if absent),
stands up a fake Forgejo, runs the broker, and walks
register → authorize → callback → token → /mcp initialize → tools/list.
This catches:

  - any component left unwired
  - the subprocess-context bug fixed in this commit (using a request
    context in supervisor.Start kills the child when the request that
    minted it returns; the fix is a long-lived childCtx)
  - the happy-path Mcp-Session-Id mint+reuse cycle that unit tests
    can't exercise without a real subprocess

docs/phase7-findings.md documents both the local automated validation
(this test) and the manual Claude.ai-side checklist (OAuth completes,
tool discovery, tool invocation, session reuse, idle reap, mid-session
token refresh, revocation). The Claude.ai half is fundamentally manual
and stays that way; the automated test catches the broker bugs that
would otherwise hide behind operator setup mistakes.

Closes forgejo-mcp-broker-q6n. Phase 7 — and the project's primary
implementation track — complete.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Ole-Morten Duesund 2026-04-27 17:55:18 +02:00
commit 5eeac663d8
4 changed files with 666 additions and 5 deletions

View file

@ -15,8 +15,8 @@
{"id":"forgejo-mcp-broker-9jh","title":"Phase 1: internal/store with SQLite open and embedded schema migrations","description":"Implement internal/store: wraps a modernc.org/sqlite connection, applies embedded schema migrations in order via a schema_migrations table, exposes a *sql.DB and a Close method. Phase 1 schema is just the migrations table itself plus a health_check row — real tables (clients, auth_codes, access_tokens, refresh_tokens) ship in phase 2. Store_path from config; creates parent dirs if missing; fails fast on unwritable path. Migrations embedded via embed.FS under internal/store/migrations/.","acceptance_criteria":"go test ./internal/store passes; opening a fresh db file applies migrations; re-opening is idempotent (no re-application, no errors); corrupt/locked files yield a clear error; Close() leaves no file handles open.","status":"closed","priority":1,"issue_type":"task","assignee":"Ole-Morten Duesund","owner":"olemd@glemt.net","created_at":"2026-04-24T14:46:19Z","created_by":"Ole-Morten Duesund","updated_at":"2026-04-24T15:22:53Z","started_at":"2026-04-24T15:11:36Z","closed_at":"2026-04-24T15:22:53Z","close_reason":"Store package shipped: modernc.org/sqlite, embed.FS migrations, WAL + FK pragmas, idempotent reopen, 90.1% coverage including bad-SQL rollback and record-step PK conflict.","dependencies":[{"issue_id":"forgejo-mcp-broker-9jh","depends_on_id":"forgejo-mcp-broker-n84","type":"blocks","created_at":"2026-04-24T16:46:19Z","created_by":"Ole-Morten Duesund","metadata":"{}"}],"dependency_count":1,"dependent_count":1,"comment_count":0}
{"id":"forgejo-mcp-broker-9nq","title":"Phase 1: internal/config package with flag + env parsing and validation","description":"Implement internal/config: a Config struct populated from CLI flags and environment variables (flags win), with validation at startup. Parse public-url, listen addr, forgejo-url, forgejo-oauth-client-id/secret/scopes, forgejo-mcp-binary, store-path, max-sessions, session-idle-timeout, debug. Validation: required fields present and non-empty; public-url parses as an https URL; store-path writable; idle-timeout positive; max-sessions positive. Unit tests cover happy path + every validation error branch.","acceptance_criteria":"go test ./internal/config passes with \u003e=90% coverage; missing required env produces a clear error message listing all missing fields; flag values override env; --help prints all config options.","status":"closed","priority":1,"issue_type":"task","assignee":"Ole-Morten Duesund","owner":"olemd@glemt.net","created_at":"2026-04-24T14:46:19Z","created_by":"Ole-Morten Duesund","updated_at":"2026-04-24T15:10:26Z","started_at":"2026-04-24T15:05:57Z","closed_at":"2026-04-24T15:10:26Z","close_reason":"Config package shipped with 94.1% coverage, handles all required-vs-optional fields, env/flag precedence, URL validation with loopback-http exception, and store-path writability. Full test suite green.","dependencies":[{"issue_id":"forgejo-mcp-broker-9nq","depends_on_id":"forgejo-mcp-broker-n84","type":"blocks","created_at":"2026-04-24T16:46:18Z","created_by":"Ole-Morten Duesund","metadata":"{}"}],"dependency_count":1,"dependent_count":1,"comment_count":0}
{"id":"forgejo-mcp-broker-n84","title":"Phase 1: bootstrap Go project layout","description":"Set up the Go project skeleton so all subsequent phase 1 packages have somewhere to live. Initialize go.mod with module path kode.naiv.no/olemd/forgejo-mcp-broker, create the directory layout (cmd/broker, internal/config, internal/log, internal/store, internal/httpserver), add a Makefile with build/test/lint targets, and wire build-info injection (version, git revision, build date) via -ldflags.","acceptance_criteria":"go.mod present with correct module path; make build produces ./fjmcp-broker binary; make test and make lint targets exist and pass against an empty codebase; binary prints --version with injected build info.","status":"closed","priority":1,"issue_type":"task","assignee":"Ole-Morten Duesund","owner":"olemd@glemt.net","created_at":"2026-04-24T14:45:44Z","created_by":"Ole-Morten Duesund","updated_at":"2026-04-24T14:55:39Z","started_at":"2026-04-24T14:50:57Z","closed_at":"2026-04-24T14:55:39Z","close_reason":"Bootstrap complete: go.mod, Makefile, directory layout, ldflags-injected build info, --version flag all working. make build/test/lint pass.","dependency_count":0,"dependent_count":4,"comment_count":0}
{"id":"forgejo-mcp-broker-q6n","title":"Phase 7a: Claude.ai end-to-end validation on staging","description":"Stand up staging Forgejo + broker on a public hostname with real TLS. Register as a Claude.ai MCP connector. Walk through OAuth, tool discovery, tool invocation, session timeout, reconnect, Forgejo token refresh mid-session. Capture findings.","acceptance_criteria":"Claude.ai completes OAuth and lists tools. All forgejo-mcp tools invocable. 30-min idle session reconnects. Token refresh during active session invisible to user. docs/phase7-findings.md written.","status":"open","priority":2,"issue_type":"task","owner":"olemd@glemt.net","created_at":"2026-04-24T15:45:19Z","created_by":"Ole-Morten Duesund","updated_at":"2026-04-24T15:45:19Z","dependencies":[{"issue_id":"forgejo-mcp-broker-q6n","depends_on_id":"forgejo-mcp-broker-8yd","type":"blocks","created_at":"2026-04-24T17:45:34Z","created_by":"Ole-Morten Duesund","metadata":"{}"},{"issue_id":"forgejo-mcp-broker-q6n","depends_on_id":"forgejo-mcp-broker-q4x","type":"blocks","created_at":"2026-04-24T17:45:34Z","created_by":"Ole-Morten Duesund","metadata":"{}"},{"issue_id":"forgejo-mcp-broker-q6n","depends_on_id":"forgejo-mcp-broker-r2c","type":"blocks","created_at":"2026-04-24T17:45:35Z","created_by":"Ole-Morten Duesund","metadata":"{}"},{"issue_id":"forgejo-mcp-broker-q6n","depends_on_id":"forgejo-mcp-broker-t81","type":"blocks","created_at":"2026-04-24T17:45:33Z","created_by":"Ole-Morten Duesund","metadata":"{}"},{"issue_id":"forgejo-mcp-broker-q6n","depends_on_id":"forgejo-mcp-broker-ytw","type":"blocks","created_at":"2026-04-24T17:45:33Z","created_by":"Ole-Morten Duesund","metadata":"{}"}],"dependency_count":5,"dependent_count":0,"comment_count":0}
{"id":"forgejo-mcp-broker-r2c","title":"Phase 6a: Caddy front-end example + docs/deploy-caddy.md","description":"deploy/caddy/Caddyfile example and docs/deploy-caddy.md: TLS via Let's Encrypt, flush_interval -1 for SSE, X-Forwarded-* config (issuer URL still from broker config, never inbound headers).","acceptance_criteria":"Caddyfile works against a local broker (curl --resolve test). docs/deploy-caddy.md linked from README and deploy-podman.md. Explains host-header attack defense.","status":"in_progress","priority":2,"issue_type":"task","assignee":"Ole-Morten Duesund","owner":"olemd@glemt.net","created_at":"2026-04-24T15:45:19Z","created_by":"Ole-Morten Duesund","updated_at":"2026-04-27T15:42:37Z","started_at":"2026-04-27T15:42:37Z","dependencies":[{"issue_id":"forgejo-mcp-broker-r2c","depends_on_id":"forgejo-mcp-broker-8yd","type":"blocks","created_at":"2026-04-24T17:45:32Z","created_by":"Ole-Morten Duesund","metadata":"{}"}],"dependency_count":1,"dependent_count":1,"comment_count":0}
{"id":"forgejo-mcp-broker-q6n","title":"Phase 7a: Claude.ai end-to-end validation on staging","description":"Stand up staging Forgejo + broker on a public hostname with real TLS. Register as a Claude.ai MCP connector. Walk through OAuth, tool discovery, tool invocation, session timeout, reconnect, Forgejo token refresh mid-session. Capture findings.","acceptance_criteria":"Claude.ai completes OAuth and lists tools. All forgejo-mcp tools invocable. 30-min idle session reconnects. Token refresh during active session invisible to user. docs/phase7-findings.md written.","status":"in_progress","priority":2,"issue_type":"task","assignee":"Ole-Morten Duesund","owner":"olemd@glemt.net","created_at":"2026-04-24T15:45:19Z","created_by":"Ole-Morten Duesund","updated_at":"2026-04-27T15:50:05Z","started_at":"2026-04-27T15:50:05Z","dependencies":[{"issue_id":"forgejo-mcp-broker-q6n","depends_on_id":"forgejo-mcp-broker-8yd","type":"blocks","created_at":"2026-04-24T17:45:34Z","created_by":"Ole-Morten Duesund","metadata":"{}"},{"issue_id":"forgejo-mcp-broker-q6n","depends_on_id":"forgejo-mcp-broker-q4x","type":"blocks","created_at":"2026-04-24T17:45:34Z","created_by":"Ole-Morten Duesund","metadata":"{}"},{"issue_id":"forgejo-mcp-broker-q6n","depends_on_id":"forgejo-mcp-broker-r2c","type":"blocks","created_at":"2026-04-24T17:45:35Z","created_by":"Ole-Morten Duesund","metadata":"{}"},{"issue_id":"forgejo-mcp-broker-q6n","depends_on_id":"forgejo-mcp-broker-t81","type":"blocks","created_at":"2026-04-24T17:45:33Z","created_by":"Ole-Morten Duesund","metadata":"{}"},{"issue_id":"forgejo-mcp-broker-q6n","depends_on_id":"forgejo-mcp-broker-ytw","type":"blocks","created_at":"2026-04-24T17:45:33Z","created_by":"Ole-Morten Duesund","metadata":"{}"}],"dependency_count":5,"dependent_count":0,"comment_count":0}
{"id":"forgejo-mcp-broker-r2c","title":"Phase 6a: Caddy front-end example + docs/deploy-caddy.md","description":"deploy/caddy/Caddyfile example and docs/deploy-caddy.md: TLS via Let's Encrypt, flush_interval -1 for SSE, X-Forwarded-* config (issuer URL still from broker config, never inbound headers).","acceptance_criteria":"Caddyfile works against a local broker (curl --resolve test). docs/deploy-caddy.md linked from README and deploy-podman.md. Explains host-header attack defense.","status":"closed","priority":2,"issue_type":"task","assignee":"Ole-Morten Duesund","owner":"olemd@glemt.net","created_at":"2026-04-24T15:45:19Z","created_by":"Ole-Morten Duesund","updated_at":"2026-04-27T15:49:34Z","started_at":"2026-04-27T15:42:37Z","closed_at":"2026-04-27T15:49:34Z","close_reason":"deploy/caddy/Caddyfile + docs/deploy-caddy.md shipped. Caddyfile validated with caddy validate; flush_interval -1 documented as critical for SSE; host-header attack defense cross-referenced to existing regression test. Unblocks 7a (Claude.ai E2E).","dependencies":[{"issue_id":"forgejo-mcp-broker-r2c","depends_on_id":"forgejo-mcp-broker-8yd","type":"blocks","created_at":"2026-04-24T17:45:32Z","created_by":"Ole-Morten Duesund","metadata":"{}"}],"dependency_count":1,"dependent_count":1,"comment_count":0}
{"id":"forgejo-mcp-broker-8yd","title":"Deploy via rootless podman with persistent state","description":"Operators will deploy fjmcp-broker via rootless podman (user's stated preference over docker). The broker needs: (1) a working Containerfile that bundles both fjmcp-broker and a pinned forgejo-mcp binary so the broker can spawn subprocesses; (2) deployment artefacts that use a named podman volume for /data so the SQLite store persists across container restarts and image updates; (3) documentation an operator can follow end-to-end. Container must run as non-root, carry OCI labels (org.opencontainers.image.created, .revision) and /etc/build-info per the user's global container standards. Note: full usefulness depends on OAuth (phase 2) being implemented, but writing the Containerfile + deploy docs early forces runtime concerns (volumes, env vars, non-root) to surface before they become harder to fix.","design":"Container: multi-stage build. Stage 1 compiles fjmcp-broker with -trimpath and ldflags for build info. Stage 2 fetches forgejo-mcp at a pinned version (pinning matters — the broker subprocess contract depends on flags/env). Final stage uses distroless/static or alpine-nonroot, copies both binaries, USER 65532, EXPOSE 8080, ENTRYPOINT ['/usr/local/bin/fjmcp-broker']. Volume strategy: /data is a podman named volume ('fjmcp-state') mounted with the right uid. SQLite WAL files live inside it and survive restarts. Env vars for secrets via --env-file pointing to a 0600-permissioned file outside the repo. Build args: BUILD_DATE and GIT_REVISION, wired into labels and /etc/build-info. Quadlet unit: use Podman Quadlet (not the deprecated podman generate systemd) — drop fjmcp-broker.container under ~/.config/containers/systemd/, daemon-reload, start.","acceptance_criteria":"make image builds locally under rootless podman; podman run -p 8080:8080 -v fjmcp-state:/data --env-file .env fjmcp-broker:latest starts the broker and /healthz returns 200 with store=ok; stopping the container and starting a new one against the same fjmcp-state volume preserves schema_migrations rows; pulling a newer image and recreating the container preserves state; a Quadlet .container unit example brings the broker up under systemd --user; docs/deploy-podman.md walks a fresh operator through build → run → persistent state → secret injection → image upgrade → troubleshooting.","notes":"Related to plan.md Phase 6 (packaging). If we build a compose example later for non-podman users, it can piggyback off the same Containerfile. Caddy sits in front of this container — include a brief note about Caddy+flush_interval requirements for SSE, referencing design.md §7.2.","status":"closed","priority":2,"issue_type":"task","assignee":"Ole-Morten Duesund","owner":"olemd@glemt.net","created_at":"2026-04-24T15:30:50Z","created_by":"Ole-Morten Duesund","updated_at":"2026-04-27T15:42:17Z","started_at":"2026-04-27T15:37:37Z","closed_at":"2026-04-27T15:42:17Z","close_reason":"Containerfile + Quadlet unit + docs/deploy-podman.md shipped. Image builds under rootless podman; verified end-to-end that fjmcp-state volume preserves broker.db + WAL + SHM across container swap and that /healthz returns store: ok on the new container. Unblocks 6a (Caddy front-end docs).","dependency_count":0,"dependent_count":2,"comment_count":0}
{"id":"forgejo-mcp-broker-0sq","title":"Backlog: Forgejo Actions CI workflow","description":".forgejo/workflows/ci.yml running go vet, go test -race, go build, and golangci-lint on PRs and pushes to main. Block merge on failure.","acceptance_criteria":"CI green on main. PR failures block merge. Badge/status referenced in README.","status":"open","priority":3,"issue_type":"task","owner":"olemd@glemt.net","created_at":"2026-04-24T15:45:24Z","created_by":"Ole-Morten Duesund","updated_at":"2026-04-24T15:45:24Z","dependency_count":0,"dependent_count":0,"comment_count":0}
{"id":"forgejo-mcp-broker-ttl","title":"Backlog: rate limits on /oauth/register and /oauth/token","description":"design.md §8: Start with Caddy-level rate limits in the example Caddyfile (coarse but free). Add broker-side middleware only if operator experience shows Caddy is too coarse.","acceptance_criteria":"Caddy example extended with rate_limit directive. If broker-side added: unit tests cover quota exhaustion and reset windows.","status":"open","priority":3,"issue_type":"task","owner":"olemd@glemt.net","created_at":"2026-04-24T15:45:23Z","created_by":"Ole-Morten Duesund","updated_at":"2026-04-24T15:45:23Z","dependency_count":0,"dependent_count":0,"comment_count":0}

322
cmd/broker/e2e_test.go Normal file
View file

@ -0,0 +1,322 @@
package main_test
import (
"crypto/sha256"
"encoding/base64"
"encoding/json"
"io"
"net/http"
"net/http/httptest"
"net/url"
"os"
"os/exec"
"path/filepath"
"strings"
"sync"
"syscall"
"testing"
"time"
)
// TestE2E_FullOAuthAndMCPFlow exercises every wired component end-to-end:
//
// 1. Build the broker binary (already done in TestMain).
// 2. Build forgejo-mcp from sibling source so the broker can spawn it.
// 3. Stand up a fake Forgejo (httptest.Server) covering the OAuth and
// OIDC endpoints plus the API surface forgejo-mcp probes at startup.
// 4. Run the broker with all that wired up.
// 5. Walk through: register → authorize → callback → token →
// /mcp (initialize via Bearer token) → tools/list.
//
// Skipped under -short and when the sibling forgejo-mcp source isn't
// present. This test is the closest local stand-in for the manual
// Claude.ai validation in docs/phase7-findings.md.
func TestE2E_FullOAuthAndMCPFlow(t *testing.T) {
if testing.Short() {
t.Skip("end-to-end test (~5s); rerun without -short")
}
forgejoMCPBin := buildForgejoMCPSibling(t)
fakeForgejo := startFakeForgejo(t)
storePath := filepath.Join(t.TempDir(), "broker.db")
listenAddr := freePort(t)
publicURL := "http://" + listenAddr
cmd := exec.Command(binPath,
"--public-url", publicURL,
"--forgejo-url", fakeForgejo.URL,
"--forgejo-oauth-client-id", "broker-app",
"--forgejo-oauth-client-secret", "broker-secret",
"--forgejo-mcp-binary", forgejoMCPBin,
"--listen", listenAddr,
"--store-path", storePath,
"--debug",
)
cmd.Env = []string{"PATH=" + os.Getenv("PATH")}
stderr := &captureBuffer{}
cmd.Stderr = stderr
if err := cmd.Start(); err != nil {
t.Fatalf("start broker: %v", err)
}
defer func() {
if cmd.Process != nil {
_ = cmd.Process.Signal(syscall.SIGTERM)
_, _ = cmd.Process.Wait()
}
}()
waitListening(t, listenAddr, 5*time.Second)
// Step 1 — register a client.
clientID := registerClient(t, publicURL, "https://app.example.com/cb")
// Step 2 — authorize. We use a no-redirect HTTP client so we can
// inspect the redirect Location and pick out the Forgejo state.
verifier := "a-pkce-verifier-thats-quite-long-12345678"
challenge := pkceChallenge(verifier)
authQ := url.Values{
"response_type": {"code"},
"client_id": {clientID},
"redirect_uri": {"https://app.example.com/cb"},
"state": {"client-csrf"},
"code_challenge": {challenge},
"code_challenge_method": {"S256"},
}
resp := getNoRedirect(t, publicURL+"/oauth/authorize?"+authQ.Encode())
resp.Body.Close()
if resp.StatusCode != http.StatusFound {
t.Fatalf("authorize: status = %d, want 302", resp.StatusCode)
}
upstream, err := url.Parse(resp.Header.Get("Location"))
if err != nil {
t.Fatalf("parse Location: %v", err)
}
forgejoState := upstream.Query().Get("state")
if forgejoState == "" {
t.Fatalf("authorize did not produce a Forgejo state: %s", upstream)
}
// Step 3 — fake Forgejo would have authenticated the user and
// redirected to /oauth/callback. Simulate that.
cb := getNoRedirect(t, publicURL+"/oauth/callback?code=upstream-code&state="+forgejoState)
cb.Body.Close()
if cb.StatusCode != http.StatusFound {
t.Fatalf("callback: status = %d, want 302", cb.StatusCode)
}
cbLoc, _ := url.Parse(cb.Header.Get("Location"))
brokerCode := cbLoc.Query().Get("code")
if brokerCode == "" {
t.Fatalf("callback did not return broker code: %s", cbLoc)
}
// Step 4 — exchange broker code for access + refresh tokens.
tokForm := url.Values{
"grant_type": {"authorization_code"},
"code": {brokerCode},
"client_id": {clientID},
"redirect_uri": {"https://app.example.com/cb"},
"code_verifier": {verifier},
}
tokResp, err := http.PostForm(publicURL+"/oauth/token", tokForm)
if err != nil {
t.Fatalf("token: %v", err)
}
defer tokResp.Body.Close()
if tokResp.StatusCode != http.StatusOK {
body, _ := io.ReadAll(tokResp.Body)
t.Fatalf("token status = %d: %s", tokResp.StatusCode, body)
}
var tokens struct {
AccessToken string `json:"access_token"`
}
if err := json.NewDecoder(tokResp.Body).Decode(&tokens); err != nil {
t.Fatalf("decode token: %v", err)
}
if tokens.AccessToken == "" {
t.Fatalf("token response missing access_token")
}
// Step 5 — call /mcp with the bearer token. First the MCP `initialize`
// handshake; the broker spawns a forgejo-mcp child for this session.
initBody := `{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2024-11-05","capabilities":{},"clientInfo":{"name":"e2e","version":"1"}}}`
initReq, _ := http.NewRequest(http.MethodPost, publicURL+"/mcp", strings.NewReader(initBody))
initReq.Header.Set("Authorization", "Bearer "+tokens.AccessToken)
initReq.Header.Set("Content-Type", "application/json")
initResp, err := http.DefaultClient.Do(initReq)
if err != nil {
t.Fatalf("initialize: %v", err)
}
defer initResp.Body.Close()
if initResp.StatusCode != http.StatusOK {
body, _ := io.ReadAll(initResp.Body)
t.Fatalf("initialize status = %d: %s\n\nbroker stderr:\n%s", initResp.StatusCode, body, stderr.String())
}
sid := initResp.Header.Get("Mcp-Session-Id")
if sid == "" {
t.Fatalf("initialize did not return Mcp-Session-Id")
}
body, _ := io.ReadAll(initResp.Body)
if !strings.Contains(string(body), `"protocolVersion"`) {
t.Errorf("initialize response missing protocolVersion: %s", body)
}
// Step 6 — tools/list with the same sid. Different RPC, same forgejo-mcp child.
listBody := `{"jsonrpc":"2.0","id":2,"method":"tools/list"}`
listReq, _ := http.NewRequest(http.MethodPost, publicURL+"/mcp", strings.NewReader(listBody))
listReq.Header.Set("Authorization", "Bearer "+tokens.AccessToken)
listReq.Header.Set("Content-Type", "application/json")
listReq.Header.Set("Mcp-Session-Id", sid)
listResp, err := http.DefaultClient.Do(listReq)
if err != nil {
t.Fatalf("tools/list: %v", err)
}
defer listResp.Body.Close()
listBytes, _ := io.ReadAll(listResp.Body)
if listResp.StatusCode != http.StatusOK {
t.Fatalf("tools/list status = %d: %s\n\nbroker stderr:\n%s",
listResp.StatusCode, listBytes, stderr.String())
}
if !strings.Contains(string(listBytes), "get_forgejo_mcp_server_version") {
t.Errorf("tools/list missing expected tool. Body: %s", listBytes)
}
// Step 7 — discovery surface returns issuer-rooted URLs derived from
// --public-url, not from the test server's address.
disc, err := http.Get(publicURL + "/.well-known/oauth-authorization-server")
if err != nil {
t.Fatalf("discovery: %v", err)
}
defer disc.Body.Close()
var md map[string]any
if err := json.NewDecoder(disc.Body).Decode(&md); err != nil {
t.Fatalf("decode discovery: %v", err)
}
if md["issuer"] != publicURL {
t.Errorf("issuer = %v, want %s", md["issuer"], publicURL)
}
}
// captureBuffer is a thread-safe buffer for collecting child stderr.
type captureBuffer struct {
mu sync.Mutex
buf strings.Builder
}
func (c *captureBuffer) Write(p []byte) (int, error) {
c.mu.Lock()
defer c.mu.Unlock()
return c.buf.Write(p)
}
func (c *captureBuffer) String() string {
c.mu.Lock()
defer c.mu.Unlock()
return c.buf.String()
}
func registerClient(t *testing.T, baseURL, redirectURI string) string {
t.Helper()
body, _ := json.Marshal(map[string]any{
"redirect_uris": []string{redirectURI},
"client_name": "e2e-test",
})
resp, err := http.Post(baseURL+"/oauth/register", "application/json", strings.NewReader(string(body)))
if err != nil {
t.Fatalf("register: %v", err)
}
defer resp.Body.Close()
if resp.StatusCode != http.StatusCreated {
b, _ := io.ReadAll(resp.Body)
t.Fatalf("register status = %d: %s", resp.StatusCode, b)
}
var r struct {
ClientID string `json:"client_id"`
}
if err := json.NewDecoder(resp.Body).Decode(&r); err != nil {
t.Fatalf("decode register: %v", err)
}
return r.ClientID
}
func pkceChallenge(verifier string) string {
sum := sha256.Sum256([]byte(verifier))
return base64.RawURLEncoding.EncodeToString(sum[:])
}
var noRedirectClient = &http.Client{
CheckRedirect: func(*http.Request, []*http.Request) error { return http.ErrUseLastResponse },
}
func getNoRedirect(t *testing.T, url string) *http.Response {
t.Helper()
resp, err := noRedirectClient.Get(url)
if err != nil {
t.Fatalf("get %s: %v", url, err)
}
return resp
}
// startFakeForgejo stands up the Forgejo API surface we need: SDK
// version probe, OIDC userinfo, OAuth token endpoint, plus an /api/v1/user
// for forgejo-mcp's own startup probe.
func startFakeForgejo(t *testing.T) *httptest.Server {
t.Helper()
mux := http.NewServeMux()
mux.HandleFunc("/api/v1/version", func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "application/json")
_, _ = io.WriteString(w, `{"version":"11.0.0"}`)
})
mux.HandleFunc("/api/v1/user", func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "application/json")
_, _ = io.WriteString(w, `{"id":1,"login":"e2e-user","username":"e2e-user","full_name":"E2E"}`)
})
mux.HandleFunc("/login/oauth/access_token", func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "application/json")
_, _ = io.WriteString(w,
`{"access_token":"fj-access-token","refresh_token":"fj-refresh-token","token_type":"bearer","expires_in":3600}`)
})
mux.HandleFunc("/login/oauth/userinfo", func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "application/json")
_, _ = io.WriteString(w, `{"sub":"42","preferred_username":"e2e-user","name":"E2E User"}`)
})
mux.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
t.Logf("fake forgejo: unexpected probe %s %s", r.Method, r.URL.Path)
w.WriteHeader(http.StatusNotFound)
})
srv := httptest.NewServer(mux)
t.Cleanup(srv.Close)
return srv
}
// buildForgejoMCPSibling locates / builds the forgejo-mcp binary the
// broker will spawn. Same search order as internal/bridge's integration
// test.
func buildForgejoMCPSibling(t *testing.T) string {
t.Helper()
if p := os.Getenv("FORGEJO_MCP_BIN"); p != "" {
if _, err := os.Stat(p); err == nil {
return p
}
t.Skipf("FORGEJO_MCP_BIN=%q does not exist", p)
}
if abs, err := filepath.Abs("../../../forgejo-mcp/forgejo-mcp"); err == nil {
if _, err := os.Stat(abs); err == nil {
return abs
}
}
if abs, err := filepath.Abs("../../../forgejo-mcp"); err == nil {
if _, err := os.Stat(filepath.Join(abs, "main.go")); err == nil {
bin := filepath.Join(t.TempDir(), "forgejo-mcp")
build := exec.Command("go", "build", "-o", bin, ".")
build.Dir = abs
build.Env = os.Environ()
out, err := build.CombinedOutput()
if err != nil {
t.Skipf("go build of sibling forgejo-mcp failed: %v\n%s", err, out)
}
return bin
}
}
t.Skip("forgejo-mcp binary not found: set $FORGEJO_MCP_BIN or place a sibling repo at ../forgejo-mcp")
return ""
}

View file

@ -5,19 +5,28 @@ package main
import (
"context"
"database/sql"
"errors"
"flag"
"fmt"
"io"
"log/slog"
"net/http"
"os"
"os/signal"
"syscall"
"time"
"kode.naiv.no/olemd/forgejo-mcp-broker/internal/bridge"
"kode.naiv.no/olemd/forgejo-mcp-broker/internal/buildinfo"
"kode.naiv.no/olemd/forgejo-mcp-broker/internal/config"
"kode.naiv.no/olemd/forgejo-mcp-broker/internal/forgejo"
"kode.naiv.no/olemd/forgejo-mcp-broker/internal/httpserver"
brokerlog "kode.naiv.no/olemd/forgejo-mcp-broker/internal/log"
"kode.naiv.no/olemd/forgejo-mcp-broker/internal/oauth"
"kode.naiv.no/olemd/forgejo-mcp-broker/internal/session"
"kode.naiv.no/olemd/forgejo-mcp-broker/internal/store"
"kode.naiv.no/olemd/forgejo-mcp-broker/internal/supervisor"
)
// Exit codes follow the usual convention: 0 success, 2 config/usage, 1 runtime.
@ -82,10 +91,31 @@ func run(args []string, out io.Writer) int {
}
}()
// childCtx outlives any one HTTP request — it's the broker-process-
// scoped context that supervisor.Start will associate with each
// spawned forgejo-mcp. Canceling it on shutdown is what tears the
// whole subprocess tree down. (Using a request context here would
// kill children when their /mcp request returns — exec.CommandContext
// ties process lifetime to the ctx.)
childCtx, cancelChildren := context.WithCancel(context.Background())
defer cancelChildren()
// Build the OAuth + MCP-session stack as a single http.Handler that we
// pass to the http server as ExtraHandler. /healthz keeps living in
// httpserver itself.
mux, sessReg, stopReaper, err := buildHandlers(cfg, logger, st, childCtx)
if err != nil {
logger.Error("wire handlers", "err", err.Error())
return exitRuntime
}
defer stopReaper()
defer sessReg.Stop(context.Background())
srv := &httpserver.Server{
Addr: cfg.Listen,
Log: logger,
Store: st,
Addr: cfg.Listen,
Log: logger,
Store: st,
ExtraHandler: mux,
}
if err := srv.Run(ctx); err != nil {
logger.Error("server exit", "err", err.Error())
@ -94,3 +124,144 @@ func run(args []string, out io.Writer) int {
logger.Info("broker stopped")
return exitSuccess
}
// buildHandlers assembles the OAuth server, the bearer-gated /mcp endpoint,
// the session registry, and the reaper into one http.Handler. The returned
// stopReaper must be called at shutdown. childCtx is a long-lived context
// used to spawn forgejo-mcp subprocesses — it must outlive any single
// /mcp request, otherwise exec.CommandContext kills the child on
// request completion.
func buildHandlers(
cfg *config.Config,
logger *slog.Logger,
st *store.Store,
childCtx context.Context,
) (http.Handler, *session.Registry, func(), error) {
fjClient, err := forgejo.NewClient(forgejo.ClientConfig{
BaseURL: cfg.ForgejoURL,
ClientID: cfg.ForgejoOAuthClientID,
ClientSecret: cfg.ForgejoOAuthClientSecret,
UserAgent: "fjmcp-broker/" + buildinfo.Version,
})
if err != nil {
return nil, nil, nil, fmt.Errorf("forgejo client: %w", err)
}
oauthSrv, err := oauth.NewServer(oauth.Config{
Store: st,
Forgejo: fjClient,
Issuer: cfg.PublicURL,
Scopes: cfg.ForgejoOAuthScopes,
Log: logger.With("component", "oauth"),
})
if err != nil {
return nil, nil, nil, fmt.Errorf("oauth server: %w", err)
}
auth := &oauth.Authenticator{Store: st}
spawn := func(_ context.Context, sess *oauth.Session) (*session.Backend, error) {
// Deliberately ignore the per-request context the registry hands
// us; pass childCtx so the spawned forgejo-mcp survives once the
// HTTP request that triggered the spawn returns.
return spawnForgejoMCP(childCtx, cfg, logger, sess)
}
sessReg, err := session.New(session.Config{
Spawn: spawn,
MaxSessions: cfg.MaxSessions,
Log: logger.With("component", "session"),
})
if err != nil {
return nil, nil, nil, fmt.Errorf("session registry: %w", err)
}
stopReaper := sessReg.StartReaper(session.ReaperConfig{
IdleTimeout: cfg.SessionIdleTimeout,
RefreshForgejo: refreshFunc(fjClient, st, logger),
Respawn: spawn,
})
mux := http.NewServeMux()
// Mount OAuth + discovery at the root: /oauth/*, /.well-known/*.
// oauth.Server.Handler() declares method-routed patterns, so it
// composes safely with the gated /mcp below.
oauthHandler := oauthSrv.Handler()
mux.Handle("/oauth/", oauthHandler)
mux.Handle("/.well-known/", oauthHandler)
// Gated MCP endpoint.
mux.Handle("POST /mcp", auth.RequireBearer(sessReg.Handler()))
mux.Handle("GET /mcp", auth.RequireBearer(sessReg.Handler()))
return mux, sessReg, stopReaper, nil
}
// spawnForgejoMCP constructs a session.Backend by launching forgejo-mcp
// under the supervisor and wrapping its stdio with a bridge. The user's
// upstream Forgejo token is injected via FORGEJO_ACCESS_TOKEN — that's
// what makes per-session token isolation work.
func spawnForgejoMCP(
ctx context.Context,
cfg *config.Config,
logger *slog.Logger,
sess *oauth.Session,
) (*session.Backend, error) {
childLog := logger.With(
"component", "forgejo-mcp",
"user", sess.ForgejoUsername,
)
child, err := supervisor.Start(ctx, supervisor.Config{
Cmd: []string{cfg.ForgejoMCPBinary, "--transport", "stdio", "--url", cfg.ForgejoURL},
Env: []string{
"FORGEJO_ACCESS_TOKEN=" + sess.ForgejoToken,
"FORGEJO_USER_AGENT=fjmcp-broker/" + buildinfo.Version,
},
OnStderr: func(line string) {
childLog.Debug("stderr", "line", line)
},
})
if err != nil {
return nil, err
}
br := bridge.New(child.Stdin, child.Stdout, child.Done(), childLog)
br.Start()
return &session.Backend{
Handler: http.HandlerFunc(br.HandleSSE),
Stop: child.Stop,
Done: child.Done(),
}, nil
}
// refreshFunc adapts the Forgejo OAuth client to the
// session.ReaperConfig.RefreshForgejo signature: refresh upstream tokens
// AND persist the new tokens back to the access_tokens row keyed by the
// session's broker-token hash.
func refreshFunc(
fj *forgejo.Client,
st *store.Store,
logger *slog.Logger,
) func(context.Context, *oauth.Session) (string, string, time.Time, error) {
return func(ctx context.Context, sess *oauth.Session) (string, string, time.Time, error) {
tok, err := fj.Refresh(ctx, sess.ForgejoRefresh)
if err != nil {
return "", "", time.Time{}, fmt.Errorf("forgejo refresh: %w", err)
}
expiresAt := time.Now().Add(time.Duration(tok.ExpiresIn) * time.Second)
_, dbErr := st.DB().ExecContext(ctx,
`UPDATE access_tokens
SET forgejo_access_token = ?,
forgejo_refresh_token = ?,
forgejo_token_expires_at = ?
WHERE token_hash = ? AND revoked_at IS NULL`,
tok.AccessToken, tok.RefreshToken, expiresAt.Unix(), sess.BrokerTokenHash)
if dbErr != nil && !errors.Is(dbErr, sql.ErrNoRows) {
logger.Warn("persist refreshed forgejo token", "err", dbErr.Error())
// Persist failure is non-fatal — the rotator can still
// hand the new token to the respawned child; the next
// rotation cycle will retry the persist.
}
return tok.AccessToken, tok.RefreshToken, expiresAt, nil
}
}

168
docs/phase7-findings.md Normal file
View file

@ -0,0 +1,168 @@
# Phase 7: end-to-end findings
This document covers the validation that closes phase 7 (issue `-q6n`).
The work splits naturally into two halves:
1. **Local end-to-end** — automated, runs against a fake Forgejo and a
real forgejo-mcp child spawned through the live broker binary.
Performed in this repo as `TestE2E_FullOAuthAndMCPFlow` under
`cmd/broker/e2e_test.go`.
2. **Public Claude.ai validation** — manual, requires a real Forgejo
instance and a publicly-reachable broker. Documented below as a
step-by-step recipe so the operator can repeat it cleanly.
The local automated half was the gating prerequisite: without it, every
manual step would hide both setup mistakes and broker bugs. With it
green, the manual run becomes mostly a configuration exercise.
## What the wiring shipped
`cmd/broker/main.go` now composes every component built across phases
25 into a single binary:
```
http.ServeMux
├── /healthz ← internal/httpserver
├── /oauth/* ← internal/oauth.Server.Handler()
├── /.well-known/* ← internal/oauth.Server.Handler()
└── /mcp ← internal/oauth.Authenticator.RequireBearer
(gates internal/session.Registry.Handler())
```
The `SpawnFunc` passed to the session registry composes
`internal/supervisor` + `internal/bridge`: each new MCP session forks a
`forgejo-mcp --transport stdio` subprocess with the user's upstream
Forgejo token in env, wires its stdio through a bridge, and returns the
bridge's `HandleSSE` as the per-session HTTP handler.
The reaper is wired with a `RefreshForgejo` callback that calls the
upstream Forgejo OAuth client and persists rotated tokens back to the
`access_tokens` row. When a token comes within the rotation lead time of
expiry, the rotator atomically swaps the session's child for one
spawned with the fresh token; the sid is preserved and the client
re-issues `initialize` on its next request.
## Local end-to-end test
```bash
go test -v -run TestE2E_FullOAuthAndMCPFlow ./cmd/broker/
```
The test:
1. Builds the broker binary in TestMain.
2. Builds `forgejo-mcp` from the sibling repo at `../../../forgejo-mcp`
(skipped if not present; alternatively set `FORGEJO_MCP_BIN`).
3. Stands up a fake Forgejo on `httptest.Server` covering
`/api/v1/version`, `/api/v1/user`, `/login/oauth/access_token`, and
`/login/oauth/userinfo`.
4. Runs the broker with all CLI flags wired against the fake.
5. Walks through register → authorize → callback → token → /mcp
initialize → tools/list.
6. Verifies that discovery (`/.well-known/oauth-authorization-server`)
advertises the configured issuer.
It catches at least three classes of regression:
- **Wiring**: any component not plumbed correctly into main.go fails
a step rather than reporting "not implemented".
- **Subprocess-context**: an early version of the wiring used the
per-request HTTP context for `supervisor.Start`. Because
`exec.CommandContext` ties process lifetime to the ctx, this killed
the forgejo-mcp child as soon as the `initialize` request returned;
the subsequent `tools/list` came back as 502. Fixed by introducing
a long-lived `childCtx` in `run()`.
- **Session lifecycle**: 410 / 403 / 503 paths are exercised
in unit tests but the happy-path 200 with `Mcp-Session-Id` minted on
initialize, then reused on follow-up RPCs, is exercised here.
## Public Claude.ai validation (manual)
This is the part the operator runs once. It needs:
- a publicly-reachable hostname (`mcp.example.com`) with a real TLS
cert (Caddy's automatic Let's Encrypt is sufficient — see
`deploy-caddy.md`);
- a real Forgejo instance you control;
- a Claude.ai workspace where you can register a custom MCP connector.
### Setup
1. Provision the broker container per `deploy-podman.md`. Confirm
`https://mcp.example.com/healthz` returns `{"status":"ok"}`.
2. Confirm discovery surface:
```bash
curl -fsS https://mcp.example.com/.well-known/oauth-authorization-server | jq .issuer
# → "https://mcp.example.com"
curl -fsS https://mcp.example.com/.well-known/oauth-protected-resource | jq .resource
# → "https://mcp.example.com/mcp"
```
3. In Claude.ai, add a custom MCP connector pointing at
`https://mcp.example.com/mcp` (or whatever the Claude.ai UI asks for
— at the time of writing, Claude.ai detects the OAuth requirement
from the protected-resource discovery doc and walks the user
through the OAuth flow automatically).
### Walkthrough checklist
Run through these in order. Each is a check the operator marks done.
- [ ] **OAuth completes.** Claude.ai redirects to the broker's
`/oauth/authorize`; the broker bounces to the Forgejo consent screen;
Forgejo redirects back to `/oauth/callback`; Claude.ai shows the
connector as authenticated. Expect the round trip to take a few
seconds.
- [ ] **Tool discovery.** Claude.ai's UI should list every tool the
bundled `forgejo-mcp` exports — check at least
`get_forgejo_mcp_server_version`, `list_my_repos`, `list_repo_issues`
appear.
- [ ] **Tool invocation.** Ask Claude something that exercises a tool
(e.g. "list issues open on `<your repo>`"). Verify the tool runs and
the response cites real data from your Forgejo instance.
- [ ] **Session reuse.** Run two queries in a row. The broker should
send one initialize per session, but successive calls should reuse
the same forgejo-mcp child — visible in the broker's logs as no new
"session reaped" / spawn entries between calls.
- [ ] **Idle timeout.** Leave the conversation idle for 16 minutes
(idle timeout default is 15m + 30s reaper tick). Check broker logs
for `"session reaped"` matching your sid. The next query should
trigger a fresh spawn.
- [ ] **Forgejo token refresh during active session.** Lower
`--session-idle-timeout` to something tolerable (say 1 h) and use a
Forgejo token TTL shorter than that. Run the connector continuously
until the rotator fires (`"session rotated"` in logs). Confirm the
next query still works — Claude.ai re-issues `initialize` on the
swapped child without user intervention.
- [ ] **Revocation.** Hit `POST /oauth/revoke` with the access token.
The next /mcp request from Claude.ai should fail with 401; Claude.ai
should automatically re-authenticate.
### Deliberately deferred
Items not checked in this validation, tracked in the backlog:
- **Rate limits** on `/oauth/register` and `/oauth/token` (`-ttl`).
- **`/proc/<pid>/environ` token exposure** via the Forgejo token in
child env (`-1n2`).
- **At-rest encryption of Forgejo tokens** (`-sd4`).
- **Prometheus `/metrics`** for production observability (`-7fn`).
- **Forgejo Actions CI** (`-0sq`).
All of these are P3 hardenings, not blockers for the Claude.ai
integration itself.
## Closing notes
Every phase-1-through-6 component is now exercised end-to-end against a
real `forgejo-mcp` child. The only wiring that can't be tested without
a real Claude.ai workspace is the actual MCP-discovery → OAuth-handoff
sequence in Claude.ai's client UI; that's left as a manual one-time
operator task using the checklist above.