feat(docker): supported single-host self-hosting stack and install guide (#1395)

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Author: FelixTJDietrich

.changeset/blessed-self-host-install.md

@@ -0,0 +1,13 @@
+---
+"hephaestus": minor
+---
+
+You can now self-host Hephaestus on a single Linux server. `docker/self-host/` adds one
+supported Docker Compose stack — application server, webhook receiver, PostgreSQL and NATS
+behind a TLS reverse proxy — that reuses the maintainers' service definitions, so it has no
+second copy to fall out of date. Follow the new [install guide](https://ls1intum.github.io/Hephaestus/admin/install);
+GitHub App setup, manual webhook creation, and backup/restore each have a companion page.
+
+Existing deployments are unaffected: the reference Compose files are unchanged apart from two
+safe additions — the NATS JetStream limits become overridable (defaults unchanged), and a
+release-pin sanity check that rejected every valid pin is fixed.

.changeset/fix-release-pin-grep.md

@@ -0,0 +1,6 @@
+---
+"hephaestus": patch
+---
+
+Fixes the release-image pin check rejecting every valid pinned digest, which stopped the
+application server from starting on a fresh deploy that enforces the digest pin.

.github/workflows/ci-compose-validate.yml

@@ -0,0 +1,102 @@
+name: Validate Compose
+
+# The self-hosted stack (docker/self-host/) composes the reference deployment's
+# service definitions via `include`, so a change to docker/compose.*.yaml can
+# break a stranger's install without touching a single self-host file. Rendering
+# both stacks here turns interpolation- and merge-level breakage into a red check
+# instead of a bad first boot. (Semantic breakage — a renamed service silently
+# joining the stack, an inherited runtime bug — still needs a real boot.)
+#
+# Runs on every PR (not just docker/ changes) so it can be a required check,
+# matching how verify-changesets.yml is wired. It is cheap: only `docker compose
+# config`, no image pulls.
+
+on:
+  pull_request:
+  push:
+    branches: [main]
+
+permissions:
+  contents: read
+
+jobs:
+  validate:
+    name: "Render compose stacks"
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
+
+      - name: Render the self-hosted stack
+        working-directory: docker/self-host
+        run: |
+          set -euo pipefail
+          # Fill the required values the way an operator would; `docker compose
+          # config` fails on any variable the stack requires but .env.example
+          # does not carry, which is exactly the drift we want to catch.
+          cp .env.example .env
+          sed -i \
+            -e 's|^APP_HOSTNAME=$|APP_HOSTNAME=hephaestus.example.com|' \
+            -e 's|^ACME_EMAIL=$|ACME_EMAIL=operator@example.com|' \
+            -e 's|^POSTGRES_PASSWORD=$|POSTGRES_PASSWORD=ci-not-a-real-password|' \
+            -e 's|^HEPHAESTUS_SECURITY_ENCRYPTION_KEY=$|HEPHAESTUS_SECURITY_ENCRYPTION_KEY=0123456789abcdef0123456789abcdef|' \
+            -e 's|^HEPHAESTUS_AUTH_STATE_COOKIE_KEY=$|HEPHAESTUS_AUTH_STATE_COOKIE_KEY=Y2ktbm90LWEtcmVhbC1zdGF0ZS1jb29raWUta2V5|' \
+            -e 's|^WEBHOOK_SECRET=$|WEBHOOK_SECRET=ci000000000000000000000000000000000|' \
+            .env
+          docker compose config > /dev/null
+          echo "Rendered services:"
+          docker compose config --services | sort
+
+      - name: Fail on unset variables
+        working-directory: docker/self-host
+        run: |
+          set -euo pipefail
+          # `config` only warns about variables missing from .env; a warning here
+          # means .env.example has fallen behind the stack it renders.
+          if docker compose config 2>&1 >/dev/null | grep "variable is not set"; then
+            echo "::error::docker/self-host/.env.example is missing variables the stack references (see warnings above)"
+            exit 1
+          fi
+
+      - name: Assert the merged stack matches single-host intent
+        working-directory: docker/self-host
+        run: |
+          set -euo pipefail
+          services=$(docker compose config --services)
+          for unwanted in application-worker maintenance; do
+            grep -qx "$unwanted" <<< "$services" && {
+              echo "::error::'$unwanted' is a reference-deployment service and must not run on a single host"; exit 1; } || true
+          done
+          for required in application-server webhook-server postgres nats-server webapp reverse-proxy; do
+            grep -qx "$required" <<< "$services" || {
+              echo "::error::'$required' is missing from the self-hosted stack"; exit 1; }
+          done
+          # Compose 2.21-2.23 parse `!override` but silently ignore it, which would
+          # publish the reference's dashboard port and keep the maintainers' ACME
+          # email. Assert the merged result rather than trust the runner's version.
+          rendered=$(docker compose config)
+          grep -q "admin@tum.de" <<< "$rendered" && {
+            echo "::error::the maintainers' ACME email survived the override — Compose is too old to honour !override"; exit 1; } || true
+          published=$(docker compose config --format json \
+            | jq -r '.services["reverse-proxy"].ports[].published' | sort -n | tr '\n' ' ')
+          [ "$published" = "80 443 " ] || {
+            echo "::error::reverse-proxy publishes '$published', expected '80 443 ' — !override was ignored"; exit 1; }
+
+      - name: Assert the pinned release version has not drifted
+        run: |
+          set -euo pipefail
+          # scripts/sync-selfhost-version.mjs keeps these equal to the release on
+          # every Version PR; a mismatch means someone edited one by hand.
+          pkg=$(node -p "require('./package.json').version")
+          env_tag=$(grep -m1 '^IMAGE_TAG=' docker/self-host/.env.example | cut -d= -f2)
+          [ "$pkg" = "$env_tag" ] || {
+            echo "::error::IMAGE_TAG=$env_tag in .env.example != package.json $pkg (run scripts/sync-selfhost-version.mjs)"; exit 1; }
+
+      - name: Render the reference deployment
+        working-directory: docker
+        run: |
+          set -euo pipefail
+          # Values mirror what the deploy workflow supplies; only proves the files
+          # still render, not that they are correct for production.
+          docker compose \
+            -f compose.proxy.yaml -f compose.core.yaml -f compose.app.yaml \
+            --env-file self-host/.env config --quiet

.github/workflows/version-pr.yml

@@ -49,7 +49,10 @@ jobs:
       - name: Maintain Version PR
         uses: changesets/action@a45c4d594aa4e2c509dc14a9f2b3b67ba3780d0d # v1.9.0
         with:
-          version: pnpm changeset version
+          # Also syncs the self-host IMAGE_TAG / install-guide version to the new
+          # release (scripts/sync-selfhost-version.mjs), so those literals never
+          # drift. The changesets action commits whatever the version step writes.
+          version: pnpm run changeset:version
           # No `publish:` — tagging, GitHub Release, and deploys are release.yml's
           # job (it uses `v`-prefixed tags to stay consistent with existing tags,
           # docker IMAGE_TAG, and the release-pin asset; `changeset tag` would emit

.gitignore

@@ -130,6 +130,9 @@ celerybeat.pid
 .env
 .venv
 env/
+
+# Self-host: Traefik writes cert material here inside the operator's checkout
+docker/self-host/letsencrypt/
 venv/
 ENV/
 env.bak/

INSTALL.md

@@ -0,0 +1,10 @@
+# Installing Hephaestus
+
+There is exactly **one supported install**: the Docker Compose stack in
+[`docker/self-host/`](docker/self-host/) on one 64-bit Linux host
+(4 vCPUs / 8 GB RAM / 40 GB SSD recommended). Other setups may work but are unsupported.
+
+**→ Follow the install guide: <https://ls1intum.github.io/Hephaestus/admin/install>**
+
+For contributor/development setup, see the
+[contributor docs](https://ls1intum.github.io/Hephaestus/contributor/local-development) instead.

README.md

@@ -51,7 +51,8 @@ UI component docs: [Storybook](https://main--66a8981a27ced8fef3190d41.chromatic.
 
 ### Setup
 
-Read the [local development guide](https://ls1intum.github.io/Hephaestus/contributor/local-development) for the Spring Boot application server (with the in-process Pi mentor agent) and the React client in `webapp`.
+- **Self-hosting:** follow the [install guide](https://ls1intum.github.io/Hephaestus/admin/install) ([INSTALL.md](INSTALL.md)) — one supported Docker Compose path.
+- **Development:** read the [local development guide](https://ls1intum.github.io/Hephaestus/contributor/local-development) for the Spring Boot application server (with the in-process Pi mentor agent) and the React client in `webapp`.
 
 ## Contributing
 

docker/compose.app.yaml

@@ -78,7 +78,7 @@ services:
           --certificate-identity 'https://github.com/ls1intum/Hephaestus/.github/workflows/release.yml@refs/heads/main' \
           --certificate-oidc-issuer 'https://token.actions.githubusercontent.com' \
           "$$TMP/pin.yaml"
-        grep -Eq '^[[:space:]]+reference:[[:space:]]+[a-z0-9][a-z0-9.\-_/:]+@sha256:[a-f0-9]{64}$$' "$$TMP/pin.yaml" \
+        grep -Eq '^[[:space:]]+reference:[[:space:]]+[a-z0-9][a-z0-9._/:-]+@sha256:[a-f0-9]{64}$$' "$$TMP/pin.yaml" \
           || { echo "pin asset is missing a digest-pinned reference" >&2; exit 1; }
         install -m 0644 "$$TMP/pin.yaml" /pin/release-pin.yaml.new
         mv /pin/release-pin.yaml.new /pin/release-pin.yaml

docker/compose.core.yaml

@@ -143,6 +143,6 @@ configs:
 
       jetstream {
         store_dir: "/data"
-        max_mem: 4G
-        max_file: 50G
+        max_mem: ${NATS_JS_MAX_MEM:-4G}
+        max_file: ${NATS_JS_MAX_FILE:-50G}
       }

docker/self-host/.env.example

@@ -0,0 +1,131 @@
+# =============================================================================
+# Hephaestus self-hosted — environment
+# =============================================================================
+#
+#   cp .env.example .env    # then fill in every REQUIRED value below
+#   docker compose up -d
+#
+# Full guide (read it first): https://ls1intum.github.io/Hephaestus/admin/install
+#
+# Hardware honesty: 4 vCPUs / 8 GB RAM / 40 GB SSD recommended. The stack runs
+# two JVM services plus Postgres and NATS; 4 GB RAM is the absolute floor and
+# only without AI practice review. Each concurrent AI review sandbox may use
+# up to 4 GiB on top.
+# =============================================================================
+
+# --- General (REQUIRED) ------------------------------------------------------
+
+# Public hostname the instance is served on (DNS A record -> this host).
+# No scheme, no trailing slash. Example: hephaestus.example.com
+APP_HOSTNAME=
+
+# Release to run: an exact version, no leading `v`, never `latest`.
+# Must match the release you checked out. See the install guide.
+IMAGE_TAG=0.73.2
+
+# Email for Let's Encrypt certificate-expiry notices.
+ACME_EMAIL=
+
+# --- Secrets (REQUIRED — generate once, then never change) -------------------
+
+# Database password. Applied only when the data volume is first initialized.
+# Generate: openssl rand -hex 16
+POSTGRES_PASSWORD=
+
+# AES-256 key encrypting credentials at rest (and sealing the JWT signing key).
+# EXACTLY 32 characters. Losing or changing it makes every stored provider
+# token unreadable — treat it like the database itself and back it up.
+# Generate: openssl rand -base64 24 | cut -c1-32
+HEPHAESTUS_SECURITY_ENCRYPTION_KEY=
+
+# Base64-encoded 32-byte AES key sealing the short-lived OAuth state cookies.
+# Generate: openssl rand -base64 32
+HEPHAESTUS_AUTH_STATE_COOKIE_KEY=
+
+# Shared secret verifying inbound GitHub/GitLab webhooks (min 32 chars).
+# You will enter this same value on the GitHub side — see the install guide.
+# Generate: openssl rand -hex 32
+WEBHOOK_SECRET=
+
+# --- Login (at least one provider REQUIRED, or nobody can sign in) -----------
+
+# GitHub OAuth App (https://github.com/settings/developers -> "New OAuth App").
+# Authorization callback URL: https://<APP_HOSTNAME>/api/login/oauth2/code/github
+# Named GH_OAUTH_* (not GITHUB_OAUTH_*) because GitHub Actions reserves the
+# GITHUB_ prefix; the container still presents the right name to the server.
+GH_OAUTH_CLIENT_ID=
+GH_OAUTH_CLIENT_SECRET=
+
+# GitLab OAuth application (optional; gitlab.com or self-hosted; scope: read_user).
+# Callback: https://<APP_HOSTNAME>/api/login/oauth2/code/gitlab
+#GITLAB_OAUTH_CLIENT_ID=
+#GITLAB_OAUTH_CLIENT_SECRET=
+#GITLAB_OAUTH_BASE_URL=https://gitlab.com
+#GITLAB_OAUTH_DISPLAY_NAME=GitLab
+
+# --- First admin (REQUIRED — set BEFORE first boot) ---------------------------
+
+# Who becomes instance admin on their first sign-in. Comma-separated
+# <provider>:@<username> or <provider>:<numeric-id>, e.g. github:@octocat
+# On public github.com prefer the numeric id (from
+# https://api.github.com/users/<login>) — handles can be reclaimed.
+HEPHAESTUS_AUTH_BOOTSTRAP_ADMINS=
+
+# Break-glass fallback: enables one-time POST /auth/bootstrap-admin while no
+# admin exists. Leave unset unless you need it; unset again after use.
+#HEPHAESTUS_AUTH_BOOTSTRAP_TOKEN=
+
+# --- GitHub App (optional — PAT-only mode works without any of these) --------
+# Needed for posting AI review feedback back to GitHub and higher rate limits.
+# Setup guide: https://ls1intum.github.io/Hephaestus/admin/github-integration
+GH_APP_PRIVATE_KEY=
+#GH_APP_ID=
+#GH_APP_INSTALLATION_URL=
+
+# Legacy GitHub token used for contributor metadata only. Leave blank.
+GH_AUTH_TOKEN=
+
+# --- Optional integrations ----------------------------------------------------
+# GitLab sync, Slack, Outline, AI practice review: enable by uncommenting here.
+# See https://ls1intum.github.io/Hephaestus/admin/production-setup for each bundle.
+#GITLAB_ENABLED=false
+#GITLAB_WORKSPACE_CREATION=false
+# Default GitLab instance for sync + workspace creation (compose otherwise defaults
+# this to the maintainers' gitlab.lrz.de). Point it at your instance.
+#GITLAB_DEFAULT_SERVER_URL=https://gitlab.com
+#HEPHAESTUS_INTEGRATION_SLACK_ENABLED=false
+#HEPHAESTUS_INTEGRATION_SLACK_CLIENT_ID=
+#HEPHAESTUS_INTEGRATION_SLACK_CLIENT_SECRET=
+#HEPHAESTUS_INTEGRATION_SLACK_SIGNING_SECRET=
+#HEPHAESTUS_INTEGRATION_OUTLINE_ENABLED=false
+
+# AI practice review bundle (enable all three together + an LLM upstream), plus
+# a model and API key per workspace in the UI. Budget ~4 GiB RAM per sandbox.
+#AGENT_NATS_ENABLED=false
+#GIT_CHECKOUT_ENABLED=false
+#PRACTICE_REVIEW_FOR_ALL=false
+#SANDBOX_MAX_CONCURRENT=1
+#LLM_PROXY_OPENAI_URL=https://api.openai.com
+
+# --- Misc (optional) -----------------------------------------------------------
+
+# Imprint/privacy pages (required for public instances in e.g. Germany):
+# see https://ls1intum.github.io/Hephaestus/admin/legal-pages
+#LEGAL_PROFILE=
+
+# Sentry error tracking. Leave blank to disable.
+SENTRY_DSN=
+SENTRY_ENVIRONMENT=production
+# PostHog product analytics — disabled unless POSTHOG_ENABLED=true; the keys below
+# do nothing on their own. Leave the two keys present (blank is fine) either way.
+#POSTHOG_ENABLED=false
+POSTHOG_PROJECT_API_KEY=
+POSTHOG_API_HOST=
+
+# NATS JetStream limits, sized for a single host (reference deployment uses 4G/50G).
+NATS_JS_MAX_MEM=1G
+NATS_JS_MAX_FILE=10G
+
+# Proxy-trust regex; the default matches reverse-proxy's fixed IP (172.29.47.2).
+# Only set when you changed that IP or the subnet in compose.single-host.yaml.
+#HEPHAESTUS_TRUSTED_PROXIES=172\.29\.47\.2