feedback from latest session : docker / traefik

skills/crowdsec/references/configure/bouncers/web-servers.md

@@ -8,6 +8,10 @@ verified:
     version: "1.7.8"
     env: k8s
     notes: "k8s with traefik + AppSec"
+  - date: 2026-07-10
+    version: "1.7.8"
+    env: docker
+    notes: "Traefik plugin: crowdsecLapiKeyFile (compose secret) auth + bouncer registration; crowdsecAppsecFailureBlock/UnreachableBlock fail closed (403 while AppSec down); AppSec block .env/CVE -> 403, benign 200"
 ---
 
 # Bouncers — Web servers (nginx, haproxy, apache, Traefik, Caddy)
@@ -292,10 +296,14 @@ http:
 | Key | Set to | Notes |
 |---|---|---|
 | `crowdsecMode` | `stream` | `live` = query LAPI per request; `stream` = poll list (lower latency, prod default); `appsec` = WAF only; `none`/`alone`. |
-| `crowdsecLapiKey` | (bouncer key) | Serves both decisions and AppSec. |
+| `crowdsecLapiKey` | (bouncer key) | Serves both decisions and AppSec. Lands in the dynamic config file — for anything you commit, use `crowdsecLapiKeyFile` instead (see secrets pitfall). |
+| `crowdsecLapiKeyFile` | (path) | Read the key from a file (e.g. a mounted Docker/K8s secret) instead of inline. Mutually exclusive with `crowdsecLapiKey`. |
 | `crowdsecLapiHost` | `crowdsec:8080` | Host:port, no scheme (scheme is `crowdsecLapiScheme`). |
 | `crowdsecAppsecEnabled` | `false` | **WAF is off by default.** `true` to forward requests to AppSec. |
 | `crowdsecAppsecHost` | `crowdsec:7422` | AppSec must `listen_addr: 0.0.0.0:7422` so the Traefik container can reach it. |
+| `crowdsecAppsecFailureBlock` | `true` | Fail-**closed** if AppSec *errors* on a request (block vs allow). Set `false` to fail open. |
+| `crowdsecAppsecUnreachableBlock` | `true` | Fail-**closed** if AppSec is *unreachable* (block vs allow). Set `false` to fail open. |
+| `crowdsecAppsecBodyLimit` | `10485760` | Max request-body bytes forwarded to AppSec (10 MB default). A DoS-hardening cap. |
 | `forwardedHeadersTrustedIPs` | `[]` | Plugin-side trust for `X-Forwarded-For`. **Not sufficient alone — see real-IP pitfall.** |
 | `clientTrustedIPs` | `[]` | IPs that **bypass the bouncer entirely**. Do **not** put your proxy/Docker range here or every request is allowed. |
 
@@ -341,6 +349,32 @@ docker exec crowdsec cscli metrics show appsec     # Processed/Blocked increment
   `0.0.0.0:7422` (not loopback) for a containerized Traefik to reach it.
 - **`stream` lag:** a fresh ban lands within `updateIntervalSeconds`; immediate ban-then-curl
   looks like a failure. (See [../../debug/symptoms/not-blocked.md](../../debug/symptoms/not-blocked.md).)
+- **Key committed in plaintext:** the literal `crowdsecLapiKey` lives in the dynamic config file,
+  which you usually don't want in version control. For anything you commit or ship, mount the key
+  as a file and point `crowdsecLapiKeyFile` at it. With Docker Compose, a `secret` mounts at
+  `/run/secrets/<name>`:
+
+  ```yaml
+  # docker-compose.yml (traefik service)
+  services:
+    traefik:
+      secrets: [lapi_key]
+  secrets:
+    lapi_key:
+      file: ./secrets/lapi_key          # the raw bouncer key, no trailing newline
+  ```
+
+  Then set `crowdsecLapiKeyFile: /run/secrets/lapi_key` in the middleware (drop `crowdsecLapiKey`).
+  The official Traefik bouncer guide treats the inline key as the quick-validation path only.
+- **Fail-open by default on AppSec trouble:** if AppSec errors or is unreachable, whether the
+  request is blocked depends on `crowdsecAppsecFailureBlock` / `crowdsecAppsecUnreachableBlock`.
+  Set both `true` to fail **closed** (a request 403s while AppSec is down) rather than pass
+  traffic through un-inspected.
+- **Non-XFF real IP (situational):** this walkthrough assumes the client IP arrives in
+  `X-Forwarded-For`. If your upstream sends a different header (e.g. `X-Real-Ip`), set the plugin's
+  `forwardedHeadersCustomName`; if it uses the PROXY protocol (AWS NLB, some HAProxy/keepalived
+  setups), trust it on the Traefik entrypoint with `proxyProtocol.trustedIPs` (the L4 analogue of
+  `forwardedHeaders.trustedIPs`). Skip both if your upstream sends plain XFF.
 
 ### Kubernetes (Helm) — extra gotchas
 

skills/crowdsec/references/install/console.md

@@ -91,7 +91,7 @@ local-vs-Console allowlist distinction.
 | Env | Enroll via |
 |---|---|
 | systemd / bare-metal | `sudo cscli console enroll …` then `systemctl reload crowdsec` |
-| Docker | `ENROLL_KEY` (and `ENROLL_INSTANCE_NAME`/`ENROLL_TAGS`) env vars on the crowdsec container, **or** `docker exec crowdsec cscli console enroll …` then restart the container |
+| Docker | `ENROLL_KEY` (and `ENROLL_INSTANCE_NAME`/`ENROLL_TAGS`) env vars on the crowdsec container, **or** `docker exec crowdsec cscli console enroll …` then restart the container. Enrollment writes `console.yaml` + `online_api_credentials.yaml` under `/etc/crowdsec` — persist that path or a recreate un-enrolls you (see pitfall). |
 | Kubernetes | `config.console.enroll_key` (and name/tags) in the Helm chart values; the LAPI pod enrolls on start |
 
 ## Pitfalls
@@ -101,6 +101,12 @@ local-vs-Console allowlist distinction.
 - **Token reuse across engines** without `--name` → every engine collides under
   the machine-ID default and they're indistinguishable in the Console. Set
   `--name` per engine.
+- **Docker recreate un-enrolls you**: interactive `cscli console enroll` writes
+  its state under `/etc/crowdsec`. If that path isn't a persisted volume, a
+  `docker compose down && up` (or image upgrade) recreates the container and drops
+  the enrollment — you re-enroll and re-Accept. Persist `/etc/crowdsec` (see
+  [../install/docker.md](../install/docker.md)), or use the `ENROLL_KEY` env
+  instead: it re-runs on every boot and is recreate-safe.
 - **Egress**: the engine must reach `api.crowdsec.net` (HTTPS). Behind a proxy,
   set the proxy env for the crowdsec service; behind egress filtering, allow
   that host. `cscli capi status` failing right after enroll is almost always

skills/crowdsec/references/install/docker.md

@@ -1,9 +1,9 @@
 ---
 verified:
-  - date: 2026-05-22
+  - date: 2026-07-10
     version: "1.7.8"
     env: docker
-    notes: "compose up, COLLECTIONS install on boot, /logs/auth.log container-path acquisition, 8081:8080 coexistence, bouncers add; teardown -v"
+    notes: "compose up on empty cs-config volume; acquis.d must be read-write (a :ro submount aborts boot with rsync exit 23); /etc/crowdsec persists creds+enrollment across down&&up; COLLECTIONS install on boot; /logs/auth.log container-path acquisition; 8081:8080 coexistence; bouncers add; teardown -v"
 ---
 
 # Install — Docker / docker-compose
@@ -27,7 +27,7 @@ services:
       - cs-config:/etc/crowdsec
       - cs-data:/var/lib/crowdsec/data
       - /var/log/auth.log:/logs/auth.log:ro          # host log, read-only
-      - ./acquis.d:/etc/crowdsec/acquis.d:ro          # your acquisition
+      - ./acquis.d:/etc/crowdsec/acquis.d            # your acquisition — must be read-write (§6)
     ports:
       - "8081:8080"      # LAPI (see port-conflict note)
       - "7423:7422"      # AppSec (only if you run the WAF)
@@ -42,6 +42,16 @@ startup — and re-runs the install on **every** start, not just the first (see
 §2). `sshd`, `appsec-virtual-patching`, and `appsec-generic-rules` all end up
 `enabled` after startup.
 
+**Persist both volumes — they hold different state.** `cs-config:/etc/crowdsec`
+holds `console.yaml` and the `local_api_credentials.yaml` /
+`online_api_credentials.yaml` files; `cs-data:/var/lib/crowdsec/data` holds the
+DB (decisions, alerts). `docker restart` keeps everything, but
+`docker compose down && up` — or an image upgrade — *recreates* the container and
+loses whatever isn't on a volume. Without `cs-config`, a container recreated after
+`cscli console enroll` comes back **un-enrolled** (you re-enroll and re-accept);
+see [console.md](console.md). The canonical Docker doc lists
+`crowdsec-config:/etc/crowdsec` among the paths that must be persisted.
+
 ## Gotchas
 
 ### 1. Acquisition paths are *container* paths, not host paths
@@ -131,6 +141,23 @@ or the published port reaches nothing. Confirm with a host
 - **IPv6**: the AppSec/firewall behaviour mirrors bare-metal; container
   networking is v4 by default unless you enable v6 on the daemon/network.
 
+### 6. `acquis.d` must be read-write when `/etc/crowdsec` is a persisted volume
+
+Mount `./acquis.d` **read-write** (as the compose above does), not `:ro`. On a
+fresh `cs-config` volume the entrypoint rsyncs its staging config into
+`/etc/crowdsec` and `chown`s everything under it, including the `acquis.d`
+submount. A `:ro` bind there makes that `chown` fail and the container dies on
+first boot:
+
+```
+rsync: [generator] chown "/etc/crowdsec/acquis.d" failed: Read-only file system (30)
+rsync error: some files/attrs were not transferred (see previous errors) (code 23)
+```
+
+The container exits **23** and never starts. This only affects the `acquis.d`
+submount *inside* the persisted `/etc/crowdsec` volume — host-log mounts elsewhere
+(e.g. `/logs/auth.log:ro`) are untouched.
+
 ## Bouncer key bootstrap
 
 ```bash