[gateway] Add the Gateway runtime, common HTTP layer, lifecycle, and test infrastructure - #3963
[gateway] Add the Gateway runtime, common HTTP layer, lifecycle, and test infrastructure#3963beryllw wants to merge 11 commits into
Conversation
fc487d0 to
f6e94e7
Compare
|
@fresh-borzoni This is the first PR for FIP-49 — could you help review it when you have time? Thanks a lot! 🙏 |
There was a problem hiding this comment.
Pull request overview
This PR introduces the initial Rust-based Fluss Gateway module (fluss-gateway/) as a standalone Cargo workspace, providing the runtime/lifecycle foundation, a shared HTTP layer (error envelope, request IDs, limits/timeouts), baseline observability (logging + Prometheus metrics), and integration test infrastructure for future gateway capabilities described in FIP-49 / issue #3958.
Changes:
- Adds gateway runtime + lifecycle (listener binding, readiness/draining, SIGTERM handling, stable exit codes).
- Adds REST foundation with
GET /health,GET /v1/openapi.json, shared error envelope, request IDs, body limits, and request deadlines. - Adds gateway-focused CI jobs (build/test, fmt/clippy/rustdoc, OpenAPI drift + lint, MSRV gate) and supporting test suites.
Reviewed changes
Copilot reviewed 28 out of 29 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| fluss-gateway/tests/support/mod.rs | Shared HTTP + process helpers for gateway integration tests. |
| fluss-gateway/tests/process.rs | End-to-end tests for the compiled binary (exit codes, SIGTERM drain, bind failures). |
| fluss-gateway/tests/http_api.rs | End-to-end HTTP contract tests against a real listener (health, OpenAPI, envelopes, limits). |
| fluss-gateway/tests/e2e_cluster.rs | Feature-gated dockerized cluster smoke test wiring (integration scaffolding). |
| fluss-gateway/src/protocol/rest/openapi.rs | Router-derived OpenAPI generation + serving and drift checks. |
| fluss-gateway/src/protocol/rest/mod.rs | REST router assembly and middleware (request IDs, envelopes, deadlines, body limits, acceptance guard). |
| fluss-gateway/src/protocol/rest/health.rs | GET /health endpoint and schema. |
| fluss-gateway/src/protocol/mod.rs | Protocol module root exporting REST surface. |
| fluss-gateway/src/observability.rs | Logging + Prometheus metrics inventory and runtime/process metric sampling. |
| fluss-gateway/src/main.rs | Binary entrypoint (CLI, config load/validate, init logging, lifecycle run + exit codes). |
| fluss-gateway/src/lifecycle.rs | Listener/task supervision, readiness state machine, graceful shutdown/draining. |
| fluss-gateway/src/lib.rs | Gateway library crate root (internal implementation modules). |
| fluss-gateway/src/error.rs | Error taxonomy + REST error envelope and OpenAPI-friendly schema generation. |
| fluss-gateway/src/config.rs | Gateway configuration loading (CLI/env/YAML precedence), parsing, validation, and warnings. |
| fluss-gateway/rustfmt.toml | Gateway-local rustfmt configuration. |
| fluss-gateway/rust-toolchain.toml | Gateway-local toolchain selection (stable + components). |
| fluss-gateway/openapi.yaml | Checked-in generated OpenAPI document. |
| fluss-gateway/justfile | Developer recipes for build/test/fmt/clippy/doc/openapi/licenses. |
| fluss-gateway/deny.toml | cargo-deny license allowlist/config. |
| fluss-gateway/copyright.txt | License header template for tooling. |
| fluss-gateway/clippy.toml | Clippy configuration (cognitive complexity threshold). |
| fluss-gateway/Cargo.toml | Standalone workspace/package definition, deps, features, tests, MSRV. |
| fluss-gateway/.licenserc.yaml | License header checking configuration (with generated-file exclusions). |
| fluss-gateway/.gitignore | Gateway-local ignore rules. |
| .github/workflows/rust-license-and-format.yml | Extends Rust license/format workflow to include gateway checks. |
| .github/workflows/rust-build-and-test.yml | Adds gateway build/test/OpenAPI/MSRV/E2E jobs and triggers. |
| .github/workflows/license-check.yml | Excludes gateway from the generic license-check workflow (gateway has its own checks). |
| .github/workflows/ci.yaml | Excludes gateway-only changes from the main CI workflow (gateway has dedicated Rust workflows). |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Introduces fluss-gateway as an independent Cargo workspace with library and executable entry points: strict configuration with stable exit codes, the shared error envelope, request-id/body-size/deadline middleware, the FIP-49 GET /health endpoint, REST and metrics listeners, task supervision with SIGTERM draining, and the OpenAPI 3.1 document generated from the typed router. Co-authored-by: gstamatakis95 <126914070+gstamatakis95@users.noreply.github.com>
…uster harness Covers the shared HTTP contract over a real listener, the compiled binary's startup, health, SIGTERM draining and exit codes, and a self-test of the fixed-version dockerized Fluss cluster harness that later capabilities reuse for their end-to-end suites. Co-authored-by: gstamatakis95 <126914070+gstamatakis95@users.noreply.github.com>
Runs the gateway workspace's build, unit tests, license headers, formatting, clippy and rustdoc, checks the checked-in OpenAPI document for drift and validates it externally, and runs the end-to-end suite in a job that fails rather than skipping silently when a selected scenario cannot start. Co-authored-by: gstamatakis95 <126914070+gstamatakis95@users.noreply.github.com>
… rename the harness job Co-authored-by: gstamatakis95 <126914070+gstamatakis95@users.noreply.github.com>
… artifact release
…ines against overflow
795c831 to
cbd4ab8
Compare
…e rust workflows Move the gateway jobs out of rust-build-and-test.yml and rust-license-and-format.yml into the dedicated gateway-ci.yml introduced in apache#4002, and install protoc in every compiling job: the gateway now depends on fluss-rs, whose proto codegen needs protoc, which the scaffold-era workflow steps did not install. The rust workflows are restored to their upstream state so gateway changes never build the fluss-rust workspace and vice versa.
… and slim the gateway CI
fresh-borzoni
left a comment
fresh-borzoni
left a comment
There was a problem hiding this comment.
@beryllw Thank you for the PR!
I looked quickly with a limited time I had, left some quick comments, PTAL
| router: Router, | ||
| shutdown: CancellationToken, | ||
| ) -> Result<(), String> { | ||
| let server = axum::serve(listener, router).with_graceful_shutdown(async move { |
There was a problem hiding this comment.
A client can open a connection, send half a request head, and we hold it forever. No header-read timeout here, and the request-timeout middleware never sees these because it only runs after hyper has a complete head. There's no auth on the listener either, so anyone who can reach the port can tie up sockets and tasks until we run out of them.
Mb set one on the hyper builder?
| // Port 0 asks the OS for a free port, so two ephemeral listeners never collide even though the | ||
| // configured addresses are equal. | ||
| if server.metrics.enabled | ||
| && server.metrics.bind_address == rest_address |
There was a problem hiding this comment.
You can put both listeners on the same port and this won't notice, e.g 0.0.0.0:8080 for REST, 127.0.0.1:8080 for metrics. The bad part is that it starts up clean and the process looks healthy while nothing can use it, combine it with readiness checks and K8s and we are asking for the problems
| .unwrap_or_default(); | ||
| request | ||
| .extensions_mut() | ||
| .insert(deadline_from_now(request_timeout)); |
There was a problem hiding this comment.
This deadline and the timeout below expire together, so a handler that answers at the deadline has its response thrown away and the caller gets a bare 504. Should be smth like a grace to the middleware timeout, or just subtract it here?
| pub(crate) async fn health(State(state): State<RestState>) -> Response { | ||
| // The response type is the documented schema, so the payload cannot drift from the contract. | ||
| json_response(&HealthResponse { | ||
| status: "ok", |
There was a problem hiding this comment.
/health says ok while draining and before we start serving, the two windows where every other route returns 503, so LB keeps sending traffic we reject. I'm confused here, can you clarify?
Purpose
Linked issue: close #3958
First PR of FIP-49: the Gateway runtime and HTTP foundation. Authentication, the Fluss backend and the data
APIs come in follow-up issues.
Distilled from the FIP-49 PoC; the foundation was written by @gstamatakis95 and is credited via
Co-authored-by.Brief change log
fluss-gateway/as an independent Cargo workspace, library plus executable.gateway.yamlconfiguration: flat dotted keys, env and CLI overrides, validation, stable exit codes.GET /healthreturning{status, uptime_ms}.Tests
66 tests: unit tests, the HTTP contract over a real listener, and the compiled binary's startup, health, SIGTERM and exit codes. A
--features integration_testssuite drives the gateway against a dockerized Fluss cluster on demand; it is not a pull-request gate yet.API and Format
New endpoints
GET /healthandGET /v1/openapi.json. No change to existing APIs or storage formats.Documentation
Module-level rustdoc, enforced by
cargo doc -D warnings.