Skip to content

Add Release Smoke Gates For Examples And Benchmarks #206

Description

@GefMar

Goal

Add lightweight CI/release smoke gates that catch broken examples and obvious performance regressions without turning CI into a flaky benchmark suite.

Context

The global review identified two release-quality gaps: examples are not smoke-checked, and performance claims are protected only by external benchmark discipline. FogHTTP's positioning depends on performance, but hard benchmark thresholds in normal CI can be noisy. The goal is a small, stable smoke layer plus clear benchmark workflow integration.

Scope

  • Add a cheap examples smoke check, at minimum syntax/import validation and preferably execution of examples that can run against a local test server.
  • Define a benchmark smoke strategy for critical paths that is stable enough for CI or release gating.
  • Avoid strict microbenchmark thresholds in normal PR CI unless they are proven stable on GitHub runners.
  • Link the benchmark repo workflow where deeper performance validation belongs.
  • Ensure release tasks know which smoke checks are required before publishing.

Out Of Scope

  • Full benchmark suite in every PR.
  • Optimizing a detected performance issue; create a separate task when a regression is confirmed.
  • Rewriting examples.

Acceptance Criteria

  • Broken example imports/syntax are caught before release.
  • Critical benchmark smoke strategy is documented and runnable.
  • Normal PR CI remains stable and reasonably fast.
  • Release-readiness audit includes example and benchmark smoke status.

Test And Check Strategy

  • Example smoke command added by this issue.
  • Benchmark smoke command from the benchmark repo or a dedicated lightweight repo script.
  • uv run pre-commit run --all-files after CI config changes.

Documentation Impact

  • Update contributing/release docs if a new local check is required.
  • Update or release notes with benchmark smoke expectations.

Risks

  • Risk: benchmark checks become noisy and block unrelated work.
    Mitigation: keep hard thresholds out of normal PR CI until stable; prefer release-only or trend-based checks.

Metadata

Metadata

Assignees

No one assigned

    Labels

    P2Ergonomics, documentation, CI, or polish

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions