Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 10 additions & 1 deletion pages/dashboard/PLAN.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,13 +17,14 @@
- [x] Slice: `DLS-PAGE-002` and `DLS-PAGE-006` conservative required-source validation for built-in page definitions.
- [x] Slice: `DLS-PAGE-003` through `DLS-PAGE-013` conservative required-field coverage validation for built-in page definitions.
- [x] Slice: `DLS-PAGE-014` conservative built-in data-state exposure validation via implementation-local declarative markers.
- [x] Slice: `DLS-PAGE-006` conservative run-link coverage validation for the `runs` built-in page.
- [ ] **Security, privacy, accessibility** — Section 13 including escaping, redaction, and keyboard and screen-reader behavior verified with Playwright.
- [ ] **Compliance suite** — Section 14 test suite, the compliance checklist, Appendix A as a passing fixture, and Appendix C as failing fixtures.
- [ ] **Parity** — inventory the features of the existing dashboard in `.github/scripts/pages-report/report.mjs`, record them in `PLAN.md` as a parity checklist, then express each one as YAML configuration plus data fixtures, closing the checklist incrementally.

## Specification questions

- 2026-08-28: Section 10 requires every built-in page to be expressed as declarative page definitions built from the custom-view primitives, but Section 4.2 and Section 10 define no YAML vocabulary for embedding those declarative built-in definitions alongside `kind: built-in` / `page`. The current validator implements the most conservative reading available in this slice by accepting an implementation-local `definition.views` mapping on built-in pages so the required-source coverage can be validated, but this key is not yet specification-backed and may need to change if the YAML vocabulary is clarified.
- 2026-08-28: Section 10 requires every built-in page to be expressed as declarative page definitions built from the custom-view primitives, but Section 4.2 and Section 10 define no YAML vocabulary for embedding those declarative built-in definitions alongside `kind: built-in` / `page`. The current validator implements the most conservative reading available in this slice by accepting an implementation-local `definition.views` sequence on built-in pages so required source, field, and run-link coverage can be validated, but this key is not yet specification-backed and may need to change if the YAML vocabulary is clarified.
- 2026-08-28: `DLS-PAGE-014` says every built-in page must expose availability, completeness, and freshness independently, but Section 10 does not define a declarative YAML shape for asserting that exposure inside a built-in page definition. The current validator uses a conservative implementation-local `definition.data-state` marker with canonical boolean `true` for each axis; this is only a coverage declaration, not presenter behavior.
- 2026-08-28: Section 4.3 requires `language-version` to be the quoted string `"0.1.0"`, but YAML parsing does not preserve whether a scalar was quoted. The current validator enforces string type and exact canonical value, which is the most conservative check available without relying on parser-specific CST details.
- 2026-08-28: Section 8 defines required logical-source metadata outside the dashboard YAML, while Section 4.2 omits any YAML vocabulary for carrying that metadata inside a dashboard document. The current validator now accepts a conservative `data.source-metadata` structure so Section 8 metadata shape can be validated in-document, but the presenter-side runtime contract and the exact source of truth between YAML and external inputs remain ambiguous.
Expand All @@ -36,6 +37,14 @@

## Run log

### 2026-08-29 (built-in runs run-link coverage slice)

- Extended the Built-in pages milestone with a narrow `DLS-PAGE-006` validator increment for conservative run-link coverage on the `runs` built-in page.
- Updated `src/specification.js` so the implementation-local declarative definition for the `runs` built-in page now requires an `outcomes` source view exposing `run-link`, matching the most conservative reading of the Section 10 requirement to expose run links only when available.
- Expanded `test/unit/validator.test.js` with a negative `runs` built-in fixture that now fails when no declarative `outcomes` run-link coverage exists, and updated the positive built-in coverage fixture to include an `outcomes` view carrying `run-link`.
- Verified `npm install`, `npm run typecheck`, `npm run lint`, and `npm test`; `npm run test:e2e` remains blocked in this environment because the Playwright Chromium executable is not provisioned (`browserType.launch: Executable doesn't exist`).
- Next milestone: Built-in pages, next slice for conservative validation of `overview` findings-link and operational-value timeline coverage without inventing presenter semantics.

### 2026-08-28 (built-in data-state exposure slice)

- Extended the Built-in pages milestone with a narrow `DLS-PAGE-014` validator increment for independent availability, completeness, and freshness exposure on built-in pages.
Expand Down
5 changes: 3 additions & 2 deletions pages/dashboard/src/specification.js
Original file line number Diff line number Diff line change
Expand Up @@ -107,7 +107,7 @@ export const BUILT_IN_PAGE_REQUIRED_SOURCES = {
organizations: ['organizations', 'repositories', 'workflows', 'runs', 'usage'],
repositories: ['repositories', 'runs', 'usage', 'operational-values'],
workflows: ['workflows', 'runs', 'outcomes', 'usage', 'findings', 'operational-values'],
runs: ['runs'],
runs: ['runs', 'outcomes'],
experiments: ['experiments', 'experiment-assignments', 'grader-observations', 'eval-observations', 'outcomes', 'usage', 'operational-values'],
graders: ['graders', 'grader-observations'],
evals: ['evals', 'eval-observations'],
Expand Down Expand Up @@ -147,7 +147,8 @@ export const BUILT_IN_PAGE_REQUIRED_FIELDS = {
'operational-values': ['operational-value']
},
runs: {
runs: ['run', 'run-status', 'run-conclusion', 'organization', 'repository', 'workflow', 'rollout-mode', 'engine', 'requested-model', 'resolved-model', 'started-at']
runs: ['run', 'run-status', 'run-conclusion', 'organization', 'repository', 'workflow', 'rollout-mode', 'engine', 'requested-model', 'resolved-model', 'started-at'],
outcomes: ['run-link']
},
experiments: {
experiments: ['experiment'],
Expand Down
35 changes: 27 additions & 8 deletions pages/dashboard/test/unit/validator.test.js
Original file line number Diff line number Diff line change
Expand Up @@ -458,17 +458,24 @@ dashboard:

expect(result.ok).toBe(false);
if (!result.ok) {
expect(result.errors).toEqual([
expect.objectContaining({
code: 'DLS-E003',
path: '$.dashboard.pages[0].definition',
message: 'built-in page "runs" requires declarative definitions for source "runs".'
})
]);
expect(result.errors).toEqual(
expect.arrayContaining([
expect.objectContaining({
code: 'DLS-E003',
path: '$.dashboard.pages[0].definition',
message: 'built-in page "runs" requires declarative definitions for source "runs".'
}),
expect.objectContaining({
code: 'DLS-E003',
path: '$.dashboard.pages[0].definition',
message: 'built-in page "runs" requires declarative definitions for source "outcomes".'
})
])
);
}
});

it('DLS-PAGE-006 DLS-PAGE-014 rejects a runs built-in page definition that omits required run fields with DLS-E003', () => {
it('DLS-PAGE-006 DLS-PAGE-014 rejects a runs built-in page definition that omits required run fields and run links with DLS-E003', () => {
const result = validateDashboardDocument(`language-version: "0.1.0"
dashboard:
id: incomplete-runs-page
Expand Down Expand Up @@ -538,6 +545,11 @@ dashboard:
code: 'DLS-E003',
path: '$.dashboard.pages[0].definition.views',
message: 'built-in page "runs" definition must expose field "started-at" for source "runs".'
}),
expect.objectContaining({
code: 'DLS-E003',
path: '$.dashboard.pages[0].definition.views',
message: 'built-in page "runs" definition must include at least one view for source "outcomes".'
})
])
);
Expand Down Expand Up @@ -689,6 +701,13 @@ dashboard:
- field: requested-model
- field: resolved-model
- field: started-at
- id: run-links
data:
source: outcomes
mark: table
encoding:
columns:
- field: run-link
- id: usage
kind: built-in
page: usage
Expand Down