Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 0 additions & 2 deletions .npmrc
Original file line number Diff line number Diff line change
@@ -1,3 +1 @@
@poolsideai:registry=https://registry.npmjs.org/
@hugeicons-pro:registry=https://npm.hugeicons.com/
//npm.hugeicons.com/:_authToken=${HUGEICONS_TOKEN}
46 changes: 46 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,45 @@ Numbers count up, bars rise from the floor. Hover any column for the model name.

## Examples

### Base + thinking results

Pass `stacked` to coalesce a base model and its thinking variant into one column.
The thinking result is automatically 40% darker. The larger value stays above the
full bar, while the smaller value is shown inside the inset bar. Stacked charts
apply the same minimum width to every column so decimal labels remain readable.
Their tooltips put the model name on its own line, followed by color-keyed Base
and Thinking values.

```svelte
<PoolEval
data={[
{ value: 46.3, provider: 'poolside', label: 'Poolside Nova', highlight: true },
{ value: 57.9, provider: 'poolside', label: 'Poolside Nova Thinking' },
{ value: 55.4, provider: 'deepseek', label: 'DeepSeek V4' },
]}
stacked
max={100}
title="SWE-bench Pro"
/>
```

Matching is case-insensitive and recognizes these suffixes by default:
`Thinking`, `Reasoning`, `Reasoner`, and `Extended Thinking` (with spaces,
hyphens, underscores, or parentheses). Override them with `stackSuffixes`:

```svelte
<PoolEval {data} stacked stackSuffixes={['deliberate', 'thinking']} />
```

For model names that do not share a suffix convention, pair them explicitly:

```js
const data = [
{ value: 46, provider: 'acme', label: 'Acme Fast', stackKey: 'acme-v2', stackRole: 'base' },
{ value: 58, provider: 'acme', label: 'Acme Deep', stackKey: 'acme-v2', stackRole: 'thinking' },
];
```

### Monochromatic mode + theming

Pass `monochromatic` to drop the canonical brand colors and let icons inherit your theme. The highlighted icon picks up `--pe-icon-highlight` alongside its bar.
Expand Down Expand Up @@ -140,6 +179,8 @@ const data = [
| `monochromatic` | `boolean` | `false` | Use single-color icons that follow `--pe-icon-color` / `--pe-icon-highlight`. |
| `labels` | `boolean` | `false` | Show rotated text labels below provider icons. |
| `counts` | `boolean` | `true` | Show the numeric value above each bar. |
| `stacked` | `boolean` | `false` | Coalesce matching base/thinking results into one column. |
| `stackSuffixes` | `string[]` | thinking/reasoning defaults | Suffixes used to infer thinking variants when `stacked` is enabled. |
| `chartHeight` | `number` | `150` | Bar lane height in pixels. |
| `providerIcons` | `Record<string, Component>` | `{}` | Custom Svelte components keyed by provider. Used before built-in SVG lookup. |
| `title` | `string` | `"Model evaluation chart"` | Accessible title. |
Expand All @@ -159,6 +200,8 @@ type Datum = {
label?: string; // model name shown in the tooltip and rotated label
highlight?: boolean; // marks this row as the highlighted (accent) bar
unit?: string; // appended to the count, e.g. "ms" or "%"
stackKey?: string; // explicit pairing key for a base/thinking result
stackRole?: 'base' | 'thinking'; // explicit role within the pair
};
```

Expand Down Expand Up @@ -188,6 +231,9 @@ All visual controls go through CSS variables on the root element. Pass them eith
|---|---|---|
| `bar-color` | `#d6d6d6` | Default bar fill. |
| `bar-highlight` | `#7c3aed` | Highlighted bar fill and focus ring. |
| `stack-color` | 40% darker `bar-color` | Thinking variant fill. |
| `stack-highlight` | 40% darker `bar-highlight` | Highlighted thinking variant fill. |
| `stack-count-color` | `#fff` | Value label inside the inset bar. |
| `bar-width` | `22px` | Width of each bar column. |
| `bar-radius` | `0px` | Corner radius for bars. |
| `bar-gap` | `18px` | Space between bar columns. |
Expand Down
2 changes: 1 addition & 1 deletion package.json
Original file line number Diff line number Diff line change
Expand Up @@ -66,7 +66,7 @@
"@chenglou/pretext": "^0.0.6"
},
"devDependencies": {
"@hugeicons-pro/core-stroke-standard": "^4.1.0",
"@hugeicons/core-free-icons": "^3.1.1",
"@pierre/diffs": "^1.1.19",
"@sveltejs/package": "^2.3.7",
"@sveltejs/vite-plugin-svelte": "^5.0.3",
Expand Down
12 changes: 6 additions & 6 deletions pnpm-lock.yaml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

133 changes: 118 additions & 15 deletions src/HomePage.svelte
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@
import {
SparklesIcon,
Chart01Icon,
ChartColumnBigIcon,
ChartColumnBig as ChartColumnBigIcon,
HashtagIcon,
Tag01Icon,
TextFontIcon,
Expand All @@ -30,7 +30,7 @@
Tick02Icon,
Download01Icon,
File01Icon,
} from '@hugeicons-pro/core-stroke-standard';
} from '@hugeicons/core-free-icons';

// Beautiful eval charts — illustrative numbers
const sweBench = [
Expand Down Expand Up @@ -84,6 +84,15 @@
{ value: 80, provider: 'gemini', label: 'Atlas 2.5 Pro' },
];

const thinkingComparison = [
{ value: 46.3, provider: 'poolside', label: 'Poolside Nova', highlight: true },
{ value: 57.9, provider: 'poolside', label: 'Poolside Nova Thinking' },
{ value: 55.4, provider: 'deepseek', label: 'DeepSeek V4' },
{ value: 57.5, provider: 'gemini', label: 'Gemini 2.5 Pro' },
{ value: 60.6, provider: 'openai', label: 'GPT-5' },
{ value: 62.1, provider: 'grok', label: 'Grok 5' },
];

const providerIcons = {
acme: AcmeProviderIcon,
};
Expand Down Expand Up @@ -147,6 +156,31 @@

<PoolEval {data} {providerIcons} max={100} title="Custom provider icon" />`;

const thinkingCode = `<script>
import PoolEval from '@poolsideai/pooleval';

const data = [
{ value: 46.3, provider: 'poolside', label: 'Poolside Nova', highlight: true },
{ value: 57.9, provider: 'poolside', label: 'Poolside Nova Thinking' },
{ value: 55.4, provider: 'deepseek', label: 'DeepSeek V4' },
{ value: 57.5, provider: 'gemini', label: 'Gemini 2.5 Pro' },
{ value: 60.6, provider: 'openai', label: 'GPT-5' },
{ value: 62.1, provider: 'grok', label: 'Grok 5' },
];
<\/script>

<PoolEval
{data}
stacked
max={100}
title="SWE-bench Pro — base + thinking"
theme={{
'bar-highlight': '#2455ff',
'text-highlight': '#2455ff',
'bar-radius': '4px',
}}
/>`;

const latencyCode = `<PoolEval
data={[
{ value: 142, provider: 'claude', label: 'Claude Haiku 4.5', highlight: true, unit: 'ms' },
Expand All @@ -163,10 +197,18 @@
/>`;

const sandboxPresets = {
thinking: {
title: 'SWE-bench Pro',
max: 100,
unit: '',
stacked: true,
rows: thinkingComparison.map((row) => ({ ...row, enabled: true })),
},
quality: {
title: 'Live Benchmark',
max: 100,
unit: '',
stacked: false,
rows: [
{ value: 76, provider: 'poolside', label: 'Poolside Nova', enabled: true },
{ value: 71, provider: 'openai', label: 'GPT-5', enabled: true },
Expand All @@ -184,6 +226,7 @@
title: 'Latency test',
max: 400,
unit: 'ms',
stacked: false,
rows: [
{ value: 135, provider: 'claude', label: 'Claude Haiku 4.5', enabled: true },
{ value: 166, provider: 'openai', label: 'GPT-5 mini', enabled: true },
Expand All @@ -203,12 +246,13 @@
return sandboxPresets[key].rows.map((row) => ({ ...row }));
}

let sandboxPreset = $state('quality');
let sandboxTitle = $state(sandboxPresets.quality.title);
let sandboxMax = $state(sandboxPresets.quality.max);
let sandboxUnit = $state(sandboxPresets.quality.unit);
let sandboxRows = $state(clonePresetRows('quality'));
let sandboxHighlight = $state('poolside');
let sandboxPreset = $state('thinking');
let sandboxTitle = $state(sandboxPresets.thinking.title);
let sandboxMax = $state(sandboxPresets.thinking.max);
let sandboxUnit = $state(sandboxPresets.thinking.unit);
let sandboxRows = $state(clonePresetRows('thinking'));
let sandboxHighlight = $state(sandboxPresets.thinking.rows[0].label);
let sandboxStacked = $state(true);
let sandboxMonochrome = $state(false);
let sandboxAnimate = $state(true);
let sandboxDuration = $state(900);
Expand All @@ -219,7 +263,7 @@
let sandboxHeight = $state(170);
let sandboxBarWidth = $state(22);
let sandboxBarGap = $state(18);
let sandboxRadius = $state(2);
let sandboxRadius = $state(4);
let sandboxIconSize = $state(20);
let sandboxLineWidth = $state(1);
let sandboxLinePadding = $state(10);
Expand Down Expand Up @@ -315,7 +359,7 @@
...row,
value: Number(row.value) || 0,
unit: sandboxUnit,
highlight: row.provider === sandboxHighlight,
highlight: row.label === sandboxHighlight,
}))
);

Expand Down Expand Up @@ -399,6 +443,9 @@
// Bars
{ name: 'bar-color', default: '#d6d6d6', description: 'Default bar fill.', category: 'Bars' },
{ name: 'bar-highlight', default: '#7c3aed', description: 'Highlighted bar fill and focus ring.', category: 'Bars' },
{ name: 'stack-color', default: 'bar-color −40%', description: 'Thinking variant fill; defaults to a 40% darker bar color.', category: 'Bars' },
{ name: 'stack-highlight', default: 'bar-highlight −40%', description: 'Highlighted thinking variant fill.', category: 'Bars' },
{ name: 'stack-count-color', default: '#fff', description: 'Count shown inside the inset stacked bar.', category: 'Bars' },
{ name: 'bar-width', default: '22px', description: 'Width of each bar column.', category: 'Bars' },
{ name: 'bar-radius', default: '0px', description: 'Corner radius for bars.', category: 'Bars' },
{ name: 'bar-gap', default: '18px', description: 'Space between bar columns.', category: 'Bars' },
Expand Down Expand Up @@ -487,7 +534,8 @@
sandboxMax = preset.max;
sandboxUnit = preset.unit;
sandboxRows = clonePresetRows(sandboxPreset);
sandboxHighlight = preset.rows[0].provider;
sandboxHighlight = preset.rows[0].label;
sandboxStacked = !!preset.stacked;
}

const panelBg = 'bg-[#f6f6f5] dark:bg-[#161616]';
Expand All @@ -503,6 +551,7 @@
];

const presetOptions = [
{ value: 'thinking', label: 'Thinking vs Base' },
{ value: 'quality', label: 'Quality Benchmark' },
{ value: 'latency', label: 'Latency Benchmark' },
];
Expand Down Expand Up @@ -581,6 +630,7 @@
if (sandboxMonochrome) props.push(` monochromatic`);
if (sandboxLabels) props.push(` labels`);
if (!sandboxCounts) props.push(` counts={false}`);
if (sandboxStacked) props.push(` stacked`);

const themeKeys = Object.keys(sandboxTheme);
if (themeKeys.length) props.push(` theme={${indentObject(sandboxTheme, 2)}}`);
Expand Down Expand Up @@ -841,6 +891,47 @@ ${props.join('\n')}

<section class="grid grid-cols-[40px_minmax(0,1fr)] gap-x-6 gap-y-8 items-start max-[560px]:grid-cols-1">
<div class="mono text-muted text-[13px] pt-1.5">06</div>
<div>
<h2 class="text-2xl tracking-[-0.01em] m-0 mb-2 font-semibold">Base + thinking, together</h2>
<p class="text-muted m-0 max-w-[560px]">
Pass <code>stacked</code> to pair base and thinking variants in one column.
The thinking result is 40% darker, while both scores stay visible. Hover
the pair for color-keyed Base and Thinking values beneath the model name.
</p>
<p class="text-muted m-0 mt-3 max-w-[680px]">
Matching is provider-scoped and case-insensitive. A base name pairs with
<code>Thinking</code>, <code>Reasoning</code>, <code>Reasoner</code>, or
<code>Extended Thinking</code> suffixes, including hyphens, underscores,
and parentheses. For irregular names, set <code>stackKey</code> and
<code>stackRole</code> explicitly.
</p>
</div>
<div class="col-start-2 grid grid-cols-[minmax(260px,0.9fr)_minmax(0,1.1fr)] gap-[22px] items-stretch mt-2 max-[560px]:col-start-1 max-[560px]:grid-cols-1">
<div class="min-w-0 pt-6 px-1 pb-2">
<PoolEval
data={thinkingComparison}
stacked
max={100}
title="SWE-bench Pro — base + thinking"
theme={{
'bar-highlight': '#2455ff',
'text-highlight': '#2455ff',
'bar-radius': '4px',
}}
/>
</div>
<CodeFile
code={thinkingCode}
filename="thinking.svelte"
ariaLabel="Base and thinking PoolEval code example"
/>
</div>
</section>

<hr class="border-0 border-t border-dashed border-rule my-20" />

<section class="grid grid-cols-[40px_minmax(0,1fr)] gap-x-6 gap-y-8 items-start max-[560px]:grid-cols-1">
<div class="mono text-muted text-[13px] pt-1.5">07</div>
<div>
<h2 class="text-2xl tracking-[-0.01em] m-0 mb-2 font-semibold">Interactive sandbox</h2>
<p class="text-muted m-0 max-w-[520px]">
Expand Down Expand Up @@ -940,6 +1031,7 @@ ${props.join('\n')}
monochromatic={sandboxMonochrome}
labels={sandboxLabels}
counts={sandboxCounts}
stacked={sandboxStacked}
chartHeight={sandboxHeight}
title={sandboxTitle}
theme={sandboxTheme}
Expand Down Expand Up @@ -1138,6 +1230,17 @@ ${props.join('\n')}
{:else if sandboxTab === 'data'}
<SandboxSection icon={DatabaseIcon} title="Type">
<SandboxSelect label="Preset" token="prop.data" bind:value={sandboxPreset} options={presetOptions} onchange={loadSandboxPreset} />
<SandboxTile
bind:enabled={sandboxStacked}
label="Coalesce thinking variants"
token="prop.stacked"
/>
{#if sandboxStacked}
<p class="m-0 text-[11px] leading-relaxed text-muted">
Within each provider, matches Thinking, Reasoning, Reasoner, and
Extended Thinking suffixes. Use stackKey/stackRole for custom names.
</p>
{/if}
</SandboxSection>

<SandboxSection icon={ArrowExpand01Icon} title="Metrics">
Expand All @@ -1147,16 +1250,16 @@ ${props.join('\n')}
<SandboxSection icon={AppleIntelligenceIcon} title="Models">
<SandboxSelect label="Highlighted Model" token="data[].highlight" bind:value={sandboxHighlight}>
{#snippet children()}
{#each sandboxRows.filter((row) => row.enabled !== false) as row (row.provider)}
<option value={row.provider}>{row.label}</option>
{#each sandboxRows.filter((row) => row.enabled !== false) as row (row.label)}
<option value={row.label}>{row.label}</option>
{/each}
{/snippet}
</SandboxSelect>

<div class="grid gap-1.5 text-xs">
<span class="text-fg font-medium">Values</span>
<div class="grid gap-1.5 text-xs text-muted">
{#each sandboxRows as row (row.provider)}
{#each sandboxRows as row (row.label)}
<SandboxTile
bind:enabled={row.enabled}
label={row.label}
Expand Down Expand Up @@ -1191,7 +1294,7 @@ ${props.join('\n')}
<hr class="border-0 border-t border-dashed border-rule my-20" />

<section class="grid grid-cols-[40px_1fr] gap-x-6 gap-y-8 items-start max-[560px]:grid-cols-1">
<div class="mono text-muted text-[13px] pt-1.5">07</div>
<div class="mono text-muted text-[13px] pt-1.5">08</div>
<div>
<h2 class="text-2xl tracking-[-0.01em] m-0 mb-2 font-semibold">Design tokens</h2>
<p class="text-muted m-0 max-w-[520px]">
Expand Down
2 changes: 1 addition & 1 deletion src/docs-ui/SandboxSelect.svelte
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
</script>

<script>
import { ArrowDown01Icon } from '@hugeicons-pro/core-stroke-standard';
import { ArrowDown01Icon } from '@hugeicons/core-free-icons';
import { renderIcon } from './render-icon.js';

let {
Expand Down
Loading