diff --git a/CLAUDE.md b/CLAUDE.md index 7bf4299..099ea6c 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -24,14 +24,18 @@ uv pip install -e . ### Running Tests ```bash -# Run all tests -pytest +# Run all unit tests (integration tests are deselected by default) +uv run pytest -# Run specific test file -pytest tests/api/test_something.py +# Run a specific test file +uv run pytest tests/payloads/test_execution_requests.py # Run with verbose output -pytest -v +uv run pytest -v + +# Run the integration tests that hit live EL/Beacon endpoints +# (override endpoints with EXPB_TEST_RPC_URL / EXPB_TEST_BEACON_URL) +uv run pytest -m integration ``` ## Core Architecture @@ -40,7 +44,7 @@ pytest -v The application uses [Typer](https://typer.tiangolo.com/) for the CLI interface. The main entry point is in [src/expb/\_\_init\_\_.py](src/expb/__init__.py), which aggregates sub-commands from: -- `generate_payloads` - Extracts historical payloads from an Ethereum RPC endpoint +- `generate_payloads` - Reconstructs the exact `engine_newPayload` / `engine_forkchoiceUpdated` requests for a block range by sourcing them from a Consensus client's Beacon API (execution RPC is used only for block→slot mapping) - `execute_scenario` - Runs a single benchmark scenario - `execute_scenarios` - Runs multiple benchmark scenarios (optionally in a loop) - `compress_payloads` - Compresses multiple smaller payloads into larger blocks @@ -142,18 +146,24 @@ Per-payload metrics can be enabled with `--per-payload-metrics` flag, generating ### Generate Payloads -Extract historical Ethereum payloads from an RPC endpoint: +Reconstruct the Engine API requests for a block range from a Consensus (Beacon) API and an +execution RPC. Both endpoints must serve the requested range (archive node for older ranges): ```bash expb generate-payloads \ - --rpc-url https://eth-mainnet.g.alchemy.com/v2/YOUR_KEY \ + --rpc-url http://localhost:8545 \ + --beacon-url http://localhost:5052 \ --start-block 19000000 \ --end-block 19001000 \ --output-dir ./payloads \ - --threads 10 \ - --workers 30 + --threads 10 ``` +Payloads are sourced from the beacon block, so the generated `ExecutionPayload`, +`executionRequests` (EIP-7685), blob versioned hashes and parent beacon block root exactly match +what the consensus client sends the execution client, across all forks through Osaka. The +`forkchoiceUpdated` requests set `safeBlockHash` and `finalizedBlockHash` to the parent block hash. + ### Run Single Scenario Execute a benchmark scenario defined in a config file: diff --git a/README.md b/README.md index 28ff286..500c54a 100644 --- a/README.md +++ b/README.md @@ -19,6 +19,28 @@ uv tool install --from git+https://github.com/NethermindEth/execution-payloads-b 2. Edit the configuration file. 3. Execute one or multiple scenarios. +### Generating Payloads + +Benchmarks replay the `engine_newPayload` / `engine_forkchoiceUpdated` requests that a consensus +client sends to an execution client. Generate them for a block range from a Consensus (Beacon) API +and an execution RPC: + +```bash +expb generate-payloads \ + --rpc-url http://localhost:8545 \ + --beacon-url http://localhost:5052 \ + --network mainnet \ + --start-block 21000000 \ + --end-block 21001000 \ + --output-dir ./payloads +``` + +The requests are sourced from the beacon block, so the generated payloads, execution requests +(EIP-7685), blob versioned hashes and parent beacon block root match exactly what the consensus +client sends the execution client — across all forks through Osaka. Both endpoints must serve the +requested range (an archive node is required for older ranges). This produces `payloads.jsonl` and +`fcus.jsonl`, consumed by scenario execution and by `send-payloads`. + ### Scenarios Execution #### Single Scenario diff --git a/docs/USAGE.md b/docs/USAGE.md index 014d359..9e29973 100644 --- a/docs/USAGE.md +++ b/docs/USAGE.md @@ -22,7 +22,16 @@ expb [OPTIONS] COMMAND [ARGS]... ## `expb generate-payloads` -Generate execution payloads for a given block range. +Generate `engine_newPayload` / `engine_forkchoiceUpdated` requests for a given block range. + +The requests are sourced from the Consensus client's Beacon API so the generated +`ExecutionPayload`, `executionRequests` (EIP-7685), blob versioned hashes and parent beacon +block root exactly match what a consensus client sends the execution client — including +post-Deneb forks (Prague/Electra, Osaka). The execution RPC is used only to map block numbers +to beacon slots and to resolve the latest block. + +Both endpoints must serve the requested block range (an archive node is required for older +ranges). **Usage**: @@ -32,7 +41,8 @@ expb generate-payloads [OPTIONS] **Options**: -* `--rpc-url TEXT`: Ethereum RPC URL [required] +* `--rpc-url TEXT`: Ethereum execution RPC URL [required] +* `--beacon-url TEXT`: Ethereum consensus (Beacon) API URL [required] * `--network [mainnet]`: Network [default: mainnet] * `--start-block INTEGER`: Start block [default: 0] * `--end-block INTEGER`: End block @@ -40,9 +50,20 @@ expb generate-payloads [OPTIONS] * `--join-payloads / --no-join-payloads`: Join payloads and FCUs into a single file (payloads.jsonl and fcus.jsonl) [default: join-payloads] * `--log-level TEXT`: Log level (e.g., DEBUG, INFO, WARNING) [default: INFO] * `--threads INTEGER`: Number of threads for parallel processing [default: 10] -* `--workers INTEGER`: Number of workers per thread for parallel processing [default: 30] * `--help`: Show this message and exit. +**Example**: + +```bash +expb generate-payloads \ + --rpc-url http://localhost:8545 \ + --beacon-url http://localhost:5052 \ + --network mainnet \ + --start-block 21000000 \ + --end-block 21001000 \ + --output-dir ./payloads +``` + ## `expb execute-scenario` Execute payloads for a given execution client using Grafana K6. diff --git a/example-expb.yaml b/example-expb.yaml index 59cd4a5..d813b24 100644 --- a/example-expb.yaml +++ b/example-expb.yaml @@ -13,7 +13,7 @@ pull_images: true # Docker images to use (pin to specific versions for reproducible benchmarks) images: - k6: grafana/k6:1.6.1 + k6: grafana/k6:2.1.0 alloy: grafana/alloy:v1.13.2 payload_server: python:3.12-slim diff --git a/grafana/dashboard.json b/grafana/dashboard.json index daa7bcf..3e0da09 100644 --- a/grafana/dashboard.json +++ b/grafana/dashboard.json @@ -1,3540 +1,4529 @@ { - "annotations": { - "list": [ - { - "builtIn": 1, - "datasource": { - "type": "grafana", - "uid": "-- Grafana --" - }, - "enable": true, - "hide": true, - "iconColor": "rgba(0, 211, 255, 1)", - "name": "Annotations & Alerts", - "type": "dashboard" - } - ] - }, - "editable": true, - "fiscalYearStartMonth": 0, - "graphTooltip": 2, - "id": 30, - "links": [], - "panels": [ - { - "datasource": { - "type": "prometheus", - "uid": "prometheus_ds_1" - }, - "fieldConfig": { - "defaults": { - "color": { - "mode": "thresholds" - }, - "custom": { - "align": "left", - "cellOptions": { - "type": "auto", - "wrapText": false - }, - "filterable": true, - "inspect": false - }, - "fieldMinMax": false, - "mappings": [], - "thresholds": { - "mode": "absolute", - "steps": [ - { - "color": "green" - } - ] - } - }, - "overrides": [ - { - "matcher": { - "id": "byRegexp", - "options": "/avg|p90|p95|p99|med|max|min/" - }, - "properties": [ - { - "id": "unit", - "value": "s" - }, - { - "id": "custom.width", - "value": 80 - } - ] - }, - { - "matcher": { - "id": "byName", - "options": "Client" - }, - "properties": [ - { - "id": "custom.width", - "value": 100 - } - ] - }, - { - "matcher": { - "id": "byName", - "options": "Method" - }, - "properties": [ - { - "id": "custom.width", - "value": 220 - } - ] - } - ] - }, - "gridPos": { - "h": 11, - "w": 24, - "x": 0, - "y": 0 - }, - "id": 10, - "options": { - "cellHeight": "sm", - "footer": { - "countRows": false, - "fields": "", - "reducer": [ - "sum" - ], - "show": false - }, - "showHeader": true, - "sortBy": [ - { - "desc": true, - "displayName": "avg" - } - ] - }, - "pluginVersion": "11.6.0", - "targets": [ - { - "datasource": { - "type": "prometheus", - "uid": "prometheus_ds_1" - }, - "editorMode": "code", - "expr": "k6_http_req_duration_p90{tool=\"expb\", testid=~\"$testid\", name!=\"payload_fetch\", group!~\"::setup::.*\"}", - "format": "table", - "hide": false, - "instant": false, - "legendFormat": "__auto", - "range": true, - "refId": "A" - }, - { - "datasource": { - "type": "prometheus", - "uid": "prometheus_ds_1" - }, - "editorMode": "code", - "expr": "k6_http_req_duration_p95{tool=\"expb\", testid=~\"$testid\", name!=\"payload_fetch\", group!~\"::setup::.*\"}", - "format": "table", - "hide": false, - "instant": false, - "legendFormat": "__auto", - "range": true, - "refId": "B" - }, - { - "datasource": { - "type": "prometheus", - "uid": "prometheus_ds_1" - }, - "editorMode": "code", - "expr": "k6_http_req_duration_p99{tool=\"expb\", testid=~\"$testid\", name!=\"payload_fetch\", group!~\"::setup::.*\"}", - "format": "table", - "hide": false, - "instant": false, - "legendFormat": "__auto", - "range": true, - "refId": "C" - }, - { - "datasource": { - "type": "prometheus", - "uid": "prometheus_ds_1" - }, - "editorMode": "code", - "expr": "k6_http_req_duration_med{tool=\"expb\", testid=~\"$testid\", name!=\"payload_fetch\", group!~\"::setup::.*\"}", - "format": "table", - "hide": false, - "instant": false, - "legendFormat": "__auto", - "range": true, - "refId": "D" - }, - { - "datasource": { - "type": "prometheus", - "uid": "prometheus_ds_1" - }, - "editorMode": "code", - "expr": "k6_http_req_duration_max{tool=\"expb\", testid=~\"$testid\", name!=\"payload_fetch\", group!~\"::setup::.*\"}", - "format": "table", - "hide": false, - "instant": false, - "legendFormat": "__auto", - "range": true, - "refId": "E" - }, - { - "datasource": { - "type": "prometheus", - "uid": "prometheus_ds_1" - }, - "editorMode": "code", - "expr": "k6_http_req_duration_min{tool=\"expb\", testid=~\"$testid\", name!=\"payload_fetch\", group!~\"::setup::.*\"}", - "format": "table", - "hide": false, - "instant": false, - "legendFormat": "__auto", - "range": true, - "refId": "F" - }, - { - "datasource": { - "type": "prometheus", - "uid": "prometheus_ds_1" - }, - "editorMode": "code", - "expr": "k6_http_req_duration_avg{tool=\"expb\", testid=~\"$testid\", name!=\"payload_fetch\", group!~\"::setup::.*\"}", - "format": "table", - "hide": false, - "instant": false, - "legendFormat": "__auto", - "range": true, - "refId": "G" - } - ], - "title": "Performance Summary", - "transformations": [ - { - "id": "merge", - "options": {} - }, - { - "id": "groupBy", - "options": { - "fields": { - "Time": { - "aggregations": [ - "lastNotNull" - ], - "operation": "aggregate" - }, - "Value #A": { - "aggregations": [ - "lastNotNull" - ], - "operation": "aggregate" - }, - "Value #B": { - "aggregations": [ - "lastNotNull" - ], - "operation": "aggregate" - }, - "Value #C": { - "aggregations": [ - "lastNotNull" - ], - "operation": "aggregate" - }, - "Value #D": { - "aggregations": [ - "lastNotNull" - ], - "operation": "aggregate" - }, - "Value #E": { - "aggregations": [ - "lastNotNull" - ], - "operation": "aggregate" - }, - "Value #F": { - "aggregations": [ - "lastNotNull" - ], - "operation": "aggregate" - }, - "Value #G": { - "aggregations": [ - "lastNotNull" - ], - "operation": "aggregate" - }, - "client_type": { - "aggregations": [], - "operation": "groupby" - }, - "group": { - "aggregations": [] - }, - "jrpc_method": { - "aggregations": [], - "operation": "groupby" - }, - "testid": { - "aggregations": [], - "operation": "groupby" - } - } - } - }, - { - "id": "organize", - "options": { - "excludeByName": {}, - "includeByName": {}, - "indexByName": { - "Value #A (lastNotNull)": 4, - "Value #B (lastNotNull)": 5, - "Value #C (lastNotNull)": 6, - "Value #D (lastNotNull)": 7, - "Value #E (lastNotNull)": 8, - "Value #F (lastNotNull)": 9, - "Value #G (lastNotNull)": 3, - "client_type": 1, - "jrpc_method": 2, - "testid": 0 - }, - "renameByName": { - "Time (lastNotNull)": "Time", - "Value #A (lastNotNull)": "p90", - "Value #B (lastNotNull)": "p95", - "Value #C (lastNotNull)": "p99", - "Value #D (lastNotNull)": "med", - "Value #E (lastNotNull)": "max", - "Value #F (lastNotNull)": "min", - "Value #G (lastNotNull)": "avg", - "client_type": "Client", - "jrpc_method": "Method", - "testid": "TestID" - } - } - }, - { - "id": "sortBy", - "options": { - "fields": {}, - "sort": [ - { - "desc": true, - "field": "Method" - } - ] - } - } - ], - "type": "table" - }, + "annotations": [ { - "datasource": { - "type": "prometheus", - "uid": "prometheus_ds_1" - }, - "fieldConfig": { - "defaults": { - "color": { - "mode": "palette-classic" - }, - "custom": { - "axisBorderShow": false, - "axisCenteredZero": false, - "axisColorMode": "text", - "axisLabel": "", - "axisPlacement": "auto", - "barAlignment": 0, - "barWidthFactor": 0.6, - "drawStyle": "line", - "fillOpacity": 0, - "gradientMode": "none", - "hideFrom": { - "legend": false, - "tooltip": false, - "viz": false - }, - "insertNulls": false, - "lineInterpolation": "linear", - "lineWidth": 1, - "pointSize": 5, - "scaleDistribution": { - "type": "linear" - }, - "showPoints": "auto", - "spanNulls": false, - "stacking": { - "group": "A", - "mode": "none" - }, - "thresholdsStyle": { - "mode": "off" - } - }, - "mappings": [], - "thresholds": { - "mode": "absolute", - "steps": [ - { - "color": "green" - } - ] - } - }, - "overrides": [ - { - "matcher": { - "id": "byName", - "options": "http_req_s_errors" - }, - "properties": [ - { - "id": "color", - "value": { - "fixedColor": "red", - "mode": "fixed" - } - }, - { - "id": "custom.lineStyle", - "value": { - "dash": [ - 10, - 10 - ], - "fill": "dash" - } - }, - { - "id": "custom.axisPlacement", - "value": "right" - }, - { - "id": "unit", - "value": "reqps" - } - ] - }, - { - "matcher": { - "id": "byName", - "options": "http_req_s" - }, - "properties": [ - { - "id": "unit", - "value": "reqps" - }, - { - "id": "custom.axisPlacement", - "value": "right" - }, - { - "id": "color", - "value": { - "fixedColor": "yellow", - "mode": "fixed" - } - }, - { - "id": "custom.lineStyle", - "value": { - "dash": [ - 10, - 10 - ], - "fill": "dash" - } - } - ] - }, - { - "matcher": { - "id": "byName", - "options": "vus" - }, - "properties": [ - { - "id": "color", - "value": { - "mode": "fixed" - } - }, - { - "id": "unit", - "value": "VUs" - } - ] - }, - { - "matcher": { - "id": "byRegexp", - "options": "http_req_duration_[a-zA-Z0-9_]+" - }, - "properties": [ - { - "id": "unit", - "value": "s" - }, - { - "id": "custom.axisPlacement", - "value": "right" - }, - { - "id": "color", - "value": { - "fixedColor": "blue", - "mode": "fixed" - } - } - ] - } - ] - }, - "gridPos": { - "h": 11, - "w": 24, - "x": 0, - "y": 11 - }, - "id": 29, - "options": { - "legend": { - "calcs": [ - "mean" - ], - "displayMode": "list", - "placement": "bottom", - "showLegend": true - }, - "tooltip": { - "hideZeros": false, - "mode": "multi", - "sort": "none" - } - }, - "pluginVersion": "11.6.0", - "targets": [ - { - "datasource": { - "type": "prometheus", - "uid": "prometheus_ds_1" - }, - "editorMode": "code", - "expr": "sum by (testid) (k6_vus{tool=\"expb\", testid=~\"$testid\", name!=\"payload_fetch\", group!~\"::setup::.*\"})", - "hide": true, - "instant": false, - "legendFormat": "vus - {{testid}}", - "range": true, - "refId": "A" - }, - { - "datasource": { - "type": "prometheus", - "uid": "prometheus_ds_1" - }, - "editorMode": "code", - "expr": "avg by (testid) (k6_iteration_duration_$quantile_stat{tool=\"expb\", testid=~\"$testid\", name!=\"payload_fetch\", group!~\"::setup::.*\"})", - "hide": false, - "instant": false, - "legendFormat": "http_req_duration_$quantile_stat - {{testid}}", - "range": true, - "refId": "C" - }, - { - "datasource": { - "type": "prometheus", - "uid": "prometheus_ds_1" - }, - "editorMode": "code", - "expr": "sum by (testid) (irate(k6_http_reqs_total{tool=\"expb\", testid=~\"$testid\", name!=\"payload_fetch\", group!~\"::setup::.*\"}[$__rate_interval]))", - "hide": false, - "instant": false, - "legendFormat": "http_req_s - {{testid}}", - "range": true, - "refId": "B" - }, - { - "datasource": { - "type": "prometheus", - "uid": "prometheus_ds_1" - }, - "editorMode": "code", - "expr": "avg by (testid) (round(k6_http_req_failed_rate{tool=\"expb\", testid=~\"$testid\", name!=\"payload_fetch\", group!~\"::setup::.*\"}, 0.1)*100)", - "hide": true, - "instant": false, - "legendFormat": "http_req_failed - {{testid}}", - "range": true, - "refId": "E" - }, - { - "datasource": { - "type": "prometheus", - "uid": "prometheus_ds_1" - }, - "editorMode": "code", - "expr": "sum by (testid) (irate(k6_http_reqs_total{tool=\"expb\", testid=~\"$testid\", expected_response=\"false\", name!=\"payload_fetch\", group!~\"::setup::.*\"}[$__rate_interval]))", - "hide": false, - "instant": false, - "legendFormat": "http_req_s_errors - {{testid}}", - "range": true, - "refId": "D" - } - ], - "title": "Performance Overview", - "type": "timeseries" - }, - { - "collapsed": false, - "gridPos": { - "h": 1, - "w": 24, - "x": 0, - "y": 22 - }, - "id": 1, - "panels": [], - "title": "Performance Overview", - "type": "row" - }, - { - "datasource": { - "type": "prometheus", - "uid": "prometheus_ds_1" - }, - "fieldConfig": { - "defaults": { - "color": { - "mode": "thresholds" - }, - "mappings": [], - "thresholds": { - "mode": "absolute", - "steps": [ - { - "color": "green" - } - ] - }, - "unit": "none" - }, - "overrides": [] - }, - "gridPos": { - "h": 5, - "w": 12, - "x": 0, - "y": 23 - }, - "id": 4, - "options": { - "colorMode": "value", - "graphMode": "none", - "justifyMode": "auto", - "orientation": "auto", - "percentChangeColorMode": "standard", - "reduceOptions": { - "calcs": [], - "fields": "", - "values": false - }, - "showPercentChange": false, - "textMode": "auto", - "wideLayout": true - }, - "pluginVersion": "11.6.0", - "targets": [ - { - "datasource": { - "type": "prometheus", - "uid": "prometheus_ds_1" - }, - "disableTextWrap": false, - "editorMode": "code", - "expr": "sum by(testid) (k6_iterations_total{tool=\"expb\", testid=~\"$testid\", name!=\"payload_fetch\", group!~\"::setup::.*\"})", - "fullMetaSearch": false, - "includeNullMetadata": true, - "instant": false, - "legendFormat": "{{testid}}", - "range": true, - "refId": "A", - "useBackend": false - } - ], - "title": "Iterations", - "type": "stat" - }, - { - "datasource": { - "type": "prometheus", - "uid": "prometheus_ds_1" - }, - "fieldConfig": { - "defaults": { - "color": { - "mode": "thresholds" - }, - "mappings": [], - "thresholds": { - "mode": "absolute", - "steps": [ - { - "color": "green" - } - ] - }, - "unit": "blocks/s" - }, - "overrides": [] - }, - "gridPos": { - "h": 5, - "w": 12, - "x": 12, - "y": 23 - }, - "id": 20, - "options": { - "colorMode": "value", - "graphMode": "none", - "justifyMode": "auto", - "orientation": "auto", - "percentChangeColorMode": "standard", - "reduceOptions": { - "calcs": [], - "fields": "", - "values": false - }, - "showPercentChange": false, - "textMode": "auto", - "wideLayout": true - }, - "pluginVersion": "11.6.0", - "targets": [ - { - "datasource": { - "type": "prometheus", - "uid": "prometheus_ds_1" - }, - "disableTextWrap": false, - "editorMode": "code", - "expr": "sum by(testid) (irate(k6_iterations_total{tool=\"expb\", testid=~\"$testid\", name!=\"payload_fetch\", group!~\"::setup::.*\"}[$__rate_interval]))", - "fullMetaSearch": false, - "includeNullMetadata": true, - "instant": false, - "interval": "", - "legendFormat": "{{testid}}", - "range": true, - "refId": "A", - "useBackend": false - } - ], - "title": "Blocks per second", - "type": "stat" - }, - { - "datasource": { - "type": "prometheus", - "uid": "prometheus_ds_1" - }, - "fieldConfig": { - "defaults": { - "color": { - "fixedColor": "red", - "mode": "fixed" - }, - "mappings": [], - "thresholds": { - "mode": "absolute", - "steps": [ - { - "color": "green" - } - ] - }, - "unit": "none" - }, - "overrides": [] - }, - "gridPos": { - "h": 5, - "w": 12, - "x": 0, - "y": 28 - }, - "id": 22, - "options": { - "colorMode": "value", - "graphMode": "none", - "justifyMode": "auto", - "orientation": "auto", - "percentChangeColorMode": "standard", - "reduceOptions": { - "calcs": [], - "fields": "", - "values": false - }, - "showPercentChange": false, - "textMode": "auto", - "wideLayout": true - }, - "pluginVersion": "11.6.0", - "targets": [ - { - "datasource": { - "type": "prometheus", - "uid": "prometheus_ds_1" - }, - "editorMode": "code", - "expr": "sum by (testid) (k6_http_reqs_total{tool=\"expb\", testid=~\"$testid\", expected_response=\"false\", name!=\"payload_fetch\", group!~\"::setup::.*\"})", - "instant": false, - "legendFormat": "{{testid}}", - "range": true, - "refId": "A" - } - ], - "title": "HTTP request failures", - "type": "stat" - }, - { - "datasource": { - "type": "prometheus", - "uid": "prometheus_ds_1" - }, - "description": "Select a different Stat to change the query", - "fieldConfig": { - "defaults": { - "color": { - "mode": "thresholds" - }, - "mappings": [], - "thresholds": { - "mode": "absolute", - "steps": [ - { - "color": "green" - } - ] - }, - "unit": "s" - }, - "overrides": [] - }, - "gridPos": { - "h": 5, - "w": 12, - "x": 12, - "y": 28 - }, - "id": 21, - "options": { - "colorMode": "value", - "graphMode": "none", - "justifyMode": "auto", - "orientation": "auto", - "percentChangeColorMode": "standard", - "reduceOptions": { - "calcs": [], - "fields": "", - "values": false - }, - "showPercentChange": false, - "textMode": "auto", - "wideLayout": true - }, - "pluginVersion": "11.6.0", - "targets": [ - { - "datasource": { - "type": "prometheus", - "uid": "prometheus_ds_1" - }, - "disableTextWrap": false, - "editorMode": "code", - "expr": "avg by(testid) ({\"k6_http_req_duration_$quantile_stat\", tool=\"expb\", testid=~\"$testid\", jrpc_method=\"engine_newPayloadV3\", name!=\"payload_fetch\", group!~\"::setup::.*\"})", - "fullMetaSearch": false, - "includeNullMetadata": true, - "instant": false, - "legendFormat": "{{testid}}", - "range": true, - "refId": "A", - "useBackend": false - } - ], - "title": "HTTP Request Duration", - "type": "stat" - }, - { - "datasource": { - "type": "prometheus", - "uid": "prometheus_ds_1" - }, - "fieldConfig": { - "defaults": { - "color": { - "mode": "palette-classic" - }, - "custom": { - "axisBorderShow": false, - "axisCenteredZero": false, - "axisColorMode": "text", - "axisLabel": "", - "axisPlacement": "auto", - "barAlignment": 0, - "barWidthFactor": 0.6, - "drawStyle": "line", - "fillOpacity": 0, - "gradientMode": "none", - "hideFrom": { - "legend": false, - "tooltip": false, - "viz": false - }, - "insertNulls": false, - "lineInterpolation": "linear", - "lineWidth": 1, - "pointSize": 5, - "scaleDistribution": { - "type": "linear" - }, - "showPoints": "auto", - "spanNulls": false, - "stacking": { - "group": "A", - "mode": "none" - }, - "thresholdsStyle": { - "mode": "off" - } - }, - "mappings": [], - "thresholds": { - "mode": "absolute", - "steps": [ - { - "color": "green" - } - ] - }, - "unit": "bytes" - }, - "overrides": [] - }, - "gridPos": { - "h": 8, - "w": 12, - "x": 0, - "y": 33 - }, - "id": 8, - "options": { - "legend": { - "calcs": [], - "displayMode": "list", - "placement": "bottom", - "showLegend": true - }, - "tooltip": { - "hideZeros": false, - "mode": "single", - "sort": "none" - } - }, - "pluginVersion": "11.6.0", - "targets": [ - { - "datasource": { - "type": "prometheus", - "uid": "prometheus_ds_1" - }, - "editorMode": "code", - "expr": "avg by (testid) (irate(k6_data_sent_total{tool=\"expb\", testid=~\"$testid\", name!=\"payload_fetch\", group!~\"::setup::.*\"}[$__rate_interval]))", - "instant": false, - "legendFormat": "data_sent - {{testid}}", - "range": true, - "refId": "A" - }, - { - "datasource": { - "type": "prometheus", - "uid": "prometheus_ds_1" - }, - "editorMode": "code", - "exemplar": false, - "expr": "avg by (testid) (irate(k6_data_received_total{tool=\"expb\", testid=~\"$testid\", name!=\"payload_fetch\", group!~\"::setup::.*\"}[$__rate_interval]))", - "hide": false, - "instant": false, - "legendFormat": "data_received - {{testid}}", - "range": true, - "refId": "B" - } - ], - "title": "Transfer Rate", - "type": "timeseries" - }, - { - "datasource": { - "type": "prometheus", - "uid": "prometheus_ds_1" - }, - "fieldConfig": { - "defaults": { - "color": { - "mode": "palette-classic" - }, - "custom": { - "axisBorderShow": false, - "axisCenteredZero": false, - "axisColorMode": "text", - "axisLabel": "", - "axisPlacement": "auto", - "barAlignment": 0, - "barWidthFactor": 0.6, - "drawStyle": "line", - "fillOpacity": 0, - "gradientMode": "none", - "hideFrom": { - "legend": false, - "tooltip": false, - "viz": false - }, - "insertNulls": false, - "lineInterpolation": "linear", - "lineWidth": 1, - "pointSize": 5, - "scaleDistribution": { - "type": "linear" - }, - "showPoints": "auto", - "spanNulls": false, - "stacking": { - "group": "A", - "mode": "none" - }, - "thresholdsStyle": { - "mode": "off" - } - }, - "mappings": [], - "thresholds": { - "mode": "absolute", - "steps": [ - { - "color": "green" - } - ] - }, - "unit": "s" - }, - "overrides": [ - { - "matcher": { - "id": "byName", - "options": "dropped_iterations" - }, - "properties": [ - { - "id": "unit", - "value": "none" - } - ] - } - ] - }, - "gridPos": { - "h": 8, - "w": 12, - "x": 12, - "y": 33 - }, - "id": 9, - "options": { - "legend": { - "calcs": [], - "displayMode": "list", - "placement": "bottom", - "showLegend": true - }, - "tooltip": { - "hideZeros": false, - "mode": "single", - "sort": "none" - } - }, - "pluginVersion": "11.6.0", - "targets": [ - { - "datasource": { - "type": "prometheus", - "uid": "prometheus_ds_1" - }, - "editorMode": "code", - "expr": "avg by (testid) (k6_iteration_duration_$quantile_stat{tool=\"expb\", testid=~\"$testid\", name!=\"payload_fetch\", group!~\"::setup::.*\"})", - "instant": false, - "legendFormat": "iteration_duration_$quantile_stat - {{testid}}", - "range": true, - "refId": "A" - }, - { - "datasource": { - "type": "prometheus", - "uid": "prometheus_ds_1" - }, - "editorMode": "code", - "exemplar": false, - "expr": "avg by (testid) (k6_dropped_iterations_total{tool=\"expb\", testid=~\"$testid\", name!=\"payload_fetch\", group!~\"::setup::.*\"})", - "hide": false, - "instant": false, - "legendFormat": "dropped_iterations - {{testid}}", - "range": true, - "refId": "B" - } - ], - "title": "Iterations", - "type": "timeseries" - }, - { - "collapsed": false, - "gridPos": { - "h": 1, - "w": 24, - "x": 0, - "y": 41 - }, - "id": 16, - "panels": [], - "title": "HTTP", - "type": "row" - }, - { - "datasource": { - "type": "prometheus", - "uid": "prometheus_ds_1" - }, - "description": "Select a different Stat to change the query\n\nHTTP-specific built-in metrics", - "fieldConfig": { - "defaults": { - "color": { - "mode": "palette-classic" - }, - "custom": { - "axisBorderShow": false, - "axisCenteredZero": false, - "axisColorMode": "text", - "axisLabel": "", - "axisPlacement": "auto", - "barAlignment": 0, - "barWidthFactor": 0.6, - "drawStyle": "line", - "fillOpacity": 0, - "gradientMode": "none", - "hideFrom": { - "legend": false, - "tooltip": false, - "viz": false - }, - "insertNulls": false, - "lineInterpolation": "linear", - "lineWidth": 1, - "pointSize": 5, - "scaleDistribution": { - "type": "linear" - }, - "showPoints": "auto", - "spanNulls": false, - "stacking": { - "group": "A", - "mode": "none" - }, - "thresholdsStyle": { - "mode": "off" - } - }, - "mappings": [], - "thresholds": { - "mode": "absolute", - "steps": [ - { - "color": "green" - } - ] - }, - "unit": "s" - }, - "overrides": [ - { - "matcher": { - "id": "byRegexp", - "options": "http_req_duration_[a-zA-Z0-9_]+" - }, - "properties": [ - { - "id": "color", - "value": { - "fixedColor": "blue", - "mode": "fixed" - } - } - ] - } - ] - }, - "gridPos": { - "h": 9, - "w": 24, - "x": 0, - "y": 42 - }, - "id": 14, - "options": { - "legend": { - "calcs": [ - "last", - "max" - ], - "displayMode": "table", - "placement": "right", - "showLegend": true, - "sortBy": "Last", - "sortDesc": true - }, - "tooltip": { - "hideZeros": false, - "mode": "single", - "sort": "none" - } - }, - "pluginVersion": "11.6.0", - "targets": [ - { - "datasource": { - "type": "prometheus", - "uid": "prometheus_ds_1" - }, - "editorMode": "code", - "expr": "avg by (testid, group) (k6_http_req_blocked_$quantile_stat{tool=\"expb\", testid=~\"$testid\", name!=\"payload_fetch\", group!~\"::setup::.*\"})", - "hide": false, - "instant": false, - "legendFormat": "http_req_blocked_$quantile_stat - {{testid}}{{group}}", - "range": true, - "refId": "B" - }, - { - "datasource": { - "type": "prometheus", - "uid": "prometheus_ds_1" - }, - "editorMode": "code", - "expr": "avg by (testid, group) (k6_http_req_tls_handshaking_$quantile_stat{tool=\"expb\", testid=~\"$testid\", name!=\"payload_fetch\", group!~\"::setup::.*\"})", - "hide": false, - "instant": false, - "legendFormat": "http_req_tls_handshaking_$quantile_stat - {{testid}}{{group}}", - "range": true, - "refId": "C" - }, - { - "datasource": { - "type": "prometheus", - "uid": "prometheus_ds_1" - }, - "editorMode": "code", - "expr": "avg by (testid, group) (k6_http_req_sending_$quantile_stat{tool=\"expb\", testid=~\"$testid\", name!=\"payload_fetch\", group!~\"::setup::.*\"})", - "hide": false, - "instant": false, - "legendFormat": "http_req_sending_$quantile_stat - {{testid}}{{group}}", - "range": true, - "refId": "D" - }, - { - "datasource": { - "type": "prometheus", - "uid": "prometheus_ds_1" - }, - "editorMode": "code", - "expr": "avg by (testid, group) (k6_http_req_waiting_$quantile_stat{tool=\"expb\", testid=~\"$testid\", name!=\"payload_fetch\", group!~\"::setup::.*\"})", - "hide": false, - "instant": false, - "legendFormat": "http_req_waiting_$quantile_stat - {{testid}}{{group}}", - "range": true, - "refId": "E" - }, - { - "datasource": { - "type": "prometheus", - "uid": "prometheus_ds_1" - }, - "editorMode": "code", - "expr": "avg by (testid, group) (k6_http_req_receiving_$quantile_stat{tool=\"expb\", testid=~\"$testid\", name!=\"payload_fetch\", group!~\"::setup::.*\"})", - "hide": false, - "instant": false, - "legendFormat": "http_req_receiving_$quantile_stat - {{testid}}{{group}}", - "range": true, - "refId": "F" - } - ], - "title": "HTTP Latency Timings", - "type": "timeseries" - }, - { - "datasource": { - "type": "prometheus", - "uid": "prometheus_ds_1" - }, - "description": "Select a different Stat to change the query", - "fieldConfig": { - "defaults": { - "color": { - "mode": "palette-classic" - }, - "custom": { - "axisBorderShow": false, - "axisCenteredZero": false, - "axisColorMode": "text", - "axisLabel": "", - "axisPlacement": "auto", - "axisSoftMin": 0, - "barAlignment": 0, - "barWidthFactor": 0.6, - "drawStyle": "line", - "fillOpacity": 0, - "gradientMode": "none", - "hideFrom": { - "legend": false, - "tooltip": false, - "viz": false - }, - "insertNulls": false, - "lineInterpolation": "linear", - "lineWidth": 1, - "pointSize": 5, - "scaleDistribution": { - "type": "linear" - }, - "showPoints": "auto", - "spanNulls": false, - "stacking": { - "group": "A", - "mode": "none" - }, - "thresholdsStyle": { - "mode": "off" - } - }, - "mappings": [], - "thresholds": { - "mode": "absolute", - "steps": [ - { - "color": "green" - } - ] - }, - "unit": "s" - }, - "overrides": [ - { - "matcher": { - "id": "byRegexp", - "options": "errors_http_req_duration_[a-zA-Z0-9_]+" - }, - "properties": [ - { - "id": "color", - "value": { - "fixedColor": "red", - "mode": "fixed" - } - } - ] + "kind": "AnnotationQuery", + "spec": { + "builtIn": true, + "enable": true, + "hide": true, + "iconColor": "rgba(0, 211, 255, 1)", + "name": "Annotations & Alerts", + "query": { + "datasource": { + "name": "-- Grafana --" }, - { - "matcher": { - "id": "byRegexp", - "options": "success_http_req_duration_[a-zA-Z0-9_]+" - }, - "properties": [ + "group": "grafana", + "kind": "DataQuery", + "spec": {}, + "version": "v0" + } + } + } + ], + "cursorSync": "Tooltip", + "editable": true, + "elements": { + "panel-10": { + "kind": "Panel", + "spec": { + "data": { + "kind": "QueryGroup", + "spec": { + "queries": [ { - "id": "color", - "value": { - "fixedColor": "green", - "mode": "fixed" + "kind": "PanelQuery", + "spec": { + "hidden": false, + "query": { + "datasource": { + "name": "${DS_PROMETHEUS}" + }, + "group": "prometheus", + "kind": "DataQuery", + "spec": { + "editorMode": "code", + "expr": "k6_http_req_duration_p90{tool=\"expb\", testid=~\"$testid\", name!=\"payload_fetch\", group!~\"::setup::.*\"}", + "format": "table", + "instant": false, + "legendFormat": "__auto", + "range": true + }, + "version": "v0" + }, + "refId": "A" } - } - ] - }, - { - "matcher": { - "id": "byRegexp", - "options": "http_req_duration_[a-zA-Z0-9_]+" - }, - "properties": [ + }, { - "id": "color", - "value": { - "fixedColor": "yellow", - "mode": "fixed" + "kind": "PanelQuery", + "spec": { + "hidden": false, + "query": { + "datasource": { + "name": "${DS_PROMETHEUS}" + }, + "group": "prometheus", + "kind": "DataQuery", + "spec": { + "editorMode": "code", + "expr": "k6_http_req_duration_p95{tool=\"expb\", testid=~\"$testid\", name!=\"payload_fetch\", group!~\"::setup::.*\"}", + "format": "table", + "instant": false, + "legendFormat": "__auto", + "range": true + }, + "version": "v0" + }, + "refId": "B" } - } - ] - }, - { - "matcher": { - "id": "byRegexp", - "options": "http_req_duration_[a-zA-Z0-9_]+" - }, - "properties": [ + }, { - "id": "color", - "value": { - "fixedColor": "blue", - "mode": "fixed" + "kind": "PanelQuery", + "spec": { + "hidden": false, + "query": { + "datasource": { + "name": "${DS_PROMETHEUS}" + }, + "group": "prometheus", + "kind": "DataQuery", + "spec": { + "editorMode": "code", + "expr": "k6_http_req_duration_p99{tool=\"expb\", testid=~\"$testid\", name!=\"payload_fetch\", group!~\"::setup::.*\"}", + "format": "table", + "instant": false, + "legendFormat": "__auto", + "range": true + }, + "version": "v0" + }, + "refId": "C" } - } - ] - } - ] - }, - "gridPos": { - "h": 9, - "w": 24, - "x": 0, - "y": 51 - }, - "id": 15, - "options": { - "legend": { - "calcs": [ - "last" - ], - "displayMode": "table", - "placement": "right", - "showLegend": true, - "sortBy": "Last", - "sortDesc": true - }, - "tooltip": { - "hideZeros": false, - "mode": "single", - "sort": "none" - } - }, - "pluginVersion": "11.6.0", - "targets": [ - { - "datasource": { - "type": "prometheus", - "uid": "prometheus_ds_1" - }, - "editorMode": "code", - "expr": "avg by (testid, group) (k6_http_req_duration_$quantile_stat{tool=\"expb\", testid=~\"$testid\", name!=\"payload_fetch\", group!~\"::setup::.*\"})", - "hide": false, - "instant": false, - "legendFormat": "http_req_duration_$quantile_stat - {{testid}}{{group}}", - "range": true, - "refId": "A" - }, - { - "datasource": { - "type": "prometheus", - "uid": "prometheus_ds_1" - }, - "editorMode": "code", - "expr": "avg by (testid, group) (k6_http_req_duration_$quantile_stat{tool=\"expb\", testid=~\"$testid\", expected_response=\"true\", name!=\"payload_fetch\", group!~\"::setup::.*\"})", - "instant": false, - "legendFormat": "success_http_req_duration_$quantile_stat - {{testid}}{{group}}", - "range": true, - "refId": "C" - }, - { - "datasource": { - "type": "prometheus", - "uid": "prometheus_ds_1" - }, - "editorMode": "code", - "expr": "avg by (testid, group) (k6_http_req_duration_$quantile_stat{tool=\"expb\", testid=~\"$testid\", expected_response=\"false\", name!=\"payload_fetch\", group!~\"::setup::.*\"})", - "hide": false, - "instant": false, - "legendFormat": "errors_http_req_duration_$quantile_stat - {{testid}}{{group}}", - "range": true, - "refId": "B" - } - ], - "title": "HTTP Latency Stats", - "type": "timeseries" - }, - { - "datasource": { - "type": "prometheus", - "uid": "${DS_PROMETHEUS}" - }, - "description": "Gas Usage calculation for engine_newPayload request waiting time.", - "fieldConfig": { - "defaults": { - "color": { - "mode": "palette-classic" - }, - "custom": { - "axisBorderShow": false, - "axisCenteredZero": false, - "axisColorMode": "text", - "axisLabel": "", - "axisPlacement": "auto", - "fillOpacity": 50, - "hideFrom": { - "legend": false, - "tooltip": false, - "viz": false - }, - "pointShape": "circle", - "pointSize": { - "fixed": 5 - }, - "pointStrokeWidth": 1, - "scaleDistribution": { - "type": "linear" - }, - "show": "points+lines" - }, - "mappings": [], - "thresholds": { - "mode": "absolute", - "steps": [ - { - "color": "green" }, { - "color": "red", - "value": 80 - } - ] - }, - "unit": "ms" - }, - "overrides": [ - { - "matcher": { - "id": "byName", - "options": "gas_used_mgas" - }, - "properties": [ - { - "id": "unit", - "value": "MGas" + "kind": "PanelQuery", + "spec": { + "hidden": false, + "query": { + "datasource": { + "name": "${DS_PROMETHEUS}" + }, + "group": "prometheus", + "kind": "DataQuery", + "spec": { + "editorMode": "code", + "expr": "k6_http_req_duration_med{tool=\"expb\", testid=~\"$testid\", name!=\"payload_fetch\", group!~\"::setup::.*\"}", + "format": "table", + "instant": false, + "legendFormat": "__auto", + "range": true + }, + "version": "v0" + }, + "refId": "D" + } }, { - "id": "custom.axisPlacement", - "value": "right" + "kind": "PanelQuery", + "spec": { + "hidden": false, + "query": { + "datasource": { + "name": "${DS_PROMETHEUS}" + }, + "group": "prometheus", + "kind": "DataQuery", + "spec": { + "editorMode": "code", + "expr": "k6_http_req_duration_max{tool=\"expb\", testid=~\"$testid\", name!=\"payload_fetch\", group!~\"::setup::.*\"}", + "format": "table", + "instant": false, + "legendFormat": "__auto", + "range": true + }, + "version": "v0" + }, + "refId": "E" + } }, { - "id": "custom.lineStyle", - "value": { - "dash": [ - 10, - 10 - ], - "fill": "dash" + "kind": "PanelQuery", + "spec": { + "hidden": false, + "query": { + "datasource": { + "name": "${DS_PROMETHEUS}" + }, + "group": "prometheus", + "kind": "DataQuery", + "spec": { + "editorMode": "code", + "expr": "k6_http_req_duration_min{tool=\"expb\", testid=~\"$testid\", name!=\"payload_fetch\", group!~\"::setup::.*\"}", + "format": "table", + "instant": false, + "legendFormat": "__auto", + "range": true + }, + "version": "v0" + }, + "refId": "F" } }, { - "id": "custom.hideFrom", - "value": { - "legend": true, - "tooltip": false, - "viz": false + "kind": "PanelQuery", + "spec": { + "hidden": false, + "query": { + "datasource": { + "name": "${DS_PROMETHEUS}" + }, + "group": "prometheus", + "kind": "DataQuery", + "spec": { + "editorMode": "code", + "expr": "k6_http_req_duration_avg{tool=\"expb\", testid=~\"$testid\", name!=\"payload_fetch\", group!~\"::setup::.*\"}", + "format": "table", + "instant": false, + "legendFormat": "__auto", + "range": true + }, + "version": "v0" + }, + "refId": "G" } } - ] - }, - { - "matcher": { - "id": "byName", - "options": "jrpc_id" - }, - "properties": [ - { - "id": "unit", - "value": "none" - } - ] - }, - { - "matcher": { - "id": "byName", - "options": "client_ggas_s" - }, - "properties": [ + ], + "queryOptions": {}, + "transformations": [ { - "id": "unit", - "value": "GGas/s" + "group": "merge", + "kind": "Transformation", + "spec": { + "options": {} + } }, { - "id": "custom.axisPlacement", - "value": "left" - } - ] - } - ] - }, - "gridPos": { - "h": 10, - "w": 24, - "x": 0, - "y": 60 - }, - "id": 27, - "options": { - "legend": { - "calcs": [ - "min", - "mean", - "max" - ], - "displayMode": "table", - "placement": "bottom", - "showLegend": true - }, - "mapping": "auto", - "series": [ - {} - ], - "tooltip": { - "hideZeros": false, - "mode": "single", - "sort": "none" - } - }, - "pluginVersion": "11.6.0", - "targets": [ - { - "editorMode": "code", - "expr": "k6_expb_slowest_processing{tool=\"expb\", testid=~\"$testid\", name!=\"payload_fetch\", group!~\"::setup::.*\"}", - "format": "table", - "instant": false, - "legendFormat": "{{testid}}", - "range": true, - "refId": "A" - } - ], - "title": "Gas Usage per Payload (Processing)", - "transformations": [ - { - "id": "convertFieldType", - "options": { - "conversions": [ - { - "destinationType": "number", - "targetField": "jrpc_id" + "group": "groupBy", + "kind": "Transformation", + "spec": { + "options": { + "fields": { + "Time": { + "aggregations": [ + "lastNotNull" + ], + "operation": "aggregate" + }, + "Value #A": { + "aggregations": [ + "lastNotNull" + ], + "operation": "aggregate" + }, + "Value #B": { + "aggregations": [ + "lastNotNull" + ], + "operation": "aggregate" + }, + "Value #C": { + "aggregations": [ + "lastNotNull" + ], + "operation": "aggregate" + }, + "Value #D": { + "aggregations": [ + "lastNotNull" + ], + "operation": "aggregate" + }, + "Value #E": { + "aggregations": [ + "lastNotNull" + ], + "operation": "aggregate" + }, + "Value #F": { + "aggregations": [ + "lastNotNull" + ], + "operation": "aggregate" + }, + "Value #G": { + "aggregations": [ + "lastNotNull" + ], + "operation": "aggregate" + }, + "client_type": { + "aggregations": [], + "operation": "groupby" + }, + "group": { + "aggregations": [] + }, + "jrpc_method": { + "aggregations": [], + "operation": "groupby" + }, + "testid": { + "aggregations": [], + "operation": "groupby" + } + } + } + } }, { - "destinationType": "number", - "targetField": "gas_used" - } - ], - "fields": {} - } - }, - { - "id": "groupBy", - "options": { - "fields": { - "Value": { - "aggregations": [ - "mean" - ], - "operation": "aggregate" - }, - "Value #A": { - "aggregations": [ - "mean" - ], - "operation": "aggregate" - }, - "Value #B": { - "aggregations": [ - "mean" - ], - "operation": "aggregate" - }, - "Value #C": { - "aggregations": [ - "mean" - ], - "operation": "aggregate" - }, - "Value #D": { - "aggregations": [ - "mean" - ], - "operation": "aggregate" - }, - "client_type": { - "aggregations": [], - "operation": "groupby" - }, - "gas_used": { - "aggregations": [ - "last" - ], - "operation": "aggregate" - }, - "jrpc_id": { - "aggregations": [], - "operation": "groupby" - }, - "jrpc_method": { - "aggregations": [], - "operation": "groupby" - }, - "scenario": { - "aggregations": [] - }, - "testid": { - "aggregations": [], - "operation": "groupby" - } - } - } - }, - { - "id": "calculateField", - "options": { - "alias": "gas_used_mgas", - "binary": { - "left": { - "matcher": { - "id": "byName", - "options": "gas_used (last)" - } - }, - "operator": "/", - "right": { - "fixed": "1000000.0" - } - }, - "mode": "binary", - "replaceFields": false - } - }, - { - "id": "calculateField", - "options": { - "alias": "client_ggas_s", - "binary": { - "left": { - "matcher": { - "id": "byName", - "options": "gas_used_mgas" - } - }, - "operator": "/", - "right": { - "matcher": { - "id": "byName", - "options": "Value (mean)" + "group": "organize", + "kind": "Transformation", + "spec": { + "options": { + "excludeByName": {}, + "includeByName": {}, + "indexByName": { + "Value #A (lastNotNull)": 4, + "Value #B (lastNotNull)": 5, + "Value #C (lastNotNull)": 6, + "Value #D (lastNotNull)": 7, + "Value #E (lastNotNull)": 8, + "Value #F (lastNotNull)": 9, + "Value #G (lastNotNull)": 3, + "client_type": 1, + "jrpc_method": 2, + "testid": 0 + }, + "renameByName": { + "Time (lastNotNull)": "Time", + "Value #A (lastNotNull)": "p90", + "Value #B (lastNotNull)": "p95", + "Value #C (lastNotNull)": "p99", + "Value #D (lastNotNull)": "med", + "Value #E (lastNotNull)": "max", + "Value #F (lastNotNull)": "min", + "Value #G (lastNotNull)": "avg", + "client_type": "Client", + "jrpc_method": "Method", + "testid": "TestID" + } + } } - } - }, - "mode": "binary", - "reduce": { - "reducer": "sum" - } - } - }, - { - "id": "sortBy", - "options": { - "fields": {}, - "sort": [ + }, { - "field": "jrpc_id" + "group": "sortBy", + "kind": "Transformation", + "spec": { + "options": { + "fields": {}, + "sort": [ + { + "desc": true, + "field": "Method" + } + ] + } + } } ] } }, - { - "id": "organize", - "options": { - "excludeByName": { - "Value (mean)": true, - "gas_used (last)": true + "description": "", + "id": 10, + "links": [], + "title": "Performance Summary", + "vizConfig": { + "group": "table", + "kind": "VizConfig", + "spec": { + "fieldConfig": { + "defaults": { + "color": { + "mode": "thresholds" + }, + "custom": { + "align": "left", + "cellOptions": { + "type": "auto" + }, + "filterable": true, + "footer": { + "reducers": [] + }, + "inspect": false, + "wrapText": false + }, + "fieldMinMax": false, + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green", + "value": 0 + } + ] + } + }, + "overrides": [ + { + "matcher": { + "id": "byRegexp", + "options": "/avg|p90|p95|p99|med|max|min/" + }, + "properties": [ + { + "id": "unit", + "value": "s" + }, + { + "id": "custom.width", + "value": 80 + } + ] + }, + { + "matcher": { + "id": "byName", + "options": "Client" + }, + "properties": [ + { + "id": "custom.width", + "value": 100 + } + ] + }, + { + "matcher": { + "id": "byName", + "options": "Method" + }, + "properties": [ + { + "id": "custom.width", + "value": 227 + } + ] + }, + { + "matcher": { + "id": "byName", + "options": "TestID", + "scope": "series" + }, + "properties": [ + { + "id": "custom.width", + "value": 276 + } + ] + } + ] }, - "includeByName": {}, - "indexByName": {}, - "renameByName": { - "Value (mean)": "", - "client_ggas_s": "GGas/s", - "client_type": "Client Type", - "gas_used (last)": "", - "gas_used_mgas": "Gas Used", - "jrpc_id": "Payload", - "jrpc_method": "Method", - "testid": "TestID" - } - } - }, - { - "id": "partitionByValues", - "options": { - "fields": [ - "TestID" - ], - "keepFields": false, - "naming": { - "asLabels": false + "options": { + "cellHeight": "sm", + "showHeader": true, + "sortBy": [ + { + "desc": false, + "displayName": "Client" + } + ] } - } + }, + "version": "13.2.0-28666480772" } - ], - "type": "xychart" + } }, - { - "datasource": { - "type": "prometheus", - "uid": "${DS_PROMETHEUS}" - }, - "description": "Gas Usage calculation for full engine_newPayload request duration.", - "fieldConfig": { - "defaults": { - "color": { - "mode": "palette-classic" - }, - "custom": { - "axisBorderShow": false, - "axisCenteredZero": false, - "axisColorMode": "text", - "axisLabel": "", - "axisPlacement": "auto", - "fillOpacity": 50, - "hideFrom": { - "legend": false, - "tooltip": false, - "viz": false - }, - "pointShape": "circle", - "pointSize": { - "fixed": 5 - }, - "pointStrokeWidth": 1, - "scaleDistribution": { - "type": "linear" - }, - "show": "points+lines" - }, - "mappings": [], - "thresholds": { - "mode": "absolute", - "steps": [ - { - "color": "green" - }, + "panel-12": { + "kind": "Panel", + "spec": { + "data": { + "kind": "QueryGroup", + "spec": { + "queries": [ { - "color": "red", - "value": 80 + "kind": "PanelQuery", + "spec": { + "hidden": false, + "query": { + "datasource": { + "name": "${DS_PROMETHEUS}" + }, + "group": "prometheus", + "kind": "DataQuery", + "spec": { + "editorMode": "code", + "exemplar": false, + "expr": "round(k6_checks_rate{tool=\"expb\", testid=~\"$testid\", name!=\"payload_fetch\", group!~\"::setup::.*\"}, 0.1)", + "format": "table", + "instant": false, + "legendFormat": "__auto", + "range": true + }, + "version": "v0" + }, + "refId": "A" + } } - ] - }, - "unit": "ms" - }, - "overrides": [ - { - "matcher": { - "id": "byName", - "options": "gas_used_mgas" - }, - "properties": [ - { - "id": "unit", - "value": "MGas" - }, + ], + "queryOptions": {}, + "transformations": [ { - "id": "custom.axisPlacement", - "value": "right" + "group": "labelsToFields", + "kind": "Transformation", + "spec": { + "options": { + "keepLabels": [ + "__name__", + "check" + ], + "mode": "columns" + } + } }, { - "id": "custom.lineStyle", - "value": { - "dash": [ - 10, - 10 - ], - "fill": "dash" + "group": "groupBy", + "kind": "Transformation", + "spec": { + "options": { + "fields": { + "Value": { + "aggregations": [ + "mean" + ], + "operation": "aggregate" + }, + "check": { + "aggregations": [], + "operation": "groupby" + }, + "k6_checks_rate": { + "aggregations": [ + "sum", + "count" + ], + "operation": "aggregate" + }, + "testid": { + "aggregations": [], + "operation": "groupby" + } + } + } } }, { - "id": "custom.hideFrom", - "value": { - "legend": true, - "tooltip": false, - "viz": false + "group": "calculateField", + "kind": "Transformation", + "spec": { + "options": { + "alias": "Success Rate", + "binary": { + "left": { + "matcher": { + "id": "byName", + "options": "Value (mean)" + } + }, + "operator": "*", + "right": { + "fixed": "100" + } + }, + "mode": "binary", + "reduce": { + "reducer": "sum" + } + } } - } - ] - }, - { - "matcher": { - "id": "byName", - "options": "jrpc_id" - }, - "properties": [ - { - "id": "unit", - "value": "none" - } - ] - }, - { - "matcher": { - "id": "byName", - "options": "client_ggas_s" - }, - "properties": [ - { - "id": "unit", - "value": "GGas/s" }, { - "id": "custom.axisPlacement", - "value": "left" + "group": "convertFieldType", + "kind": "Transformation", + "spec": { + "options": { + "conversions": [], + "fields": {} + } + } } ] } - ] - }, - "gridPos": { - "h": 10, - "w": 24, - "x": 0, - "y": 70 - }, - "id": 28, - "options": { - "legend": { - "calcs": [ - "min", - "mean", - "max" - ], - "displayMode": "table", - "placement": "bottom", - "showLegend": true }, - "mapping": "auto", - "series": [ - {} - ], - "tooltip": { - "hideZeros": false, - "mode": "single", - "sort": "none" - } - }, - "pluginVersion": "11.6.0", - "targets": [ - { - "editorMode": "code", - "expr": "k6_expb_slowest_payloads{tool=\"expb\", testid=~\"$testid\", name!=\"payload_fetch\", group!~\"::setup::.*\"}", - "format": "table", - "instant": false, - "legendFormat": "{{testid}}", - "range": true, - "refId": "A" - } - ], - "title": "Gas Usage per Payload (Full)", - "transformations": [ - { - "id": "convertFieldType", - "options": { - "conversions": [ - { - "destinationType": "number", - "targetField": "jrpc_id" + "description": "", + "id": 12, + "links": [], + "title": "Checks list", + "vizConfig": { + "group": "table", + "kind": "VizConfig", + "spec": { + "fieldConfig": { + "defaults": { + "color": { + "mode": "thresholds" + }, + "custom": { + "align": "auto", + "cellOptions": { + "type": "auto" + }, + "footer": { + "reducers": [] + }, + "inspect": false + }, + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green", + "value": 0 + } + ] + } }, + "overrides": [ + { + "matcher": { + "id": "byName", + "options": "Success Rate" + }, + "properties": [ + { + "id": "custom.hideFrom.viz", + "value": false + }, + { + "id": "unit", + "value": "%" + } + ] + }, + { + "matcher": { + "id": "byName", + "options": "Value (mean)" + }, + "properties": [ + { + "id": "custom.hideFrom.viz", + "value": true + } + ] + }, + { + "matcher": { + "id": "byName", + "options": "check" + }, + "properties": [ + { + "id": "filterable", + "value": false + } + ] + } + ] + }, + "options": { + "cellHeight": "sm", + "enablePagination": true, + "frameIndex": 2, + "showHeader": true, + "sortBy": [ + { + "desc": true, + "displayName": "Value (count)" + } + ] + } + }, + "version": "13.2.0-28666480772" + } + } + }, + "panel-13": { + "kind": "Panel", + "spec": { + "data": { + "kind": "QueryGroup", + "spec": { + "queries": [ { - "destinationType": "number", - "targetField": "gas_used" + "kind": "PanelQuery", + "spec": { + "hidden": false, + "query": { + "datasource": { + "name": "${DS_PROMETHEUS}" + }, + "group": "prometheus", + "kind": "DataQuery", + "spec": { + "editorMode": "code", + "expr": "avg by (testid) (round(k6_checks_rate{tool=\"expb\", testid=~\"$testid\", name!=\"payload_fetch\", group!~\"::setup::.*\"}, 0.1)*100)", + "instant": false, + "legendFormat": "k6_checks_rate - {{testid}}", + "range": true + }, + "version": "v0" + }, + "refId": "A" + } } ], - "fields": {} + "queryOptions": {}, + "transformations": [] } }, - { - "id": "groupBy", - "options": { - "fields": { - "Value": { - "aggregations": [ - "mean" - ], - "operation": "aggregate" - }, - "Value #A": { - "aggregations": [ - "mean" - ], - "operation": "aggregate" - }, - "Value #B": { - "aggregations": [ - "mean" - ], - "operation": "aggregate" - }, - "Value #C": { - "aggregations": [ - "mean" - ], - "operation": "aggregate" - }, - "Value #D": { - "aggregations": [ - "mean" - ], - "operation": "aggregate" - }, - "client_type": { - "aggregations": [], - "operation": "groupby" - }, - "gas_used": { - "aggregations": [ - "last" - ], - "operation": "aggregate" - }, - "jrpc_id": { - "aggregations": [], - "operation": "groupby" + "description": "Filter by check name to query a particular check", + "id": 13, + "links": [], + "title": "Checks Success Rate (aggregate individual checks)", + "vizConfig": { + "group": "timeseries", + "kind": "VizConfig", + "spec": { + "fieldConfig": { + "defaults": { + "color": { + "mode": "palette-classic" + }, + "custom": { + "axisBorderShow": false, + "axisCenteredZero": false, + "axisColorMode": "text", + "axisLabel": "", + "axisPlacement": "auto", + "axisSoftMax": 100, + "axisSoftMin": 0, + "barAlignment": -1, + "barWidthFactor": 0.6, + "drawStyle": "line", + "fillOpacity": 0, + "gradientMode": "none", + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + }, + "insertNulls": false, + "lineInterpolation": "linear", + "lineWidth": 2, + "pointSize": 5, + "scaleDistribution": { + "type": "linear" + }, + "showPoints": "auto", + "showValues": false, + "spanNulls": false, + "stacking": { + "group": "A", + "mode": "none" + }, + "thresholdsStyle": { + "mode": "off" + } + }, + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green", + "value": 0 + } + ] + }, + "unit": "%" }, - "jrpc_method": { - "aggregations": [], - "operation": "groupby" + "overrides": [] + }, + "options": { + "annotations": { + "clustering": -1, + "multiLane": false }, - "scenario": { - "aggregations": [] + "legend": { + "calcs": [], + "displayMode": "list", + "enableFacetedFilter": false, + "overflow": "ellipsis", + "placement": "bottom", + "showLegend": true }, - "testid": { - "aggregations": [], - "operation": "groupby" - } - } - } - }, - { - "id": "calculateField", - "options": { - "alias": "gas_used_mgas", - "binary": { - "left": { - "matcher": { - "id": "byName", - "options": "gas_used (last)" - } - }, - "operator": "/", - "right": { - "fixed": "1000000.0" - } - }, - "mode": "binary", - "replaceFields": false - } - }, - { - "id": "calculateField", - "options": { - "alias": "client_ggas_s", - "binary": { - "left": { - "matcher": { - "id": "byName", - "options": "gas_used_mgas" - } - }, - "operator": "/", - "right": { - "matcher": { - "id": "byName", - "options": "Value (mean)" - } - } - }, - "mode": "binary", - "reduce": { - "reducer": "sum" - } - } - }, - { - "id": "sortBy", - "options": { - "fields": {}, - "sort": [ - { - "field": "jrpc_id" + "tooltip": { + "hideZeros": false, + "mode": "single", + "sort": "none" } - ] - } - }, - { - "id": "organize", - "options": { - "excludeByName": { - "Value (mean)": true, - "gas_used (last)": true - }, - "includeByName": {}, - "indexByName": {}, - "renameByName": { - "Value (mean)": "", - "client_ggas_s": "GGas/s", - "client_type": "Client Type", - "gas_used (last)": "", - "gas_used_mgas": "Gas Used", - "jrpc_id": "Payload", - "jrpc_method": "Method", - "testid": "TestID" - } - } - }, - { - "id": "partitionByValues", - "options": { - "fields": [ - "TestID" - ], - "keepFields": false, - "naming": { - "asLabels": false } - } + }, + "version": "13.2.0-28666480772" } - ], - "type": "xychart" + } }, - { - "datasource": { - "type": "prometheus", - "uid": "${DS_PROMETHEUS}" - }, - "fieldConfig": { - "defaults": { - "color": { - "mode": "thresholds" - }, - "custom": { - "align": "auto", - "cellOptions": { - "type": "auto" - }, - "inspect": false - }, - "mappings": [], - "thresholds": { - "mode": "absolute", - "steps": [ + "panel-14": { + "kind": "Panel", + "spec": { + "data": { + "kind": "QueryGroup", + "spec": { + "queries": [ + { + "kind": "PanelQuery", + "spec": { + "hidden": false, + "query": { + "datasource": { + "name": "${DS_PROMETHEUS}" + }, + "group": "prometheus", + "kind": "DataQuery", + "spec": { + "editorMode": "code", + "expr": "avg by (testid, group) (k6_http_req_blocked_$quantile_stat{tool=\"expb\", testid=~\"$testid\", name!=\"payload_fetch\", group!~\"::setup::.*\"})", + "instant": false, + "legendFormat": "http_req_blocked_$quantile_stat - {{testid}}{{group}}", + "range": true + }, + "version": "v0" + }, + "refId": "B" + } + }, + { + "kind": "PanelQuery", + "spec": { + "hidden": false, + "query": { + "datasource": { + "name": "${DS_PROMETHEUS}" + }, + "group": "prometheus", + "kind": "DataQuery", + "spec": { + "editorMode": "code", + "expr": "avg by (testid, group) (k6_http_req_tls_handshaking_$quantile_stat{tool=\"expb\", testid=~\"$testid\", name!=\"payload_fetch\", group!~\"::setup::.*\"})", + "instant": false, + "legendFormat": "http_req_tls_handshaking_$quantile_stat - {{testid}}{{group}}", + "range": true + }, + "version": "v0" + }, + "refId": "C" + } + }, + { + "kind": "PanelQuery", + "spec": { + "hidden": false, + "query": { + "datasource": { + "name": "${DS_PROMETHEUS}" + }, + "group": "prometheus", + "kind": "DataQuery", + "spec": { + "editorMode": "code", + "expr": "avg by (testid, group) (k6_http_req_sending_$quantile_stat{tool=\"expb\", testid=~\"$testid\", name!=\"payload_fetch\", group!~\"::setup::.*\"})", + "instant": false, + "legendFormat": "http_req_sending_$quantile_stat - {{testid}}{{group}}", + "range": true + }, + "version": "v0" + }, + "refId": "D" + } + }, { - "color": "green" + "kind": "PanelQuery", + "spec": { + "hidden": false, + "query": { + "datasource": { + "name": "${DS_PROMETHEUS}" + }, + "group": "prometheus", + "kind": "DataQuery", + "spec": { + "editorMode": "code", + "expr": "avg by (testid, group) (k6_http_req_waiting_$quantile_stat{tool=\"expb\", testid=~\"$testid\", name!=\"payload_fetch\", group!~\"::setup::.*\"})", + "instant": false, + "legendFormat": "http_req_waiting_$quantile_stat - {{testid}}{{group}}", + "range": true + }, + "version": "v0" + }, + "refId": "E" + } }, { - "color": "red", - "value": 80 + "kind": "PanelQuery", + "spec": { + "hidden": false, + "query": { + "datasource": { + "name": "${DS_PROMETHEUS}" + }, + "group": "prometheus", + "kind": "DataQuery", + "spec": { + "editorMode": "code", + "expr": "avg by (testid, group) (k6_http_req_receiving_$quantile_stat{tool=\"expb\", testid=~\"$testid\", name!=\"payload_fetch\", group!~\"::setup::.*\"})", + "instant": false, + "legendFormat": "http_req_receiving_$quantile_stat - {{testid}}{{group}}", + "range": true + }, + "version": "v0" + }, + "refId": "F" + } } - ] + ], + "queryOptions": {}, + "transformations": [] } }, - "overrides": [ - { - "matcher": { - "id": "byName", - "options": "Duration" + "description": "Select a different Stat to change the query\n\nHTTP-specific built-in metrics", + "id": 14, + "links": [], + "title": "HTTP Latency Timings", + "vizConfig": { + "group": "timeseries", + "kind": "VizConfig", + "spec": { + "fieldConfig": { + "defaults": { + "color": { + "mode": "palette-classic" + }, + "custom": { + "axisBorderShow": false, + "axisCenteredZero": false, + "axisColorMode": "text", + "axisLabel": "", + "axisPlacement": "auto", + "barAlignment": 0, + "barWidthFactor": 0.6, + "drawStyle": "line", + "fillOpacity": 0, + "gradientMode": "none", + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + }, + "insertNulls": false, + "lineInterpolation": "linear", + "lineWidth": 1, + "pointSize": 5, + "scaleDistribution": { + "type": "linear" + }, + "showPoints": "auto", + "showValues": false, + "spanNulls": false, + "stacking": { + "group": "A", + "mode": "none" + }, + "thresholdsStyle": { + "mode": "off" + } + }, + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green", + "value": 0 + } + ] + }, + "unit": "s" + }, + "overrides": [ + { + "matcher": { + "id": "byRegexp", + "options": "http_req_duration_[a-zA-Z0-9_]+" + }, + "properties": [ + { + "id": "color", + "value": { + "fixedColor": "blue", + "mode": "fixed" + } + } + ] + } + ] }, - "properties": [ - { - "id": "unit", - "value": "ms" + "options": { + "annotations": { + "clustering": -1, + "multiLane": false + }, + "legend": { + "calcs": [ + "lastNotNull", + "max" + ], + "displayMode": "table", + "enableFacetedFilter": true, + "overflow": "ellipsis", + "placement": "right", + "showLegend": true, + "sortBy": "Last", + "sortDesc": true + }, + "tooltip": { + "hideZeros": false, + "mode": "single", + "sort": "none" } - ] + } }, - { - "matcher": { - "id": "byName", - "options": "Processing" - }, - "properties": [ + "version": "13.2.0-28666480772" + } + } + }, + "panel-15": { + "kind": "Panel", + "spec": { + "data": { + "kind": "QueryGroup", + "spec": { + "queries": [ { - "id": "unit", - "value": "ms" - } - ] - }, - { - "matcher": { - "id": "byName", - "options": "Gas Used" - }, - "properties": [ + "kind": "PanelQuery", + "spec": { + "hidden": false, + "query": { + "datasource": { + "name": "${DS_PROMETHEUS}" + }, + "group": "prometheus", + "kind": "DataQuery", + "spec": { + "editorMode": "code", + "expr": "avg by (testid, group) (k6_http_req_duration_$quantile_stat{tool=\"expb\", testid=~\"$testid\", name!=\"payload_fetch\", group!~\"::setup::.*\"})", + "instant": false, + "legendFormat": "http_req_duration_$quantile_stat - {{testid}}{{group}}", + "range": true + }, + "version": "v0" + }, + "refId": "A" + } + }, { - "id": "unit", - "value": "MGas" + "kind": "PanelQuery", + "spec": { + "hidden": false, + "query": { + "datasource": { + "name": "${DS_PROMETHEUS}" + }, + "group": "prometheus", + "kind": "DataQuery", + "spec": { + "editorMode": "code", + "expr": "avg by (testid, group) (k6_http_req_duration_$quantile_stat{tool=\"expb\", testid=~\"$testid\", expected_response=\"true\", name!=\"payload_fetch\", group!~\"::setup::.*\"})", + "instant": false, + "legendFormat": "success_http_req_duration_$quantile_stat - {{testid}}{{group}}", + "range": true + }, + "version": "v0" + }, + "refId": "C" + } + }, + { + "kind": "PanelQuery", + "spec": { + "hidden": false, + "query": { + "datasource": { + "name": "${DS_PROMETHEUS}" + }, + "group": "prometheus", + "kind": "DataQuery", + "spec": { + "editorMode": "code", + "expr": "avg by (testid, group) (k6_http_req_duration_$quantile_stat{tool=\"expb\", testid=~\"$testid\", expected_response=\"false\", name!=\"payload_fetch\", group!~\"::setup::.*\"})", + "instant": false, + "legendFormat": "errors_http_req_duration_$quantile_stat - {{testid}}{{group}}", + "range": true + }, + "version": "v0" + }, + "refId": "B" + } } - ] - } - ] - }, - "gridPos": { - "h": 14, - "w": 24, - "x": 0, - "y": 80 - }, - "id": 24, - "options": { - "cellHeight": "sm", - "footer": { - "countRows": false, - "enablePagination": true, - "fields": "", - "reducer": [ - "sum" - ], - "show": false - }, - "showHeader": true, - "sortBy": [ - { - "desc": true, - "displayName": "Duration" + ], + "queryOptions": {}, + "transformations": [] } - ] - }, - "pluginVersion": "11.6.0", - "targets": [ - { - "datasource": { - "type": "prometheus", - "uid": "${DS_PROMETHEUS}" - }, - "editorMode": "code", - "expr": "k6_expb_slowest_payloads{tool=\"expb\", testid=~\"$testid\", name!=\"payload_fetch\", group!~\"::setup::.*\"}", - "format": "table", - "legendFormat": "__auto", - "range": true, - "refId": "A" - }, - { - "datasource": { - "type": "prometheus", - "uid": "${DS_PROMETHEUS}" - }, - "editorMode": "code", - "expr": "k6_expb_slowest_processing{tool=\"expb\", testid=~\"$testid\", name!=\"payload_fetch\", group!~\"::setup::.*\"}", - "format": "table", - "hide": false, - "instant": false, - "legendFormat": "__auto", - "range": true, - "refId": "B" - } - ], - "title": "Payloads Durations", - "transformations": [ - { - "id": "merge", - "options": {} }, - { - "id": "groupBy", - "options": { - "fields": { - "Value": { - "aggregations": [ - "max" - ], - "operation": "aggregate" + "description": "Select a different Stat to change the query", + "id": 15, + "links": [], + "title": "HTTP Latency Stats", + "vizConfig": { + "group": "timeseries", + "kind": "VizConfig", + "spec": { + "fieldConfig": { + "defaults": { + "color": { + "mode": "palette-classic" + }, + "custom": { + "axisBorderShow": false, + "axisCenteredZero": false, + "axisColorMode": "text", + "axisLabel": "", + "axisPlacement": "auto", + "axisSoftMin": 0, + "barAlignment": 0, + "barWidthFactor": 0.6, + "drawStyle": "line", + "fillOpacity": 0, + "gradientMode": "none", + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + }, + "insertNulls": false, + "lineInterpolation": "linear", + "lineWidth": 1, + "pointSize": 5, + "scaleDistribution": { + "type": "linear" + }, + "showPoints": "auto", + "showValues": false, + "spanNulls": false, + "stacking": { + "group": "A", + "mode": "none" + }, + "thresholdsStyle": { + "mode": "off" + } + }, + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green", + "value": 0 + } + ] + }, + "unit": "s" }, - "Value #A": { - "aggregations": [ - "mean" - ], - "operation": "aggregate" + "overrides": [ + { + "matcher": { + "id": "byRegexp", + "options": "errors_http_req_duration_[a-zA-Z0-9_]+" + }, + "properties": [ + { + "id": "color", + "value": { + "fixedColor": "red", + "mode": "fixed" + } + } + ] + }, + { + "matcher": { + "id": "byRegexp", + "options": "success_http_req_duration_[a-zA-Z0-9_]+" + }, + "properties": [ + { + "id": "color", + "value": { + "fixedColor": "green", + "mode": "fixed" + } + } + ] + }, + { + "matcher": { + "id": "byRegexp", + "options": "http_req_duration_[a-zA-Z0-9_]+" + }, + "properties": [ + { + "id": "color", + "value": { + "fixedColor": "yellow", + "mode": "fixed" + } + } + ] + }, + { + "matcher": { + "id": "byRegexp", + "options": "http_req_duration_[a-zA-Z0-9_]+" + }, + "properties": [ + { + "id": "color", + "value": { + "fixedColor": "blue", + "mode": "fixed" + } + } + ] + } + ] + }, + "options": { + "annotations": { + "clustering": -1, + "multiLane": false }, - "Value #B": { - "aggregations": [ - "mean" + "legend": { + "calcs": [ + "lastNotNull" ], - "operation": "aggregate" + "displayMode": "table", + "enableFacetedFilter": false, + "overflow": "ellipsis", + "placement": "right", + "showLegend": true, + "sortBy": "Last", + "sortDesc": true + }, + "tooltip": { + "hideZeros": false, + "mode": "single", + "sort": "none" + } + } + }, + "version": "13.2.0-28666480772" + } + } + }, + "panel-17": { + "kind": "Panel", + "spec": { + "data": { + "kind": "QueryGroup", + "spec": { + "queries": [ + { + "kind": "PanelQuery", + "spec": { + "hidden": false, + "query": { + "datasource": { + "name": "${DS_PROMETHEUS}" + }, + "group": "prometheus", + "kind": "DataQuery", + "spec": { + "editorMode": "code", + "expr": "k6_http_req_duration_min{tool=\"expb\", testid=~\"$testid\", name!=\"payload_fetch\", group!~\"::setup::.*\"}", + "format": "table", + "instant": false, + "legendFormat": "min", + "range": true + }, + "version": "v0" + }, + "refId": "A" + } + }, + { + "kind": "PanelQuery", + "spec": { + "hidden": false, + "query": { + "datasource": { + "name": "${DS_PROMETHEUS}" + }, + "group": "prometheus", + "kind": "DataQuery", + "spec": { + "editorMode": "code", + "expr": "k6_http_req_duration_max{tool=\"expb\", testid=~\"$testid\", name!=\"payload_fetch\", group!~\"::setup::.*\"}", + "format": "table", + "instant": false, + "legendFormat": "max", + "range": true + }, + "version": "v0" + }, + "refId": "B" + } }, - "client_type": { - "aggregations": [], - "operation": "groupby" + { + "kind": "PanelQuery", + "spec": { + "hidden": false, + "query": { + "datasource": { + "name": "${DS_PROMETHEUS}" + }, + "group": "prometheus", + "kind": "DataQuery", + "spec": { + "editorMode": "code", + "expr": "k6_http_req_duration_p95{tool=\"expb\", testid=~\"$testid\", name!=\"payload_fetch\", group!~\"::setup::.*\"}", + "format": "table", + "instant": false, + "legendFormat": "p95", + "range": true + }, + "version": "v0" + }, + "refId": "C" + } }, - "gas_used": { - "aggregations": [], - "operation": "groupby" + { + "kind": "PanelQuery", + "spec": { + "hidden": false, + "query": { + "datasource": { + "name": "${DS_PROMETHEUS}" + }, + "group": "prometheus", + "kind": "DataQuery", + "spec": { + "editorMode": "code", + "expr": "k6_http_req_duration_p99{tool=\"expb\", testid=~\"$testid\", name!=\"payload_fetch\", group!~\"::setup::.*\"}", + "format": "table", + "instant": false, + "legendFormat": "__auto", + "range": true + }, + "version": "v0" + }, + "refId": "D" + } }, - "group": { - "aggregations": [] + { + "kind": "PanelQuery", + "spec": { + "hidden": false, + "query": { + "datasource": { + "name": "${DS_PROMETHEUS}" + }, + "group": "prometheus", + "kind": "DataQuery", + "spec": { + "editorMode": "code", + "expr": "k6_http_req_duration_avg{tool=\"expb\",testid=~\"$testid\", name!=\"payload_fetch\", group!~\"::setup::.*\"}", + "format": "table", + "instant": false, + "legendFormat": "avg", + "range": true + }, + "version": "v0" + }, + "refId": "E" + } }, - "jrpc_id": { - "aggregations": [], - "operation": "groupby" + { + "kind": "PanelQuery", + "spec": { + "hidden": false, + "query": { + "datasource": { + "name": "${DS_PROMETHEUS}" + }, + "group": "prometheus", + "kind": "DataQuery", + "spec": { + "editorMode": "code", + "expr": "k6_http_req_duration_med{tool=\"expb\",testid=~\"$testid\", name!=\"payload_fetch\", group!~\"::setup::.*\"}", + "format": "table", + "instant": false, + "legendFormat": "med", + "range": true + }, + "version": "v0" + }, + "refId": "F" + } + } + ], + "queryOptions": {}, + "transformations": [ + { + "group": "merge", + "kind": "Transformation", + "spec": { + "options": {} + } }, - "jrpc_method": { - "aggregations": [], - "operation": "groupby" + { + "group": "groupBy", + "kind": "Transformation", + "spec": { + "options": { + "fields": { + "Value #B": { + "aggregations": [ + "min" + ], + "operation": "aggregate" + }, + "Value #C": { + "aggregations": [ + "max" + ], + "operation": "aggregate" + }, + "Value #D": { + "aggregations": [ + "mean" + ], + "operation": "aggregate" + }, + "Value #E": { + "aggregations": [ + "mean" + ], + "operation": "aggregate" + }, + "Value #F": { + "aggregations": [ + "mean" + ], + "operation": "aggregate" + }, + "Value #G": { + "aggregations": [ + "mean" + ], + "operation": "aggregate" + }, + "group": { + "aggregations": [], + "operation": "groupby" + }, + "method": { + "aggregations": [], + "operation": "groupby" + }, + "name": { + "aggregations": [], + "operation": "groupby" + }, + "status": { + "aggregations": [], + "operation": "groupby" + }, + "testid": { + "aggregations": [], + "operation": "groupby" + } + } + } + } }, - "testid": { - "aggregations": [], - "operation": "groupby" + { + "group": "organize", + "kind": "Transformation", + "spec": { + "options": { + "excludeByName": { + "Time": true + }, + "includeByName": {}, + "indexByName": { + "Time": 0, + "Value #B": 4, + "Value #C": 5, + "Value #D": 6, + "Value #E": 7, + "method": 2, + "name": 1, + "status": 3 + }, + "renameByName": { + "Value #B": "min", + "Value #B (min)": "min", + "Value #C": "max", + "Value #C (max)": "max", + "Value #D": "p95", + "Value #D (mean)": "p95", + "Value #E": "p99", + "Value #E (mean)": "p99", + "Value #F (mean)": "avg", + "Value #G (mean)": "med" + } + } + } } - } + ] } }, - { - "id": "calculateField", - "options": { - "alias": "gas_used_mgas", - "binary": { - "left": { - "matcher": { - "id": "byName", - "options": "gas_used" - } - }, - "operator": "/", - "right": { - "fixed": "1000000" - } - }, - "mode": "binary", - "reduce": { - "reducer": "sum" + "description": "min/max/p95/p99 depends on the available Quantile Stats", + "id": 17, + "links": [], + "title": "Requests by URL", + "vizConfig": { + "group": "table", + "kind": "VizConfig", + "spec": { + "fieldConfig": { + "defaults": { + "color": { + "mode": "thresholds" + }, + "custom": { + "align": "left", + "cellOptions": { + "type": "auto" + }, + "footer": { + "reducers": [] + }, + "inspect": false, + "wrapText": true + }, + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green", + "value": 0 + } + ] + } + }, + "overrides": [ + { + "matcher": { + "id": "byName", + "options": "name" + }, + "properties": [ + { + "id": "filterable", + "value": false + } + ] + }, + { + "matcher": { + "id": "byName", + "options": "method" + }, + "properties": [ + { + "id": "filterable", + "value": false + } + ] + }, + { + "matcher": { + "id": "byName", + "options": "status" + }, + "properties": [ + { + "id": "filterable", + "value": false + }, + { + "id": "custom.width", + "value": 62 + } + ] + }, + { + "matcher": { + "id": "byName", + "options": "min" + }, + "properties": [ + { + "id": "unit", + "value": "s" + }, + { + "id": "custom.width", + "value": 105 + } + ] + }, + { + "matcher": { + "id": "byName", + "options": "max" + }, + "properties": [ + { + "id": "unit", + "value": "s" + }, + { + "id": "custom.width", + "value": 91 + } + ] + }, + { + "matcher": { + "id": "byName", + "options": "p95" + }, + "properties": [ + { + "id": "unit", + "value": "s" + }, + { + "id": "custom.width", + "value": 96 + } + ] + }, + { + "matcher": { + "id": "byName", + "options": "p99" + }, + "properties": [ + { + "id": "unit", + "value": "s" + }, + { + "id": "custom.width", + "value": 105 + } + ] + }, + { + "matcher": { + "id": "byName", + "options": "avg" + }, + "properties": [ + { + "id": "unit", + "value": "s" + }, + { + "id": "custom.width", + "value": 102 + } + ] + }, + { + "matcher": { + "id": "byName", + "options": "med" + }, + "properties": [ + { + "id": "unit", + "value": "s" + } + ] + }, + { + "matcher": { + "id": "byName", + "options": "group" + }, + "properties": [ + { + "id": "custom.width", + "value": 232 + } + ] + } + ] }, - "replaceFields": false, - "unary": { - "fieldName": "Gas Used", - "operator": "abs" + "options": { + "cellHeight": "sm", + "enablePagination": true, + "frameIndex": 2, + "showHeader": true, + "sortBy": [ + { + "desc": false, + "displayName": "group" + } + ] } + }, + "version": "13.2.0-28666480772" + } + } + }, + "panel-18": { + "kind": "Panel", + "spec": { + "data": { + "kind": "QueryGroup", + "spec": { + "queries": [ + { + "kind": "PanelQuery", + "spec": { + "hidden": false, + "query": { + "datasource": { + "name": "${DS_PROMETHEUS}" + }, + "group": "prometheus", + "kind": "DataQuery", + "spec": { + "editorMode": "code", + "expr": "sum by (testid) (irate(k6_http_reqs_total{tool=\"expb\", testid=~\"$testid\", name!=\"payload_fetch\", group!~\"::setup::.*\"}[$__rate_interval]))", + "instant": false, + "legendFormat": "http_req_s - {{testid}}", + "range": true + }, + "version": "v0" + }, + "refId": "A" + } + }, + { + "kind": "PanelQuery", + "spec": { + "hidden": false, + "query": { + "datasource": { + "name": "${DS_PROMETHEUS}" + }, + "group": "prometheus", + "kind": "DataQuery", + "spec": { + "editorMode": "code", + "expr": "sum by (testid) (irate(k6_http_reqs_total{tool=\"expb\", testid=~\"$testid\", expected_response=\"false\", name!=\"payload_fetch\", group!~\"::setup::.*\"}[$__rate_interval]))", + "instant": false, + "legendFormat": "http_req_s_errors - {{testid}}", + "range": true + }, + "version": "v0" + }, + "refId": "B" + } + }, + { + "kind": "PanelQuery", + "spec": { + "hidden": false, + "query": { + "datasource": { + "name": "${DS_PROMETHEUS}" + }, + "group": "prometheus", + "kind": "DataQuery", + "spec": { + "editorMode": "code", + "expr": "sum by (testid) (irate(k6_http_reqs_total{tool=\"expb\", testid=~\"$testid\", expected_response=\"true\", name!=\"payload_fetch\", group!~\"::setup::.*\"}[$__rate_interval]))", + "instant": false, + "legendFormat": "http_req_s_success - {{testid}}", + "range": true + }, + "version": "v0" + }, + "refId": "C" + } + } + ], + "queryOptions": {}, + "transformations": [] } }, - { - "id": "organize", - "options": { - "excludeByName": { - "gas_used": true - }, - "includeByName": {}, - "indexByName": { - "Value #A (mean)": 6, - "Value #B (mean)": 5, - "client_type": 1, - "gas_used": 4, - "gas_used_mgas": 7, - "jrpc_id": 2, - "jrpc_method": 3, - "testid": 0 + "description": "", + "id": 18, + "links": [], + "title": "HTTP Request Rate", + "vizConfig": { + "group": "timeseries", + "kind": "VizConfig", + "spec": { + "fieldConfig": { + "defaults": { + "color": { + "mode": "palette-classic" + }, + "custom": { + "axisBorderShow": false, + "axisCenteredZero": false, + "axisColorMode": "text", + "axisLabel": "", + "axisPlacement": "auto", + "axisSoftMin": 0, + "barAlignment": 0, + "barWidthFactor": 0.6, + "drawStyle": "line", + "fillOpacity": 0, + "gradientMode": "none", + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + }, + "insertNulls": false, + "lineInterpolation": "linear", + "lineWidth": 1, + "pointSize": 5, + "scaleDistribution": { + "type": "linear" + }, + "showPoints": "auto", + "showValues": false, + "spanNulls": false, + "stacking": { + "group": "A", + "mode": "none" + }, + "thresholdsStyle": { + "mode": "off" + } + }, + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green", + "value": 0 + } + ] + }, + "unit": "reqps" + }, + "overrides": [ + { + "matcher": { + "id": "byName", + "options": "http_req_s_errors" + }, + "properties": [ + { + "id": "color", + "value": { + "fixedColor": "red", + "mode": "fixed" + } + }, + { + "id": "custom.lineStyle", + "value": { + "dash": [ + 10, + 10 + ], + "fill": "dash" + } + } + ] + }, + { + "matcher": { + "id": "byName", + "options": "http_req_s" + }, + "properties": [ + { + "id": "color", + "value": { + "fixedColor": "yellow", + "mode": "fixed" + } + }, + { + "id": "custom.lineStyle", + "value": { + "dash": [ + 10, + 10 + ], + "fill": "dash" + } + } + ] + }, + { + "matcher": { + "id": "byName", + "options": "http_req_s_success" + }, + "properties": [ + { + "id": "custom.lineStyle", + "value": { + "dash": [ + 10, + 10 + ], + "fill": "dash" + } + }, + { + "id": "color", + "value": { + "fixedColor": "green", + "mode": "fixed" + } + } + ] + } + ] }, - "renameByName": { - "Value #A (mean)": "Duration", - "Value #B (mean)": "Processing", - "Value (max)": "Duration", - "client_type": "Client", - "gas_used": "", - "gas_used_mgas": "Gas Used", - "jrpc_id": "Payload", - "jrpc_method": "JRPC Method", - "testid": "Test ID" + "options": { + "annotations": { + "clustering": -1, + "multiLane": false + }, + "legend": { + "calcs": [], + "displayMode": "list", + "enableFacetedFilter": false, + "overflow": "ellipsis", + "placement": "bottom", + "showLegend": true + }, + "tooltip": { + "hideZeros": false, + "mode": "single", + "sort": "none" + } } - } - }, - { - "id": "sortBy", - "options": { - "fields": {}, - "sort": [ + }, + "version": "13.2.0-28666480772" + } + } + }, + "panel-20": { + "kind": "Panel", + "spec": { + "data": { + "kind": "QueryGroup", + "spec": { + "queries": [ { - "desc": true, - "field": "Duration" + "kind": "PanelQuery", + "spec": { + "hidden": false, + "query": { + "datasource": { + "name": "${DS_PROMETHEUS}" + }, + "group": "prometheus", + "kind": "DataQuery", + "spec": { + "disableTextWrap": false, + "editorMode": "code", + "expr": "sum by(testid) (irate(k6_iterations_total{tool=\"expb\", testid=~\"$testid\"}[$__rate_interval]))", + "fullMetaSearch": false, + "includeNullMetadata": true, + "instant": false, + "interval": "", + "legendFormat": "{{testid}}", + "range": true, + "useBackend": false + }, + "version": "v0" + }, + "refId": "A" + } } - ] + ], + "queryOptions": {}, + "transformations": [] } - } - ], - "type": "table" - }, - { - "datasource": { - "type": "prometheus", - "uid": "prometheus_ds_1" - }, - "description": "", - "fieldConfig": { - "defaults": { - "color": { - "mode": "palette-classic" - }, - "custom": { - "axisBorderShow": false, - "axisCenteredZero": false, - "axisColorMode": "text", - "axisLabel": "", - "axisPlacement": "auto", - "axisSoftMin": 0, - "barAlignment": 0, - "barWidthFactor": 0.6, - "drawStyle": "line", - "fillOpacity": 0, - "gradientMode": "none", - "hideFrom": { - "legend": false, - "tooltip": false, - "viz": false - }, - "insertNulls": false, - "lineInterpolation": "linear", - "lineWidth": 1, - "pointSize": 5, - "scaleDistribution": { - "type": "linear" - }, - "showPoints": "auto", - "spanNulls": false, - "stacking": { - "group": "A", - "mode": "none" + }, + "description": "", + "id": 20, + "links": [], + "title": "Blocks per second", + "vizConfig": { + "group": "stat", + "kind": "VizConfig", + "spec": { + "fieldConfig": { + "defaults": { + "color": { + "mode": "thresholds" + }, + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green", + "value": 0 + } + ] + }, + "unit": "blocks/s" + }, + "overrides": [] }, - "thresholdsStyle": { - "mode": "off" + "options": { + "colorMode": "value", + "graphMode": "none", + "justifyMode": "auto", + "orientation": "auto", + "percentChangeColorMode": "standard", + "reduceOptions": { + "calcs": [], + "fields": "", + "values": false + }, + "showPercentChange": false, + "textMode": "auto", + "wideLayout": true } }, - "mappings": [], - "thresholds": { - "mode": "absolute", - "steps": [ + "version": "13.2.0-28666480772" + } + } + }, + "panel-21": { + "kind": "Panel", + "spec": { + "data": { + "kind": "QueryGroup", + "spec": { + "queries": [ { - "color": "green" + "kind": "PanelQuery", + "spec": { + "hidden": false, + "query": { + "datasource": { + "name": "${DS_PROMETHEUS}" + }, + "group": "prometheus", + "kind": "DataQuery", + "spec": { + "disableTextWrap": false, + "editorMode": "code", + "expr": "avg by(testid) ({\"k6_http_req_duration_$quantile_stat\", tool=\"expb\", testid=~\"$testid\", jrpc_method=~\"engine_newPayloadV.*\", name!=\"payload_fetch\", group!~\"::setup::.*\"})", + "fullMetaSearch": false, + "includeNullMetadata": true, + "instant": false, + "legendFormat": "{{testid}}", + "range": true, + "useBackend": false + }, + "version": "v0" + }, + "refId": "A" + } } - ] - }, - "unit": "reqps" + ], + "queryOptions": {}, + "transformations": [] + } }, - "overrides": [ - { - "matcher": { - "id": "byName", - "options": "http_req_s_errors" + "description": "Select a different Stat to change the query", + "id": 21, + "links": [], + "title": "HTTP Request Duration", + "vizConfig": { + "group": "stat", + "kind": "VizConfig", + "spec": { + "fieldConfig": { + "defaults": { + "color": { + "mode": "thresholds" + }, + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green", + "value": 0 + } + ] + }, + "unit": "s" + }, + "overrides": [] }, - "properties": [ + "options": { + "colorMode": "value", + "graphMode": "none", + "justifyMode": "auto", + "orientation": "auto", + "percentChangeColorMode": "standard", + "reduceOptions": { + "calcs": [], + "fields": "", + "values": false + }, + "showPercentChange": false, + "textMode": "auto", + "wideLayout": true + } + }, + "version": "13.2.0-28666480772" + } + } + }, + "panel-22": { + "kind": "Panel", + "spec": { + "data": { + "kind": "QueryGroup", + "spec": { + "queries": [ { - "id": "color", - "value": { + "kind": "PanelQuery", + "spec": { + "hidden": false, + "query": { + "datasource": { + "name": "${DS_PROMETHEUS}" + }, + "group": "prometheus", + "kind": "DataQuery", + "spec": { + "editorMode": "code", + "expr": "sum by (testid) (k6_http_reqs_failed_total{tool=\"expb\", testid=~\"$testid\", name!=\"payload_fetch\", group!~\"::setup::.*\"})", + "instant": false, + "legendFormat": "{{testid}}", + "range": true + }, + "version": "v0" + }, + "refId": "A" + } + } + ], + "queryOptions": {}, + "transformations": [] + } + }, + "description": "", + "id": 22, + "links": [], + "title": "HTTP request failures", + "vizConfig": { + "group": "stat", + "kind": "VizConfig", + "spec": { + "fieldConfig": { + "defaults": { + "color": { "fixedColor": "red", "mode": "fixed" + }, + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green", + "value": 0 + } + ] + }, + "unit": "none" + }, + "overrides": [] + }, + "options": { + "colorMode": "value", + "graphMode": "none", + "justifyMode": "auto", + "orientation": "auto", + "percentChangeColorMode": "standard", + "reduceOptions": { + "calcs": [], + "fields": "", + "values": false + }, + "showPercentChange": false, + "textMode": "auto", + "wideLayout": true + } + }, + "version": "13.2.0-28666480772" + } + } + }, + "panel-24": { + "kind": "Panel", + "spec": { + "data": { + "kind": "QueryGroup", + "spec": { + "queries": [ + { + "kind": "PanelQuery", + "spec": { + "hidden": false, + "query": { + "datasource": { + "name": "${DS_PROMETHEUS}" + }, + "group": "prometheus", + "kind": "DataQuery", + "spec": { + "editorMode": "code", + "expr": "k6_expb_slowest_payloads{tool=\"expb\", testid=~\"$testid\", name!=\"payload_fetch\", group!~\"::setup::.*\"}", + "format": "table", + "legendFormat": "__auto", + "range": true + }, + "version": "v0" + }, + "refId": "A" } }, { - "id": "custom.lineStyle", - "value": { - "dash": [ - 10, - 10 - ], - "fill": "dash" + "kind": "PanelQuery", + "spec": { + "hidden": false, + "query": { + "datasource": { + "name": "${DS_PROMETHEUS}" + }, + "group": "prometheus", + "kind": "DataQuery", + "spec": { + "editorMode": "code", + "expr": "k6_expb_slowest_processing{tool=\"expb\", testid=~\"$testid\", name!=\"payload_fetch\", group!~\"::setup::.*\"}", + "format": "table", + "instant": false, + "legendFormat": "__auto", + "range": true + }, + "version": "v0" + }, + "refId": "B" } } - ] - }, - { - "matcher": { - "id": "byName", - "options": "http_req_s" - }, - "properties": [ + ], + "queryOptions": {}, + "transformations": [ { - "id": "color", - "value": { - "fixedColor": "yellow", - "mode": "fixed" + "group": "merge", + "kind": "Transformation", + "spec": { + "options": {} } }, { - "id": "custom.lineStyle", - "value": { - "dash": [ - 10, - 10 - ], - "fill": "dash" + "group": "groupBy", + "kind": "Transformation", + "spec": { + "options": { + "fields": { + "Value": { + "aggregations": [ + "max" + ], + "operation": "aggregate" + }, + "Value #A": { + "aggregations": [ + "mean" + ], + "operation": "aggregate" + }, + "Value #B": { + "aggregations": [ + "mean" + ], + "operation": "aggregate" + }, + "client_type": { + "aggregations": [], + "operation": "groupby" + }, + "gas_used": { + "aggregations": [], + "operation": "groupby" + }, + "group": { + "aggregations": [] + }, + "jrpc_id": { + "aggregations": [], + "operation": "groupby" + }, + "jrpc_method": { + "aggregations": [], + "operation": "groupby" + }, + "testid": { + "aggregations": [], + "operation": "groupby" + } + } + } } - } - ] - }, - { - "matcher": { - "id": "byName", - "options": "http_req_s_success" - }, - "properties": [ + }, { - "id": "custom.lineStyle", - "value": { - "dash": [ - 10, - 10 - ], - "fill": "dash" + "group": "calculateField", + "kind": "Transformation", + "spec": { + "options": { + "alias": "gas_used_mgas", + "binary": { + "left": { + "matcher": { + "id": "byName", + "options": "gas_used" + } + }, + "operator": "/", + "right": { + "fixed": "1000000" + } + }, + "mode": "binary", + "reduce": { + "reducer": "sum" + }, + "replaceFields": false, + "unary": { + "fieldName": "Gas Used", + "operator": "abs" + } + } } }, { - "id": "color", - "value": { - "fixedColor": "green", - "mode": "fixed" + "group": "organize", + "kind": "Transformation", + "spec": { + "options": { + "excludeByName": { + "gas_used": true + }, + "includeByName": {}, + "indexByName": { + "Value #A (mean)": 6, + "Value #B (mean)": 5, + "client_type": 1, + "gas_used": 4, + "gas_used_mgas": 7, + "jrpc_id": 2, + "jrpc_method": 3, + "testid": 0 + }, + "renameByName": { + "Value #A (mean)": "Duration", + "Value #B (mean)": "Processing", + "Value (max)": "Duration", + "client_type": "Client", + "gas_used": "", + "gas_used_mgas": "Gas Used", + "jrpc_id": "Payload", + "jrpc_method": "JRPC Method", + "testid": "Test ID" + } + } + } + }, + { + "group": "sortBy", + "kind": "Transformation", + "spec": { + "options": { + "fields": {}, + "sort": [ + { + "desc": true, + "field": "Duration" + } + ] + } } } ] } - ] - }, - "gridPos": { - "h": 9, - "w": 24, - "x": 0, - "y": 94 - }, - "id": 18, - "options": { - "legend": { - "calcs": [], - "displayMode": "list", - "placement": "bottom", - "showLegend": true - }, - "tooltip": { - "hideZeros": false, - "mode": "single", - "sort": "none" - } - }, - "pluginVersion": "11.6.0", - "targets": [ - { - "datasource": { - "type": "prometheus", - "uid": "prometheus_ds_1" - }, - "editorMode": "code", - "expr": "sum by (testid) (irate(k6_http_reqs_total{tool=\"expb\", testid=~\"$testid\", name!=\"payload_fetch\", group!~\"::setup::.*\"}[$__rate_interval]))", - "instant": false, - "legendFormat": "http_req_s - {{testid}}", - "range": true, - "refId": "A" - }, - { - "datasource": { - "type": "prometheus", - "uid": "prometheus_ds_1" - }, - "editorMode": "code", - "expr": "sum by (testid) (irate(k6_http_reqs_total{tool=\"expb\", testid=~\"$testid\", expected_response=\"false\", name!=\"payload_fetch\", group!~\"::setup::.*\"}[$__rate_interval]))", - "hide": false, - "instant": false, - "legendFormat": "http_req_s_errors - {{testid}}", - "range": true, - "refId": "B" }, - { - "datasource": { - "type": "prometheus", - "uid": "prometheus_ds_1" + "description": "", + "id": 24, + "links": [], + "title": "Payloads Durations", + "vizConfig": { + "group": "table", + "kind": "VizConfig", + "spec": { + "fieldConfig": { + "defaults": { + "color": { + "mode": "thresholds" + }, + "custom": { + "align": "auto", + "cellOptions": { + "type": "auto" + }, + "footer": { + "reducers": [] + }, + "inspect": false + }, + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green", + "value": 0 + }, + { + "color": "red", + "value": 80 + } + ] + } + }, + "overrides": [ + { + "matcher": { + "id": "byName", + "options": "Duration" + }, + "properties": [ + { + "id": "unit", + "value": "ms" + } + ] + }, + { + "matcher": { + "id": "byName", + "options": "Processing" + }, + "properties": [ + { + "id": "unit", + "value": "ms" + } + ] + }, + { + "matcher": { + "id": "byName", + "options": "Gas Used" + }, + "properties": [ + { + "id": "unit", + "value": "MGas" + } + ] + }, + { + "matcher": { + "id": "byName", + "options": "Test ID", + "scope": "series" + }, + "properties": [ + { + "id": "custom.width", + "value": 289 + } + ] + } + ] + }, + "options": { + "cellHeight": "sm", + "enablePagination": true, + "showHeader": true, + "sortBy": [ + { + "desc": true, + "displayName": "Duration" + } + ] + } }, - "editorMode": "code", - "expr": "sum by (testid) (irate(k6_http_reqs_total{tool=\"expb\", testid=~\"$testid\", expected_response=\"true\", name!=\"payload_fetch\", group!~\"::setup::.*\"}[$__rate_interval]))", - "hide": false, - "instant": false, - "legendFormat": "http_req_s_success - {{testid}}", - "range": true, - "refId": "C" + "version": "13.2.0-28666480772" } - ], - "title": "HTTP Request Rate", - "type": "timeseries" + } }, - { - "datasource": { - "type": "prometheus", - "uid": "prometheus_ds_1" - }, - "description": "", - "fieldConfig": { - "defaults": { - "color": { - "mode": "palette-classic" - }, - "custom": { - "axisBorderShow": false, - "axisCenteredZero": false, - "axisColorMode": "text", - "axisLabel": "", - "axisPlacement": "auto", - "axisSoftMin": 0, - "barAlignment": 0, - "barWidthFactor": 0.6, - "drawStyle": "line", - "fillOpacity": 0, - "gradientMode": "none", - "hideFrom": { - "legend": false, - "tooltip": false, - "viz": false - }, - "insertNulls": false, - "lineInterpolation": "linear", - "lineWidth": 1, - "pointSize": 5, - "scaleDistribution": { - "type": "linear" - }, - "showPoints": "auto", - "spanNulls": false, - "stacking": { - "group": "A", - "mode": "none" + "panel-25": { + "kind": "Panel", + "spec": { + "data": { + "kind": "QueryGroup", + "spec": { + "queries": [ + { + "kind": "PanelQuery", + "spec": { + "hidden": false, + "query": { + "datasource": { + "name": "${DS_PROMETHEUS}" + }, + "group": "prometheus", + "kind": "DataQuery", + "spec": { + "disableTextWrap": false, + "editorMode": "code", + "expr": "sum by(testid) (irate(k6_iterations_total{tool=\"expb\", testid=~\"$testid\"}[$__rate_interval]))", + "fullMetaSearch": false, + "includeNullMetadata": true, + "instant": false, + "legendFormat": "{{testid}}", + "range": true, + "useBackend": false + }, + "version": "v0" + }, + "refId": "A" + } + } + ], + "queryOptions": {}, + "transformations": [] + } + }, + "description": "", + "id": 25, + "links": [], + "title": "Blocks Rate", + "vizConfig": { + "group": "timeseries", + "kind": "VizConfig", + "spec": { + "fieldConfig": { + "defaults": { + "color": { + "mode": "palette-classic" + }, + "custom": { + "axisBorderShow": false, + "axisCenteredZero": false, + "axisColorMode": "text", + "axisLabel": "", + "axisPlacement": "auto", + "axisSoftMin": 0, + "barAlignment": 0, + "barWidthFactor": 0.6, + "drawStyle": "line", + "fillOpacity": 0, + "gradientMode": "none", + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + }, + "insertNulls": false, + "lineInterpolation": "linear", + "lineWidth": 1, + "pointSize": 5, + "scaleDistribution": { + "type": "linear" + }, + "showPoints": "auto", + "showValues": false, + "spanNulls": false, + "stacking": { + "group": "A", + "mode": "none" + }, + "thresholdsStyle": { + "mode": "off" + } + }, + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green", + "value": 0 + } + ] + }, + "unit": "reqps" + }, + "overrides": [ + { + "matcher": { + "id": "byName", + "options": "http_req_s_errors" + }, + "properties": [ + { + "id": "color", + "value": { + "fixedColor": "red", + "mode": "fixed" + } + }, + { + "id": "custom.lineStyle", + "value": { + "dash": [ + 10, + 10 + ], + "fill": "dash" + } + } + ] + }, + { + "matcher": { + "id": "byName", + "options": "http_req_s" + }, + "properties": [ + { + "id": "color", + "value": { + "fixedColor": "yellow", + "mode": "fixed" + } + }, + { + "id": "custom.lineStyle", + "value": { + "dash": [ + 10, + 10 + ], + "fill": "dash" + } + } + ] + }, + { + "matcher": { + "id": "byName", + "options": "http_req_s_success" + }, + "properties": [ + { + "id": "custom.lineStyle", + "value": { + "dash": [ + 10, + 10 + ], + "fill": "dash" + } + }, + { + "id": "color", + "value": { + "fixedColor": "green", + "mode": "fixed" + } + } + ] + } + ] }, - "thresholdsStyle": { - "mode": "off" + "options": { + "annotations": { + "clustering": -1, + "multiLane": false + }, + "legend": { + "calcs": [], + "displayMode": "list", + "enableFacetedFilter": false, + "overflow": "ellipsis", + "placement": "bottom", + "showLegend": true + }, + "tooltip": { + "hideZeros": false, + "mode": "single", + "sort": "none" + } } }, - "mappings": [], - "thresholds": { - "mode": "absolute", - "steps": [ + "version": "13.2.0-28666480772" + } + } + }, + "panel-27": { + "kind": "Panel", + "spec": { + "data": { + "kind": "QueryGroup", + "spec": { + "queries": [ { - "color": "green" + "kind": "PanelQuery", + "spec": { + "hidden": false, + "query": { + "datasource": { + "name": "${DS_PROMETHEUS}" + }, + "group": "prometheus", + "kind": "DataQuery", + "spec": { + "editorMode": "code", + "expr": "k6_expb_slowest_processing{tool=\"expb\", testid=~\"$testid\", name!=\"payload_fetch\", group!~\"::setup::.*\"}", + "format": "table", + "instant": false, + "legendFormat": "{{testid}}", + "range": true + }, + "version": "v0" + }, + "refId": "A" + } } - ] - }, - "unit": "reqps" - }, - "overrides": [ - { - "matcher": { - "id": "byName", - "options": "http_req_s_errors" - }, - "properties": [ + ], + "queryOptions": {}, + "transformations": [ { - "id": "color", - "value": { - "fixedColor": "red", - "mode": "fixed" + "group": "convertFieldType", + "kind": "Transformation", + "spec": { + "options": { + "conversions": [ + { + "destinationType": "number", + "targetField": "jrpc_id" + }, + { + "destinationType": "number", + "targetField": "gas_used" + } + ], + "fields": {} + } } }, { - "id": "custom.lineStyle", - "value": { - "dash": [ - 10, - 10 - ], - "fill": "dash" + "group": "groupBy", + "kind": "Transformation", + "spec": { + "options": { + "fields": { + "Value": { + "aggregations": [ + "mean" + ], + "operation": "aggregate" + }, + "Value #A": { + "aggregations": [ + "mean" + ], + "operation": "aggregate" + }, + "Value #B": { + "aggregations": [ + "mean" + ], + "operation": "aggregate" + }, + "Value #C": { + "aggregations": [ + "mean" + ], + "operation": "aggregate" + }, + "Value #D": { + "aggregations": [ + "mean" + ], + "operation": "aggregate" + }, + "client_type": { + "aggregations": [], + "operation": "groupby" + }, + "gas_used": { + "aggregations": [ + "last" + ], + "operation": "aggregate" + }, + "jrpc_id": { + "aggregations": [], + "operation": "groupby" + }, + "jrpc_method": { + "aggregations": [], + "operation": "groupby" + }, + "scenario": { + "aggregations": [] + }, + "testid": { + "aggregations": [], + "operation": "groupby" + } + } + } } - } - ] - }, - { - "matcher": { - "id": "byName", - "options": "http_req_s" - }, - "properties": [ + }, { - "id": "color", - "value": { - "fixedColor": "yellow", - "mode": "fixed" + "group": "calculateField", + "kind": "Transformation", + "spec": { + "options": { + "alias": "gas_used_mgas", + "binary": { + "left": { + "matcher": { + "id": "byName", + "options": "gas_used (last)" + } + }, + "operator": "/", + "right": { + "fixed": "1000000.0" + } + }, + "mode": "binary", + "replaceFields": false + } } }, { - "id": "custom.lineStyle", - "value": { - "dash": [ - 10, - 10 - ], - "fill": "dash" + "group": "calculateField", + "kind": "Transformation", + "spec": { + "options": { + "alias": "client_ggas_s", + "binary": { + "left": { + "matcher": { + "id": "byName", + "options": "gas_used_mgas" + } + }, + "operator": "/", + "right": { + "matcher": { + "id": "byName", + "options": "Value (mean)" + } + } + }, + "mode": "binary", + "reduce": { + "reducer": "sum" + } + } } - } - ] - }, - { - "matcher": { - "id": "byName", - "options": "http_req_s_success" - }, - "properties": [ + }, { - "id": "custom.lineStyle", - "value": { - "dash": [ - 10, - 10 - ], - "fill": "dash" + "group": "sortBy", + "kind": "Transformation", + "spec": { + "options": { + "fields": {}, + "sort": [ + { + "field": "jrpc_id" + } + ] + } } }, { - "id": "color", - "value": { - "fixedColor": "green", - "mode": "fixed" + "group": "organize", + "kind": "Transformation", + "spec": { + "options": { + "excludeByName": { + "Value (mean)": true, + "gas_used (last)": true + }, + "includeByName": {}, + "indexByName": {}, + "renameByName": { + "Value (mean)": "", + "client_ggas_s": "GGas/s", + "client_type": "Client Type", + "gas_used (last)": "", + "gas_used_mgas": "Gas Used", + "jrpc_id": "Payload", + "jrpc_method": "Method", + "testid": "TestID" + } + } } - } - ] - } - ] - }, - "gridPos": { - "h": 9, - "w": 24, - "x": 0, - "y": 103 - }, - "id": 25, - "options": { - "legend": { - "calcs": [], - "displayMode": "list", - "placement": "bottom", - "showLegend": true - }, - "tooltip": { - "hideZeros": false, - "mode": "single", - "sort": "none" - } - }, - "pluginVersion": "11.6.0", - "targets": [ - { - "datasource": { - "type": "prometheus", - "uid": "prometheus_ds_1" - }, - "disableTextWrap": false, - "editorMode": "code", - "expr": "sum by(testid) (irate(k6_iteration_total{tool=\"expb\", testid=~\"$testid\", expected_response=\"true\", name!=\"payload_fetch\", group!~\"::setup::.*\"}[$__rate_interval]))", - "fullMetaSearch": false, - "hide": false, - "includeNullMetadata": true, - "instant": false, - "legendFormat": "http_req_s_success - {{testid}}", - "range": true, - "refId": "A", - "useBackend": false - } - ], - "title": "Blocks Rate", - "type": "timeseries" - }, - { - "datasource": { - "type": "prometheus", - "uid": "prometheus_ds_1" - }, - "description": "min/max/p95/p99 depends on the available Quantile Stats", - "fieldConfig": { - "defaults": { - "color": { - "mode": "thresholds" - }, - "custom": { - "align": "left", - "cellOptions": { - "type": "auto", - "wrapText": true - }, - "inspect": false - }, - "mappings": [], - "thresholds": { - "mode": "absolute", - "steps": [ + }, { - "color": "green" + "group": "partitionByValues", + "kind": "Transformation", + "spec": { + "options": { + "fields": [ + "TestID" + ], + "keepFields": false, + "naming": { + "asLabels": false + } + } + } } ] } }, - "overrides": [ - { - "matcher": { - "id": "byName", - "options": "name" + "description": "Gas Usage calculation for engine_newPayload request waiting time.", + "id": 27, + "links": [], + "title": "Gas Usage per Payload (Processing)", + "vizConfig": { + "group": "xychart", + "kind": "VizConfig", + "spec": { + "fieldConfig": { + "defaults": { + "color": { + "mode": "palette-classic" + }, + "custom": { + "axisBorderShow": false, + "axisCenteredZero": false, + "axisColorMode": "text", + "axisLabel": "", + "axisPlacement": "auto", + "fillOpacity": 50, + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + }, + "pointShape": "circle", + "pointSize": { + "fixed": 5 + }, + "pointStrokeWidth": 1, + "scaleDistribution": { + "type": "linear" + }, + "show": "points+lines" + }, + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green", + "value": 0 + }, + { + "color": "red", + "value": 80 + } + ] + }, + "unit": "ms" + }, + "overrides": [ + { + "matcher": { + "id": "byName", + "options": "gas_used_mgas" + }, + "properties": [ + { + "id": "unit", + "value": "MGas" + }, + { + "id": "custom.axisPlacement", + "value": "right" + }, + { + "id": "custom.lineStyle", + "value": { + "dash": [ + 10, + 10 + ], + "fill": "dash" + } + }, + { + "id": "custom.hideFrom", + "value": { + "legend": true, + "tooltip": false, + "viz": false + } + } + ] + }, + { + "matcher": { + "id": "byName", + "options": "jrpc_id" + }, + "properties": [ + { + "id": "unit", + "value": "none" + } + ] + }, + { + "matcher": { + "id": "byName", + "options": "client_ggas_s" + }, + "properties": [ + { + "id": "unit", + "value": "GGas/s" + }, + { + "id": "custom.axisPlacement", + "value": "left" + } + ] + } + ] }, - "properties": [ - { - "id": "filterable", - "value": false + "options": { + "legend": { + "calcs": [ + "min", + "mean", + "max" + ], + "displayMode": "table", + "overflow": "ellipsis", + "placement": "bottom", + "showLegend": true + }, + "mapping": "auto", + "series": [ + {} + ], + "tooltip": { + "hideZeros": false, + "mode": "single", + "sort": "none" } - ] + } }, - { - "matcher": { - "id": "byName", - "options": "method" - }, - "properties": [ + "version": "13.2.0-28666480772" + } + } + }, + "panel-28": { + "kind": "Panel", + "spec": { + "data": { + "kind": "QueryGroup", + "spec": { + "queries": [ { - "id": "filterable", - "value": false + "kind": "PanelQuery", + "spec": { + "hidden": false, + "query": { + "datasource": { + "name": "${DS_PROMETHEUS}" + }, + "group": "prometheus", + "kind": "DataQuery", + "spec": { + "editorMode": "code", + "expr": "k6_expb_slowest_payloads{tool=\"expb\", testid=~\"$testid\", name!=\"payload_fetch\", group!~\"::setup::.*\"}", + "format": "table", + "instant": false, + "legendFormat": "{{testid}}", + "range": true + }, + "version": "v0" + }, + "refId": "A" + } } - ] - }, - { - "matcher": { - "id": "byName", - "options": "status" - }, - "properties": [ + ], + "queryOptions": {}, + "transformations": [ { - "id": "filterable", - "value": false + "group": "convertFieldType", + "kind": "Transformation", + "spec": { + "options": { + "conversions": [ + { + "destinationType": "number", + "targetField": "jrpc_id" + }, + { + "destinationType": "number", + "targetField": "gas_used" + } + ], + "fields": {} + } + } }, { - "id": "custom.width", - "value": 62 - } - ] - }, - { - "matcher": { - "id": "byName", - "options": "min" - }, - "properties": [ - { - "id": "unit", - "value": "s" + "group": "groupBy", + "kind": "Transformation", + "spec": { + "options": { + "fields": { + "Value": { + "aggregations": [ + "mean" + ], + "operation": "aggregate" + }, + "Value #A": { + "aggregations": [ + "mean" + ], + "operation": "aggregate" + }, + "Value #B": { + "aggregations": [ + "mean" + ], + "operation": "aggregate" + }, + "Value #C": { + "aggregations": [ + "mean" + ], + "operation": "aggregate" + }, + "Value #D": { + "aggregations": [ + "mean" + ], + "operation": "aggregate" + }, + "client_type": { + "aggregations": [], + "operation": "groupby" + }, + "gas_used": { + "aggregations": [ + "last" + ], + "operation": "aggregate" + }, + "jrpc_id": { + "aggregations": [], + "operation": "groupby" + }, + "jrpc_method": { + "aggregations": [], + "operation": "groupby" + }, + "scenario": { + "aggregations": [] + }, + "testid": { + "aggregations": [], + "operation": "groupby" + } + } + } + } }, { - "id": "custom.width", - "value": 105 - } - ] - }, - { - "matcher": { - "id": "byName", - "options": "max" - }, - "properties": [ + "group": "calculateField", + "kind": "Transformation", + "spec": { + "options": { + "alias": "gas_used_mgas", + "binary": { + "left": { + "matcher": { + "id": "byName", + "options": "gas_used (last)" + } + }, + "operator": "/", + "right": { + "fixed": "1000000.0" + } + }, + "mode": "binary", + "replaceFields": false + } + } + }, { - "id": "unit", - "value": "s" + "group": "calculateField", + "kind": "Transformation", + "spec": { + "options": { + "alias": "client_ggas_s", + "binary": { + "left": { + "matcher": { + "id": "byName", + "options": "gas_used_mgas" + } + }, + "operator": "/", + "right": { + "matcher": { + "id": "byName", + "options": "Value (mean)" + } + } + }, + "mode": "binary", + "reduce": { + "reducer": "sum" + } + } + } }, { - "id": "custom.width", - "value": 91 - } - ] - }, - { - "matcher": { - "id": "byName", - "options": "p95" - }, - "properties": [ + "group": "sortBy", + "kind": "Transformation", + "spec": { + "options": { + "fields": {}, + "sort": [ + { + "field": "jrpc_id" + } + ] + } + } + }, { - "id": "unit", - "value": "s" + "group": "organize", + "kind": "Transformation", + "spec": { + "options": { + "excludeByName": { + "Value (mean)": true, + "gas_used (last)": true + }, + "includeByName": {}, + "indexByName": {}, + "renameByName": { + "Value (mean)": "", + "client_ggas_s": "GGas/s", + "client_type": "Client Type", + "gas_used (last)": "", + "gas_used_mgas": "Gas Used", + "jrpc_id": "Payload", + "jrpc_method": "Method", + "testid": "TestID" + } + } + } }, { - "id": "custom.width", - "value": 96 + "group": "partitionByValues", + "kind": "Transformation", + "spec": { + "options": { + "fields": [ + "TestID" + ], + "keepFields": false, + "naming": { + "asLabels": false + } + } + } } ] - }, - { - "matcher": { - "id": "byName", - "options": "p99" + } + }, + "description": "Gas Usage calculation for full engine_newPayload request duration.", + "id": 28, + "links": [], + "title": "Gas Usage per Payload (Full)", + "vizConfig": { + "group": "xychart", + "kind": "VizConfig", + "spec": { + "fieldConfig": { + "defaults": { + "color": { + "mode": "palette-classic" + }, + "custom": { + "axisBorderShow": false, + "axisCenteredZero": false, + "axisColorMode": "text", + "axisLabel": "", + "axisPlacement": "auto", + "fillOpacity": 50, + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + }, + "pointShape": "circle", + "pointSize": { + "fixed": 5 + }, + "pointStrokeWidth": 1, + "scaleDistribution": { + "type": "linear" + }, + "show": "points+lines" + }, + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green", + "value": 0 + }, + { + "color": "red", + "value": 80 + } + ] + }, + "unit": "ms" + }, + "overrides": [ + { + "matcher": { + "id": "byName", + "options": "gas_used_mgas" + }, + "properties": [ + { + "id": "unit", + "value": "MGas" + }, + { + "id": "custom.axisPlacement", + "value": "right" + }, + { + "id": "custom.lineStyle", + "value": { + "dash": [ + 10, + 10 + ], + "fill": "dash" + } + }, + { + "id": "custom.hideFrom", + "value": { + "legend": true, + "tooltip": false, + "viz": false + } + } + ] + }, + { + "matcher": { + "id": "byName", + "options": "jrpc_id" + }, + "properties": [ + { + "id": "unit", + "value": "none" + } + ] + }, + { + "matcher": { + "id": "byName", + "options": "client_ggas_s" + }, + "properties": [ + { + "id": "unit", + "value": "GGas/s" + }, + { + "id": "custom.axisPlacement", + "value": "left" + } + ] + } + ] }, - "properties": [ - { - "id": "unit", - "value": "s" + "options": { + "legend": { + "calcs": [ + "min", + "mean", + "max" + ], + "displayMode": "table", + "overflow": "ellipsis", + "placement": "bottom", + "showLegend": true }, - { - "id": "custom.width", - "value": 105 + "mapping": "auto", + "series": [ + {} + ], + "tooltip": { + "hideZeros": false, + "mode": "single", + "sort": "none" } - ] + } }, - { - "matcher": { - "id": "byName", - "options": "avg" - }, - "properties": [ + "version": "13.2.0-28666480772" + } + } + }, + "panel-29": { + "kind": "Panel", + "spec": { + "data": { + "kind": "QueryGroup", + "spec": { + "queries": [ { - "id": "unit", - "value": "s" + "kind": "PanelQuery", + "spec": { + "hidden": true, + "query": { + "datasource": { + "name": "${DS_PROMETHEUS}" + }, + "group": "prometheus", + "kind": "DataQuery", + "spec": { + "editorMode": "code", + "expr": "sum by (testid) (k6_vus{tool=\"expb\", testid=~\"$testid\", name!=\"payload_fetch\", group!~\"::setup::.*\"})", + "instant": false, + "legendFormat": "vus - {{testid}}", + "range": true + }, + "version": "v0" + }, + "refId": "A" + } }, { - "id": "custom.width", - "value": 102 - } - ] - }, - { - "matcher": { - "id": "byName", - "options": "med" - }, - "properties": [ + "kind": "PanelQuery", + "spec": { + "hidden": false, + "query": { + "datasource": { + "name": "${DS_PROMETHEUS}" + }, + "group": "prometheus", + "kind": "DataQuery", + "spec": { + "editorMode": "code", + "expr": "avg by (testid) (k6_iteration_duration_$quantile_stat{tool=\"expb\", testid=~\"$testid\", name!=\"payload_fetch\", group!~\"::setup::.*\"})", + "instant": false, + "legendFormat": "http_req_duration_$quantile_stat - {{testid}}", + "range": true + }, + "version": "v0" + }, + "refId": "C" + } + }, { - "id": "unit", - "value": "s" - } - ] - }, - { - "matcher": { - "id": "byName", - "options": "group" - }, - "properties": [ + "kind": "PanelQuery", + "spec": { + "hidden": false, + "query": { + "datasource": { + "name": "${DS_PROMETHEUS}" + }, + "group": "prometheus", + "kind": "DataQuery", + "spec": { + "editorMode": "code", + "expr": "sum by (testid) (irate(k6_http_reqs_total{tool=\"expb\", testid=~\"$testid\", name!=\"payload_fetch\", group!~\"::setup::.*\"}[$__rate_interval]))", + "instant": false, + "legendFormat": "http_req_s - {{testid}}", + "range": true + }, + "version": "v0" + }, + "refId": "B" + } + }, { - "id": "custom.width", - "value": 232 + "kind": "PanelQuery", + "spec": { + "hidden": true, + "query": { + "datasource": { + "name": "${DS_PROMETHEUS}" + }, + "group": "prometheus", + "kind": "DataQuery", + "spec": { + "editorMode": "code", + "expr": "avg by (testid) (round(k6_http_req_failed_rate{tool=\"expb\", testid=~\"$testid\", name!=\"payload_fetch\", group!~\"::setup::.*\"}, 0.1)*100)", + "instant": false, + "legendFormat": "http_req_failed - {{testid}}", + "range": true + }, + "version": "v0" + }, + "refId": "E" + } + }, + { + "kind": "PanelQuery", + "spec": { + "hidden": false, + "query": { + "datasource": { + "name": "${DS_PROMETHEUS}" + }, + "group": "prometheus", + "kind": "DataQuery", + "spec": { + "editorMode": "code", + "expr": "sum by (testid) (irate(k6_http_reqs_total{tool=\"expb\", testid=~\"$testid\", expected_response=\"false\", name!=\"payload_fetch\", group!~\"::setup::.*\"}[$__rate_interval]))", + "instant": false, + "legendFormat": "http_req_s_errors - {{testid}}", + "range": true + }, + "version": "v0" + }, + "refId": "D" + } } - ] - } - ] - }, - "gridPos": { - "h": 12, - "w": 24, - "x": 0, - "y": 112 - }, - "id": 17, - "options": { - "cellHeight": "sm", - "footer": { - "countRows": false, - "enablePagination": true, - "fields": "", - "reducer": [ - "sum" - ], - "show": false - }, - "frameIndex": 2, - "showHeader": true, - "sortBy": [ - { - "desc": false, - "displayName": "group" + ], + "queryOptions": {}, + "transformations": [] } - ] - }, - "pluginVersion": "11.6.0", - "targets": [ - { - "datasource": { - "type": "prometheus", - "uid": "prometheus_ds_1" - }, - "editorMode": "code", - "expr": "k6_http_req_duration_min{tool=\"expb\", testid=~\"$testid\", name!=\"payload_fetch\", group!~\"::setup::.*\"}", - "format": "table", - "hide": false, - "instant": false, - "legendFormat": "min", - "range": true, - "refId": "A" - }, - { - "datasource": { - "type": "prometheus", - "uid": "prometheus_ds_1" - }, - "editorMode": "code", - "expr": "k6_http_req_duration_max{tool=\"expb\", testid=~\"$testid\", name!=\"payload_fetch\", group!~\"::setup::.*\"}", - "format": "table", - "hide": false, - "instant": false, - "legendFormat": "max", - "range": true, - "refId": "B" - }, - { - "datasource": { - "type": "prometheus", - "uid": "prometheus_ds_1" - }, - "editorMode": "code", - "expr": "k6_http_req_duration_p95{tool=\"expb\", testid=~\"$testid\", name!=\"payload_fetch\", group!~\"::setup::.*\"}", - "format": "table", - "hide": false, - "instant": false, - "legendFormat": "p95", - "range": true, - "refId": "C" - }, - { - "datasource": { - "type": "prometheus", - "uid": "prometheus_ds_1" - }, - "editorMode": "code", - "expr": "k6_http_req_duration_p99{tool=\"expb\", testid=~\"$testid\", name!=\"payload_fetch\", group!~\"::setup::.*\"}", - "format": "table", - "hide": false, - "instant": false, - "legendFormat": "__auto", - "range": true, - "refId": "D" - }, - { - "datasource": { - "type": "prometheus", - "uid": "prometheus_ds_1" - }, - "editorMode": "code", - "expr": "k6_http_req_duration_avg{tool=\"expb\",testid=~\"$testid\", name!=\"payload_fetch\", group!~\"::setup::.*\"}", - "format": "table", - "hide": false, - "instant": false, - "legendFormat": "avg", - "range": true, - "refId": "E" - }, - { - "datasource": { - "type": "prometheus", - "uid": "prometheus_ds_1" - }, - "editorMode": "code", - "expr": "k6_http_req_duration_med{tool=\"expb\",testid=~\"$testid\", name!=\"payload_fetch\", group!~\"::setup::.*\"}", - "format": "table", - "hide": false, - "instant": false, - "legendFormat": "med", - "range": true, - "refId": "F" - } - ], - "title": "Requests by URL", - "transformations": [ - { - "id": "merge", - "options": {} }, - { - "id": "groupBy", - "options": { - "fields": { - "Value #B": { - "aggregations": [ - "min" - ], - "operation": "aggregate" - }, - "Value #C": { - "aggregations": [ - "max" - ], - "operation": "aggregate" - }, - "Value #D": { - "aggregations": [ - "mean" - ], - "operation": "aggregate" - }, - "Value #E": { - "aggregations": [ - "mean" - ], - "operation": "aggregate" + "description": "", + "id": 29, + "links": [], + "title": "Performance Overview", + "vizConfig": { + "group": "timeseries", + "kind": "VizConfig", + "spec": { + "fieldConfig": { + "defaults": { + "color": { + "mode": "palette-classic" + }, + "custom": { + "axisBorderShow": false, + "axisCenteredZero": false, + "axisColorMode": "text", + "axisLabel": "", + "axisPlacement": "auto", + "barAlignment": 0, + "barWidthFactor": 0.6, + "drawStyle": "line", + "fillOpacity": 0, + "gradientMode": "none", + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + }, + "insertNulls": false, + "lineInterpolation": "linear", + "lineWidth": 1, + "pointSize": 5, + "scaleDistribution": { + "type": "linear" + }, + "showPoints": "auto", + "showValues": false, + "spanNulls": false, + "stacking": { + "group": "A", + "mode": "none" + }, + "thresholdsStyle": { + "mode": "off" + } + }, + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green", + "value": 0 + } + ] + } }, - "Value #F": { - "aggregations": [ - "mean" - ], - "operation": "aggregate" + "overrides": [ + { + "matcher": { + "id": "byName", + "options": "http_req_s_errors" + }, + "properties": [ + { + "id": "color", + "value": { + "fixedColor": "red", + "mode": "fixed" + } + }, + { + "id": "custom.lineStyle", + "value": { + "dash": [ + 10, + 10 + ], + "fill": "dash" + } + }, + { + "id": "custom.axisPlacement", + "value": "right" + }, + { + "id": "unit", + "value": "reqps" + } + ] + }, + { + "matcher": { + "id": "byName", + "options": "http_req_s" + }, + "properties": [ + { + "id": "unit", + "value": "reqps" + }, + { + "id": "custom.axisPlacement", + "value": "right" + }, + { + "id": "color", + "value": { + "fixedColor": "yellow", + "mode": "fixed" + } + }, + { + "id": "custom.lineStyle", + "value": { + "dash": [ + 10, + 10 + ], + "fill": "dash" + } + } + ] + }, + { + "matcher": { + "id": "byName", + "options": "vus" + }, + "properties": [ + { + "id": "color", + "value": { + "mode": "fixed" + } + }, + { + "id": "unit", + "value": "VUs" + } + ] + }, + { + "matcher": { + "id": "byRegexp", + "options": "http_req_duration_[a-zA-Z0-9_]+" + }, + "properties": [ + { + "id": "unit", + "value": "s" + }, + { + "id": "custom.axisPlacement", + "value": "right" + }, + { + "id": "color", + "value": { + "fixedColor": "blue", + "mode": "fixed" + } + } + ] + } + ] + }, + "options": { + "annotations": { + "clustering": -1, + "multiLane": false }, - "Value #G": { - "aggregations": [ + "legend": { + "calcs": [ "mean" ], - "operation": "aggregate" + "displayMode": "list", + "enableFacetedFilter": false, + "overflow": "ellipsis", + "placement": "bottom", + "showLegend": true }, - "group": { - "aggregations": [], - "operation": "groupby" - }, - "method": { - "aggregations": [], - "operation": "groupby" - }, - "name": { - "aggregations": [], - "operation": "groupby" - }, - "status": { - "aggregations": [], - "operation": "groupby" - }, - "testid": { - "aggregations": [], - "operation": "groupby" + "tooltip": { + "hideZeros": false, + "mode": "multi", + "sort": "none" } } + }, + "version": "13.2.0-28666480772" + } + } + }, + "panel-4": { + "kind": "Panel", + "spec": { + "data": { + "kind": "QueryGroup", + "spec": { + "queries": [ + { + "kind": "PanelQuery", + "spec": { + "hidden": false, + "query": { + "datasource": { + "name": "${DS_PROMETHEUS}" + }, + "group": "prometheus", + "kind": "DataQuery", + "spec": { + "disableTextWrap": false, + "editorMode": "code", + "expr": "sum by(testid) (k6_iterations_total{tool=\"expb\", testid=~\"$testid\"})", + "fullMetaSearch": false, + "includeNullMetadata": true, + "instant": false, + "legendFormat": "{{testid}}", + "range": true, + "useBackend": false + }, + "version": "v0" + }, + "refId": "A" + } + } + ], + "queryOptions": {}, + "transformations": [] } }, - { - "id": "organize", - "options": { - "excludeByName": { - "Time": true - }, - "includeByName": {}, - "indexByName": { - "Time": 0, - "Value #B": 4, - "Value #C": 5, - "Value #D": 6, - "Value #E": 7, - "method": 2, - "name": 1, - "status": 3 + "description": "", + "id": 4, + "links": [], + "title": "Iterations", + "vizConfig": { + "group": "stat", + "kind": "VizConfig", + "spec": { + "fieldConfig": { + "defaults": { + "color": { + "mode": "thresholds" + }, + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green", + "value": 0 + } + ] + }, + "unit": "none" + }, + "overrides": [] }, - "renameByName": { - "Value #B": "min", - "Value #B (min)": "min", - "Value #C": "max", - "Value #C (max)": "max", - "Value #D": "p95", - "Value #D (mean)": "p95", - "Value #E": "p99", - "Value #E (mean)": "p99", - "Value #F (mean)": "avg", - "Value #G (mean)": "med" + "options": { + "colorMode": "value", + "graphMode": "none", + "justifyMode": "auto", + "orientation": "auto", + "percentChangeColorMode": "standard", + "reduceOptions": { + "calcs": [], + "fields": "", + "values": false + }, + "showPercentChange": false, + "textMode": "auto", + "wideLayout": true } - } + }, + "version": "13.2.0-28666480772" } - ], - "type": "table" - }, - { - "collapsed": false, - "gridPos": { - "h": 1, - "w": 24, - "x": 0, - "y": 124 - }, - "id": 11, - "panels": [], - "title": "Checks", - "type": "row" + } }, - { - "datasource": { - "type": "prometheus", - "uid": "prometheus_ds_1" - }, - "fieldConfig": { - "defaults": { - "color": { - "mode": "thresholds" - }, - "custom": { - "align": "auto", - "cellOptions": { - "type": "auto" - }, - "inspect": false - }, - "mappings": [], - "thresholds": { - "mode": "absolute", - "steps": [ + "panel-8": { + "kind": "Panel", + "spec": { + "data": { + "kind": "QueryGroup", + "spec": { + "queries": [ { - "color": "green" + "kind": "PanelQuery", + "spec": { + "hidden": false, + "query": { + "datasource": { + "name": "${DS_PROMETHEUS}" + }, + "group": "prometheus", + "kind": "DataQuery", + "spec": { + "editorMode": "code", + "expr": "avg by (testid) (irate(k6_data_sent_total{tool=\"expb\", testid=~\"$testid\"}[$__rate_interval]))", + "instant": false, + "legendFormat": "data_sent - {{testid}}", + "range": true + }, + "version": "v0" + }, + "refId": "A" + } + }, + { + "kind": "PanelQuery", + "spec": { + "hidden": false, + "query": { + "datasource": { + "name": "${DS_PROMETHEUS}" + }, + "group": "prometheus", + "kind": "DataQuery", + "spec": { + "editorMode": "code", + "exemplar": false, + "expr": "avg by (testid) (irate(k6_data_received_total{tool=\"expb\", testid=~\"$testid\"}[$__rate_interval]))", + "instant": false, + "legendFormat": "data_received - {{testid}}", + "range": true + }, + "version": "v0" + }, + "refId": "B" + } } - ] + ], + "queryOptions": {}, + "transformations": [] } }, - "overrides": [ - { - "matcher": { - "id": "byName", - "options": "Success Rate" + "description": "", + "id": 8, + "links": [], + "title": "Transfer Rate", + "vizConfig": { + "group": "timeseries", + "kind": "VizConfig", + "spec": { + "fieldConfig": { + "defaults": { + "color": { + "mode": "palette-classic" + }, + "custom": { + "axisBorderShow": false, + "axisCenteredZero": false, + "axisColorMode": "text", + "axisLabel": "", + "axisPlacement": "auto", + "barAlignment": 0, + "barWidthFactor": 0.6, + "drawStyle": "line", + "fillOpacity": 0, + "gradientMode": "none", + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + }, + "insertNulls": false, + "lineInterpolation": "linear", + "lineWidth": 1, + "pointSize": 5, + "scaleDistribution": { + "type": "linear" + }, + "showPoints": "auto", + "showValues": false, + "spanNulls": false, + "stacking": { + "group": "A", + "mode": "none" + }, + "thresholdsStyle": { + "mode": "off" + } + }, + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green", + "value": 0 + } + ] + }, + "unit": "bytes" + }, + "overrides": [] }, - "properties": [ - { - "id": "custom.hidden", - "value": false + "options": { + "annotations": { + "clustering": -1, + "multiLane": false }, - { - "id": "unit", - "value": "%" + "legend": { + "calcs": [], + "displayMode": "list", + "enableFacetedFilter": false, + "overflow": "ellipsis", + "placement": "bottom", + "showLegend": true + }, + "tooltip": { + "hideZeros": false, + "mode": "single", + "sort": "none" } - ] + } }, - { - "matcher": { - "id": "byName", - "options": "Value (mean)" - }, - "properties": [ + "version": "13.2.0-28666480772" + } + } + }, + "panel-9": { + "kind": "Panel", + "spec": { + "data": { + "kind": "QueryGroup", + "spec": { + "queries": [ { - "id": "custom.hidden", - "value": true - } - ] - }, - { - "matcher": { - "id": "byName", - "options": "check" - }, - "properties": [ + "kind": "PanelQuery", + "spec": { + "hidden": false, + "query": { + "datasource": { + "name": "${DS_PROMETHEUS}" + }, + "group": "prometheus", + "kind": "DataQuery", + "spec": { + "editorMode": "code", + "expr": "avg by (testid) (k6_iteration_duration_$quantile_stat{tool=\"expb\", testid=~\"$testid\", name!=\"payload_fetch\", group!~\"::setup::.*\"})", + "instant": false, + "legendFormat": "iteration_duration_$quantile_stat - {{testid}}", + "range": true + }, + "version": "v0" + }, + "refId": "A" + } + }, { - "id": "filterable", - "value": false + "kind": "PanelQuery", + "spec": { + "hidden": false, + "query": { + "datasource": { + "name": "${DS_PROMETHEUS}" + }, + "group": "prometheus", + "kind": "DataQuery", + "spec": { + "editorMode": "code", + "exemplar": false, + "expr": "avg by (testid) (k6_dropped_iterations_total{tool=\"expb\", testid=~\"$testid\", name!=\"payload_fetch\", group!~\"::setup::.*\"})", + "instant": false, + "legendFormat": "dropped_iterations - {{testid}}", + "range": true + }, + "version": "v0" + }, + "refId": "B" + } } - ] - } - ] - }, - "gridPos": { - "h": 8, - "w": 12, - "x": 0, - "y": 125 - }, - "id": 12, - "options": { - "cellHeight": "sm", - "footer": { - "countRows": false, - "enablePagination": true, - "fields": "", - "reducer": [ - "sum" - ], - "show": false - }, - "frameIndex": 2, - "showHeader": true, - "sortBy": [ - { - "desc": true, - "displayName": "Value (count)" - } - ] - }, - "pluginVersion": "11.6.0", - "targets": [ - { - "datasource": { - "type": "prometheus", - "uid": "prometheus_ds_1" - }, - "editorMode": "code", - "exemplar": false, - "expr": "round(k6_checks_rate{tool=\"expb\", testid=~\"$testid\", name!=\"payload_fetch\", group!~\"::setup::.*\"}, 0.1)", - "format": "table", - "instant": false, - "legendFormat": "__auto", - "range": true, - "refId": "A" - } - ], - "title": "Checks list", - "transformations": [ - { - "id": "labelsToFields", - "options": { - "keepLabels": [ - "__name__", - "check" ], - "mode": "columns" + "queryOptions": {}, + "transformations": [] } }, - { - "id": "groupBy", - "options": { - "fields": { - "Value": { - "aggregations": [ - "mean" - ], - "operation": "aggregate" + "description": "", + "id": 9, + "links": [], + "title": "Iterations", + "vizConfig": { + "group": "timeseries", + "kind": "VizConfig", + "spec": { + "fieldConfig": { + "defaults": { + "color": { + "mode": "palette-classic" + }, + "custom": { + "axisBorderShow": false, + "axisCenteredZero": false, + "axisColorMode": "text", + "axisLabel": "", + "axisPlacement": "auto", + "barAlignment": 0, + "barWidthFactor": 0.6, + "drawStyle": "line", + "fillOpacity": 0, + "gradientMode": "none", + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + }, + "insertNulls": false, + "lineInterpolation": "linear", + "lineWidth": 1, + "pointSize": 5, + "scaleDistribution": { + "type": "linear" + }, + "showPoints": "auto", + "showValues": false, + "spanNulls": false, + "stacking": { + "group": "A", + "mode": "none" + }, + "thresholdsStyle": { + "mode": "off" + } + }, + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green", + "value": 0 + } + ] + }, + "unit": "s" }, - "check": { - "aggregations": [], - "operation": "groupby" + "overrides": [ + { + "matcher": { + "id": "byName", + "options": "dropped_iterations" + }, + "properties": [ + { + "id": "unit", + "value": "none" + } + ] + } + ] + }, + "options": { + "annotations": { + "clustering": -1, + "multiLane": false }, - "k6_checks_rate": { - "aggregations": [ - "sum", - "count" - ], - "operation": "aggregate" + "legend": { + "calcs": [], + "displayMode": "list", + "enableFacetedFilter": false, + "overflow": "ellipsis", + "placement": "bottom", + "showLegend": true }, - "testid": { - "aggregations": [], - "operation": "groupby" + "tooltip": { + "hideZeros": false, + "mode": "single", + "sort": "none" } } + }, + "version": "13.2.0-28666480772" + } + } + } + }, + "layout": { + "kind": "RowsLayout", + "spec": { + "rows": [ + { + "kind": "RowsLayoutRow", + "spec": { + "collapse": false, + "hideHeader": true, + "layout": { + "kind": "GridLayout", + "spec": { + "items": [ + { + "kind": "GridLayoutItem", + "spec": { + "element": { + "kind": "ElementReference", + "name": "panel-10" + }, + "height": 11, + "width": 24, + "x": 0, + "y": 0 + } + }, + { + "kind": "GridLayoutItem", + "spec": { + "element": { + "kind": "ElementReference", + "name": "panel-29" + }, + "height": 11, + "width": 24, + "x": 0, + "y": 11 + } + } + ] + } + }, + "title": "" } }, { - "id": "calculateField", - "options": { - "alias": "Success Rate", - "binary": { - "left": { - "matcher": { - "id": "byName", - "options": "Value (mean)" - } - }, - "operator": "*", - "right": { - "fixed": "100" + "kind": "RowsLayoutRow", + "spec": { + "collapse": false, + "layout": { + "kind": "GridLayout", + "spec": { + "items": [ + { + "kind": "GridLayoutItem", + "spec": { + "element": { + "kind": "ElementReference", + "name": "panel-4" + }, + "height": 5, + "width": 12, + "x": 0, + "y": 0 + } + }, + { + "kind": "GridLayoutItem", + "spec": { + "element": { + "kind": "ElementReference", + "name": "panel-20" + }, + "height": 5, + "width": 12, + "x": 12, + "y": 0 + } + }, + { + "kind": "GridLayoutItem", + "spec": { + "element": { + "kind": "ElementReference", + "name": "panel-22" + }, + "height": 5, + "width": 12, + "x": 0, + "y": 5 + } + }, + { + "kind": "GridLayoutItem", + "spec": { + "element": { + "kind": "ElementReference", + "name": "panel-21" + }, + "height": 5, + "width": 12, + "x": 12, + "y": 5 + } + }, + { + "kind": "GridLayoutItem", + "spec": { + "element": { + "kind": "ElementReference", + "name": "panel-8" + }, + "height": 8, + "width": 12, + "x": 0, + "y": 10 + } + }, + { + "kind": "GridLayoutItem", + "spec": { + "element": { + "kind": "ElementReference", + "name": "panel-9" + }, + "height": 8, + "width": 12, + "x": 12, + "y": 10 + } + } + ] } }, - "mode": "binary", - "reduce": { - "reducer": "sum" - } + "title": "Performance Overview" } }, { - "id": "convertFieldType", - "options": { - "conversions": [], - "fields": {} - } - } - ], - "type": "table" - }, - { - "datasource": { - "type": "prometheus", - "uid": "prometheus_ds_1" - }, - "description": "Filter by check name to query a particular check", - "fieldConfig": { - "defaults": { - "color": { - "mode": "palette-classic" - }, - "custom": { - "axisBorderShow": false, - "axisCenteredZero": false, - "axisColorMode": "text", - "axisLabel": "", - "axisPlacement": "auto", - "axisSoftMax": 100, - "axisSoftMin": 0, - "barAlignment": -1, - "barWidthFactor": 0.6, - "drawStyle": "line", - "fillOpacity": 0, - "gradientMode": "none", - "hideFrom": { - "legend": false, - "tooltip": false, - "viz": false - }, - "insertNulls": false, - "lineInterpolation": "linear", - "lineWidth": 2, - "pointSize": 5, - "scaleDistribution": { - "type": "linear" - }, - "showPoints": "auto", - "spanNulls": false, - "stacking": { - "group": "A", - "mode": "none" - }, - "thresholdsStyle": { - "mode": "off" - } - }, - "mappings": [], - "thresholds": { - "mode": "absolute", - "steps": [ - { - "color": "green" + "kind": "RowsLayoutRow", + "spec": { + "collapse": false, + "layout": { + "kind": "GridLayout", + "spec": { + "items": [ + { + "kind": "GridLayoutItem", + "spec": { + "element": { + "kind": "ElementReference", + "name": "panel-14" + }, + "height": 9, + "width": 24, + "x": 0, + "y": 0 + } + }, + { + "kind": "GridLayoutItem", + "spec": { + "element": { + "kind": "ElementReference", + "name": "panel-15" + }, + "height": 9, + "width": 24, + "x": 0, + "y": 9 + } + }, + { + "kind": "GridLayoutItem", + "spec": { + "element": { + "kind": "ElementReference", + "name": "panel-27" + }, + "height": 10, + "width": 24, + "x": 0, + "y": 18 + } + }, + { + "kind": "GridLayoutItem", + "spec": { + "element": { + "kind": "ElementReference", + "name": "panel-28" + }, + "height": 10, + "width": 24, + "x": 0, + "y": 28 + } + }, + { + "kind": "GridLayoutItem", + "spec": { + "element": { + "kind": "ElementReference", + "name": "panel-24" + }, + "height": 14, + "width": 24, + "x": 0, + "y": 38 + } + }, + { + "kind": "GridLayoutItem", + "spec": { + "element": { + "kind": "ElementReference", + "name": "panel-18" + }, + "height": 9, + "width": 24, + "x": 0, + "y": 52 + } + }, + { + "kind": "GridLayoutItem", + "spec": { + "element": { + "kind": "ElementReference", + "name": "panel-25" + }, + "height": 9, + "width": 24, + "x": 0, + "y": 61 + } + }, + { + "kind": "GridLayoutItem", + "spec": { + "element": { + "kind": "ElementReference", + "name": "panel-17" + }, + "height": 12, + "width": 24, + "x": 0, + "y": 70 + } + } + ] } - ] - }, - "unit": "%" - }, - "overrides": [] - }, - "gridPos": { - "h": 8, - "w": 12, - "x": 12, - "y": 125 - }, - "id": 13, - "options": { - "legend": { - "calcs": [], - "displayMode": "list", - "placement": "bottom", - "showLegend": true + }, + "title": "HTTP" + } }, - "tooltip": { - "hideZeros": false, - "mode": "single", - "sort": "none" - } - }, - "pluginVersion": "11.6.0", - "targets": [ { - "datasource": { - "type": "prometheus", - "uid": "prometheus_ds_1" - }, - "editorMode": "code", - "expr": "avg by (testid) (round(k6_checks_rate{tool=\"expb\", testid=~\"$testid\", name!=\"payload_fetch\", group!~\"::setup::.*\"}, 0.1)*100)", - "instant": false, - "legendFormat": "k6_checks_rate - {{testid}}", - "range": true, - "refId": "A" + "kind": "RowsLayoutRow", + "spec": { + "collapse": false, + "layout": { + "kind": "GridLayout", + "spec": { + "items": [ + { + "kind": "GridLayoutItem", + "spec": { + "element": { + "kind": "ElementReference", + "name": "panel-12" + }, + "height": 8, + "width": 12, + "x": 0, + "y": 0 + } + }, + { + "kind": "GridLayoutItem", + "spec": { + "element": { + "kind": "ElementReference", + "name": "panel-13" + }, + "height": 8, + "width": 12, + "x": 12, + "y": 0 + } + } + ] + } + }, + "title": "Checks" + } } - ], - "title": "Checks Success Rate (aggregate individual checks)", - "type": "timeseries" + ] } - ], + }, + "links": [], + "liveNow": false, "preload": false, - "refresh": "", - "schemaVersion": 41, "tags": [], - "templating": { - "list": [ - { + "timeSettings": { + "autoRefresh": "", + "autoRefreshIntervals": [ + "5s", + "10s", + "30s", + "1m", + "5m", + "15m", + "30m", + "1h", + "2h", + "1d" + ], + "fiscalYearStartMonth": 0, + "from": "2026-07-09T14:51:44.794Z", + "hideTimepicker": false, + "timezone": "browser", + "to": "2026-07-09T14:59:45.055Z" + }, + "title": "Execution Payloads Benchmarks Dashboard", + "variables": [ + { + "kind": "DatasourceVariable", + "spec": { + "allowCustomValue": true, "current": { - "text": "Prometheus", - "value": "prometheus_ds_1" + "text": "Core-Central-Prometheus", + "value": "bfq8nnuwzm3uob" }, "description": "Choose a Prometheus Data Source", + "hide": "dontHide", "includeAll": false, "label": "Prometheus DS", + "multi": false, "name": "DS_PROMETHEUS", "options": [], - "query": "prometheus", - "refresh": 1, + "pluginId": "prometheus", + "refresh": "onDashboardLoad", "regex": "", - "type": "datasource" - }, - { + "skipUrlSync": false + } + }, + { + "kind": "QueryVariable", + "spec": { + "allowCustomValue": true, "current": { "text": "All", "value": [ "$__all" ] }, - "datasource": { - "type": "prometheus", - "uid": "prometheus_ds_1" - }, "definition": "label_values(testid)", "description": "Filter by \"testid\" tag. Define it by tagging: k6 run --tag testid=xyz", + "hide": "dontHide", "includeAll": true, "label": "Test ID", "multi": true, "name": "testid", "options": [], "query": { - "query": "label_values(testid)", - "refId": "PrometheusVariableQueryEditor-VariableQuery" + "datasource": { + "name": "${DS_PROMETHEUS}" + }, + "group": "prometheus", + "kind": "DataQuery", + "spec": { + "query": "label_values(testid)", + "refId": "PrometheusVariableQueryEditor-VariableQuery" + }, + "version": "v0" }, - "refresh": 2, + "refresh": "onTimeRangeChanged", "regex": "", - "type": "query" - }, - { + "regexApplyTo": "value", + "skipUrlSync": false, + "sort": "disabled" + } + }, + { + "kind": "QueryVariable", + "spec": { + "allowCustomValue": true, "current": { "text": "p99", "value": "p99" }, - "datasource": { - "type": "prometheus", - "uid": "prometheus_ds_1" - }, - "definition": "metrics(k6_http_req_duration_)", + "definition": "metrics(k6_http_req_duration_.*)", "description": "Statistic for Trend Metrics Queries. The available options depend on the values of the K6_PROMETHEUS_RW_TREND_STATS setting.", + "hide": "dontHide", "includeAll": false, "label": "Trend Metrics Query", + "multi": false, "name": "quantile_stat", "options": [], "query": { - "query": "metrics(k6_http_req_duration_)", - "refId": "PrometheusVariableQueryEditor-VariableQuery" + "datasource": { + "name": "${DS_PROMETHEUS}" + }, + "group": "prometheus", + "kind": "DataQuery", + "spec": { + "qryType": 2, + "query": "metrics(k6_http_req_duration_.*)", + "refId": "PrometheusVariableQueryEditor-VariableQuery" + }, + "version": "v0" }, - "refresh": 2, + "refresh": "onTimeRangeChanged", "regex": "/http_req_duration_(min|max|count|sum|avg|med|p[0-9]+)/g", - "sort": 2, - "type": "query" + "regexApplyTo": "text", + "skipUrlSync": false, + "sort": "alphabeticalDesc" + } + }, + { + "datasource": { + "name": "prometheus_ds_1" }, - { + "group": "prometheus", + "kind": "AdhocVariable", + "spec": { + "allowCustomValue": true, "baseFilters": [], - "datasource": { - "type": "prometheus", - "uid": "prometheus_ds_1" - }, + "defaultKeys": [], "description": "Adhoc filters are applied to all panels. To enable it, go to Dashboard Settings / Variables / adhoc_filter and select the target Prometheus data source.", + "enableGroupBy": false, "filters": [], + "hide": "dontHide", "label": "AdhocFilter", "name": "adhoc_filter", - "type": "adhoc" + "skipUrlSync": false } - ] - }, - "time": { - "from": "now-3h", - "to": "now" - }, - "timepicker": {}, - "timezone": "", - "title": "Execution Payloads Benchmarks Dashboard", - "uid": "ccbb2351-2ae2-462f-ae0e-f2c893ad1028", - "version": 34 + } + ] } \ No newline at end of file diff --git a/pyproject.toml b/pyproject.toml index f6ba9c9..c21afca 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -13,6 +13,7 @@ dependencies = [ "jinja2>=3.1.6", "pydantic>=2.11.7", "pyyaml>=6.0.2", + "requests>=2.32.0", "rich>=14.0.0", "structlog>=25.4.0", "typer>=0.16.0", @@ -37,4 +38,12 @@ packages = ["src/expb"] [dependency-groups] dev = [ "hatchling>=1.29.0", + "pytest>=8.0.0", ] + +[tool.pytest.ini_options] +testpaths = ["tests"] +markers = [ + "integration: tests that require live EL/Beacon endpoints (deselected by default)", +] +addopts = "-m 'not integration'" diff --git a/src/expb/_version.py b/src/expb/_version.py index 12fb7ed..0594ff5 100644 --- a/src/expb/_version.py +++ b/src/expb/_version.py @@ -1,2 +1,2 @@ __version__ = "0.1.0" -__commit__ = "4d5423b" +__commit__ = "7621458" diff --git a/src/expb/configs/defaults.py b/src/expb/configs/defaults.py index ba82dd0..1d358ef 100644 --- a/src/expb/configs/defaults.py +++ b/src/expb/configs/defaults.py @@ -2,7 +2,7 @@ # Images ## Grafana K6 -K6_DEFAULT_IMAGE = "grafana/k6:1.6.1" +K6_DEFAULT_IMAGE = "grafana/k6:2.1.0" ## Grafana Alloy ALLOY_DEFAULT_IMAGE = "grafana/alloy:v1.13.2" ## Payload Server diff --git a/src/expb/configs/networks.py b/src/expb/configs/networks.py index 0f89ded..6638187 100644 --- a/src/expb/configs/networks.py +++ b/src/expb/configs/networks.py @@ -55,9 +55,16 @@ def __init__( self, name: str, forks_timestamps: dict[Fork, int], + genesis_time: int, + seconds_per_slot: int = 12, ): self.name = name self.forks_timestamps = forks_timestamps + self.genesis_time = genesis_time + self.seconds_per_slot = seconds_per_slot + + def slot_from_timestamp(self, timestamp: int) -> int: + return (timestamp - self.genesis_time) // self.seconds_per_slot def get_fork_timestamp(self, fork: Fork) -> int: return self.forks_timestamps.get(fork, -1) @@ -86,6 +93,8 @@ class Network(Enum): Fork.PRAGUE: 1746612311, Fork.OSAKA: 1764798551, }, + genesis_time=1606824023, + seconds_per_slot=12, ) @classmethod diff --git a/src/expb/generate_payloads.py b/src/expb/generate_payloads.py index 520986c..f80edfa 100644 --- a/src/expb/generate_payloads.py +++ b/src/expb/generate_payloads.py @@ -13,7 +13,10 @@ @app.command() def generate_payloads( - rpc_url: Annotated[str, typer.Option(help="Ethereum RPC URL")], + rpc_url: Annotated[str, typer.Option(help="Ethereum execution RPC URL")], + beacon_url: Annotated[ + str, typer.Option(help="Ethereum consensus (Beacon) API URL") + ], network: Annotated[Network, typer.Option(help="Network")] = Network.MAINNET, start_block: Annotated[int, typer.Option(help="Start block")] = 0, end_block: Annotated[int | None, typer.Option(help="End block")] = None, @@ -32,9 +35,6 @@ def generate_payloads( threads: Annotated[ int, typer.Option(help="Number of threads for parallel processing") ] = 10, - workers: Annotated[ - int, typer.Option(help="Number of workers per thread for parallel processing") - ] = 30, ) -> None: """ Generate execution payloads requests for a given block range. @@ -49,13 +49,13 @@ def generate_payloads( generator = Generator( rpc_url=rpc_url, + beacon_url=beacon_url, network=network, start_block=start_block, end_block=end_block, output_dir=output_dir, join_payloads=join_payloads, threads=threads, - workers=workers, logger=logger, ) @@ -63,6 +63,7 @@ def generate_payloads( "Starting payloads generation", network=network.value, rpc_url=rpc_url, + beacon_url=beacon_url, start_block=start_block, end_block=end_block, ) diff --git a/src/expb/payloads/executor/executor_config.py b/src/expb/payloads/executor/executor_config.py index df83fb0..a03cb07 100644 --- a/src/expb/payloads/executor/executor_config.py +++ b/src/expb/payloads/executor/executor_config.py @@ -518,6 +518,18 @@ def get_k6_command( f"--env=EXPB_WARMUP_WAIT={self.k6_warmup_wait}", f"--env=testid={self.test_id}", ] + # Run-general tags: pass as output-level --tag so they attach to EVERY + # exported series, including k6's self-emitted engine metrics + # (k6_iterations_total, k6_vus, k6_data_*), which do not inherit the + # script-side options.tags / scenario tags in the Prometheus + # remote-write output. + run_tags = { + "testid": self.test_id, + "tool": "expb", + "client_type": self.execution_client.value.name, + } + for key, value in run_tags.items(): + command.append(f"--tag={key}={value}") if self.exports is not None and self.exports.prometheus_rw is not None: command.append("--out=experimental-prometheus-rw") for tag in self.exports.prometheus_rw.tags: diff --git a/src/expb/payloads/generator.py b/src/expb/payloads/generator.py index 6d414b3..8bb1648 100644 --- a/src/expb/payloads/generator.py +++ b/src/expb/payloads/generator.py @@ -1,12 +1,13 @@ -import asyncio +import hashlib import json import os import re from concurrent.futures import ThreadPoolExecutor from pathlib import Path +import requests from web3 import Web3 -from web3.types import BlockData, HexBytes, TxData +from web3.types import BlockData from expb.configs.networks import Fork, Network from expb.logging import Logger @@ -17,21 +18,22 @@ def __init__( self, network: Network, rpc_url: str, + beacon_url: str, start_block: int, output_dir: Path, end_block: int | None = None, # if None, will use the latest block join_payloads: bool = True, threads: int = 10, - workers: int = 30, logger=Logger(), ): self.w3 = Web3(Web3.HTTPProvider(rpc_url)) + self.beacon_url = beacon_url.rstrip("/") + self.beacon = requests.Session() self.network = network.value self.start_block = start_block self.output_dir = output_dir self.join_payloads = join_payloads self.threads = threads - self.workers = workers self.log = logger if end_block is None: self.end_block = self.w3.eth.get_block("latest")["number"] @@ -68,169 +70,132 @@ def get_fcu_version(self, block: BlockData) -> int: else: raise ValueError(f"Unknown fork: {fork}") - def compose_payload_v1( - self, - block: BlockData, - transactions: list[str], - ) -> dict: - return { - "parentHash": block["parentHash"].to_0x_hex(), - "feeRecipient": block["miner"], - "stateRoot": block["stateRoot"].to_0x_hex(), - "receiptsRoot": block["receiptsRoot"].to_0x_hex(), - "logsBloom": block["logsBloom"].to_0x_hex(), - "prevRandao": block["mixHash"].to_0x_hex(), - "blockNumber": hex(block["number"]), - "gasLimit": hex(block["gasLimit"]), - "gasUsed": hex(block["gasUsed"]), - "timestamp": hex(block["timestamp"]), - "extraData": block["extraData"].to_0x_hex(), - "baseFeePerGas": hex(block["baseFeePerGas"]), - "blockHash": block["hash"].to_0x_hex(), - "transactions": transactions, - } - - def compose_payload_v2( - self, - block: BlockData, - transactions: list[str], - withdrawals: list[dict], - ) -> dict: - payload = self.compose_payload_v1(block, transactions) - payload.update( - { - "withdrawals": withdrawals, - } + def get_beacon_block(self, slot: int) -> dict: + resp = self.beacon.get( + f"{self.beacon_url}/eth/v2/beacon/blocks/{slot}", + timeout=120, ) - return payload + if resp.status_code == 404: + raise ValueError(f"No beacon block found for slot {slot}") + resp.raise_for_status() + return resp.json()["data"]["message"] - def compose_payload_v3( - self, - block: BlockData, - transactions: list[str], - withdrawals: list[dict], - ) -> dict: - payload = self.compose_payload_v2(block, transactions, withdrawals) - block["parentBeaconBlockRoot"] - payload.update( + @staticmethod + def _to_hex(dec_str: str | int) -> str: + return hex(int(dec_str)) + + @staticmethod + def _to_bytes(hex_str: str) -> bytes: + return bytes.fromhex(hex_str[2:] if hex_str.startswith("0x") else hex_str) + + @staticmethod + def _u64_le(dec_str: str | int) -> bytes: + return int(dec_str).to_bytes(8, "little") + + def map_withdrawals(self, withdrawals: list[dict]) -> list[dict]: + return [ { - "blobGasUsed": ( - hex(block["blobGasUsed"]) - if hasattr(block, "blobGasUsed") - else "0x0" - ), - "excessBlobGas": ( - hex(block["excessBlobGas"]) - if hasattr(block, "excessBlobGas") - else "0x0" - ), + "index": self._to_hex(wd["index"]), + "validatorIndex": self._to_hex(wd["validator_index"]), + "address": wd["address"], + "amount": self._to_hex(wd["amount"]), } - ) - return payload + for wd in withdrawals + ] - def compose_payload_v4( - self, - block: BlockData, - transactions: list[str], - withdrawals: list[dict], - ) -> dict: - payload = self.compose_payload_v3(block, transactions, withdrawals) - # payload.update({}) + def compose_payload(self, ep: dict, version: int) -> dict: + payload = { + "parentHash": ep["parent_hash"], + "feeRecipient": ep["fee_recipient"], + "stateRoot": ep["state_root"], + "receiptsRoot": ep["receipts_root"], + "logsBloom": ep["logs_bloom"], + "prevRandao": ep["prev_randao"], + "blockNumber": self._to_hex(ep["block_number"]), + "gasLimit": self._to_hex(ep["gas_limit"]), + "gasUsed": self._to_hex(ep["gas_used"]), + "timestamp": self._to_hex(ep["timestamp"]), + "extraData": ep["extra_data"], + "baseFeePerGas": self._to_hex(ep["base_fee_per_gas"]), + "blockHash": ep["block_hash"], + "transactions": ep["transactions"], + } + if version >= 2: + payload["withdrawals"] = self.map_withdrawals(ep.get("withdrawals", [])) + if version >= 3: + payload["blobGasUsed"] = self._to_hex(ep["blob_gas_used"]) + payload["excessBlobGas"] = self._to_hex(ep["excess_blob_gas"]) return payload - async def get_raw_tx( - self, - tx_semaphore: asyncio.Semaphore, - tx_hash: str, - ) -> str: - async with tx_semaphore: - raw = self.w3.eth.get_raw_transaction(tx_hash) # ty:ignore[invalid-argument-type] - return self.w3.to_hex(raw) + def get_blobs_versioned_hashes(self, commitments: list[str]) -> list[str]: + versioned_hashes = [] + for commitment in commitments: + digest = hashlib.sha256(self._to_bytes(commitment)).digest() + versioned_hashes.append("0x01" + digest[1:].hex()) + return versioned_hashes - async def get_block_transactions( - self, - block: BlockData, - ) -> list[str]: - tasks = [] - tx_semaphore = asyncio.Semaphore(self.workers) - tx_hashes = [ - tx.to_0x_hex() if isinstance(tx, HexBytes) else tx["hash"].to_0x_hex() - for tx in block["transactions"] - ] - for tx_hash in tx_hashes: - tx_task = self.get_raw_tx(tx_semaphore, tx_hash) - tasks.append(tx_task) - transactions = await asyncio.gather(*tasks) - return transactions + def encode_deposit_request(self, d: dict) -> bytes: + return ( + self._to_bytes(d["pubkey"]) + + self._to_bytes(d["withdrawal_credentials"]) + + self._u64_le(d["amount"]) + + self._to_bytes(d["signature"]) + + self._u64_le(d["index"]) + ) - def get_block_withdrawals(self, block: BlockData) -> list[dict]: - if hasattr(block, "withdrawals") and block["withdrawals"] is not None: - return [ - { - "index": hex(wd["index"]), - "validatorIndex": hex(wd["validatorIndex"]), - "address": wd["address"], - "amount": hex(wd["amount"]), - } - for wd in block["withdrawals"] - ] - return [] + def encode_withdrawal_request(self, w: dict) -> bytes: + return ( + self._to_bytes(w["source_address"]) + + self._to_bytes(w["validator_pubkey"]) + + self._u64_le(w["amount"]) + ) - def get_blobs_versioned_hashes(self, block: BlockData) -> list[str]: - blob_versioned_hashes: list[str] = [] - for tx in block["transactions"]: - tx_data: TxData | None = None - if isinstance(tx, HexBytes): - tx_data = self.w3.eth.get_transaction(tx) - else: - tx_data = tx - if hasattr(tx_data, "blobVersionedHashes"): - for hash in tx_data["blobVersionedHashes"]: - blob_versioned_hashes.append(hash.to_0x_hex()) - return blob_versioned_hashes + def encode_consolidation_request(self, c: dict) -> bytes: + return ( + self._to_bytes(c["source_address"]) + + self._to_bytes(c["source_pubkey"]) + + self._to_bytes(c["target_pubkey"]) + ) - async def get_execution_requests(self, block: BlockData) -> list[dict]: - # TODO: implement this! - return [] + def get_execution_requests(self, execution_requests: dict) -> list[str]: + """EIP-7685 encoding: list of ``request_type ++ request_data`` byte arrays, + ordered ascending by type, with empty types excluded. Each request type is a + fixed-size SSZ container, so ``request_data`` is the concatenation of its + elements.""" + requests_list: list[str] = [] + deposits = execution_requests.get("deposits", []) + withdrawals = execution_requests.get("withdrawals", []) + consolidations = execution_requests.get("consolidations", []) + if deposits: + data = b"".join(self.encode_deposit_request(d) for d in deposits) + requests_list.append("0x00" + data.hex()) + if withdrawals: + data = b"".join(self.encode_withdrawal_request(w) for w in withdrawals) + requests_list.append("0x01" + data.hex()) + if consolidations: + data = b"".join(self.encode_consolidation_request(c) for c in consolidations) + requests_list.append("0x02" + data.hex()) + return requests_list - async def get_new_payload_request(self, block: BlockData) -> dict: - block_number = block["number"] - txs_task = self.get_block_transactions(block) - version = self.get_payload_version(block) - params = [] - ( - payload, - blobs_versioned_hashes, - parent_beacon_block_root, - execution_requests, - ) = None, None, None, None - # get engine_newPayload params for each version - transactions = await txs_task - if version == 1: - payload = self.compose_payload_v1(block, transactions) - elif version == 2: - withdrawals = self.get_block_withdrawals(block) - payload = self.compose_payload_v2(block, transactions, withdrawals) - elif version == 3: - withdrawals = self.get_block_withdrawals(block) - payload = self.compose_payload_v3(block, transactions, withdrawals) - blobs_versioned_hashes = self.get_blobs_versioned_hashes(block) - parent_beacon_block_root = block["parentBeaconBlockRoot"].to_0x_hex() - elif version == 4: - withdrawals = self.get_block_withdrawals(block) - payload = self.compose_payload_v4(block, transactions, withdrawals) - blobs_versioned_hashes = self.get_blobs_versioned_hashes(block) - parent_beacon_block_root = block["parentBeaconBlockRoot"].to_0x_hex() - execution_requests = await self.get_execution_requests(block) - else: - raise ValueError(f"Unknown payload version: {version}") - params.append(payload) - if blobs_versioned_hashes is not None: - params.append(blobs_versioned_hashes) - if parent_beacon_block_root is not None: - params.append(parent_beacon_block_root) - if execution_requests is not None: - params.append(execution_requests) + def get_new_payload_request( + self, + block_number: int, + message: dict, + version: int, + ) -> dict: + body = message["body"] + ep = body["execution_payload"] + payload = self.compose_payload(ep, version) + params: list = [payload] + if version >= 3: + params.append( + self.get_blobs_versioned_hashes(body.get("blob_kzg_commitments", [])) + ) + params.append(message["parent_root"]) + if version >= 4: + params.append( + self.get_execution_requests(body.get("execution_requests", {})) + ) return { "id": block_number, "jsonrpc": "2.0", @@ -238,18 +203,21 @@ async def get_new_payload_request(self, block: BlockData) -> dict: "params": params, } - async def get_fcu_request(self, block: BlockData) -> dict: - version = self.get_fcu_version(block) - block_number = block["number"] + def get_fcu_request( + self, + block_number: int, + ep: dict, + version: int, + ) -> dict: return { "id": block_number, "jsonrpc": "2.0", "method": f"engine_forkchoiceUpdatedV{version}", "params": [ { - "headBlockHash": block["hash"].to_0x_hex(), - "safeBlockHash": "0x0000000000000000000000000000000000000000000000000000000000000000", - "finalizedBlockHash": "0x0000000000000000000000000000000000000000000000000000000000000000", + "headBlockHash": ep["block_hash"], + "safeBlockHash": ep["parent_hash"], + "finalizedBlockHash": ep["parent_hash"], } ], } @@ -260,15 +228,34 @@ def generate_payload( ) -> None: self.log.info("Generating payload", block_number=block_number) block = self.w3.eth.get_block(block_number) + slot = self.network.slot_from_timestamp(block["timestamp"]) + self.log.debug( + "Fetching beacon block", block_number=block_number, slot=slot + ) + message = self.get_beacon_block(slot) + ep = message["body"]["execution_payload"] + if int(ep["block_number"]) != block_number: + self.log.error( + "Beacon block number mismatch, skipping", + block_number=block_number, + slot=slot, + beacon_block_number=int(ep["block_number"]), + ) + return + + payload_version = self.get_payload_version(block) + fcu_version = self.get_fcu_version(block) self.log.debug( "Generating engine_newPayload request", block_number=block_number ) - engine_new_payload_request = asyncio.run(self.get_new_payload_request(block)) + engine_new_payload_request = self.get_new_payload_request( + block_number, message, payload_version + ) self.log.debug( "Generating engine_forkChoiceUpdated request", block_number=block_number ) - engine_new_payload_request["gasUsed"] = block["gasUsed"] - fcu_request = asyncio.run(self.get_fcu_request(block)) + fcu_request = self.get_fcu_request(block_number, ep, fcu_version) + enp_req_file_name = os.path.join( self.output_dir, f"payload_{block_number}.json" ) diff --git a/tests/__init__.py b/tests/__init__.py new file mode 100644 index 0000000..e69de29 diff --git a/tests/conftest.py b/tests/conftest.py new file mode 100644 index 0000000..97c6a09 --- /dev/null +++ b/tests/conftest.py @@ -0,0 +1,31 @@ +import hashlib +import json +from pathlib import Path + +import pytest + +from expb.configs.networks import Network +from expb.payloads.generator import Generator + +FIXTURES_DIR = Path(__file__).parent / "fixtures" + + +def load_fixture(name: str) -> dict: + return json.loads((FIXTURES_DIR / f"{name}.json").read_text()) + + +def compute_requests_hash(encoded_requests: list[str]) -> str: + """EIP-7685: sha256(concat(sha256(request) for each non-empty request)).""" + m = hashlib.sha256() + for req in encoded_requests: + m.update(hashlib.sha256(bytes.fromhex(req[2:])).digest()) + return "0x" + m.hexdigest() + + +@pytest.fixture +def generator() -> Generator: + """A Generator instance whose network config is set but whose EL/Beacon URLs + are never contacted (only pure helper methods are exercised).""" + gen = Generator.__new__(Generator) + gen.network = Network.MAINNET.value + return gen diff --git a/tests/fixtures/block_with_deposit.json b/tests/fixtures/block_with_deposit.json new file mode 100644 index 0000000..8186d20 --- /dev/null +++ b/tests/fixtures/block_with_deposit.json @@ -0,0 +1,30 @@ +{ + "block_number": 25488796, + "slot": 14725091, + "requests_hash": "0x9a5a870aadd216c94354e690d224cd3077293f908abb927dc99b4ebc6fe2df63", + "parent_beacon_block_root": "0x72a58fe1a2683cf0323a11cd97a07d4d2ace7620b14f8d39575df5fcef6d7b4f", + "beacon_parent_root": "0x72a58fe1a2683cf0323a11cd97a07d4d2ace7620b14f8d39575df5fcef6d7b4f", + "execution_requests": { + "deposits": [ + { + "pubkey": "0xb084ae8321ed266a515b971ab5eaa835e4efbfb78529ac407797a90828fdcd4fb99e9b40421cd118a99a341cba8c0ce4", + "withdrawal_credentials": "0x0100000000000000000000009bd494dcf339c599d814e473441d468a3341e998", + "amount": "32000000000", + "signature": "0xb5a8c66f3fd90079db088bd61153b79c10f1822f01e0d585f4cefcc620ed41ac3f0649fe27d4fb4cda7756760f75f308160373dcf4dc6d7cb5971a00451be56c0c1cba15ffe3318121c7827da8959f752f3548373f4802c54a2ad7ec4014f810", + "index": "2511876" + } + ], + "withdrawals": [], + "consolidations": [] + }, + "blob_kzg_commitments": [ + "0x9496be285ced033266f5863ec0a8f5072c5c7bfbeee91b07308d3d3bbd4b4fc5f5d22f57670e1d493df5af6d0248bc42", + "0xb33ade5fd138762fc3641fdd62aef3ea066bb0ca1d1e042199b1bce8dcb1e36e91ffa2045d2c95a9c59545cf4ecb5262", + "0x82b77c3149cae320c40512d56c22731796b2588c831e89997cc2b264e6072dcd2ee345862f98ac3451351b3641936eb8" + ], + "expected_blob_versioned_hashes": [ + "0x01dae16130e88d431530049d236a7a8221d5a20d7724564e23c3574a774fa593", + "0x01e90af0e3c6a1c2940d876286e383f708cbd22b7e6dba469c3c9da35ff30040", + "0x01c8b73acc29afb97fe81d30c255b81e0daadb307796ead17c21532ee08ed3fd" + ] +} \ No newline at end of file diff --git a/tests/fixtures/block_with_withdrawal.json b/tests/fixtures/block_with_withdrawal.json new file mode 100644 index 0000000..58ca0c1 --- /dev/null +++ b/tests/fixtures/block_with_withdrawal.json @@ -0,0 +1,24 @@ +{ + "block_number": 25488822, + "slot": 14725117, + "requests_hash": "0x3a6e3886487a88df07bd5a33781fef778b6c6825f3d83983ec5dc7c3892e61d5", + "parent_beacon_block_root": "0x32f70dae5b4ea5081d08d4449e88753031925d9114588280dbbf715e56bb0cd7", + "beacon_parent_root": "0x32f70dae5b4ea5081d08d4449e88753031925d9114588280dbbf715e56bb0cd7", + "execution_requests": { + "deposits": [], + "withdrawals": [ + { + "source_address": "0xa4093d37a8ce69fb156841b531b7b41219e5834c", + "validator_pubkey": "0x8a8b6c3820a247106f8fa8af47aaed902d5e939cf3d6d478bc3b1fead9a74326ed98c8b75a49ee12384ba03f7f22794a", + "amount": "0" + } + ], + "consolidations": [] + }, + "blob_kzg_commitments": [ + "0x9557827927641f960c67f793cdcd577e1727bb654dd830853328273964e9f18e8dd127dea09d2e126026e76135fca245" + ], + "expected_blob_versioned_hashes": [ + "0x019d1a44fc00003977e0bfe7d4590ab2410a1e576e1a2a24dd001158d10f9496" + ] +} \ No newline at end of file diff --git a/tests/payloads/__init__.py b/tests/payloads/__init__.py new file mode 100644 index 0000000..e69de29 diff --git a/tests/payloads/test_execution_requests.py b/tests/payloads/test_execution_requests.py new file mode 100644 index 0000000..744b8b4 --- /dev/null +++ b/tests/payloads/test_execution_requests.py @@ -0,0 +1,63 @@ +"""Validate EIP-7685 execution-requests encoding and blob-versioned-hash derivation +against real mainnet data whose expected values come from the EL block header +(independent of the encoder under test).""" +import pytest + +from tests.conftest import compute_requests_hash, load_fixture + +FIXTURES = ["block_with_deposit", "block_with_withdrawal"] + + +@pytest.mark.parametrize("fixture_name", FIXTURES) +def test_execution_requests_match_header_requests_hash(generator, fixture_name): + fx = load_fixture(fixture_name) + encoded = generator.get_execution_requests(fx["execution_requests"]) + assert compute_requests_hash(encoded).lower() == fx["requests_hash"].lower() + + +@pytest.mark.parametrize("fixture_name", FIXTURES) +def test_blob_versioned_hashes_match_el(generator, fixture_name): + fx = load_fixture(fixture_name) + derived = generator.get_blobs_versioned_hashes(fx["blob_kzg_commitments"]) + assert [h.lower() for h in derived] == [ + h.lower() for h in fx["expected_blob_versioned_hashes"] + ] + + +def test_empty_execution_requests_encode_to_empty_list(generator): + assert generator.get_execution_requests({}) == [] + assert generator.get_execution_requests( + {"deposits": [], "withdrawals": [], "consolidations": []} + ) == [] + + +def test_requests_are_ordered_by_type_and_prefixed(generator): + """A block carrying every request type must emit them ordered 0x00,0x01,0x02 + and exclude empty types.""" + execution_requests = { + "withdrawals": [ + { + "source_address": "0x" + "11" * 20, + "validator_pubkey": "0x" + "22" * 48, + "amount": "1", + } + ], + "deposits": [ + { + "pubkey": "0x" + "33" * 48, + "withdrawal_credentials": "0x" + "44" * 32, + "amount": "32000000000", + "signature": "0x" + "55" * 96, + "index": "7", + } + ], + "consolidations": [], + } + encoded = generator.get_execution_requests(execution_requests) + assert len(encoded) == 2 + assert encoded[0].startswith("0x00") # deposit first + assert encoded[1].startswith("0x01") # withdrawal second + # deposit request body is fixed-size 192 bytes (+1 type byte) => 386 hex chars + "0x" + assert len(encoded[0]) == 2 + 2 + 192 * 2 + # withdrawal request body is fixed-size 76 bytes (+1 type byte) + assert len(encoded[1]) == 2 + 2 + 76 * 2 diff --git a/tests/payloads/test_field_mapping.py b/tests/payloads/test_field_mapping.py new file mode 100644 index 0000000..a7653b2 --- /dev/null +++ b/tests/payloads/test_field_mapping.py @@ -0,0 +1,75 @@ +"""Unit tests for the CL(JSON) -> Engine-API field mapping.""" + + +def _sample_ep() -> dict: + return { + "parent_hash": "0x" + "aa" * 32, + "fee_recipient": "0x" + "bb" * 20, + "state_root": "0x" + "cc" * 32, + "receipts_root": "0x" + "dd" * 32, + "logs_bloom": "0x" + "00" * 256, + "prev_randao": "0x" + "ee" * 32, + "block_number": "12345", + "gas_limit": "30000000", + "gas_used": "21000", + "timestamp": "1746612311", + "extra_data": "0xabcdef", + "base_fee_per_gas": "1000000000", + "block_hash": "0x" + "ff" * 32, + "transactions": ["0x02aabb", "0x03ccdd"], + "withdrawals": [ + { + "index": "10", + "validator_index": "256", + "address": "0x" + "12" * 20, + "amount": "1000000000", + } + ], + "blob_gas_used": "131072", + "excess_blob_gas": "0", + } + + +def test_compose_payload_v1_fields(generator): + payload = generator.compose_payload(_sample_ep(), version=1) + assert payload["parentHash"] == "0x" + "aa" * 32 + assert payload["feeRecipient"] == "0x" + "bb" * 20 + assert payload["prevRandao"] == "0x" + "ee" * 32 + assert payload["blockNumber"] == hex(12345) + assert payload["gasLimit"] == hex(30000000) + assert payload["gasUsed"] == hex(21000) + assert payload["timestamp"] == hex(1746612311) + assert payload["baseFeePerGas"] == hex(1000000000) + assert payload["blockHash"] == "0x" + "ff" * 32 + assert payload["transactions"] == ["0x02aabb", "0x03ccdd"] + # v1 must not carry later-fork fields + assert "withdrawals" not in payload + assert "blobGasUsed" not in payload + + +def test_compose_payload_v2_adds_withdrawals(generator): + payload = generator.compose_payload(_sample_ep(), version=2) + assert payload["withdrawals"] == [ + { + "index": hex(10), + "validatorIndex": hex(256), + "address": "0x" + "12" * 20, + "amount": hex(1000000000), + } + ] + assert "blobGasUsed" not in payload + + +def test_compose_payload_v3_adds_blob_fields(generator): + payload = generator.compose_payload(_sample_ep(), version=3) + assert payload["blobGasUsed"] == hex(131072) + assert payload["excessBlobGas"] == hex(0) + + +def test_versioned_hash_algorithm(generator): + # 0x01 prefix followed by sha256(commitment)[1:] + commitment = "0x" + "00" * 48 + import hashlib + + expected = "0x01" + hashlib.sha256(bytes(48)).digest()[1:].hex() + assert generator.get_blobs_versioned_hashes([commitment]) == [expected] diff --git a/tests/payloads/test_generator_integration.py b/tests/payloads/test_generator_integration.py new file mode 100644 index 0000000..cf2cabb --- /dev/null +++ b/tests/payloads/test_generator_integration.py @@ -0,0 +1,80 @@ +"""End-to-end fidelity test against a live mainnet archive (EL + Beacon). + +Deselected by default; run with: + uv run pytest -m integration +Endpoints can be overridden via EXPB_TEST_RPC_URL / EXPB_TEST_BEACON_URL. +""" +import os +from pathlib import Path + +import pytest + +from expb.configs.networks import Network +from expb.payloads.generator import Generator +from tests.conftest import compute_requests_hash + +pytestmark = pytest.mark.integration + +RPC_URL = os.environ.get("EXPB_TEST_RPC_URL", "http://38.154.254.162:8545") +BEACON_URL = os.environ.get("EXPB_TEST_BEACON_URL", "http://38.154.254.162:4000") + +# Prague blocks validated previously: (block_number, note) +BLOCKS = [25488796, 25488822, 25488855] + + +@pytest.fixture(scope="module") +def gen(tmp_path_factory) -> Generator: + return Generator( + network=Network.MAINNET, + rpc_url=RPC_URL, + beacon_url=BEACON_URL, + start_block=BLOCKS[0], + end_block=BLOCKS[0], + output_dir=Path(tmp_path_factory.mktemp("payloads")), + ) + + +@pytest.mark.parametrize("block_number", BLOCKS) +def test_generated_new_payload_matches_canonical_block(gen, block_number): + block = gen.w3.eth.get_block(block_number) + el_block = gen.w3.eth.get_block(block_number, full_transactions=True) + slot = gen.network.slot_from_timestamp(block["timestamp"]) + message = gen.get_beacon_block(slot) + ep = message["body"]["execution_payload"] + assert int(ep["block_number"]) == block_number + + version = gen.get_payload_version(block) + req = gen.get_new_payload_request(block_number, message, version) + payload = req["params"][0] + + # block hash + tx count == canonical block + assert payload["blockHash"].lower() == block["hash"].to_0x_hex().lower() + assert len(payload["transactions"]) == len(el_block["transactions"]) + + if version >= 3: + blob_hashes, parent_beacon_root = req["params"][1], req["params"][2] + assert parent_beacon_root.lower() == block["parentBeaconBlockRoot"].to_0x_hex().lower() + el_blob_hashes = [ + vh.to_0x_hex().lower() + for tx in el_block["transactions"] + for vh in (tx.get("blobVersionedHashes") or []) + ] + assert [h.lower() for h in blob_hashes] == el_blob_hashes + + if version >= 4: + execution_requests = req["params"][3] + computed = compute_requests_hash(execution_requests) + assert computed.lower() == block["requestsHash"].to_0x_hex().lower() + + +@pytest.mark.parametrize("block_number", BLOCKS) +def test_generated_fcu_uses_parent_hash(gen, block_number): + block = gen.w3.eth.get_block(block_number) + slot = gen.network.slot_from_timestamp(block["timestamp"]) + message = gen.get_beacon_block(slot) + ep = message["body"]["execution_payload"] + fcu = gen.get_fcu_request(block_number, ep, gen.get_fcu_version(block)) + params = fcu["params"][0] + assert params["headBlockHash"].lower() == block["hash"].to_0x_hex().lower() + assert params["safeBlockHash"].lower() == block["parentHash"].to_0x_hex().lower() + assert params["finalizedBlockHash"] == params["safeBlockHash"] diff --git a/uv.lock b/uv.lock index dc6535f..7043d7d 100644 --- a/uv.lock +++ b/uv.lock @@ -333,6 +333,7 @@ dependencies = [ { name = "jinja2" }, { name = "pydantic" }, { name = "pyyaml" }, + { name = "requests" }, { name = "rich" }, { name = "structlog" }, { name = "typer" }, @@ -342,6 +343,7 @@ dependencies = [ [package.dev-dependencies] dev = [ { name = "hatchling" }, + { name = "pytest" }, ] [package.metadata] @@ -351,6 +353,7 @@ requires-dist = [ { name = "jinja2", specifier = ">=3.1.6" }, { name = "pydantic", specifier = ">=2.11.7" }, { name = "pyyaml", specifier = ">=6.0.2" }, + { name = "requests", specifier = ">=2.32.0" }, { name = "rich", specifier = ">=14.0.0" }, { name = "structlog", specifier = ">=25.4.0" }, { name = "typer", specifier = ">=0.16.0" }, @@ -358,7 +361,10 @@ requires-dist = [ ] [package.metadata.requires-dev] -dev = [{ name = "hatchling", specifier = ">=1.29.0" }] +dev = [ + { name = "hatchling", specifier = ">=1.29.0" }, + { name = "pytest", specifier = ">=8.0.0" }, +] [[package]] name = "filelock" @@ -445,6 +451,15 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/76/c6/c88e154df9c4e1a2a66ccf0005a88dfb2650c1dffb6f5ce603dfbd452ce3/idna-3.10-py3-none-any.whl", hash = "sha256:946d195a0d259cbba61165e88e65941f16e9b36ea6ddb97f00452bae8b1287d3", size = 70442, upload-time = "2024-09-15T18:07:37.964Z" }, ] +[[package]] +name = "iniconfig" +version = "2.3.0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/72/34/14ca021ce8e5dfedc35312d08ba8bf51fdd999c576889fc2c24cb97f4f10/iniconfig-2.3.0.tar.gz", hash = "sha256:c76315c77db068650d49c5b56314774a7804df16fee4402c1f19d6d15d8c4730", size = 20503, upload-time = "2025-10-18T21:55:43.219Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/cb/b1/3846dd7f199d53cb17f49cba7e651e9ce294d8497c8c150530ed11865bb8/iniconfig-2.3.0-py3-none-any.whl", hash = "sha256:f631c04d2c48c52b84d0d0549c99ff3859c98df65b3101406327ecc7d53fbf12", size = 7484, upload-time = "2025-10-18T21:55:41.639Z" }, +] + [[package]] name = "jinja2" version = "3.1.6" @@ -713,6 +728,22 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/8a/0b/9fcc47d19c48b59121088dd6da2488a49d5f72dacf8262e2790a1d2c7d15/pygments-2.19.1-py3-none-any.whl", hash = "sha256:9ea1544ad55cecf4b8242fab6dd35a93bbce657034b0611ee383099054ab6d8c", size = 1225293, upload-time = "2025-01-06T17:26:25.553Z" }, ] +[[package]] +name = "pytest" +version = "9.1.1" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "colorama", marker = "sys_platform == 'win32'" }, + { name = "iniconfig" }, + { name = "packaging" }, + { name = "pluggy" }, + { name = "pygments" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/e4/47/b9efed96c114afcfa3c9d3fe98a76a1d14c74a9e266d397cf6eb64be5e01/pytest-9.1.1.tar.gz", hash = "sha256:1088fbde8f2b49d95a549a195707afa7a76a3ce9bcadc26b6d71f0ffda5fe313", size = 1636369, upload-time = "2026-06-19T10:58:32.857Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/24/25/1de2678b631f5a49215c6c96fff41ba892b0a34df68d6d80292b1b48aa7f/pytest-9.1.1-py3-none-any.whl", hash = "sha256:37a86b45efb9a47a61a36449063e8e18d0cab3161329fc099eb21783169c4f0c", size = 386536, upload-time = "2026-06-19T10:58:31.347Z" }, +] + [[package]] name = "pyunormalize" version = "16.0.0"