Skip to content

Commit 72e1c07

Browse files
matt-aitkenmatt-aitken
committed
fix: label and group errors without a message by their name
Errors thrown without a message (e.g. tagged errors) and non-Error throws (strings, plain objects) were all shown as "Unknown error" and collapsed into a single fingerprint, so unrelated failures across different tasks shared one group. Fall back message -> name -> raw when computing both the error fingerprint and the displayed message. Message-bearing errors are unaffected, so existing groups don't split; only errors that currently have no message get a meaningful title and their own group going forward. Adds a ClickHouse migration (MODIFY QUERY on the two error materialized views) for the display fallback; existing rows are left unchanged.
1 parent 7faa525 commit 72e1c07

4 files changed

Lines changed: 242 additions & 1 deletion

File tree

‎.server-changes/errors-message-fallback.md‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
---
2+
area: webapp
3+
type: fix
4+
---
5+
6+
The Errors page now shows a meaningful title for failures that don't carry a message — such as errors thrown without a message, or values thrown that aren't `Error` objects — instead of labelling them all "Unknown error". These errors are now also grouped by their name (or value) rather than being collapsed into a single group.

‎apps/webapp/app/utils/errorFingerprinting.ts‎

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -12,7 +12,11 @@ export function calculateErrorFingerprint(error: unknown): string {
1212
// 2. It won't be an instanceof Error because it's from the database.
1313
const errorObj = error as any;
1414
const errorType = String(errorObj.type || errorObj.name || "Error");
15-
const message = String(errorObj.message || "");
15+
// Fall back to the error class name, then the raw serialized value, so
16+
// messageless errors (e.g. tagged errors) and non-Error throws (strings,
17+
// plain objects) still group by something distinctive instead of collapsing
18+
// into a single fingerprint. Message-bearing errors are unaffected.
19+
const message = String(errorObj.message || errorObj.name || errorObj.raw || "");
1620
const stack = String(errorObj.stack || errorObj.stacktrace || "");
1721

1822
// Normalize message to group similar errors

‎apps/webapp/test/errorFingerprinting.test.ts‎

Lines changed: 44 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -402,4 +402,48 @@ describe("calculateErrorFingerprint", () => {
402402
};
403403
expect(calculateErrorFingerprint(error1)).toBe(calculateErrorFingerprint(error2));
404404
});
405+
406+
describe("message fallback to name / raw", () => {
407+
it("distinguishes messageless built-in errors by their class name", () => {
408+
const error1 = { type: "BUILT_IN_ERROR", name: "ListMessagesError", message: "" };
409+
const error2 = { type: "BUILT_IN_ERROR", name: "SendMessageError", message: "" };
410+
expect(calculateErrorFingerprint(error1)).not.toBe(calculateErrorFingerprint(error2));
411+
});
412+
413+
it("groups two messageless errors of the same class together", () => {
414+
const error1 = { type: "BUILT_IN_ERROR", name: "ListMessagesError", message: "" };
415+
const error2 = { type: "BUILT_IN_ERROR", name: "ListMessagesError", message: "" };
416+
expect(calculateErrorFingerprint(error1)).toBe(calculateErrorFingerprint(error2));
417+
});
418+
419+
it("does not change the fingerprint of a message-bearing error", () => {
420+
// Fingerprint must stay stable for the common path, otherwise existing
421+
// error groups would split on deploy.
422+
const withMessage = {
423+
type: "BUILT_IN_ERROR",
424+
name: "SomeError",
425+
message: "Connection timeout",
426+
};
427+
const withoutName = { type: "BUILT_IN_ERROR", message: "Connection timeout" };
428+
expect(calculateErrorFingerprint(withMessage)).toBe(calculateErrorFingerprint(withoutName));
429+
});
430+
431+
it("distinguishes string errors by their raw value", () => {
432+
const error1 = { type: "STRING_ERROR", raw: "rate limited" };
433+
const error2 = { type: "STRING_ERROR", raw: "connection refused" };
434+
expect(calculateErrorFingerprint(error1)).not.toBe(calculateErrorFingerprint(error2));
435+
});
436+
437+
it("distinguishes custom errors by their raw value", () => {
438+
const error1 = { type: "CUSTOM_ERROR", raw: '{"code":"E_LIMIT"}' };
439+
const error2 = { type: "CUSTOM_ERROR", raw: '{"code":"E_AUTH"}' };
440+
expect(calculateErrorFingerprint(error1)).not.toBe(calculateErrorFingerprint(error2));
441+
});
442+
443+
it("prefers message over name and raw when all are present", () => {
444+
const withMessage = { type: "BUILT_IN_ERROR", name: "Ignored", message: "real message" };
445+
const messageOnly = { type: "BUILT_IN_ERROR", message: "real message" };
446+
expect(calculateErrorFingerprint(withMessage)).toBe(calculateErrorFingerprint(messageOnly));
447+
});
448+
});
405449
});

‎internal-packages/clickhouse/schema/035_error_message_fallback_to_name_raw.sql‎

Lines changed: 187 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,187 @@
1+
-- +goose Up
2+
-- Fall back to the error class name, then the raw serialized value, before
3+
-- 'Unknown error' so messageless errors (tagged errors) and non-Error throws
4+
-- (strings, plain objects) get a meaningful title instead of all reading
5+
-- 'Unknown error'. Only affects rows inserted after this migration.
6+
7+
ALTER TABLE trigger_dev.errors_mv_v1 MODIFY QUERY
8+
SELECT
9+
organization_id,
10+
project_id,
11+
environment_id,
12+
task_identifier,
13+
error_fingerprint,
14+
15+
any(coalesce(nullIf(toString(error.data.type), ''), nullIf(toString(error.data.name), ''), 'Error')) as error_type,
16+
any(coalesce(
17+
nullIf(substring(toString(error.data.message), 1, 500), ''),
18+
nullIf(toString(error.data.name), ''),
19+
nullIf(substring(toString(error.data.raw), 1, 500), ''),
20+
'Unknown error'
21+
)) as error_message,
22+
any(coalesce(substring(toString(error.data.stack), 1, 2000), '')) as sample_stack_trace,
23+
24+
toDateTime(max(created_at)) as last_seen_date,
25+
26+
min(created_at) as first_seen,
27+
max(created_at) as last_seen,
28+
sumState(toUInt64(1)) as occurrence_count,
29+
uniqState(task_version) as affected_task_versions,
30+
31+
anyState(run_id) as sample_run_id,
32+
anyState(friendly_id) as sample_friendly_id,
33+
34+
sumMapState([status], [toUInt64(1)]) as status_distribution
35+
FROM trigger_dev.task_runs_v2
36+
WHERE
37+
error_fingerprint != ''
38+
AND status IN ('SYSTEM_FAILURE', 'CRASHED', 'INTERRUPTED', 'COMPLETED_WITH_ERRORS', 'TIMED_OUT')
39+
AND _is_deleted = 0
40+
GROUP BY
41+
organization_id,
42+
project_id,
43+
environment_id,
44+
task_identifier,
45+
error_fingerprint;
46+
47+
ALTER TABLE trigger_dev.error_occurrences_mv_v1 MODIFY QUERY
48+
SELECT
49+
organization_id,
50+
project_id,
51+
environment_id,
52+
task_identifier,
53+
error_fingerprint,
54+
task_version,
55+
toStartOfMinute (created_at) as minute,
56+
any (
57+
coalesce(
58+
nullIf(toString (error.data.type), ''),
59+
nullIf(toString (error.data.name), ''),
60+
'Error'
61+
)
62+
) as error_type,
63+
any (
64+
coalesce(
65+
nullIf(substring(toString (error.data.message), 1, 500), ''),
66+
nullIf(toString (error.data.name), ''),
67+
nullIf(substring(toString (error.data.raw), 1, 500), ''),
68+
'Unknown error'
69+
)
70+
) as error_message,
71+
any (
72+
coalesce(
73+
substring(toString (error.data.stack), 1, 2000),
74+
''
75+
)
76+
) as stack_trace,
77+
count() as count
78+
FROM
79+
trigger_dev.task_runs_v2
80+
WHERE
81+
error_fingerprint != ''
82+
AND status IN (
83+
'SYSTEM_FAILURE',
84+
'CRASHED',
85+
'INTERRUPTED',
86+
'COMPLETED_WITH_ERRORS',
87+
'TIMED_OUT'
88+
)
89+
AND _is_deleted = 0
90+
GROUP BY
91+
organization_id,
92+
project_id,
93+
environment_id,
94+
task_identifier,
95+
error_fingerprint,
96+
task_version,
97+
minute;
98+
99+
-- +goose Down
100+
101+
ALTER TABLE trigger_dev.errors_mv_v1 MODIFY QUERY
102+
SELECT
103+
organization_id,
104+
project_id,
105+
environment_id,
106+
task_identifier,
107+
error_fingerprint,
108+
109+
any(coalesce(nullIf(toString(error.data.type), ''), nullIf(toString(error.data.name), ''), 'Error')) as error_type,
110+
any(coalesce(nullIf(substring(toString(error.data.message), 1, 500), ''), 'Unknown error')) as error_message,
111+
any(coalesce(substring(toString(error.data.stack), 1, 2000), '')) as sample_stack_trace,
112+
113+
toDateTime(max(created_at)) as last_seen_date,
114+
115+
min(created_at) as first_seen,
116+
max(created_at) as last_seen,
117+
sumState(toUInt64(1)) as occurrence_count,
118+
uniqState(task_version) as affected_task_versions,
119+
120+
anyState(run_id) as sample_run_id,
121+
anyState(friendly_id) as sample_friendly_id,
122+
123+
sumMapState([status], [toUInt64(1)]) as status_distribution
124+
FROM trigger_dev.task_runs_v2
125+
WHERE
126+
error_fingerprint != ''
127+
AND status IN ('SYSTEM_FAILURE', 'CRASHED', 'INTERRUPTED', 'COMPLETED_WITH_ERRORS', 'TIMED_OUT')
128+
AND _is_deleted = 0
129+
GROUP BY
130+
organization_id,
131+
project_id,
132+
environment_id,
133+
task_identifier,
134+
error_fingerprint;
135+
136+
ALTER TABLE trigger_dev.error_occurrences_mv_v1 MODIFY QUERY
137+
SELECT
138+
organization_id,
139+
project_id,
140+
environment_id,
141+
task_identifier,
142+
error_fingerprint,
143+
task_version,
144+
toStartOfMinute (created_at) as minute,
145+
any (
146+
coalesce(
147+
nullIf(toString (error.data.type), ''),
148+
nullIf(toString (error.data.name), ''),
149+
'Error'
150+
)
151+
) as error_type,
152+
any (
153+
coalesce(
154+
nullIf(
155+
substring(toString (error.data.message), 1, 500),
156+
''
157+
),
158+
'Unknown error'
159+
)
160+
) as error_message,
161+
any (
162+
coalesce(
163+
substring(toString (error.data.stack), 1, 2000),
164+
''
165+
)
166+
) as stack_trace,
167+
count() as count
168+
FROM
169+
trigger_dev.task_runs_v2
170+
WHERE
171+
error_fingerprint != ''
172+
AND status IN (
173+
'SYSTEM_FAILURE',
174+
'CRASHED',
175+
'INTERRUPTED',
176+
'COMPLETED_WITH_ERRORS',
177+
'TIMED_OUT'
178+
)
179+
AND _is_deleted = 0
180+
GROUP BY
181+
organization_id,
182+
project_id,
183+
environment_id,
184+
task_identifier,
185+
error_fingerprint,
186+
task_version,
187+
minute;

0 commit comments

Comments
 (0)