Add subset telemetry attrs and rows metric

[robacourt]

Summary

• add subset query span attrs for returned row count and result bytes
• record POST body params as telemetry attrs, truncating long strings to 64KB
• surface electric.subqueries.subset_result.rows from ElectricTelemetry.StackTelemetry
• add focused tests for POST body telemetry flattening/truncation and the new stack metric

Testing

• mix test test/electric/plug/utils_test.exs test/electric/plug/subset_telemetry_test.exs
• mix test test/electric/telemetry/stack_telemetry_test.exs

[codecov]

❌ 2 Tests Failed:

Tests completed	Failed	Passed	Skipped
3628	2	3626	27

View the top 2 failed test(s) by shortest run time

Elixir.Electric.Shapes.FilterTest::test optimisations where clause in the form `field1 = const1 AND field2 = const2` is optimised for lots of const2 values

Stack Traces | 0.83s run time

17) test optimisations where clause in the form `field1 = const1 AND field2 = const2` is optimised for lots of const2 values (Electric.Shapes.FilterTest)
     .../electric/shapes/filter_test.exs:761
     Assertion with < failed
     code:  assert affected_reductions < @max_reductions
     left:  1562
     right: 1300
     stacktrace:
       .../electric/shapes/filter_test.exs:775: (test)

Elixir.Electric.Shapes.FilterTest::test optimisations where clause in the form `field1 = const1 AND field2 = const2` is optimised for lots of const1 values

Stack Traces | 1s run time

2) test optimisations where clause in the form `field1 = const1 AND field2 = const2` is optimised for lots of const1 values (Electric.Shapes.FilterTest)
     .../electric/shapes/filter_test.exs:739
     Assertion with < failed
     code:  assert affected_reductions < @max_reductions
     left:  1561
     right: 1300
     stacktrace:
       .../electric/shapes/filter_test.exs:753: (test)

To view more test analytics, go to the Test Analytics Dashboard
_{📋 Got 3 mins? Take this short survey to help us improve Test Analytics.}

[alco]

record POST body params as telemetry attrs, truncating long strings to 64KB

64KB is a lot, and it's a per-value truncation limit? What's the use case for such strings in telemetry?

Normally the long strings we observe in observability data are log lines and those are generally on the order of 100 bytes.

[robacourt]

64KB is a lot, and it's a per-value truncation limit? What's the use case for such strings in telemetry?

The use case for POST over GET was because the URL params were getting too big, body params in a POST doesn't have that limit. So we should expect large values, including large subset where clauses (one of the main things I'm trying to expose in this PR). Honeycomb has a 64KB limit for string values so I thought I'd limit it to that, but perhaps we should consider a much lower value.

[alco]

@robacourt the first item in the list of Limits on that page reads

2,000 fields per event. The entire event must be less than 1 MB of uncompressed JSON.

We're probably nowhere near the worst case but 2K string fields 64KB each would be 125MiB.

I'd say we need to include a large enough prefix that will allow identifying the particular where expression or whatever else it is encoded in that long string. For that, 100 or a few hundred bytes should be sufficient.

It feels wrong to try to fit the whole body in a telemetry event if its total size is tens of KB. We just wouldn't use the whole payload anyway, we'd be looking for patterns.

[github-actions]

This PR has been released! 🚀

The following packages include changes from this PR:

• @core/sync-service@1.6.2

Thanks for contributing to Electric!