Skip to content

[PECOBLR-1709] Add SEA metadata operation logging header#1191

Merged
msrathore-db merged 10 commits intodatabricks:mainfrom
msrathore-db:sea-metadata-operation-logging
Feb 27, 2026
Merged

[PECOBLR-1709] Add SEA metadata operation logging header#1191
msrathore-db merged 10 commits intodatabricks:mainfrom
msrathore-db:sea-metadata-operation-logging

Conversation

@msrathore-db
Copy link
Copy Markdown
Collaborator

@msrathore-db msrathore-db commented Jan 30, 2026

Description

Add X-Databricks-Sea-Metadata-Operation-Type HTTP header to SEA ExecuteStatement requests for metadata operations. This enables CP-side logs (ELK, usage logs, SHR metrics) to distinguish metadata operations (getTables, getColumns, etc.) from regular SQL queries.

Changes:

  • Added metadataOperationType parameter to executeStatement() in IDatabricksClient
  • SEA client adds the header when metadataOperationType is non-null
  • Metadata client passes appropriate operation types: GetCatalogs, GetSchemas, GetTables, GetColumns, GetFunctions,
    GetPrimaryKeys, GetCrossReference
  • Thrift client signature updated (header ignored in Thrift mode)

Testing

  • DatabricksSdkClientTest - 23 tests passing
  • DatabricksMetadataSdkClientTest - 44 tests passing
  • DatabricksThriftServiceClientTest - 41 tests passing
  • DatabricksStatementTest - 49 tests passing
  • DatabricksPreparedStatementTest - 64 tests passing
  • DatabricksConnectionTest - 39 tests passing
  • Manual verification that header appears in SEA requests for
    metadata operations

Additional Notes to the Reviewer

NO_CHANGELOG=true

msrathore-db and others added 4 commits January 31, 2026 02:20
Added metadataOperationType parameter to executeStatement mock calls
in testExecuteInsertWithNonRowcountQueryPrefixes and
testExecuteInsertWithoutNonRowcountQueryPrefixes tests.

Co-Authored-By: Claude (databricks-claude-opus-4-5) <noreply@anthropic.com>
@msrathore-db msrathore-db changed the title Add SEA metadata operation logging header [PECOBLR-1709] Add SEA metadata operation logging header Feb 5, 2026
- Create MetadataOperationType enum with constants to replace string literals
  (GET_CATALOGS, GET_SCHEMAS, GET_TABLES, GET_COLUMNS, GET_FUNCTIONS,
  GET_PRIMARY_KEYS, GET_CROSS_REFERENCE)

- Refactor getHeaders() method using HeaderContext builder pattern to avoid
  parameter bloat as new headers are added

- Update all metadata operations to use enum constants

- Add comprehensive tests for enum values and header generation

- Verified header X-Databricks-Metadata-Operation-Type is correctly added
  with enum values for SEA metadata operations

Tests passing:
- MetadataOperationTypeTest: 15 tests
- DatabricksSdkClientTest: 26 tests
- DatabricksMetadataSdkClientTest: 44 tests

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Comment thread src/main/java/com/databricks/jdbc/dbclient/impl/sqlexec/DatabricksSdkClient.java Outdated
Replace HeaderContext builder pattern with simpler map-based approach
for passing additional HTTP headers. This addresses reviewer feedback
about unnecessary complexity.

Changes:
- Removed HeaderContext class and builder (~50 lines)
- Updated getHeaders() to accept Map<String, String> additionalHeaders
- Simplified call sites to use maps instead of builder pattern
- Metadata operation type header passed via map when needed
- All logging at DEBUG level for production use

Benefits:
- Simpler code: 44 lines net reduction
- More flexible for future headers
- Easier to understand and maintain
- No builder boilerplate

Example usage:
  Map<String, String> additionalHeaders = new HashMap<>();
  if (metadataOperationType != null) {
    additionalHeaders.put(HEADER_METADATA_OPERATION_TYPE,
                         metadataOperationType.getHeaderValue());
  }
  req.withHeaders(getHeaders("executeStatement", type, false, additionalHeaders));

All tests passing:
- DatabricksSdkClientTest: 26 tests ✓
- DatabricksMetadataSdkClientTest: 44 tests ✓
- MetadataOperationTypeTest: 15 tests ✓

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Copy link
Copy Markdown
Collaborator

@vikrantpuppala vikrantpuppala left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks for the change!

msrathore-db and others added 4 commits February 26, 2026 15:40
The merge from main brought 9 new DatabricksStatementTest calls to
executeStatement() that used the old 6-parameter signature. Added the
7th parameter (metadataOperationType) using any() matcher since these
are all non-metadata operations.

All 2960 tests passing.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@msrathore-db msrathore-db enabled auto-merge (squash) February 27, 2026 14:30
@msrathore-db msrathore-db merged commit 10c59d5 into databricks:main Feb 27, 2026
12 of 13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants