[improvement](spill) Monitor and log inode usage during spill gc - #67498
Draft
mrhhsg wants to merge 1 commit into
Draft
[improvement](spill) Monitor and log inode usage during spill gc#67498mrhhsg wants to merge 1 commit into
mrhhsg wants to merge 1 commit into
Conversation
Contributor
|
Thank you for your contribution to Apache Doris. Please clearly describe your PR:
|
### What problem does this PR solve?
Issue Number: None
Problem Summary:
Spill creates one directory per query and per operator plus one file per
part, so under high concurrency a spill disk can run out of inodes long
before it runs out of bytes. The spill capacity checks and the periodic
`spill gc time` log only report bytes, so an inode leak or a growing
`spill_gc` backlog cannot be diagnosed from the BE log or metrics.
This PR adds inode monitoring to the spill file manager:
- `LocalFileSystem::get_inode_info()` returns the total and available
inode count of the file system holding a path via `statvfs`.
- `SpillDataDir::update_capacity()` refreshes the inode statistics and
exposes them as `spill_disk_inode_total` and
`spill_disk_inode_available` metrics. A throttled warning is logged
when the inode usage reaches `storage_flood_stage_usage_percent`.
- The spill gc summary log now prints, per spill store, the inode usage,
the number of query directories still pending under `spill_gc`, the
number of directories/files deleted in this round and the number of
failed deletions. Deletion failures are no longer silently discarded
but logged with a throttled warning.
### Release note
None
### Check List (For Author)
- Test:
- Unit Test: `LocalFileSystemTest.GetInodeInfo`,
`SpillFileTest.UpdateCapacityTracksInodeUsage`,
`SpillFileTest.GCCleansUpGcRootBacklog`
- Regression test: No
- Behavior changed: No
- Does this need documentation: No
### Check List (For Reviewer who merge this PR)
- Confirm the release note
- Confirm test cases
- Confirm document
- Add branch pick label
Claude-Session: https://claude.ai/code/session_0149x6DMH4C9RUmHLjNKDFxS
mrhhsg
force-pushed
the
spill-gc-inode-monitor
branch
from
September 3, 2026 13:03
f37fd33 to
990d180
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What problem does this PR solve?
Issue Number: None
Problem Summary:
Spill creates one directory per query and per operator plus one file per
part, so under high concurrency a spill disk can run out of inodes long
before it runs out of bytes. The spill capacity checks and the periodic
spill gc timelog only report bytes, so an inode leak or a growingspill_gcbacklog cannot be diagnosed from the BE log or metrics.This PR adds inode monitoring to the spill file manager:
LocalFileSystem::get_inode_info()returns the total and availableinode count of the file system holding a path via
statvfs.SpillDataDir::update_capacity()refreshes the inode statistics andexposes them as
spill_disk_inode_totalandspill_disk_inode_availablemetrics. A throttled warning is loggedwhen the inode usage reaches
storage_flood_stage_usage_percent.the number of query directories still pending under
spill_gc, thenumber of directories/files deleted in this round and the number of
failed deletions. Deletion failures are no longer silently discarded
but logged with a throttled warning.
Release note
None
Check List (For Author)
LocalFileSystemTest.GetInodeInfo,SpillFileTest.UpdateCapacityTracksInodeUsage,SpillFileTest.GCCleansUpGcRootBacklogCheck List (For Reviewer who merge this PR)
https://claude.ai/code/session_0149x6DMH4C9RUmHLjNKDFxS