gh-138122: Make the tachyon profiler opcode-aware #142394
Open
+3,025
−453
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Introduces LocationInfo struct sequence with end_lineno, col_offset, and end_col_offset fields. Adds opcodes parameter to RemoteUnwinder that extracts the currently executing opcode alongside its source span. Refactors linetable parsing to correctly accumulate line numbers separately from output values, fixing edge cases in computed_line.
New opcode_utils.py maps opcode numbers to names and detects specialized variants using opcode module metadata. Adds normalize_location() and extract_lineno() helpers to collector base for uniform location handling. CLI gains --opcodes flag, validated against compatible formats (gecko, flamegraph, heatmap, live).
Stores per-node opcode counts in the tree structure. Exports opcode mapping (names and deopt relationships) in JSON so the JS renderer can show instruction names and distinguish specialized variants.
Tracks opcode state transitions per thread and emits interval markers when the executing opcode changes. Markers include opcode name, line, column, and duration. Adds Opcodes category to marker schema.
Expandable panel per hot line shows instruction-level sample breakdown with opcode names and specialization percentage. Converts call graph data structures from lists to sets for O(1) deduplication.
New widget displays instruction-level stats for selected function when --opcodes is enabled. Navigation via j/k keys with scroll support. Adds per-thread opcode tracking. Updates pstats collector for new frame format.
Frame location is now a 4-tuple (lineno, end_lineno, col_offset, end_col_offset). MockFrameInfo wraps locations in LocationInfo struct. Updates assertions throughout and adds opcode_utils coverage.
# Conflicts: # Lib/test/test_profiling/test_sampling_profiler/test_integration.py
pablogsal
requested review from
ivonastojanovic and
savannahostrowski
and removed request for
ivonastojanovic
Member
savannahostrowski
left a comment
savannahostrowski
left a comment
There was a problem hiding this comment.
First pass on most of the files (mainly the JS/CSS)! I ended up playing around with this quite a bit (flamegraph, heatmap, live mode, etc) and it looks awesome. You're on a roll 💫 !
A couple of comments here and then I also opened pablogsal#110 to move a bunch of the CSS into proper classes instead of inlining all the styles, which fixes some dark mode theming bugs (figured that'd be simpler than commenting in a bunch of places)!
|
|
||
| const opcodeLines = sortedOpcodes.map(([opcode, count]) => { | ||
| const opcodeInfo = getOpcodeInfo(parseInt(opcode, 10)); | ||
| const pct = ((count / totalOpcodeSamples) * 100).toFixed(1); |
Member
There was a problem hiding this comment.
This isn't used right now. I assume we do want to include it somewhere in the UI 😄
| const normalizedIntensity = (intensity - 0.3) / 0.7; | ||
| // Warm orange-red with increasing opacity for hotter spans | ||
| const alpha = 0.25 + normalizedIntensity * 0.35; // 0.25 to 0.6 | ||
| return `rgba(255, 100, 50, ${alpha})`; |
Member
There was a problem hiding this comment.
Maybe we can read from a CSS variable instead so we don't have to hardcode this/can reuse colors we already have defined?
| return `rgba(255, 100, 50, ${alpha})`; | ||
| } else if (intensity > 0) { | ||
| // Cold spans: very subtle gray, almost invisible | ||
| return `rgba(150, 150, 150, 0.1)`; |
|
|
||
| elif ch == curses.KEY_LEFT or ch == curses.KEY_UP: | ||
| # Navigate to previous thread in PER_THREAD mode, or switch from ALL to PER_THREAD | ||
| elif ch == ord("j") or ch == ord("J"): |
Member
There was a problem hiding this comment.
Unless I'm missing something, I think we can deduplicate a lot of logic with some helpers for up/down movement, right?
CSS fixes for opcode-aware Tachyon - pythongh-138122
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
When profiling Python code, knowing which source line is hot often isn't enough—a single line can contain multiple operations with very different performance characteristics. For example, result = obj.attr + expensive_call() has an attribute load, a function call, and an addition, but traditional line-level profiling treats them identically. This becomes especially important with the adaptive specialization, where understanding whether the interpreter has specialized an instruction (e.g.,
LOAD_ATTR→LOAD_ATTR_INSTANCE_VALUE) provides crucial optimization insights that were previously only visible through manual dis inspection.This PR threads bytecode information through the existing sampling profiler infrastructure by extending RemoteUnwinder to optionally extract the executing opcode and its precise source span (including column offsets) from the remote process's frame state. The column-level location data flows through to the visualization layers, where the heatmap can highlight the exact expression within a line, the gecko format emits interval markers for Firefox Profiler's timeline, and the live TUI displays real-time instruction breakdowns. The feature is opt-in via
--opcodesto avoid the overhead when not needed, and the opcode utilities handle the mapping between specialized instruction variants and their base forms so users can see both what's actually executing and what it was specialized from.screenrecording-2025-12-02_21-12-36.mp4