testing changes to kvasir tests - DO NOT MERGE#765
testing changes to kvasir tests - DO NOT MERGE#765markro49 wants to merge 2 commits intocodespecs:masterfrom
Conversation
📝 WalkthroughWalkthroughThis pull request updates the kvasir-tests suite: it adds a SUPPORTED_OS whitelist (Fedora, Rocky, Ubuntu) and removes the DTRACE_DIFF variable from the common Makefile, adjusts compiler and DWARF flags, and changes several Makefiles to replace COUNTS_ONLY with explicit SKIP_DTRACE_DIFF / SKIP_INVS_DIFF flags or to comment out skip flags. It adds new goal/decls files (notably setjmp-test.decls.goal and rdtsc.decls.goal), modifies test outputs/assertions and READMEs, removes some perl test artifacts, and makes small fixes (e.g., function-pointer return, default target removal). 🚥 Pre-merge checks | ❌ 1❌ Failed checks (1 warning)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Tip Issue Planner is now in beta. Read the docs and try it out! Share your feedback on Discord. Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 5
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
tests/kvasir-tests/rijndael/Makefile (1)
20-20:⚠️ Potential issue | 🟡 MinorPre-existing bug: unescaped parentheses in
sedbasic regex.Line 20 uses
s/^VERSION_ID="(.*)"/\1/— in basicsed, parentheses must be escaped as\(and\)for capture groups. This meansOS_RELis always empty for this file, making theifeqconditions on lines 22-23 dead code.Not introduced by this PR, but worth fixing while you're editing this file.
Proposed fix
-OS_REL := $(shell sed -n -e 's/^VERSION_ID="(.*)"/\1/' /etc/os-release) +OS_REL := $(shell sed -n -e 's/^VERSION_ID="\(.*\)"/\1/p' /etc/os-release)
🤖 Fix all issues with AI agents
In `@tests/kvasir-tests/Makefile.common`:
- Around line 150-151: SUPPORTED_OS is declared but not enforced; add a runtime
check in the Makefile that compares the detected OS identifier (OS_ID) against
SUPPORTED_OS and aborts with a clear error if OS_ID is not in the list. Modify
the Makefile (near where OS_ID is determined) to split SUPPORTED_OS into words,
test membership of $(OS_ID) in that list (e.g., using a simple make conditional
or shell test invoked from make), and call $(error ...) or exit with a message
listing supported OSes when the check fails so unsupported platforms stop early.
- Around line 21-25: The DWARF bump to -gdwarf-3 may change debug-info layout
for the test harness (CFLAGS includes -gdwarf-3) so run kvasir/fjalar against
DWARF3 output on all supported target platforms and compiler versions to ensure
parsing still works, and update any DWARF-dependent test expectations if needed;
also replace the misleading comment that says "clang sets -Wreturn-mismatch"
with a correct note that GCC 14+ enables -Wreturn-mismatch by default (or
mention both compilers if relevant) and keep the active flag CFLAGS +=
-Wno-return-mismatch as-is to suppress that warning.
- Around line 15-19: Remove the dead assignments to CC and CXX: the variables CC
and CXX are set to clang/clang++ and then immediately overridden by gcc/g++, so
delete the redundant "CC := clang" and "CXX := clang++" lines or replace them
with conditional logic to choose the compiler (e.g., only set clang if CC/CXX
are unset or based on an environment flag). Update uses of CC and CXX in the
Makefile to rely on the single authoritative assignment or the chosen
conditional mechanism so the clang assignments no longer get ignored.
In `@tests/kvasir-tests/pthread-test/Makefile`:
- Around line 3-8: The Makefile for the pthread-test is leaving SKIP_DTRACE_DIFF
and SKIP_OUT_DIFF commented out which causes intermittent CI failures due to
thread-scheduling-dependent output; update the Makefile used by the pthread-test
to enable at least one skip flag (uncomment and set SKIP_DTRACE_DIFF := 1 and/or
SKIP_OUT_DIFF := 1) so CI ignores non-deterministic diffs, or alternatively
implement a normalization step in the pthread-test output pipeline (e.g., sort
or canonicalize Thread 1/Thread 2 sections before diffing) so the diff is
deterministic.
In `@tests/kvasir-tests/UnionComp/Makefile`:
- Around line 3-15: The Makefile currently re-enables dtrace/invariant diffs
despite the comment that union overlay produces nondeterministic comp.name
values; re-comment or set the flags SKIP_DTRACE_DIFF and SKIP_INVS_DIFF back to
true so CI doesn't fail intermittently. Edit the Makefile to restore the
intended behavior by defining or uncommenting SKIP_DTRACE_DIFF := 1 and
SKIP_INVS_DIFF := 1 (or otherwise ensuring those variables are exported/used) so
the unstable ::comp_test_ptr..comp.name and ::comp_test_ptr..comp.next->name
differences are ignored in automated runs.
| CC := clang | ||
| CC := gcc | ||
|
|
||
| CXX := clang++ | ||
| CXX := g++ |
There was a problem hiding this comment.
🛠️ Refactor suggestion | 🟠 Major
Dead code: clang/clang++ assignments immediately overridden by gcc/g++.
Lines 15 and 18 set CC and CXX to clang/clang++, but lines 16 and 19 unconditionally override them with gcc/g++. The clang assignments have no effect. Either remove them or use conditional logic if the intent is to support switching compilers.
Proposed fix (remove dead assignments)
-CC := clang
CC := gcc
-
-CXX := clang++
CXX := g++📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| CC := clang | |
| CC := gcc | |
| CXX := clang++ | |
| CXX := g++ | |
| CC := gcc | |
| CXX := g++ |
🤖 Prompt for AI Agents
In `@tests/kvasir-tests/Makefile.common` around lines 15 - 19, Remove the dead
assignments to CC and CXX: the variables CC and CXX are set to clang/clang++ and
then immediately overridden by gcc/g++, so delete the redundant "CC := clang"
and "CXX := clang++" lines or replace them with conditional logic to choose the
compiler (e.g., only set clang if CC/CXX are unset or based on an environment
flag). Update uses of CC and CXX in the Makefile to rely on the single
authoritative assignment or the chosen conditional mechanism so the clang
assignments no longer get ignored.
| # CFLAGS += -gdwarf-2 | ||
| CFLAGS += -gdwarf-3 | ||
|
|
||
| # clang sets -Wreturn-mismatch | ||
| CFLAGS += -Wno-return-mismatch |
There was a problem hiding this comment.
DWARF version bump and -Wno-return-mismatch flag.
Two observations:
-
Switching from
-gdwarf-2to-gdwarf-3changes the debug info format. Verify that kvasir/fjalar correctly parses DWARF 3 output across all target platforms/compiler versions. -
The comment on line 24 says "clang sets -Wreturn-mismatch" but the active compiler is
gcc(line 16). GCC 14 introduced-Wreturn-mismatchas a default warning. The comment should reference GCC 14+, not clang.
Fix misleading comment
-# clang sets -Wreturn-mismatch
+# gcc 14+ enables -Wreturn-mismatch by default
CFLAGS += -Wno-return-mismatch📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| # CFLAGS += -gdwarf-2 | |
| CFLAGS += -gdwarf-3 | |
| # clang sets -Wreturn-mismatch | |
| CFLAGS += -Wno-return-mismatch | |
| # CFLAGS += -gdwarf-2 | |
| CFLAGS += -gdwarf-3 | |
| # gcc 14+ enables -Wreturn-mismatch by default | |
| CFLAGS += -Wno-return-mismatch |
🤖 Prompt for AI Agents
In `@tests/kvasir-tests/Makefile.common` around lines 21 - 25, The DWARF bump to
-gdwarf-3 may change debug-info layout for the test harness (CFLAGS includes
-gdwarf-3) so run kvasir/fjalar against DWARF3 output on all supported target
platforms and compiler versions to ensure parsing still works, and update any
DWARF-dependent test expectations if needed; also replace the misleading comment
that says "clang sets -Wreturn-mismatch" with a correct note that GCC 14+
enables -Wreturn-mismatch by default (or mention both compilers if relevant) and
keep the active flag CFLAGS += -Wno-return-mismatch as-is to suppress that
warning.
| # Define the strict whitelist | ||
| SUPPORTED_OS := Fedora Rocky Ubuntu |
There was a problem hiding this comment.
🧹 Nitpick | 🔵 Trivial
SUPPORTED_OS is defined but never enforced.
SUPPORTED_OS := Fedora Rocky Ubuntu is declared but never checked against OS_ID. Without enforcement, unsupported OSes silently proceed and may produce confusing failures. If the intent is to guard against unsupported platforms, add a check:
Proposed enforcement
# Define the strict whitelist
SUPPORTED_OS := Fedora Rocky Ubuntu
+
+ifeq ($(filter $(OS_ID),$(SUPPORTED_OS)),)
+ $(warning Unsupported OS: $(OS_ID). Supported: $(SUPPORTED_OS))
+endif📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| # Define the strict whitelist | |
| SUPPORTED_OS := Fedora Rocky Ubuntu | |
| # Define the strict whitelist | |
| SUPPORTED_OS := Fedora Rocky Ubuntu | |
| ifeq ($(filter $(OS_ID),$(SUPPORTED_OS)),) | |
| $(warning Unsupported OS: $(OS_ID). Supported: $(SUPPORTED_OS)) | |
| endif |
🤖 Prompt for AI Agents
In `@tests/kvasir-tests/Makefile.common` around lines 150 - 151, SUPPORTED_OS is
declared but not enforced; add a runtime check in the Makefile that compares the
detected OS identifier (OS_ID) against SUPPORTED_OS and aborts with a clear
error if OS_ID is not in the list. Modify the Makefile (near where OS_ID is
determined) to split SUPPORTED_OS into words, test membership of $(OS_ID) in
that list (e.g., using a simple make conditional or shell test invoked from
make), and call $(error ...) or exit with a message listing supported OSes when
the check fails so unsupported platforms stop early.
| # This test creates and runs two threads. There may or may not be | ||
| # diffs in the .dtrace and .out files depending on the order the | ||
| # threads are run. If there are diffs, you should verify that | ||
| # it is just the order of Thread 1 and Thread 2. | ||
| #SKIP_DTRACE_DIFF := 1 | ||
| #SKIP_OUT_DIFF := 1 |
There was a problem hiding this comment.
Re-enabling diffs for a thread-scheduling-dependent test will cause intermittent CI failures.
The comment on lines 3-6 correctly notes that .dtrace and .out diffs depend on thread scheduling order, yet both SKIP_DTRACE_DIFF and SKIP_OUT_DIFF are commented out. This will produce non-deterministic CI results — passes on some runs, failures on others — which undermines CI reliability.
The suggestion to "verify that it is just the order of Thread 1 and Thread 2" is a manual step that cannot be automated in CI. Consider keeping at least one of these skip flags active, or implementing a normalization step (e.g., sorting thread-order-dependent sections) before the diff.
🤖 Prompt for AI Agents
In `@tests/kvasir-tests/pthread-test/Makefile` around lines 3 - 8, The Makefile
for the pthread-test is leaving SKIP_DTRACE_DIFF and SKIP_OUT_DIFF commented out
which causes intermittent CI failures due to thread-scheduling-dependent output;
update the Makefile used by the pthread-test to enable at least one skip flag
(uncomment and set SKIP_DTRACE_DIFF := 1 and/or SKIP_OUT_DIFF := 1) so CI
ignores non-deterministic diffs, or alternatively implement a normalization step
in the pthread-test output pipeline (e.g., sort or canonicalize Thread 1/Thread
2 sections before diffing) so the diff is deterministic.
| # The ptr (next) is assigned an address giving the | ||
| # char* (name) random garbage. Hence, the .dtrace | ||
| # file will almost never match. You should verify | ||
| # it contains only differences of the form: | ||
| # ::comp_test_ptr[..].comp.name | ||
| # -[ "(4321" ] | ||
| # +[ "(`4321" ] | ||
| # ... | ||
| # ::comp_test_ptr[..].comp.next->name | ||
| # -[ "(4321" ] | ||
| # +[ "(`4321" ] | ||
| #SKIP_DTRACE_DIFF := 1 | ||
| #SKIP_INVS_DIFF := 1 |
There was a problem hiding this comment.
Contradiction: comment says dtrace "will almost never match", yet diff checks are re-enabled.
Lines 3-4 document that the union overlay gives comp.name random garbage, and the dtrace file "will almost never match." Yet SKIP_DTRACE_DIFF and SKIP_INVS_DIFF are commented out (lines 14-15), meaning these diffs will now be enforced and are expected to fail on nearly every run.
If the intent is temporary (to observe failures across platforms), this is fine for the experiment, but these flags should be re-enabled before merging to master — otherwise CI will be persistently red for this test.
🤖 Prompt for AI Agents
In `@tests/kvasir-tests/UnionComp/Makefile` around lines 3 - 15, The Makefile
currently re-enables dtrace/invariant diffs despite the comment that union
overlay produces nondeterministic comp.name values; re-comment or set the flags
SKIP_DTRACE_DIFF and SKIP_INVS_DIFF back to true so CI doesn't fail
intermittently. Edit the Makefile to restore the intended behavior by defining
or uncommenting SKIP_DTRACE_DIFF := 1 and SKIP_INVS_DIFF := 1 (or otherwise
ensuring those variables are exported/used) so the unstable
::comp_test_ptr..comp.name and ::comp_test_ptr..comp.next->name differences are
ignored in automated runs.
There was a problem hiding this comment.
Actionable comments posted: 3
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@tests/kvasir-tests/md5/README`:
- Around line 15-17: Create a tracked issue for the noted Kvasir bug ("probes
break our heuristic for determining where the function's prologue ends") and add
the issue reference to the README comment so the TODO is actionable; include in
the issue a short title, reproduction steps, expected vs actual behavior, impact
(tests/md5), and label it with appropriate areas (kvasir, heuristics), then edit
the README to replace the vague note with a one-line reference to that issue
(e.g., "tracked in ISSUE-1234") so future readers can find the detailed plan and
progress.
- Around line 1-3: Fix the grammar in the README sentence by replacing "I haved
committed" with "I have committed" so the first line reads "To temporarily
eliminate a goal file difference, I have committed an 'incorrect'
md5.dtrace.goal-Ubuntu."; locate and update the sentence in the README
(tests/kvasir-tests/md5/README) where that phrase appears.
In `@tests/kvasir-tests/partial-init/partial-init.out.goal`:
- Line 12: tests/kvasir-tests/clean-output.pl must normalize glibc wrapper
source locations so tests aren't tied to glibc line numbers; update the script
to strip or canonicalize patterns like "write (write.c:26)" by adding a regex
replacement in clean-output.pl that converts any "(\w+\.c):\d+" suffix (e.g.,
write.c:26, read.c:123) to a stable token such as "$1:XXX" or remove the
":<digits>" portion, and apply the same rule for other wrappers (read.c, open.c,
etc.) so partial-init.out.goal comparisons no longer fail across distributions.
| To temporarily eliminate a goal file difference, I haved committed an | ||
| 'incorrect' md5.dtrace.goal-Ubuntu. | ||
| markro 2/17/2026 |
There was a problem hiding this comment.
Typo: "haved" → "have"
Line 1 contains a grammatical error: "I haved committed" should be "I have committed".
📝 Proposed fix
-To temporarily eliminate a goal file difference, I haved committed an
+To temporarily eliminate a goal file difference, I have committed an📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| To temporarily eliminate a goal file difference, I haved committed an | |
| 'incorrect' md5.dtrace.goal-Ubuntu. | |
| markro 2/17/2026 | |
| To temporarily eliminate a goal file difference, I have committed an | |
| 'incorrect' md5.dtrace.goal-Ubuntu. | |
| markro 2/17/2026 |
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@tests/kvasir-tests/md5/README` around lines 1 - 3, Fix the grammar in the
README sentence by replacing "I haved committed" with "I have committed" so the
first line reads "To temporarily eliminate a goal file difference, I have
committed an 'incorrect' md5.dtrace.goal-Ubuntu."; locate and update the
sentence in the README (tests/kvasir-tests/md5/README) where that phrase
appears.
| These probes break our heuristic for determining where the function's | ||
| prologue ends. I think I know how to fix this, but it will require a | ||
| significant amount of work in Kvasir. |
There was a problem hiding this comment.
🧹 Nitpick | 🔵 Trivial
Consider tracking the known bug in an issue.
The comment acknowledges a non-trivial fix is needed in Kvasir ("significant amount of work") but doesn't link to any issue tracker entry. Adding an issue reference here would help ensure the workaround is eventually addressed and doesn't become permanently forgotten technical debt.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@tests/kvasir-tests/md5/README` around lines 15 - 17, Create a tracked issue
for the noted Kvasir bug ("probes break our heuristic for determining where the
function's prologue ends") and add the issue reference to the README comment so
the TODO is actionable; include in the issue a short title, reproduction steps,
expected vs actual behavior, impact (tests/md5), and label it with appropriate
areas (kvasir, heuristics), then edit the README to replace the vague note with
a one-line reference to that issue (e.g., "tracked in ISSUE-1234") so future
readers can find the detailed plan and progress.
| 01010000 00000000 00000000 00000000 0101 | ||
| 01010000 00000000 00000000 00000000 0101==PID== Syscall param write(buf) points to uninitialised byte(s) | ||
| ==PID== at <HEAP_ADDR>: __write_nocancel | ||
| ==PID== at <HEAP_ADDR>: write (write.c:26) |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
# Find normalization/filter scripts and Makefiles in kvasir tests
echo "=== Searching for test infrastructure scripts ==="
fd -type f \( -name "*.pl" -o -name "*.sh" -o -name "*.py" -o -name "*.awk" -o -name "Makefile*" -o -name "*.mk" \) tests/kvasir-tests | head -20
echo -e "\n=== Checking for normalization/comparison logic ==="
rg -l "write\.c|HEAP_ADDR|STATIC_ADDR|normalize|filter|comparison" tests/kvasir-tests --type-list
echo -e "\n=== Looking for Makefile patterns ==="
fd -type f -name "Makefile*" tests/kvasir-tests -exec grep -l "sed\|awk\|normalize\|filter\|write" {} \;Repository: codespecs/daikon
Length of output: 950
🏁 Script executed:
#!/bin/bash
# Find the test comparison/runner script
echo "=== Looking for test runner/comparison logic in common Makefile ==="
find tests/kvasir-tests -name "*Makefile*" -o -name "*.mk" | xargs cat 2>/dev/null | head -100
echo -e "\n=== Searching for .goal file comparison logic ==="
rg -i "\.goal|\.out|comparison|diff|normalize" tests/kvasir-tests -A 2 -B 2 --max-count=20Repository: codespecs/daikon
Length of output: 50374
🏁 Script executed:
#!/bin/bash
# Check if there's a parent Makefile or test runner
echo "=== Checking root-level test infrastructure ==="
ls -la tests/ | head -20
cat tests/Makefile 2>/dev/null || echo "No root Makefile in tests/"
echo -e "\n=== Looking for test runner scripts in root ==="
fd -type f \( -name "run*" -o -name "*test*" \) . --max-depth 2 -x file {} \;Repository: codespecs/daikon
Length of output: 6264
🏁 Script executed:
#!/bin/bash
# Look at the test comparison logic in Makefile.common
echo "=== Searching for diff/comparison targets in Makefile.common ==="
rg -n "diff|comparison|HEAP_ADDR|STATIC_ADDR|write\.c|sed|awk" tests/Makefile.common | head -50
echo -e "\n=== Looking at specific sections handling diffs ==="
sed -n '1,100p' tests/Makefile.common | head -80
echo -e "\n=== Checking for normalization logic ==="
grep -n "normalize\|substitute\|PREPROC_UTIL\|filter" tests/Makefile.common | head -20Repository: codespecs/daikon
Length of output: 7792
🏁 Script executed:
#!/bin/bash
# Search for where actual comparison happens and how patterns are handled
echo "=== Looking for the actual comparison rules ==="
sed -n '100,300p' tests/Makefile.common
echo -e "\n=== Checking for any Python/Perl scripts that might do normalization ==="
find tests -name "*.pl" -o -name "*.py" -o -name "*.awk" 2>/dev/null | xargs ls -la 2>/dev/nullRepository: codespecs/daikon
Length of output: 8512
🏁 Script executed:
#!/bin/bash
# Check the partial-init test directory specifically to see if there's a normalize script
echo "=== Checking partial-init test directory ==="
ls -la tests/kvasir-tests/partial-init/
echo -e "\n=== Looking at the .goal file ==="
head -30 tests/kvasir-tests/partial-init/partial-init.out.goal
echo -e "\n=== Checking for any pre/post processing ==="
grep -r "write\.c\|HEAP_ADDR\|normalize" tests/kvasir-tests/partial-init/ 2>/dev/nullRepository: codespecs/daikon
Length of output: 2894
🏁 Script executed:
#!/bin/bash
echo "=== Full content of clean-output.pl ==="
cat tests/kvasir-tests/clean-output.pl
echo -e "\n=== Checking what patterns are substituted ==="
grep -n "write\|sed\|s/\|sub " tests/kvasir-tests/clean-output.plRepository: codespecs/daikon
Length of output: 5474
🏁 Script executed:
#!/bin/bash
# Check the Makefile.common in kvasir-tests to see how clean-output.pl is called
echo "=== Looking for how clean-output.pl is used in kvasir test infrastructure ==="
grep -n "clean-output\|\.out\|PREPROC" tests/kvasir-tests/Makefile.common | head -40
echo -e "\n=== Checking kvasir-specific Makefile ==="
head -100 tests/kvasir-tests/Makefile.commonRepository: codespecs/daikon
Length of output: 5282
🏁 Script executed:
#!/bin/bash
# Check if there are OS-specific goal files for partial-init
echo "=== Looking for OS-specific goal files in partial-init ==="
ls -la tests/kvasir-tests/partial-init/ | grep goal
echo -e "\n=== Searching for any goal-Fedora or goal-Rocky files in kvasir-tests ==="
find tests/kvasir-tests -name "*goal-Fedora*" -o -name "*goal-Rocky*" -o -name "*goal-Ubuntu*" 2>/dev/null | head -20
echo -e "\n=== Check if any write.c pattern normalization exists elsewhere ==="
rg "write\.c" tests/ --type sh --type pl --type py 2>/dev/null | head -20Repository: codespecs/daikon
Length of output: 1977
🏁 Script executed:
#!/bin/bash
# Verify the actual glibc versions on different platforms
echo "=== Checking glibc versions in different Linux distributions ==="
echo "Ubuntu 24.04 uses glibc 2.39 (write.c line 26 is from this version)"
echo "Fedora 40 uses glibc 2.39 (may differ from Ubuntu due to patches)"
echo "Fedora 41 uses glibc 2.40 (almost certainly different line numbers)"
echo "Rocky Linux 9 uses glibc 2.34 (significantly different from 2.39)"
# Check if there are any comments in the test infrastructure about this
echo -e "\n=== Checking for comments about glibc compatibility ==="
rg -i "glibc|write\.c|line.number" tests/kvasir-tests/ | grep -i comment | head -10Repository: codespecs/daikon
Length of output: 561
The hardcoded write.c:26 line number is glibc-version-specific and will cause test failures on non-Ubuntu platforms.
With this PR, the Valgrind output now shows write (write.c:26) instead of __write_nocancel. The :26 suffix is the source line in glibc's write.c wrapper, which varies across glibc releases:
- Ubuntu 24.04 → glibc 2.39 (line 26)
- Fedora 41 → glibc 2.40 (different line number)
- Rocky Linux 9 → glibc 2.34 (significantly different)
The PR adds a SUPPORTED_OS whitelist including Fedora and Rocky Linux, but this goal file will fail comparison on those platforms due to the hardcoded line number. The test infrastructure already supports normalizing version-dependent details (e.g., vg_replace_malloc.c line numbers are normalized to XXX in clean-output.pl), so the proper fix is to extend that normalization to glibc source locations.
Update tests/kvasir-tests/clean-output.pl to normalize glibc source-file line numbers:
Example fix
s/: __write_nocancel .*/: __write_nocancel/;
+ s/\(write\.c:\d+\)/(write.c:XXX)/g;
s/Copyright \(C\) .*$/COPYRIGHT/;(and apply the same normalization to other glibc wrappers like read.c, open.c, etc., or use a more general pattern)
Alternatively, create OS-specific goal files (partial-init.out.goal-Fedora, partial-init.out.goal-Rocky) matching each platform's glibc version.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@tests/kvasir-tests/partial-init/partial-init.out.goal` at line 12,
tests/kvasir-tests/clean-output.pl must normalize glibc wrapper source locations
so tests aren't tied to glibc line numbers; update the script to strip or
canonicalize patterns like "write (write.c:26)" by adding a regex replacement in
clean-output.pl that converts any "(\w+\.c):\d+" suffix (e.g., write.c:26,
read.c:123) to a stable token such as "$1:XXX" or remove the ":<digits>"
portion, and apply the same rule for other wrappers (read.c, open.c, etc.) so
partial-init.out.goal comparisons no longer fail across distributions.
|
replaced with PR 766 |
1 participant
testing changes to kvasir tests. want to see if pass on all platforms.
DO NOT MERGE