Skip to content

fix(runner-shared): always set PYTHONPERFSUPPORT - #577

Merged
not-matthias merged 1 commit into
mainfrom
cod-3820-investigate-missing-python-valgrind-frames-in-runner-541
Oct 9, 2026
Merged

not-matthias merged 1 commit into
mainfrom
cod-3820-investigate-missing-python-valgrind-frames-in-runner-541

Conversation

@not-matthias

@not-matthias not-matthias commented Oct 9, 2026 •

Copy link
Copy Markdown
Member

TLDR: Since exec-harness 1.4.0, Python exec benchmarks in simulation mode have no Python frames in their callgraphs. The runtime env now sets PYTHONPERFSUPPORT=1 in every mode.

  • The runtime env only set PYTHONPERFSUPPORT=1 in memory mode. Without it, Python writes no /tmp/perf-<pid>.map, so valgrind can't name Python frames. exec-harness 1.3.0 set it per command in simulation mode.
  • Walltime on Linux is unchanged: PYTHON_PERF_JIT_SUPPORT=1 takes precedence in CPython, so jitdump is still used.

Review notes

  • Simulation now sets the variable for every integration, not only exec-harness. pytest-codspeed already activates the perf trampoline itself.
  • macOS walltime keeps jitdump off (COD-2645), so Python there now emits a perf map under samply. Not tested on macOS.
  • The trampoline costs instructions, so Python exec simulation values go back up. Python 3.12 running a fractal script under callgrind: 2.22e9 Ir without it, 2.51e9 Ir with it (+13%).

Closes COD-3820

@not-matthias
not-matthias force-pushed the cod-3820-investigate-missing-python-valgrind-frames-in-runner-541 branch from c2dfd4b to dfe96fc Compare October 9, 2026 10:53
@not-matthias not-matthias changed the title fix(exec-harness): emit Python and Node perf maps in simulation mode fix(exec-harness): emit the Python perf map in simulation mode Oct 9, 2026
@not-matthias
not-matthias marked this pull request as ready for review October 9, 2026 10:54
@codspeed

codspeed Bot commented Oct 9, 2026 •

Copy link
Copy Markdown

Merging this PR will not alter performance

⚠️ Unknown Walltime execution environment detected

Using the Walltime instrument on standard Hosted Runners will lead to inconsistent data.

For the most accurate results, we recommend using CodSpeed Macro Runners: bare-metal machines fine-tuned for performance measurement consistency.

✅ 31 untouched benchmarks
⏩ 6 skipped benchmarks1


Comparing cod-3820-investigate-missing-python-valgrind-frames-in-runner-541 (aecc796) with main (24f182e)

Open in CodSpeed

Footnotes

  1. 6 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

The runtime env only set PYTHONPERFSUPPORT=1 in memory mode. Since
exec-harness 1.4.0 relies on that env, Python exec targets in simulation
mode write no /tmp/perf-<pid>.map, and valgrind callgraphs lose every
Python frame. exec-harness 1.3.0 set it per command in simulation mode.

Set it in every mode. In walltime, PYTHON_PERF_JIT_SUPPORT=1 takes
precedence in CPython, so Linux still uses jitdump. On macOS walltime,
where jitdump stays off, Python now emits the perf map.

The perf trampoline adds instructions: a Python exec benchmark measured
under callgrind goes from 2.22e9 to 2.51e9 Ir (+13%), back to the 1.3.0
level.

Closes COD-3820
@not-matthias not-matthias changed the title fix(exec-harness): emit the Python perf map in simulation mode fix(runner-shared): always set PYTHONPERFSUPPORT Oct 9, 2026
@not-matthias
not-matthias force-pushed the cod-3820-investigate-missing-python-valgrind-frames-in-runner-541 branch from dfe96fc to aecc796 Compare October 9, 2026 10:57
@greptile-apps

greptile-apps Bot commented Oct 9, 2026 •

Copy link
Copy Markdown

RetriggerConfidence Score: 5/5

[Medium impact] The PR appears safe to merge; no new actionable defect was found.

Summary

Sets PYTHONPERFSUPPORT=1 in every measurement mode so Python exec simulation benchmarks can provide frame names.

  • Keeps the existing platform-dependent PYTHON_PERF_JIT_SUPPORT setting.
  • Current pytest-codspeed skips activation when the trampoline is already active.
  • not-matthias acknowledges the higher simulation instruction counts and the macOS walltime change, which has not been tested there.
  • The previous test suggestion was manually resolved without explanation. It was unnumbered, so it has no previousFindings entry.

Reviews (2) · Last reviewed commit: "fix(runner-shared): always set PYTHONPER..." · Reviewed by Greptile

Comment thread crates/exec-harness/src/runtime_env/mod.rs Outdated

@GuillaumeLagrange GuillaumeLagrange left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I could have caught this during review sorry

@not-matthias

Copy link
Copy Markdown
Member Author

I could have caught this during review sorry

Also completely missed it. I'm a bit surprised Greptile didn't catch this too 🙃

@not-matthias
not-matthias merged commit c87b3d9 into main Oct 9, 2026
62 checks passed
@not-matthias
not-matthias deleted the cod-3820-investigate-missing-python-valgrind-frames-in-runner-541 branch October 9, 2026 12:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants