fix(upload): read the persisted login token again before each upload by SuperMuel · Pull Request #567 · CodSpeedHQ/codspeed

SuperMuel · 2026-10-06T09:40:01Z

Problem

codspeed run reads the token saved by codspeed auth login once, at start. It uploads with that same token at the end.

If the token expires during the run, the upload fails with 401 Invalid token, and all the results are lost. Running codspeed auth login again during the run does not help: the running CLI never reads the new token.

For example, a setup that saves a new 1-hour token before the old one expires still loses a 53-minute run this way.

Fix

Read the saved token again right before each upload. This is what the CLI already does for OIDC tokens in CI.

Only the token saved by codspeed auth login is read again. A token from --token, CODSPEED_TOKEN, --oauth-token or CODSPEED_OAUTH_TOKEN can't change during a run, so it stays as is.
The token is still checked at start, so a missing or invalid token still stops the run before the benchmarks.
If the saved token can't be read again, the run keeps the token it started with.

Proof

A 30 s walltime run, with the saved token changed 12 s into the run:

CLI	Token change during the run	Upload
this branch	new valid token	OK, with the new token
this branch	invalid token	401, so the upload really uses the new token
5.4.0	invalid token	OK, with the start token (old behavior)
this branch	invalid token at start	stops before the benchmarks

The unit tests in api_client.rs cover the same cases.

Refs COD-3768

greptile-apps · 2026-10-06T09:43:26Z

Confidence Score: 4/5

[Critical risk] Changes how authentication tokens are loaded before uploads.

The PR should address concurrent login writes before merging, because an upload can still use an expired token and lose results.

Findings

Expired token survives concurrent login ▶

Fix with agent prompt

### Issue 1
src/api_client.rs:136-139
If `codspeed auth login` is writing a refreshed token when the upload rereads the config, the read can encounter a partial file. This branch then keeps the token from the start of the run. If that token has expired, the upload fails and the run’s results are lost-the failure this change aims to prevent. The login path uses `fs::write`, so the refresh needs an atomic read/write boundary or a retry before falling back.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Summary

The PR distinguishes persisted login tokens from explicit tokens and rereads the selected profile before each upload, while retaining the initial token when rereading fails. It also adds token-reload tests and confines the legacy-config test’s environment override.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart LR
  A[Start run with persisted token] --> B[Run benchmarks]
  B --> C[Before each upload: reread selected profile]
  C -->|New token| D[Replace client authentication]
  C -->|Read fails or token absent| E[Keep in-memory token]
  D --> F[Upload]
  E --> F

_{Reviews (1) · Last reviewed commit: "fix(upload): read the persisted login to..."}

greptile-apps · 2026-10-06T09:43:29Z

+            Err(err) => warn!(
+                "Failed to read the token of profile `{}` again, keeping the one this run started with: {err}",
+                profile.profile_name
+            ),


Expired token survives concurrent login

If codspeed auth login is writing a refreshed token when the upload rereads the config, the read can encounter a partial file. This branch then keeps the token from the start of the run. If that token has expired, the upload fails and the run’s results are lost—the failure this change aims to prevent. The login path uses fs::write, so the refresh needs an atomic read/write boundary or a retry before falling back.

Prompt To Fix With AI

This is a comment left during a code review. Path: src/api_client.rs Line: 136-139 Comment: **Expired token survives concurrent login** If `codspeed auth login` is writing a refreshed token when the upload rereads the config, the read can encounter a partial file. This branch then keeps the token from the start of the run. If that token has expired, the upload fails and the run’s results are lost—the failure this change aims to prevent. The login path uses `fs::write`, so the refresh needs an atomic read/write boundary or a retry before falling back. --- For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Valid, but very unlikely (a tiny file rewritten about once an hour), and in that case the run keeps its start token, which is the current behavior, so I'd leave an atomic write in auth login for a follow-up.

That’s fair. The race window is small, and falling back to the run-start token preserves the existing behavior rather than introducing a regression in this PR. I’ll consider the partial-read risk deferred to a follow-up that makes the auth login profile write atomic.

Tip: You can customize Greptile's behavior for this repo with .greptile/rules.md and .greptile/config.json.

codspeed · 2026-10-06T09:55:17Z

Merging this PR will not alter performance

⚠️

Unknown Walltime execution environment detected

Using the Walltime instrument on standard Hosted Runners will lead to inconsistent data.

For the most accurate results, we recommend using CodSpeed Macro Runners: bare-metal machines fine-tuned for performance measurement consistency.

✅ 31 untouched benchmarks
⏩ 6 skipped benchmarks¹

_{Comparing cod-3768-wizard-codspeed-run-uploads-fail-with-401-when-the-run (82b94ce) with main (bcad8db)}

6 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

`codspeed run` read the token persisted by `codspeed auth login` once, at start, and uploaded with it. A run that outlived that token (the Wizard refreshes its one-hour tokens by running `codspeed auth login` again while the benchmarks run) uploaded with the expired one and failed with `401 Unauthorized -> Reason: Invalid token`, losing the results of the whole run. Read the profile token again right before each upload, the same way an OIDC token is minted again. Tokens given through `--token` or `--oauth-token` cannot change while the run goes, so they are kept as they are. The token is still read at start, so a missing or invalid token fails the command before the benchmarks run. The legacy-config test now isolates `XDG_CONFIG_HOME` with `temp_env` instead of setting it for the whole process, so it does not race with the new tests. Refs COD-3768 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

fargito · 2026-10-06T11:55:45Z

+    /// A token obtained through `codspeed auth login` and passed through
+    /// `--oauth-token` / `CODSPEED_OAUTH_TOKEN`.
    CliLogin(String),
+    /// The token `codspeed auth login` persisted for the selected profile. Read
+    /// again before every upload, because a login that ran during the
+    /// benchmarks may have stored a newer token.
+    PersistedCliLogin {
+        token: String,
+        profile: ProfileLocation,
+    },


Why do we need to add a new authentication method? We can keep the same method and simply re-resolve it before upload right? Isn't it what we talked about?

This keeps the profile selected at the start of the run, to avoid this case:

codspeed profile set A

codspeed run starts and reads the token of profile A

codspeed profile set B

codspeed run finishes, and the upload reads the token of profile B

It's probably rare, but possible.

Do you think this edge case is too unlikely to be worth handling?

Yes I think it a bit far-fetched. What you are telling me here is that the existing CliLogin method does not read the profile file? If it does, then maybe the ProfileLocation should be added to CliLogin instead of creating a duplicated logic

`CliLogin` already holds the token persisted by `codspeed auth login`. Give it an optional `ProfileLocation` instead of adding a separate variant: `Some` when the token was read from a profile, so it is read again before each upload, and `None` when it came from `--oauth-token` or `CODSPEED_OAUTH_TOKEN`. No behavior change. Refs COD-3768 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

SuperMuel · 2026-10-06T13:15:38Z

Closing: instead of reading the token again from disk, the CLI will get a refresh mechanism (COD-3776).

SuperMuel requested a review from fargito October 6, 2026 09:40

greptile-apps Bot reviewed Oct 6, 2026

View reviewed changes

SuperMuel force-pushed the cod-3768-wizard-codspeed-run-uploads-fail-with-401-when-the-run branch from acac8f1 to 6be3b53 Compare October 6, 2026 10:06

fargito requested changes Oct 6, 2026

View reviewed changes

SuperMuel closed this Oct 6, 2026

Navigation Menu

Repository navigation

fix(upload): read the persisted login token again before each upload - #567

fix(upload): read the persisted login token again before each upload#567
SuperMuel wants to merge 2 commits into
mainfrom
cod-3768-wizard-codspeed-run-uploads-fail-with-401-when-the-run

SuperMuel commented Oct 6, 2026 •

edited

Loading

Uh oh!

greptile-apps Bot commented Oct 6, 2026 •

edited

Loading

Uh oh!

greptile-apps Bot Oct 6, 2026

Uh oh!

SuperMuel Oct 6, 2026

Uh oh!

greptile-apps Bot Oct 6, 2026

Uh oh!

codspeed Bot commented Oct 6, 2026 •

edited

Loading

Uh oh!

fargito Oct 6, 2026

Uh oh!

SuperMuel Oct 6, 2026 •

edited

Loading

Uh oh!

SuperMuel Oct 6, 2026 •

edited

Loading

Uh oh!

fargito Oct 6, 2026

Uh oh!

SuperMuel commented Oct 6, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

SuperMuel commented Oct 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Problem

Fix

Proof

Uh oh!

greptile-apps Bot commented Oct 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Confidence Score: 4/5

Findings

Summary

Uh oh!

greptile-apps Bot Oct 6, 2026

Choose a reason for hiding this comment

Uh oh!

SuperMuel Oct 6, 2026

Choose a reason for hiding this comment

Uh oh!

greptile-apps Bot Oct 6, 2026

Choose a reason for hiding this comment

Uh oh!

codspeed Bot commented Oct 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Merging this PR will not alter performance

Footnotes

Uh oh!

fargito Oct 6, 2026

Choose a reason for hiding this comment

Uh oh!

SuperMuel Oct 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

SuperMuel Oct 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

fargito Oct 6, 2026

Choose a reason for hiding this comment

Uh oh!

SuperMuel commented Oct 6, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

SuperMuel commented Oct 6, 2026 •

edited

Loading

greptile-apps Bot commented Oct 6, 2026 •

edited

Loading

codspeed Bot commented Oct 6, 2026 •

edited

Loading

SuperMuel Oct 6, 2026 •

edited

Loading

SuperMuel Oct 6, 2026 •

edited

Loading