Skip to content

feat: LLVM 23.1.3 native ARM64 GNU and openkal host support - #781

Draft
Sunrisepeak wants to merge 25 commits into
mainfrom
feat/llvm-2313
Draft

Sunrisepeak wants to merge 25 commits into
mainfrom
feat/llvm-2313

Conversation

@Sunrisepeak

@Sunrisepeak Sunrisepeak commented Oct 7, 2026 •

Copy link
Copy Markdown
Member

Summary

Move the LLVM line to 23.1.3 and make fresh Linux aarch64 installations select LLVM with the native GNU target. Architecture-specific managed glibc, headers and companion resources close the native host and development dependencies. Existing user settings and explicit musl targets remain available; Linux x86_64 retains GCC 16.1.0.

Host compiler dependencies and graph-supplied target runtimes remain independent. Native Linux ARM64 joins the existing Linux x86_64, macOS ARM64 and Windows x64 build hosts for openkal cross builds to Linux x86_64, Windows x64 and macOS ARM64. Target runners execute the exact produced artifacts without installing a compiler or C/C++ runtime.

Implementation and regressions

  • Preserve user-declared toolchains for graph-supplied targets, while avoiding unsupported foreign payload installation for engine-selected tools.
  • Keep ARM host std PCM and importers on consistent compiler-rt settings; suppress automatic unwinder selection when a self-contained ELF already links its static unwinder explicitly.
  • Centralize E2E compiler versions; adapt the libc++ 23 namespace fixture and accept both exact valid ARM GNU triple spellings.
  • Fix both xcode-27 shards, custom MCPP_HOME helpers, the isolated cold-registry fixture and the reserved GCC_ROOT test export. Bare Windows now passes; the diagnosed fixture conflict supersedes the earlier image/Defender attribution for E2E toolchain helper exports GCC_ROOT and redirects cold MinGW and musl driver helpers #783.
  • Update SPEC-009, bilingual usage documentation and the measured four-host declaration. Compiler installation and scanning derive from the same declaration column.
  • Supply ecosystem test emulators and the separate MinGW header fixture through three named xlings steps. E2E 738 first proves unrestricted Clang actually finds the private host headers, then checks that real graph-supplied commands exclude them. The private fixture stays outside MCPP_HOME and is exposed only to that test.

Validation

Final source: c6dca54e673c50a86edd109d5555c54743418f8f.

  • Main CI 37774864382: 55 successful, one conditional skip, zero failures or pending jobs.
  • ARM fresh installation 37774863874: one successful, one conditional skip. Both workflows conclude success; the PR rollup contains 56 successful and two skipped checks.
  • All four build hosts produce three targets. All three target-system jobs execute products from every host. Each target log contains four actual indexed JSON success lines and four TLS/destruction/concurrent-unwind success lines, in addition to the original examples: 24 products across 12 host/target combinations.
  • All four downloaded final-head matrix artifacts match the 244-row declaration across every one of the 11 fields. Counts are Linux ARM64 66, Linux x86_64 70, macOS ARM64 38 and Windows x86_64 70. Previous keys remain; 35 ARM rows are added and 18 rows change. There are no duplicate keys, build-fail or unnamed rejection reasons.
  • The earlier cold revision 3 native admission 37736041779 passed 147 unit/integration programs, GNU self-hosting, four actual index consumers and native JSON/TLS. Its exact source/index contract and missed parent caches establish fresh revision 3 consumption; installer stdout does not directly report archive hashes. Native four-consumer admission and representative JSON cross admission have separate scopes.
  • Final managed test-tool preparation takes 1/5/5 seconds. All 11 actual ecosystem scripts and the Each one RAN gate pass, including the MinGW positive control. The previous silent 43-minute preparation was cancelled rather than counted as successful; the intermediate unsupported xlings run invocation is corrected to the actual registered shim.
  • Local owning-home emulator shims, actual managed MinGW installation and exact LLVM 23.1.3 positive/negative controls pass. Workflow/coverage fixture tests pass 25/25; shell, YAML, pins, documentation and workflow assertions pass. The complete 142-file PR list contains no logs, archives, build outputs or local temporary review files.

The existing iOS reporter's informational glibc(payload) label is documented as a reporting limitation, not an iOS glibc linkage claim. Matrix rejection rows remain explicit. Actual macOS products retain their platform libSystem dependency.

Dependencies and release boundary

openxlings/xlings#647 is merged; client v2026.10.8.1 is released and mirrored. openxlings/xim-pkgindex#938 was reviewed and merged by the previously authorized bypass squash as dc8bf083; its final head had 20 successes, three conditional skips and no failures. Architecture-specific glibc revision 3 archives and the ARM LLVM resource closure are published. Full GLOBAL/CN bytes, sidecars and asset metadata match. The authenticated equal-tree squash provenance guard preserves native admission and archive checks.

This PR is ready for the maintainer's review and merge decision. It remains unmerged; no mcpp release or tag has been created. After approval and merge, release from the actual squash commit, verify the formal mirrored engine and cold CN/SubOS ecosystem consumption, then review the automatic index update and bootstrap pin. LLVM latest remains at 22.1.8 until released consumption satisfies SPEC-009 section 10.7. Source CI does not replace released acceptance.

Part2 design and execution record record the decisions and evidence. The final Chinese review report is delivered locally under .agents/docs/ without another source push.

Closes #780
Closes #783
Related: #784

@Sunrisepeak

Copy link
Copy Markdown
Member Author

CI status notes on the two red legs (both diagnosed, neither is the line move failing its own gate):

  1. openkal / build 3 targets on macos — pre-existing main regression, filed as openkal cross-build macos leg: x86_64-linux-gnu refused although openkal-linux@0.16.1 is in the graph (graph-supply release not firing) #782. The same-source example pins its own [toolchain] default = "llvm@22.1.8", the refusal path (host_can_serve + the unserved-target diagnosis) is untouched by this PR, and the openkal job has not run on main since the 2026-10-02 CI acceleration (path-gated: run 37401792942 skipped it). This PR is simply the first tree to exercise the leg in three weeks. The failure fires at the early host-serve gate for x86_64-linux-gnu on a macOS host although openkal-linux@0.16.1 is in the graph — the graph-supply release is not firing.

  2. macos / xcode-27 integration leg — ci-macos xcode-27: ld64.lld cannot parse arm64e.x1 in either available SDK (upstream, tracked) #669's failure is gone: mcpp itself built with 23.1.3 on the macOS 27 image and the leg reached the final integration step, which clones openxlings/xlings and builds it. That build resolves xlings' OWN manifest pin macos = "llvm@20.1.7" — 20.1.7's ld64.lld has the same arm64e.x1 TAPI defect — and fails with the old signature on a foreign repository's pin. The cross-repo fix is build: move the macOS self-host pin to llvm@23.1.3 openxlings/xlings#645 (macos pin → 23.1.3, windows deliberately unchanged); once it merges, this leg is expected green and ci-macos xcode-27: ld64.lld cannot parse arm64e.x1 in either available SDK (upstream, tracked) #669 closes.

@Sunrisepeak

Copy link
Copy Markdown
Member Author

Pushed the second段 of work, three parts:

1. #782 repair. The openkal leg's rerun failure was the decisive sample: llvm@23.1.3 installed successfully (the log shows the autoInstall completing and resolving) and the same refusal still fired — proving the skip was not the only overreach; the whole path treated "diagnosis held" as "unbuildable". The repair: the toolchain resolution always attempts the install now (a declared or --target toolchain is wanted exactly when the graph supplies the target's system); when the install itself fails and the diagnosis stands, it releases there, keeping one cause per message. The openkal leg of this CI run is the fix's verification.

2. e2e toolchain-version abstraction. tests/e2e/_toolchain_env.sh is now the one place a test learns a toolchain version (override env → newest installed payload → fallback constant, per family: llvm/gcc/musl-gcc/mingw-cross). The llvm family's ~90 literals across 38 scripts read it; the next line move edits one file here. _llvm_env.sh stays as an alias shim.

3. #669 closed: xlings#645 merged, both xcode-27 legs green on rerun — the issue's own two closure conditions, both met.

Also answered on the thread: llvm@latest is not accepted today (measured: it is treated as a literal version directory). The engine pins exact releases by design (SPEC-006 §2.1, SPEC-009 §3.4 — reproducible builds). The "track newest" convenience is feasible as parse-layer sugar (translate to the family's newest installed/indexed version, state the translation, keep the exact version in the cache key) — filed as a follow-up decision, not in this PR.

@Sunrisepeak

Copy link
Copy Markdown
Member Author

Round-3 fixes for the two target-matrix legs the previous push red'd, plus the origin-aware form of the #782 repair:

#782 repair, refined by its own CI. The unconditional install fired on a case it should not: linux-aarch64's restored sandbox cache holds an x86_64-only gcc@16.1.0, so an engine-chosen musl/gnu spec could extract a foreign-arch payload and die later at the hermetic check (297's hosted control on that leg). The install now distinguishes who chose the toolchain: a user-declared toolchain (manifest [toolchain], --toolchain, MCPP_TOOLCHAIN) still installs through the held diagnosis — the declaration outranks the payload matrix, same standing as the [target.X] toolchain escape hatch; an engine-chosen one (row pin, host default) skips when the target is unservable, releasing the held diagnosis with its correct sentence. This is the rule the SPEC-009 §11 distinction (recorded default vs user declaration) implies for installs.

297's gate now asks the resolver, not the file listing. The matrix contract on non-linux-x86_64 hosts is a skip naming the FACT ("gcc is not installed here"); the old gate read toolchain list, which a foreign-arch cache leftover satisfies while the toolchain cannot run. The gate now queries why toolchain for the host-arch gnu row — in a directory with a minimal manifest, since from a bare one the query refuses with no mcpp.toml found (reason other) before reaching the resolver. A real gcc resolves (reason: none, the test runs, as on linux-x86_64); a foreign-arch leftover gets host-cannot-serve and the declared skip. Measured locally: linux-x86_64 with a working gcc runs to OK.

@Sunrisepeak

Copy link
Copy Markdown
Member Author

Round-4 fix, caught by the macos-e2e leg: the heredoc unquoting from the abstraction migration had silently not landed — 14 fixtures still carried <<'EOF' with `` inside, so macOS/Windows read the literal llvm@${LLVM_VERSION} from their generated manifests and died at resolution (234's "build with bmi_schedule=on failed" was this, not a scheduler regression). Re-applied with an absolute-path pass and a re-audit: no `${LLVM_VERSION}` remains inside a quoted heredoc; 234/738/804 re-verified locally with the expansion in place. (881/182 local failures are environment-shaped — 881 is `requires: msvc` Windows-only, 182 times out on this machine's cold index init — both run in CI's own shards.)

23.1.3 is the first point release carrying the macOS 27 arm64e.x1 ld64.lld
fix (llvm-project#222721, backported to release/23.x as ee66426) — the
release #669 was blocked on. clang 23.1.0's MSVC STL std-module defect was
fixed in 23.1.1 (#640, llvm-project#218152).

The line move (single PR per SPEC-009 §10.6):
- kKnownTargets: the 17 llvm rows move 22.1.8 -> 23.1.3; the gcc rows stay.
- Host defaults: macOS and Windows-with-MSVC move to llvm@23.1.3, resolving
  the [toolchain] macos/windows deviations (SPEC-009 §12). Linux keeps
  gcc@16.1.0 by design — native glibc ABI — now recorded as a §4.1 reason
  at the single pin site, with the gcc@15.1.0-musl reason beside it.
- All four known-red legs (#669) become unconditional normal legs; the
  known-red floor assertion moves to zero. The xcode-27 legs went green on
  rerun and #669 is closed.
- Readers move with the tables: workflows, the macOS action, CI tools, e2e
  fixtures, tests/matrix/expected.tsv, examples, user docs (en/zh); SPEC-009
  v0.4 records the move.
- libc++ 23 adaptation the gate caught: _LIBCPP_BEGIN_NAMESPACE_STD opens
  under the ODR-signature abi_tag pragma, and clang refuses to add abi_tag
  on a redeclaration — e2e 133's verbose_abort override now spells the
  namespaces by hand.

The #782 repair: a held unserved-target diagnosis no longer skips the
toolchain install. A declared or --target toolchain resolves and installs
even when no payload here serves the target — a retargetable clang plus a
graph package supplying the target's system is the arrangement the openkal
rows exist for, and the skip fired before the graph release could run the
moment the suite stopped installing the pinned version out of band
(measured on the openkal macos leg, twice). When the install itself fails
and the diagnosis stands, it releases there. e2e 890 pins the contract on
the mac/windows legs.

The e2e toolchain-version abstraction: tests/e2e/_toolchain_env.sh is the
one place a test learns a toolchain version — override, newest installed,
fallback constant, per family — and the llvm family's ~90 literals across
38 scripts now read it. The next line move edits one file here. _llvm_env.sh
becomes an alias shim.

Closes #780. Closed #669 (xcode-27 legs green on rerun). Files #782 as the
remaining openkal macos regression this repair addresses.
Add the ARM64 first-run pin, native GNU host row and runtime paths; preserve explicit musl and existing user defaults. Add cold payload admission, complete xcode-27 shards, respect MCPP_HOME and capture Windows compiler failure evidence.

Prepare release 2026.10.8.1. Native ARM64 remains preview until the coordinated resources and client release pass the measured integration gates.

Refs: #784, #783
@Sunrisepeak Sunrisepeak changed the title Move the LLVM line to 23.1.3 and the macOS/Windows host defaults with it feat: LLVM 23.1.3 defaults and native Linux ARM64 GNU support (2026.10.8.1) Oct 7, 2026
Exercise standard modules, exceptions, threads, 16-byte atomics, a C shared-library consumer and relocated self-contained deployment. Record managed loader/header resolution, require native openkal execution and preserve fresh-install Windows failure diagnostics.

Document the explicit compiler-and-target migration command and its publication prerequisites.

Refs: #784
@Sunrisepeak Sunrisepeak changed the title feat: LLVM 23.1.3 defaults and native Linux ARM64 GNU support (2026.10.8.1) feat: LLVM 23.1.3 native ARM64 GNU and openkal host support Oct 7, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

2 participants