All pull requests

Pull requests

• perf(kernel): avoid the activation layout conversion in fp8 moe decode and read mxfp4 gemv weights as column pairs

  #653·jason-fxz opened Oct 9, 2026Collaborator·

  ·

• test(kernel): cover Triton activation and norm fallbacks

  #652·zihaomu opened Oct 9, 2026Contributor·

  ·

• fix(scheduler): stop the swa prefill commit from freeing pages the request still reads

  #651·ischencheng opened Oct 8, 2026·

  ·
  1

• fix(engine): apply shrinking cache resizes before growing ones in live rebuild (Closes #643)

  #650·void-mckenzie opened Oct 8, 2026Contributor·

  ·
  1

• fix(rotary): preserve fp32 inputs in mrope torch fallbacks

  #641·taking-lying-flat opened Oct 8, 2026Contributor·

  ·

• feat(server): add --api-key authentication

  #639·jason-fxz opened Oct 8, 2026Collaborator·

  ·
  2

• feat(qwen3_5_moe): tensor-parallel weight sharding for multi-GPU serving

  #637·zhangnju opened Oct 8, 2026·4 of 5 tasks·

  ·

• feat(quant): opt-in fp8-block QAT for the lm_head

  #636·zhangnju opened Oct 8, 2026·2 of 3 tasks·

  ·

• perf(moe): avoid intra-op fanout while filling NVFP4 host banks

  #635·earlvanze opened Oct 8, 2026Contributor·

  ·

• build(deps): move to torch 2.14.1, triton 3.8 and sglang-kernel 0.4.9

  #630·jason-fxz opened Oct 7, 2026Collaborator·

  ·

• feat(attention): hand-written GQA-fused extend kernel for pre-sm70 (1.8x)

  #618·wac81 opened Oct 6, 2026·

  ·

• perf(fla): fix the pre-sm70 GDN chunk-kernel launch configs (7.4x / 11.3x)

  #617·wac81 opened Oct 6, 2026·

  ·

• fix(server): name the lifecycle stage in every API's 503

  #616·YevheniiKotyrlo opened Oct 6, 2026Contributor·

  ·

• fix(engine): start a single rank without a rendezvous port

  #615·YevheniiKotyrlo opened Oct 6, 2026Contributor·

  ·

• fix(server): leak no semaphores when a signal stops the server

  #614·YevheniiKotyrlo opened Oct 6, 2026Contributor·

  ·

• fix(server)!: answer /health with 503 until the server accepts requests

  #613·YevheniiKotyrlo opened Oct 6, 2026Contributor·

  ·
  1

• fix(moe): route moe_align to the atomic-free staged path below sm_70

  #612·wac81 opened Oct 6, 2026·

  ·

• fix(models): preserve glm-5.2 router correction bias in fp32

  #609·ischencheng opened Oct 5, 2026·

  ·

• perf(pre-sm70): pipeline the block-fp8 GEMMs and halve the extend-attention K/V re-reads on Pascal

  #605·wac81 opened Oct 5, 2026·

  ·

• fix(server): soft-accept OpenAI response_format json_object/json_schema

  #604·berdachuk opened Oct 5, 2026·5 tasks·

  ·

• fix(moe): fit Qwen3.8 Flash Next FP8 banks across host and GPU memory

  #602·yunwei37 opened Oct 4, 2026Contributor·

  ·
  1

  1

• feat(engine): --embed-device cpu keeps the input embedding in RAM

  #596·KarrAcaRn opened Oct 4, 2026·

  ·

• feat(server): add ft info and a pre-load memory preflight

  #595·KarrAcaRn opened Oct 3, 2026·

  ·
  1

• feat(qwen4_exp): support ModelOpt QAD checkpoints (NVFP4 PLE tables, MXFP8 dense)

  #594·skavans opened Oct 3, 2026·

  ·

• fix(checkpoint): refuse converting into a directory that already holds an FTW

  #593·Avicennasis opened Oct 3, 2026·

  ·