b10032
b10032
View on GitHubView PackagePublished: Jul 15, 2026

Release Notes

cuda : CUDA GGML_OP_LIGHTNING_INDEXER implementation (generic vector kernel + wmma kernel) (#25545)

  • cuda : CUDA GGML_OP_LIGHTNING_INDEXER implementation (generic vector kernel + wmma kernel)

  • chore : remove indentation of #pragma unroll

  • cuda : remove unnecessary kernel template declarations

  • cuda : add WARPS_PER_BLOCK and K_VECS_PER_BLOCK template parameters in lightning indexer kernels to avoid duplication of constants.

  • cuda : relax MMA architecture requirements to Turing in lightning indexer implementation

  • chore : renamed variables

  • chore : rename ggml_cuda_op_lightning_indexer() to ggml_cuda_lightning_indexer()

  • chore : TODO for AMD rocWMMA

  • chore : whitespace formatting

  • chore : another variable rename to fix problems caused by shadowing

  • chore : yet another rename, this time uppercased all constants

  • cuda : added alignment checks for Q and K tensors in lightning indexer implementation


Co-authored-by: StanisÅ‚aw Szymczyk [email protected]

macOS/iOS:

Linux:

Android:

Windows:

openEuler:

  • DISABLED
  • openEuler x86 (310p)
  • openEuler x86 (910b, ACL Graph)
  • openEuler aarch64 (310p)
  • openEuler aarch64 (910b, ACL Graph)

UI: