b9606
b9606
View on GitHubView PackagePublished: Jun 12, 2026

Release Notes

spec: add EAGLE3 speculative decoding support (#18039)

  • llama : enable layer input extraction

  • spec: support eagle3

  • eagle3: fix params bug

  • eagle3: support Gemma4 eagle3 from RedHatAI

  • eagle3: set sync when get features from target

Co-authored-by: tnhnyzc [email protected]

  • eagle3 : fix ubatch handling in embd_layer_inp extraction and encoder

Co-authored-by: Doğaç Eldenk [email protected]

  • eagle3: adapt to upstream changes

  • eagle3: fix rebase issues and adapt to upstream changes

  • eagle3:exclude the eagle3 arch from test-llama-archs

  • eagle3: fix editorconfig check failures

  • eagle3: fix multi-seq issue in d2t vocab mapping

  • cont : minor style / clean-up

  • spec : remove common_speculative_setup_draft_model()

  • llama : clean-up unused API

  • eagle3: set d2t vocab mapping in decode graph

  • cont : assert layer inputs are configured

  • hparams : use n_embd_inp instead of n_embd_target_features

  • eagle3: make output.weight optional and inherit from target model when needed

  • haparams : generic norm-before-residual param

  • llama-ext : consistent names

  • cont : fix

  • hparams : remove target_hidden_size

  • cparams : rename output_layer_inp -> embeddings_layer_inp

  • arch : reuse ATTN_NORM_2 instead of adding new hidden norm

  • llama : clean-up names

  • cont : add assert + comment

  • Update conversion/llama.py

Co-authored-by: Sigbjørn Skjæret [email protected]


Co-authored-by: Georgi Gerganov [email protected] Co-authored-by: tnhnyzc [email protected] Co-authored-by: Doğaç Eldenk [email protected] Co-authored-by: Sigbjørn Skjæret [email protected]

macOS/iOS:

Linux:

Android:

Windows:

openEuler:

  • DISABLED
  • openEuler x86 (310p)
  • openEuler x86 (910b, ACL Graph)
  • openEuler aarch64 (310p)
  • openEuler aarch64 (910b, ACL Graph)

UI: