[WIP] phase 1 of peagle support

[shanjiaz]

PLEASE FILL IN THE PR DESCRIPTION HERE ENSURING ALL CHECKLIST ITEMS (AT THE BOTTOM) HAVE BEEN CONSIDERED.

Purpose

P-EAGLE (Parallel EAGLE) is a parallel speculative decoding method that extends EAGLE-3 with multi-token prediction, potentially offering 2-3x speedups over sequential EAGLE-3 drafting. Adding this feature would enable us to train a P-EAGLE speculator model.

See RFC 292

Description

• Added peagle model & config definition with peagle specific features like learnable hidden_mask
• COD sampling for geometric decar across depth
• Use flex attention for peagle attention mask construction
• training & trainer logging support

Related Issue

See RFC 292

Tests

Still have to verify acceptance length for qwen 30b model.

I have filled in:

• The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
• The test plan/results, such as providing test command and pasting the results.
• (Optional) The necessary documentation update.
• I (a human) have written or reviewed the code in this pr to the best of my ability.

[mergify]

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @shanjiaz.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

[github-actions]

📦 Build Artifacts Available
The build artifacts (`.whl` and `.tar.gz`) have been successfully generated and are available for download: https://github.com/vllm-project/speculators/actions/runs/23116052974/artifacts/5933194505.
They will be retained for up to 30 days.
Commit: 99cc20e

[mergify]

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @shanjiaz.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork