Skip to content

Add EagleSpeculatorConfig for EAGLE1 and HASS models with unified architecture support#34

Merged
markurtz merged 10 commits intomainfrom
eagle-speculator-config
Jun 27, 2025
Merged

Add EagleSpeculatorConfig for EAGLE1 and HASS models with unified architecture support#34
markurtz merged 10 commits intomainfrom
eagle-speculator-config

Conversation

@markurtz
Copy link
Copy Markdown
Collaborator

@markurtz markurtz commented Jun 24, 2025 •
edited
Loading

Summary

This pull request implements the EagleSpeculatorConfig class to support EAGLE1 and HASS speculator models within the Speculators framework. It introduces a unified configuration allowing both variants through configurable parameters while cleaning up related model config handling.

Details

  • Implemented EagleSpeculatorConfig:
    • Supports both EAGLE1 and HASS variants.
    • Includes parameters such as transformer_layer_architecture, transformer_layer_config, layernorms, and fusion_bias.
    • Adds a model_validator to ensure the transformer layer architecture is included in the architectures list.
  • Updated SpeculatorModelConfig:
    • Cleaned up __init__ to strip Pydantic fields before initializing PretrainedConfig.
    • Adjusted type hints and comments for clarity and compatibility.
  • Removed unused TransformerSpeculatorConfig implementation.
  • Registered new Eagle model config in module init for auto-discovery.
  • Added comprehensive unit tests in test_config.py to:
    • Validate marshalling and pretrained methods for the new config.
    • Confirm registry inclusion of EagleSpeculatorConfig.
    • Enhance coverage and validation for config save/load pathways.

Test Plan

  • Ran pytest tests/unit/test_config.py to ensure:
    • All config initialization, marshalling, and save/load methods work as expected.
    • New EagleSpeculatorConfig is correctly registered and functional.
  • Verified SpeculatorModelConfig.registered_classes() includes EagleSpeculatorConfig.

Related Issues

  • None
rahul-tuli and others added 3 commits June 23, 2025 15:14
@rahul-tuli
feat: Implement LlamaEagleSpeculator with
7895196 
serialization support

  - Add LlamaEagleSpeculator model class
  - Implement model serialization/deserialization
  - Add torch as a project dependency
  - Include unit and integration tests
  - Fixes for serialization handling

  This introduces the core LlamaEagleSpeculator
  functionality with full serialization support
@rahul-tuli
Review updates!
e1e565b
@markurtz
Rework the commits and diff to focus this just on the implementation …
12aeebb 
…for the Eagle config
@markurtz markurtz changed the base branch from eagle-speculator to main June 24, 2025 20:42
Copilot AI reviewed Jun 24, 2025
View reviewed changes
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR implements a unified configuration for EAGLE1 and HASS speculator models by introducing the EagleSpeculatorConfig class and updating related configuration utilities.

  • Added a new configuration class (EagleSpeculatorConfig) in src/speculators/models/eagle.py.
  • Updated init.py to expose EagleSpeculatorConfig.
  • Modified speculators/config.py to remove the from_pretrained method and add a from_dict override for speculators_config conversion.
  • Added "torch" as a dependency in pyproject.toml.

Reviewed Changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated no comments.

File Description
src/speculators/models/eagle.py Introduces a new unified configuration class for Eagle speculator models.
src/speculators/models/init.py Updates exports to include EagleSpeculatorConfig.
src/speculators/config.py Removes from_pretrained method and adds a from_dict override for handling speculators_config conversion.
pyproject.toml Adds "torch" as a dependency.
@nm-red-hat-upstream-automation-bot
Copy link
Copy Markdown

nm-red-hat-upstream-automation-bot bot commented Jun 24, 2025

📦 Build Artifacts Available
The build artifacts (.whl and .tar.gz) have been successfully generated and are available for download: https://github.com/neuralmagic/speculators/actions/runs/15861138985/artifacts/3395944327.
They will be retained for up to 30 days.
@nm-red-hat-upstream-automation-bot
Copy link
Copy Markdown

nm-red-hat-upstream-automation-bot bot commented Jun 24, 2025

📦 Build Artifacts Available
The build artifacts (.whl and .tar.gz) have been successfully generated and are available for download: https://github.com/neuralmagic/speculators/actions/runs/15861222688/artifacts/3395977513.
They will be retained for up to 30 days.
@nm-red-hat-upstream-automation-bot
Copy link
Copy Markdown

nm-red-hat-upstream-automation-bot bot commented Jun 24, 2025

📦 Build Artifacts Available
The build artifacts (.whl and .tar.gz) have been successfully generated and are available for download: https://github.com/neuralmagic/speculators/actions/runs/15861326944/artifacts/3396014850.
They will be retained for up to 30 days.
rahul-tuli
rahul-tuli approved these changes Jun 24, 2025
View reviewed changes
Copy link
Copy Markdown
Collaborator

@rahul-tuli rahul-tuli left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Much simpler!
MeganEFlynn
MeganEFlynn reviewed Jun 24, 2025
View reviewed changes
@nm-red-hat-upstream-automation-bot
Copy link
Copy Markdown

nm-red-hat-upstream-automation-bot bot commented Jun 24, 2025

📦 Build Artifacts Available
The build artifacts (.whl and .tar.gz) have been successfully generated and are available for download: https://github.com/neuralmagic/speculators/actions/runs/15861550975/artifacts/3396103726.
They will be retained for up to 30 days.
MeganEFlynn
MeganEFlynn approved these changes Jun 24, 2025
View reviewed changes
Copy link
Copy Markdown
Collaborator

@MeganEFlynn MeganEFlynn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me as long as we can update/build on this when adding new models beyond llama.
@nm-red-hat-upstream-automation-bot
Copy link
Copy Markdown

nm-red-hat-upstream-automation-bot bot commented Jun 24, 2025

📦 Build Artifacts Available
The build artifacts (.whl and .tar.gz) have been successfully generated and are available for download: https://github.com/neuralmagic/speculators/actions/runs/15861620973/artifacts/3396132407.
They will be retained for up to 30 days.
@nm-red-hat-upstream-automation-bot
Copy link
Copy Markdown

nm-red-hat-upstream-automation-bot bot commented Jun 24, 2025

📦 Build Artifacts Available
The build artifacts (.whl and .tar.gz) have been successfully generated and are available for download: https://github.com/neuralmagic/speculators/actions/runs/15861750022/artifacts/3396181919.
They will be retained for up to 30 days.
@rahul-tuli rahul-tuli mentioned this pull request Jun 26, 2025
Closed
@markurtz markurtz force-pushed the eagle-speculator-config branch from a56e3c4 to 6771fd1 Compare June 27, 2025 19:29
@nm-red-hat-upstream-automation-bot
Copy link
Copy Markdown

nm-red-hat-upstream-automation-bot bot commented Jun 27, 2025

📦 Build Artifacts Available
The build artifacts (.whl and .tar.gz) have been successfully generated and are available for download: https://github.com/neuralmagic/speculators/actions/runs/15934383504/artifacts/3421873540.
They will be retained for up to 30 days.
@markurtz markurtz changed the title Eagle speculator model config implementation Jun 27, 2025
@markurtz markurtz merged commit de9be4d into main Jun 27, 2025
10 of 11 checks passed
@markurtz markurtz deleted the eagle-speculator-config branch June 27, 2025 19:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

5 participants

@markurtz @anmarques @rahul-tuli @MeganEFlynn