Skip to content

reformat example #75

Merged
dsikka merged 1 commit intomainfrom
update-example-format
Aug 1, 2025
Merged

reformat example #75
dsikka merged 1 commit intomainfrom
update-example-format

Conversation

@shanjiaz
Copy link
Copy Markdown
Collaborator

@shanjiaz shanjiaz commented Aug 1, 2025

Updated the example to follow the new converter cli format

sample output:

bash examples/eagle3/apply_eagle3_eagle.sh 
2025-08-01 15:26:55.970 | INFO     | speculators.convert.eagle.eagle3_converter:convert:39 - Converting Eagle-3 checkpoint: yuhuili/EAGLE3-LLaMA3.1-Instruct-8B
2025-08-01 15:26:55.970 | INFO     | speculators.convert.eagle.utils:download_checkpoint_from_hub:32 - Downloading checkpoint from HuggingFace: yuhuili/EAGLE3-LLaMA3.1-Instruct-8B
Fetching 2 files: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 2154.24it/s]
2025-08-01 15:26:56.093 | DEBUG    | speculators.convert.eagle.utils:download_checkpoint_from_hub:39 - Downloaded to: /proving-grounds/engine/hub_cache/models--yuhuili--EAGLE3-LLaMA3.1-Instruct-8B/snapshots/607d0d5b7871cd4b89395b6af288c070cfa0a168
2025-08-01 15:26:56.093 | DEBUG    | speculators.convert.eagle.utils:load_checkpoint_config:98 - Loading config from: /proving-grounds/engine/hub_cache/models--yuhuili--EAGLE3-LLaMA3.1-Instruct-8B/snapshots/607d0d5b7871cd4b89395b6af288c070cfa0a168/config.json
2025-08-01 15:26:56.145 | DEBUG    | speculators.convert.eagle.utils:load_checkpoint_weights:133 - Loading PyTorch weights from: /proving-grounds/engine/hub_cache/models--yuhuili--EAGLE3-LLaMA3.1-Instruct-8B/snapshots/607d0d5b7871cd4b89395b6af288c070cfa0a168/pytorch_model.bin
2025-08-01 15:26:56.605 | INFO     | speculators.convert.eagle.eagle3_converter:convert:45 - Loaded 15 weights
2025-08-01 15:26:57.416 | DEBUG    | speculators.convert.eagle.eagle3_converter:_process_checkpoint_weights:80 - Processing 15 Eagle3 weights
2025-08-01 15:26:57.416 | DEBUG    | speculators.convert.eagle.eagle3_converter:_process_checkpoint_weights:89 - Remapped: midlayer.self_attn.q_proj.weight -> layers.0.self_attn.q_proj.weight
2025-08-01 15:26:57.416 | DEBUG    | speculators.convert.eagle.eagle3_converter:_process_checkpoint_weights:89 - Remapped: midlayer.self_attn.k_proj.weight -> layers.0.self_attn.k_proj.weight
2025-08-01 15:26:57.416 | DEBUG    | speculators.convert.eagle.eagle3_converter:_process_checkpoint_weights:89 - Remapped: midlayer.self_attn.v_proj.weight -> layers.0.self_attn.v_proj.weight
2025-08-01 15:26:57.416 | DEBUG    | speculators.convert.eagle.eagle3_converter:_process_checkpoint_weights:89 - Remapped: midlayer.self_attn.o_proj.weight -> layers.0.self_attn.o_proj.weight
2025-08-01 15:26:57.416 | DEBUG    | speculators.convert.eagle.eagle3_converter:_process_checkpoint_weights:89 - Remapped: midlayer.mlp.gate_proj.weight -> layers.0.mlp.gate_proj.weight
2025-08-01 15:26:57.416 | DEBUG    | speculators.convert.eagle.eagle3_converter:_process_checkpoint_weights:89 - Remapped: midlayer.mlp.up_proj.weight -> layers.0.mlp.up_proj.weight
2025-08-01 15:26:57.416 | DEBUG    | speculators.convert.eagle.eagle3_converter:_process_checkpoint_weights:89 - Remapped: midlayer.mlp.down_proj.weight -> layers.0.mlp.down_proj.weight
2025-08-01 15:26:57.416 | DEBUG    | speculators.convert.eagle.eagle3_converter:_process_checkpoint_weights:89 - Remapped: midlayer.hidden_norm.weight -> layers.0.hidden_norm.weight
2025-08-01 15:26:57.416 | DEBUG    | speculators.convert.eagle.eagle3_converter:_process_checkpoint_weights:89 - Remapped: midlayer.input_layernorm.weight -> layers.0.input_layernorm.weight
2025-08-01 15:26:57.416 | DEBUG    | speculators.convert.eagle.eagle3_converter:_process_checkpoint_weights:89 - Remapped: midlayer.post_attention_layernorm.weight -> layers.0.post_attention_layernorm.weight
2025-08-01 15:26:57.416 | INFO     | speculators.convert.eagle.eagle3_converter:_process_checkpoint_weights:98 - Eagle model missing embeddings - adding verifier embeddings
2025-08-01 15:26:57.417 | INFO     | speculators.convert.eagle.eagle3_converter:_add_verifier_embeddings:114 - Loading embeddings from verifier model: meta-llama/Meta-Llama-3.1-8B-Instruct
source /home/hezhao/speculators/env/bin/activate
Loading checkpoint shards: 100%|██████████████████████████████████████████████████████���█████████████████████████████████████████████████| 4/4 [00:02<00:00,  1.56it/s]
2025-08-01 15:27:00.809 | INFO     | speculators.convert.eagle.eagle3_converter:_add_verifier_embeddings:132 - Loaded embeddings with shape: torch.Size([128256, 4096])
[2025-08-01 15:27:08,665] [INFO] [real_accelerator.py:254:get_accelerator] Setting ds_accelerator to cuda (auto detect)
[2025-08-01 15:27:09,681] [INFO] [logging.py:107:log_dist] [Rank -1] [TorchCheckpointEngine] Initialized with serialization = False
2025-08-01 15:27:16.956 | SUCCESS  | speculators.convert.eagle.eagle3_converter:convert:62 - Saved to: eagle3-llama-3.1-8b-instruct-converted
2025-08-01 15:27:16.956 | INFO     | speculators.convert.eagle.eagle3_converter:_validate_converted_checkpoint:237 - Validating converted Eagle-3 checkpoint...
2025-08-01 15:27:16.984 | SUCCESS  | speculators.convert.eagle.eagle3_converter:_validate_converted_checkpoint:244 - Validation succeeded
Signed-off-by: shanjiaz <zsjwpianpian@gmail.com>
@github-actions
Copy link
Copy Markdown

github-actions bot commented Aug 1, 2025

📦 Build Artifacts Available
The build artifacts (`.whl` and `.tar.gz`) have been successfully generated and are available for download: https://github.com/neuralmagic/speculators/actions/runs/16678968212/artifacts/3668479672.
They will be retained for up to 30 days.
Commit: 7b04028

@shanjiaz shanjiaz marked this pull request as ready for review August 1, 2025 15:42
@shanjiaz shanjiaz requested review from dsikka and rahul-tuli August 1, 2025 15:50
Copy link
Copy Markdown
Collaborator

@dsikka dsikka left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you update your PR description with the output that's expected?

@shanjiaz
Copy link
Copy Markdown
Collaborator Author

shanjiaz commented Aug 1, 2025

Can you update your PR description with the output that's expected?

Yes!

@dsikka dsikka merged commit 8ae0271 into main Aug 1, 2025
10 checks passed
@dsikka dsikka deleted the update-example-format branch August 1, 2025 16:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

2 participants