Welcome to the official GitHub repository for HyFunc!
📢 News: this work has been accepted at the KDD 2026 !
If you find our project interesting or helpful, we would appreciate it if you could give us a star! Your support is a tremendous encouragement to us!
While agentic AI systems rely on LLMs to translate user intent into structured function calls, this process is fraught with computational redundancy, leading to high inference latency that hinders real-time applications. This paper identifies and addresses three key redundancies: (1) the redundant processing of a large library of function descriptions for every request; (2) the redundant use of a large, slow model to generate an entire, often predictable, token sequence; and (3) the redundant generation of fixed, boilerplate parameter syntax. We introduce HyFunc, a novel framework that systematically eliminates these inefficiencies. HyFunc employs a hybrid-model cascade where a large model distills user intent into a single soft token. This token guides a lightweight retriever to select relevant functions and directs a smaller, prefix-tuned model to generate the final call, thus avoiding redundant context processing and full-sequence generation by the large model. To eliminate syntactic redundancy, our dynamic templating technique injects boilerplate parameter syntax on-the-fly within an extended vLLM engine. To avoid potential limitations in generalization, we evaluate HyFunc on an unseen benchmark dataset, BFCL. Experimental results demonstrate that HyFunc achieves an excellent balance between efficiency and performance. It achieves an inference latency of 0.828 seconds, outperforming all baseline models, and reaches a performance of 80.1%, surpassing all models with a comparable parameter scale. These results suggest that HyFunc offers a more efficient paradigm for agentic AI.
We use conda to manage the environment. Please refer to the following steps to install the environment:
conda create -n HyFunc python=3.10 -y
conda activate HyFunc
pip install -r requirements.txtPlease download the training dataset from XLAM.
Please install the evaluation dataset by referring to BFCL.
Please download the LML from ToolACE.
Please download the LMS from Qwen.
To run the code, simply execute the following command:
Step 1: Use LML to synthesize function embeddings and soft token embeddings.
python LML_embedding.pyStep 2: Train the function retriever.
python function_retriever.pyStep 3: Train the LMS using soft tokens.
python LMS_SFT.pyStep 4: Perform inference using dynamic templates.
python dynamic_templating.py