Replace OpenAI GPT with another LLM in your app by changing a single line of code. Xinference gives you the freedom to use any LLM you need. With Xinference, you're empowered to run inference with any open-source language models, speech recognition models, and multimodal models, whether in the cloud, on-premises, or even on your laptop.

An Easy-to-use, Scalable and High-performance RLHF Framework based on Ray (PPO & GRPO & REINFORCE++ & LoRA & vLLM & RFT)

Data processing with ML, LLM and Vision LLM

📚A curated list of Awesome LLM/VLM Inference Papers with codes: WINT8/4, FlashAttention, PagedAttention, MLA, Parallelism etc.

Ramalama is an open-source tool that simplifies the local use and serving of AI models for inference from any source through the familiar approach of containers.

Evaluate your LLM's response with Prometheus and GPT4 💯

LLM notes, including model inference, transformer model structure, and llm framework code analysis notes.

Community maintained hardware plugin for vLLM on Ascend

A Python library powered by Language Models (LLMs) for conversational data discovery and analysis.

Make Discord your LLM frontend ● Supports any OpenAI compatible API (Ollama, LM Studio, vLLM, OpenRouter, xAI, Mistral, Groq and more)

Production ready LLM model compression/quantization toolkit with hw accelerated inference support for both cpu/gpu via HF, vLLM, and SGLang.

[EMNLP 2024 Industry Track] This is the official PyTorch implementation of "LLMC: Benchmarking Large Language Model Quantization with a Versatile Compression Toolkit".

A large-scale simulation framework for LLM inference

基于SparkTTS、OrpheusTTS等模型，提供高质量中文语音合成与声音克隆服务。

The RunPod worker template for serving our large language model endpoints. Powered by vLLM.

TopicGPT: A Prompt-Based Framework for Topic Modeling (NAACL'24)

A CPU Realtime VLM in 500M. Surpassed Moondream2 and SmolVLM. Training from scratch with ease.

[CVPR 2025] RoboBrain: A Unified Brain Model for Robotic Manipulation from Abstract to Concrete. Official Repository.

gpt_server是一个用于生产级部署LLMs、Embedding、Reranker、ASR和TTS的开源框架。

Referring any person or objects given a natural language description. Code base for RexSeek and HumanRef Benchmark