Add shimmy - Python-free Rust inference server #22

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Open

Michael-A-Kuykendall wants to merge 1 commit into vaaaaanquish:main from Michael-A-Kuykendall:main

Michael-A-Kuykendall commented Oct 18, 2025

Adds shimmy to the Interface & Pipeline & AutoML section.

Project Details:

Repository: https://github.com/Michael-A-Kuykendall/shimmy
Crate: https://crates.io/crates/shimmy
Stars: 2,918 ⭐
Description: Python-free Rust inference server — OpenAI-API compatible

Key ML Features:

OpenAI-compatible API for seamless integration
GGUF + SafeTensors format support
Hot model swapping for dynamic serving
Auto-discovery of available models
Single binary deployment (no Python dependencies)
GPU acceleration support
LLaMA, ChatML, and various model architectures

Why this belongs in Awesome Rust ML:

Pure Rust implementation for ML inference serving
Production-ready inference server with significant adoption
Fills important gap for OpenAI-compatible Rust inference solutions
Zero-dependency deployment suitable for ML production environments
Active development with regular feature additions

Placement: Added alphabetically to Interface & Pipeline & AutoML section alongside other inference servers like orkhon, wonnx, and tract.

Following contribution guidelines: "Please just update the README.md"


          Add shimmy - Python-free Rust inference server

697ad47

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet