runtime
ONNX Runtime GenAI
ONNX Runtime GenAI is an open‑source extension that lets you execute generative AI models (LLMs, diffusion, speech) on‑device using ONNX Runtime. It provides Python, C#, C/C++, Java and Objective‑C bindings and supports CPU, CUDA, DirectML, OpenVINO, WebGPU and other accelerators across Linux, Windows, macOS, Android and iOS. The library is free under the MIT license; you install it via `pip install onnxruntime-genai` or build from source for custom hardware.
What it can help with
- on-device inference
- onnx runtime
- directml
- npu
- cross platform