runtime

OpenVINO GenAI

OpenVINO GenAI is an open‑source Python/C++/Node.js library that lets you run LLMs, vision‑language models, diffusion image generators, speech‑to‑text and text‑to‑speech on top of the OpenVINO Runtime. Install it with a single pip command (pip install openvino-genai) and run inference on CPUs, Intel GPUs or NPUs without extra dependencies. The library is Apache‑2.0 licensed and includes tokenizers and optimization features such as speculative decoding and KV‑cache eviction.

• free
• self-host
• Commercial