Features

prompt caching

A feature that reuses the processed results of repeated prompt prefixes — like a shared system prompt — so subsequent calls with the same opening are faster.