Features

prompt_cache_key

An optional routing hint you attach to a request so that related requests (such as turns in the same conversation) are directed to the same cache, reducing latency.