Concepts

tokens per second

A measure of how fast the AI generates text — one token is roughly one word or word-piece, so 3,000 tokens/second means about 2,000 words generated every second.

Related areas

  • Concepts
  • Size & speed

Alternate definitions 1

Definition 2Size & speed

How fast a model generates text — higher feels snappier. It depends on both the model size and your hardware.

Size & speed · curriculum materials

Return to all terms