Tokens per second is commonly used to describe model generation speed. Output tokens per second measures how quickly a response is generated after generation begins; input processing throughput may be reported separately. Hardware, quantization, batch size, context length and concurrent requests all affect the result.
A figure of 50 tokens per second means a 500-token answer takes about ten seconds to finish generating once it starts. Long prompts, busy servers and larger models usually lower the rate. When comparing providers, note whether the figure is for one request or total throughput across many requests, since those answer different questions. Model quantization is one way to raise speed on limited hardware.
Compare benchmarks only when their measurement methods and workloads are similar. High throughput does not necessarily mean low time-to-first-token or a better experience for a single user. Related: Inference Latency, LLM Inference.