Top-p sampling, also called nucleus sampling, limits token selection to a dynamic group of candidates. The group includes the most likely tokens until their combined probability reaches the configured p threshold; the model then samples from that group. This differs from top-k sampling, which uses a fixed number of candidates.
If the threshold is low, the model chooses from only the few most likely tokens, giving steadier text. If it is higher, rarer tokens enter the pool and wording becomes more varied. Many guides suggest adjusting temperature or top-p rather than both at once, because it is hard to tell which change caused a difference. Both settings act during LLM inference.
Top-p and temperature can interact, and their effects vary by model. Change one setting at a time and evaluate outputs on the actual task rather than assuming a universal best value. Related: LLM Temperature, Tokenization.