tok/s
Tokens per Second: notes from a homelab that runs local LLMs. Written by Mike Hirsch, mostly after midnight.
$ ollama run llama3.1:70b --verbose
>>> write a tagline for my blog
"Fast enough to be useful. Slow enough to blog about."
eval rate: 14.2 tokens/s
No posts here yet.