Token Generation Speed Simulator
Simulate token generation speed for Large Language Models and understand how different speeds affect user experience.
The AI Cube presets are conservative guidance values based on publicly available GB10/DGX Spark benchmarks - not a measurement of your workload. Binding figures come from a benchmark with your target model, context length and concurrency.
0.00s
0 tok/s
Achieve These Speeds with AI Cube
Local AI inference without cloud dependency. Full data control.
WZ-IT AI Cube
ASUS/NVIDIA GB10 Appliance
Performance Comparison
| Model | WZ-IT AI Cube |
|---|---|
GPT-OSS 20B ~20 Billion Parameters | 80-90 tok/s |
GPT-OSS 120B ~120 Billion Parameters | 35-60 tok/s |
* Conservative guidance based on publicly available GB10/DGX Spark benchmarks. Performance depends on model, context length, backend and concurrency.
Typical Use Cases
Chatbot
Quick responses for fluid conversation
Sufficient for email generation
Report/Article
Longer texts with good performance
Token Speed FAQ
Everything you need to know about token generation
Topics
Basics
Token generation speed (measured in tokens per second or tok/s) indicates how fast an AI model can generate text. One token corresponds to approximately 4 characters or 0.75 words. At 100 tok/s, about 75 words are generated per second.
Simulation helps developers and businesses understand how different speeds affect user experience. Slow generation (under 30 tok/s) feels sluggish, while fast generation (over 100 tok/s) provides a fluid experience.
A token is the smallest unit an LLM processes. It can be a word, word part, or punctuation mark. 'Hello' is one token, 'championship' might be split into 'champion' + 'ship'. Most LLMs use about 1 token per 4 characters.
Influencing Factors
Key factors include: 1) GPU performance and VRAM, 2) Model size (7B, 70B, 120B parameters), 3) Quantization (FP16, INT8, INT4), 4) Batch size, 5) Context length, 6) Inference backend (vLLM, Ollama, TensorRT).
Larger models are slower. A 7B model can achieve 200+ tok/s, a 70B model about 50-100 tok/s, and a 120B model typically 30-60 tok/s on consumer hardware. However, response quality increases with model size.
No, speed varies. At the beginning (prefill phase), generation is often slower, then stabilizes. Query complexity, context length, and system load also affect speed.
Practical Application
Under 20 tok/s: Noticeably slow, frustrating. 20-50 tok/s: Acceptable for most applications. 50-100 tok/s: Good experience, feels fluid. Over 100 tok/s: Excellent, text appears almost instantly.
Chatbots: 50-100 tok/s for fluid conversation. Document generation: 30-50 tok/s sufficient. Real-time translation: 100+ tok/s recommended. Code completion: 100+ tok/s for best developer experience.
1) Use better GPU with more VRAM, 2) Quantization (INT4/INT8) for smaller models, 3) Use optimized inference engines like vLLM, 4) Batch processing for multiple requests, 5) Optimize KV-cache, 6) Enable continuous batching.
Hardware & AI Cube
WZ-IT AI Cube based on GB10: conservative guidance is around 80-90 tok/s with GPT-OSS 20B and around 35-60 tok/s with GPT-OSS 120B. Values depend on context length, quantization, backend and concurrency.
Cloud APIs have network latency, rate limits, and share resources with other users. Local hardware like the AI Cube offers dedicated resources, no network delay, and consistent performance without queues.
Cloud APIs charge per token; local hardware has a one-off purchase cost and no token costs afterwards. The volume at which that flips depends on the model, the provider price and your utilisation - we work it through with your numbers in the intro call.
Ready for fast AI inference?
The AI Cube offers local AI inference on owned hardware - without cloud dependency and external token costs.
One AI reference - and the foundation beneath it
ml&s speaks about integrating a local AI solution. The other voices cover architecture, data sovereignty and operations - exactly the maturity an AI project needs to reach production.
“While looking for a partner to integrate a local AI solution, we came across Timo and Robin - and could not have made a better decision. They quickly understood our requirements and solved technical challenges competently and pragmatically. We were particularly impressed by the straightforward collaboration, their high level of expertise and the speed of implementation. We are delighted to have Timo and Robin at our side as reliable business partners and look forward to our next projects together. (translated)”
Henrik Jeche
Head of IT Administration
ml&s - full-service provider for the electronics industry.
“Communication with WZ-IT was open, friendly and professional from the very beginning. Mr. Wevelsiep and Mr. Zins responded to every one of our questions patiently, promptly and respectfully, and contributed their own suggestions and good ideas on how to modernize our server infrastructure and adapt it even better to our needs. Throughout, WZ-IT consistently kept our need for data sovereignty and GDPR compliance in mind. Even short-notice adjustments and change requests from our side were implemented quickly and patiently and integrated consistently into the overall concept. From planning to the final handover, every step was documented transparently and comprehensively - so if a provider change should ever become necessary, the documentation would let us onboard a successor into our infrastructure very quickly. We are extremely satisfied with WZ-IT and hope to continue working together for a long time.”
Christoph Mußmann
Project Officer, DGHO e.V.
DGHO - German Society for Hematology and Medical Oncology.
“We have had very good experiences with Mr. Wevelsiep and WZ-IT. The consultation was professional, clearly understandable, and at fair prices. The team not only implemented our requirements but also thought along and proactively. Instead of just processing individual tasks, they provided us with well-founded explanations that strengthened our own understanding. WZ-IT took a lot of pressure off us with their structured approach - that was exactly what we needed and is the reason why we keep coming back. (translated)”
Matthias Zimmermann
Managing Director, Annota GmbH
Annota - healthtech startup with an AI documentation assistant.
“I got to know Timo and Robin as dedicated and professional partners who make a lot of things possible for their clients that initially seem impossible. I felt very well advised. The services offered have thoroughly convinced me. (translated)”
Andrea Pawlowski
Communications Manager, Golem.de
Golem.de - leading German IT news outlet.
“Professional, honest and technically thoroughly sound: WZ-IT set up our Proxmox and backup infrastructure securely and future-proof. The consulting from Timo and Robin was objective and needs-driven, the implementation smooth and the final documentation exemplary. You can tell immediately that business-critical infrastructure is thought through holistically from the outset here and implemented with clear ownership.”
Pascal Block
Project Manager, Stadtwerke Brühl GmbH
Stadtwerke Brühl - municipal energy and infrastructure utility.
“With Timo and Robin, you're not only on the safe side technically - you also get the best human support! Whether it's quick help in everyday life or complex IT solutions: the guys from WZ-IT think along with you, act quickly and speak a language you understand. The collaboration is uncomplicated, reliable and always on an equal footing. That makes IT fun - and above all: it works! Big thank you to the team! (translated)”
Sonja Aßer
Data Manager, ARGE
ARGE Neue Medien - master data & digital standards for plumbing and building technology.
“I recently worked with Timo and the WZ-IT team, and honestly, it turned out to be one of the best tech decisions I have made for my business. Right from the start, Timo took the time to walk me through every step in a simple and calm way. No matter how many questions I had, he never rushed me. The results speak for themselves. With WZ-IT, we reduced our monthly expenses from $1,300 down to $250. This was a huge win for us.”
Aleksandr Shuliko
CTO, EVA Real Estate
EVA Real Estate - leading real-estate agency in Dubai.
“WZ-IT manages our Proxmox cluster reliably and professionally. The team handles continuous monitoring and regular updates for us and responds very quickly to any issues or inquiries. They also configure new nodes, systems, and applications that we need to add to our cluster. With WZ-IT's proactive support, our cluster and the business-critical applications running on it remain stable, and high availability is consistently ensured. We value the professional collaboration and the noticeable relief it brings to our daily operations.”
Pascal Hakkers
Aphy AG
Aphy AG - AI platform for hotel back-office.
“WZ-IT provided very competent support in implementing our server and virtualization infrastructure. As part of the project, a Proxmox-based virtualization environment was set up along with a virtual machine for our application systems. Additionally, a structured backup concept was implemented. A particular focus was on security: the setup of a VPN tunnel and the clean implementation of encryption and access concepts were professionally executed. The structured consulting as well as the reliable and swift implementation deserve special mention. We are very satisfied with the services provided and are happy to recommend WZ-IT.”
John Hellmerichs
Managing Director, AInergy GmbH
AInergy - consultancy for energy and facility management.
“Timo and Robin from WZ-IT set up a RocketChat server for us - and I couldn't be more satisfied! From the initial consultation to the final implementation, everything was absolutely professional, efficient, and to my complete satisfaction. I particularly appreciate the clear communication, transparent pricing, and the comprehensive expertise that both bring to the table. Even after the setup, they take care of the maintenance, which frees up my time enormously and allows me to focus on other important areas of my business - with the good feeling that our IT is in the best hands. I can recommend WZ-IT without reservation and look forward to continuing our collaboration! (translated)”
Sebastian Maier
Managing Director, Yonju GmbH
Yonju - business coaching and AI products.
“Counting on WZ-IT team was crucial, their expertise and solutions gave us the pace to deploy in production our services, even suggesting and performing improvements over our configuration and setup. We expect to keep counting on them for continuous maintenance of our services and implementation of new solutions.”
Gabriel Sanz Señor
Managing Director, Odiseo Solutions
Odiseo Solutions - software engineering for the space industry.
“While looking for a partner to integrate a local AI solution, we came across Timo and Robin - and could not have made a better decision. They quickly understood our requirements and solved technical challenges competently and pragmatically. We were particularly impressed by the straightforward collaboration, their high level of expertise and the speed of implementation. We are delighted to have Timo and Robin at our side as reliable business partners and look forward to our next projects together. (translated)”
Henrik Jeche
Head of IT Administration
ml&s - full-service provider for the electronics industry.
“Communication with WZ-IT was open, friendly and professional from the very beginning. Mr. Wevelsiep and Mr. Zins responded to every one of our questions patiently, promptly and respectfully, and contributed their own suggestions and good ideas on how to modernize our server infrastructure and adapt it even better to our needs. Throughout, WZ-IT consistently kept our need for data sovereignty and GDPR compliance in mind. Even short-notice adjustments and change requests from our side were implemented quickly and patiently and integrated consistently into the overall concept. From planning to the final handover, every step was documented transparently and comprehensively - so if a provider change should ever become necessary, the documentation would let us onboard a successor into our infrastructure very quickly. We are extremely satisfied with WZ-IT and hope to continue working together for a long time.”
Christoph Mußmann
Project Officer, DGHO e.V.
DGHO - German Society for Hematology and Medical Oncology.
“We have had very good experiences with Mr. Wevelsiep and WZ-IT. The consultation was professional, clearly understandable, and at fair prices. The team not only implemented our requirements but also thought along and proactively. Instead of just processing individual tasks, they provided us with well-founded explanations that strengthened our own understanding. WZ-IT took a lot of pressure off us with their structured approach - that was exactly what we needed and is the reason why we keep coming back. (translated)”
Matthias Zimmermann
Managing Director, Annota GmbH
Annota - healthtech startup with an AI documentation assistant.
“I got to know Timo and Robin as dedicated and professional partners who make a lot of things possible for their clients that initially seem impossible. I felt very well advised. The services offered have thoroughly convinced me. (translated)”
Andrea Pawlowski
Communications Manager, Golem.de
Golem.de - leading German IT news outlet.
“Professional, honest and technically thoroughly sound: WZ-IT set up our Proxmox and backup infrastructure securely and future-proof. The consulting from Timo and Robin was objective and needs-driven, the implementation smooth and the final documentation exemplary. You can tell immediately that business-critical infrastructure is thought through holistically from the outset here and implemented with clear ownership.”
Pascal Block
Project Manager, Stadtwerke Brühl GmbH
Stadtwerke Brühl - municipal energy and infrastructure utility.
“With Timo and Robin, you're not only on the safe side technically - you also get the best human support! Whether it's quick help in everyday life or complex IT solutions: the guys from WZ-IT think along with you, act quickly and speak a language you understand. The collaboration is uncomplicated, reliable and always on an equal footing. That makes IT fun - and above all: it works! Big thank you to the team! (translated)”
Sonja Aßer
Data Manager, ARGE
ARGE Neue Medien - master data & digital standards for plumbing and building technology.
“I recently worked with Timo and the WZ-IT team, and honestly, it turned out to be one of the best tech decisions I have made for my business. Right from the start, Timo took the time to walk me through every step in a simple and calm way. No matter how many questions I had, he never rushed me. The results speak for themselves. With WZ-IT, we reduced our monthly expenses from $1,300 down to $250. This was a huge win for us.”
Aleksandr Shuliko
CTO, EVA Real Estate
EVA Real Estate - leading real-estate agency in Dubai.
“WZ-IT manages our Proxmox cluster reliably and professionally. The team handles continuous monitoring and regular updates for us and responds very quickly to any issues or inquiries. They also configure new nodes, systems, and applications that we need to add to our cluster. With WZ-IT's proactive support, our cluster and the business-critical applications running on it remain stable, and high availability is consistently ensured. We value the professional collaboration and the noticeable relief it brings to our daily operations.”
Pascal Hakkers
Aphy AG
Aphy AG - AI platform for hotel back-office.
“WZ-IT provided very competent support in implementing our server and virtualization infrastructure. As part of the project, a Proxmox-based virtualization environment was set up along with a virtual machine for our application systems. Additionally, a structured backup concept was implemented. A particular focus was on security: the setup of a VPN tunnel and the clean implementation of encryption and access concepts were professionally executed. The structured consulting as well as the reliable and swift implementation deserve special mention. We are very satisfied with the services provided and are happy to recommend WZ-IT.”
John Hellmerichs
Managing Director, AInergy GmbH
AInergy - consultancy for energy and facility management.
“Timo and Robin from WZ-IT set up a RocketChat server for us - and I couldn't be more satisfied! From the initial consultation to the final implementation, everything was absolutely professional, efficient, and to my complete satisfaction. I particularly appreciate the clear communication, transparent pricing, and the comprehensive expertise that both bring to the table. Even after the setup, they take care of the maintenance, which frees up my time enormously and allows me to focus on other important areas of my business - with the good feeling that our IT is in the best hands. I can recommend WZ-IT without reservation and look forward to continuing our collaboration! (translated)”
Sebastian Maier
Managing Director, Yonju GmbH
Yonju - business coaching and AI products.
“Counting on WZ-IT team was crucial, their expertise and solutions gave us the pace to deploy in production our services, even suggesting and performing improvements over our configuration and setup. We expect to keep counting on them for continuous maintenance of our services and implementation of new solutions.”
Gabriel Sanz Señor
Managing Director, Odiseo Solutions
Odiseo Solutions - software engineering for the space industry.
“While looking for a partner to integrate a local AI solution, we came across Timo and Robin - and could not have made a better decision. They quickly understood our requirements and solved technical challenges competently and pragmatically. We were particularly impressed by the straightforward collaboration, their high level of expertise and the speed of implementation. We are delighted to have Timo and Robin at our side as reliable business partners and look forward to our next projects together. (translated)”
Henrik Jeche
Head of IT Administration
ml&s - full-service provider for the electronics industry.
“Communication with WZ-IT was open, friendly and professional from the very beginning. Mr. Wevelsiep and Mr. Zins responded to every one of our questions patiently, promptly and respectfully, and contributed their own suggestions and good ideas on how to modernize our server infrastructure and adapt it even better to our needs. Throughout, WZ-IT consistently kept our need for data sovereignty and GDPR compliance in mind. Even short-notice adjustments and change requests from our side were implemented quickly and patiently and integrated consistently into the overall concept. From planning to the final handover, every step was documented transparently and comprehensively - so if a provider change should ever become necessary, the documentation would let us onboard a successor into our infrastructure very quickly. We are extremely satisfied with WZ-IT and hope to continue working together for a long time.”
Christoph Mußmann
Project Officer, DGHO e.V.
DGHO - German Society for Hematology and Medical Oncology.
“We have had very good experiences with Mr. Wevelsiep and WZ-IT. The consultation was professional, clearly understandable, and at fair prices. The team not only implemented our requirements but also thought along and proactively. Instead of just processing individual tasks, they provided us with well-founded explanations that strengthened our own understanding. WZ-IT took a lot of pressure off us with their structured approach - that was exactly what we needed and is the reason why we keep coming back. (translated)”
Matthias Zimmermann
Managing Director, Annota GmbH
Annota - healthtech startup with an AI documentation assistant.
“I got to know Timo and Robin as dedicated and professional partners who make a lot of things possible for their clients that initially seem impossible. I felt very well advised. The services offered have thoroughly convinced me. (translated)”
Andrea Pawlowski
Communications Manager, Golem.de
Golem.de - leading German IT news outlet.
“Professional, honest and technically thoroughly sound: WZ-IT set up our Proxmox and backup infrastructure securely and future-proof. The consulting from Timo and Robin was objective and needs-driven, the implementation smooth and the final documentation exemplary. You can tell immediately that business-critical infrastructure is thought through holistically from the outset here and implemented with clear ownership.”
Pascal Block
Project Manager, Stadtwerke Brühl GmbH
Stadtwerke Brühl - municipal energy and infrastructure utility.
“With Timo and Robin, you're not only on the safe side technically - you also get the best human support! Whether it's quick help in everyday life or complex IT solutions: the guys from WZ-IT think along with you, act quickly and speak a language you understand. The collaboration is uncomplicated, reliable and always on an equal footing. That makes IT fun - and above all: it works! Big thank you to the team! (translated)”
Sonja Aßer
Data Manager, ARGE
ARGE Neue Medien - master data & digital standards for plumbing and building technology.
“I recently worked with Timo and the WZ-IT team, and honestly, it turned out to be one of the best tech decisions I have made for my business. Right from the start, Timo took the time to walk me through every step in a simple and calm way. No matter how many questions I had, he never rushed me. The results speak for themselves. With WZ-IT, we reduced our monthly expenses from $1,300 down to $250. This was a huge win for us.”
Aleksandr Shuliko
CTO, EVA Real Estate
EVA Real Estate - leading real-estate agency in Dubai.
“WZ-IT manages our Proxmox cluster reliably and professionally. The team handles continuous monitoring and regular updates for us and responds very quickly to any issues or inquiries. They also configure new nodes, systems, and applications that we need to add to our cluster. With WZ-IT's proactive support, our cluster and the business-critical applications running on it remain stable, and high availability is consistently ensured. We value the professional collaboration and the noticeable relief it brings to our daily operations.”
Pascal Hakkers
Aphy AG
Aphy AG - AI platform for hotel back-office.
“WZ-IT provided very competent support in implementing our server and virtualization infrastructure. As part of the project, a Proxmox-based virtualization environment was set up along with a virtual machine for our application systems. Additionally, a structured backup concept was implemented. A particular focus was on security: the setup of a VPN tunnel and the clean implementation of encryption and access concepts were professionally executed. The structured consulting as well as the reliable and swift implementation deserve special mention. We are very satisfied with the services provided and are happy to recommend WZ-IT.”
John Hellmerichs
Managing Director, AInergy GmbH
AInergy - consultancy for energy and facility management.
“Timo and Robin from WZ-IT set up a RocketChat server for us - and I couldn't be more satisfied! From the initial consultation to the final implementation, everything was absolutely professional, efficient, and to my complete satisfaction. I particularly appreciate the clear communication, transparent pricing, and the comprehensive expertise that both bring to the table. Even after the setup, they take care of the maintenance, which frees up my time enormously and allows me to focus on other important areas of my business - with the good feeling that our IT is in the best hands. I can recommend WZ-IT without reservation and look forward to continuing our collaboration! (translated)”
Sebastian Maier
Managing Director, Yonju GmbH
Yonju - business coaching and AI products.
“Counting on WZ-IT team was crucial, their expertise and solutions gave us the pace to deploy in production our services, even suggesting and performing improvements over our configuration and setup. We expect to keep counting on them for continuous maintenance of our services and implementation of new solutions.”
Gabriel Sanz Señor
Managing Director, Odiseo Solutions
Odiseo Solutions - software engineering for the space industry.
Let's Talk About Your Idea
Whether a specific IT challenge or just an idea - we look forward to the exchange. In a brief conversation, we'll evaluate together if and how your project fits with WZ-IT.
Leading companies trust WZ-IT
Timo Wevelsiep & Robin Zins
Managing Directors of WZ-IT