Local, GDPR-Compliant Speech Recognition: Whisper, Voxtral and Parakeet Compared

Transcribing dictation, meetings or calls in your own network? WZ-IT runs speech-to-text models such as Whisper or Voxtral on the AI Cube in your company network or on a managed GPU server, see the AI hub. Book a meeting

Speech recognition is one of the AI applications with the clearest benefit: dictation becomes text, meetings become minutes, phone calls become searchable notes. At the same time, audio is particularly sensitive. A recording contains the voice itself in addition to the content, often names, health details or client matters, and it is frequently created in situations where the people involved do not expect it to be passed on.

Cloud transcription services process this data at the provider. Open models such as OpenAI Whisper, Mistral Voxtral and NVIDIA Parakeet, by contrast, run entirely on your own hardware. This article compares the models based on their model cards (license, size, German support), shows the common runtimes, speaker separation with pyannote and the legal framework of the GDPR, German criminal law and the EU AI Act. All model details reflect September 2026.

Table of Contents

1. What local speech recognition means
2. The models at a glance: Whisper, Voxtral, Parakeet
3. German: what the vendor figures say
4. Runtimes and interfaces
5. Speaker separation with pyannote
6. Hallucinations and quality assurance
7. Hardware: what fits on 24 GB, 96 GB and the AI Cube
8. Legal framework: GDPR, German Criminal Code and AI Act
9. Use cases: meetings, dictation, phone
10. Our approach at WZ-IT
11. Further guides

What local speech recognition means

Local means: the model and the audio files sit on a server the company controls itself, in its own network or on a dedicated server in a data centre. Neither audio nor transcript goes to a model provider. A typical chain consists of four steps:

Recognition itself is the most compute-intensive part, but not the only one that touches data. Summarisation by a language model should also run locally, otherwise the transcript leaves the building in the last step after all.

The models at a glance: Whisper, Voxtral, Parakeet

Sources: model cards of Whisper large-v3, Whisper large-v3-turbo, Voxtral Mini 3B, Voxtral Small 24B, Voxtral Mini 4B Realtime, Parakeet TDT 0.6B v3 and Canary-1B-v2; Whisper release dates from the announcements of large-v3 and turbo.

Whisper is the benchmark other models are measured against. The Whisper repository releases code and weights under the MIT license. Whisper processes audio in 30-second windows; for longer recordings the runtime stitches the segments together. large-v3-turbo reduces the decoder layers from 32 to 4, which makes it considerably faster and, according to OpenAI, comes with a minor quality degradation. turbo is not trained for translation into English.

Voxtral is a language model with audio input. Besides plain transcription it answers questions about the spoken content and summarises it without a separate language model. According to Mistral, Voxtral handles up to 30 minutes of audio for transcription and up to 40 minutes for understanding tasks with a 32,000-token context. Voxtral Mini 4B Realtime is designed for live transcription, with a configurable delay between 80 ms and 2.4 s.

Parakeet is a pure recognition model from the NVIDIA NeMo framework. It delivers punctuation, capitalisation and word- and segment-level timestamps. According to the model card, it processes up to 24 minutes of audio in one pass with full attention and up to three hours with local attention. Canary-1B-v2 from the same line additionally translates between English and the 24 other languages.

On licensing: MIT and Apache 2.0 permit commercial use with no conditions beyond the license notice. CC BY 4.0 also permits it but requires attribution. For an internal tool this is unproblematic; in a product for third parties the notice belongs in the documentation.

Not all Voxtral models are open. Voxtral Mini Transcribe V2 with built-in speaker diarization, announced in February 2026, is offered by Mistral only through its own API. From this generation, open weights exist only for Voxtral Realtime.

German: what the vendor figures say

For decision-makers, the word error rate (WER) in German is the obvious metric. The available figures, however, come from the vendors themselves:

These figures are not directly comparable. Normalisation of punctuation and numbers, decoding parameters and test setup differ, and a realtime model with 480 ms delay works under different conditions than a model that sees the entire file. FLEURS and CoVoST consist of read sentences in good recording quality. Telephone audio at 8 kHz, room reverberation in meetings, dialect and specialist vocabulary (drug names, case numbers, product names) degrade the results of every model.

Mistral states that Voxtral outperforms Whisper large-v3 in its own tests (Mistral). That, too, is a vendor statement. What counts for the selection is a test with 30 to 60 minutes of your own recordings that have been transcribed correctly by hand. The WER can be calculated from them with standard tools such as jiwer, and specialist terms belong in a separate checklist.

Runtimes and interfaces

A model on its own is not yet a service. Between model and application sits a runtime that accepts audio, runs the model and returns text with timestamps.

For integration, the interface matters more than the model. Speaches and vLLM offer an endpoint in the format of the OpenAI audio API. Applications already built for a cloud transcription service can be switched to your own server by changing the base URL and model name. WZ-IT runs Speaches as a managed service and vLLM for language and audio models. How vLLM differs from Ollama and llama.cpp is covered in vLLM, Ollama or llama.cpp.

If you use Nextcloud, transcription can also be connected through the Nextcloud Assistant; the changes in version 35 are described in the article on Nextcloud 35.

Speaker separation with pyannote

Meeting minutes need to show who said what. Whisper, Voxtral 2507 and Parakeet do not provide this information. It is produced in a second step, diarization.

The common open solution is pyannote. The current open model speaker-diarization-community-1 is licensed under CC BY 4.0, runs with pyannote.audio 4.x and can be operated fully offline once downloaded. The download requires registering on Hugging Face and sharing contact details with the developers. The stronger precision-2 model, by contrast, runs only through the pyannoteAI API and is therefore not suitable for local operation.

Two points belong in the planning:

• Placeholders instead of names. Diarization assigns labels such as SPEAKER_00. Mapping them to people happens manually or by matching against stored voice profiles.
• Voice profiles are sensitive. Data that allows the unique identification of a person based on behavioural or physiological characteristics is biometric data under Art. 4(14) GDPR. If it is processed for identification, Art. 9 GDPR applies. For most minutes, manual assignment is sufficient.

Overlapping speech remains a weakness of all methods. Mistral notes for its own API model that with overlapping speech the model typically transcribes one speaker (Mistral).

Hallucinations and quality assurance

Speech recognition with generative models can produce text that was never spoken. The Whisper large-v3 model card states this limitation explicitly. A study by Koenecke et al. (Careless Whisper, ACM FAccT 2024) found entire hallucinated phrases or sentences in roughly 1% of transcriptions; 38% of these hallucinations contained explicit harms such as inaccurate associations.

Measures that reduce the risk:

A transcript is a draft. For documents that decisions are based on, a human review step belongs in the workflow, and the audio should remain available until the review is complete.

Hardware: what fits on 24 GB, 96 GB and the AI Cube

Speech recognition needs far less GPU memory than large language models. Vendor figures:

For WZ-IT's offerings this means:

The Managed GPU Server 24 costs €699 net per month plus €499 one-time setup, the Managed GPU Server 96 €1,799 net per month plus €999 setup. The AI Cube costs €6,490 net one-time, AI Cube Care for ongoing operations €349.90 net per month. How many concurrent users and tasks a system can handle is described in the article on sizing a local AI server. The calculation for language models is shown in GPU VRAM sizing.

Throughput depends on model, runtime, batch size and audio quality. Reliable figures come only from measuring on the target hardware with realistic recordings. For the AI Cube with its Arm processor, it is also necessary to check whether the chosen runtime is available for the platform.

Legal framework: GDPR, German Criminal Code and AI Act

Local operation solves the question of the data path, not every data protection obligation. The following overview is a classification, not legal advice.

For professionals bound by secrecy, Section 203(3) StGB permits disclosure to other persons contributing to their work insofar as this is necessary for that work; subsection (4) makes the professional responsible for ensuring those persons have been bound to confidentiality. With local processing on your own hardware, there is no disclosure to a model provider. Maintenance by an IT service provider still falls under these rules and needs a corresponding agreement.

For recordings in an employment context, the works council's co-determination right under Section 87(1) no. 6 of the Works Constitution Act applies if the recording is suitable for monitoring performance or behaviour. How an AI assistant can be agreed with the works council is described in AI assistant and the works council. The other obligations under the AI Act are summarised in EU AI Act for companies.

Three decisions have proven useful in practice: a retention period for the audio (for example until the transcript is approved), separate access rights for audio and text, and logging of who transcribed and accessed which recording.

Use cases: meetings, dictation, phone

For law firms and medical practices, the articles Local AI for law firms and Local AI for medical practices describe the wider framework. Preparing conversations with departing employees is part of the knowledge transfer offering, which includes transcription where needed.

Our approach at WZ-IT

1. Assessment. Audio sources, volume per day, languages and specialist vocabulary, protection needs and further processing. This determines whether batch transcription, live operation or both are needed.
2. Model test with your own recordings. A selection of Whisper, Voxtral and Parakeet against hand-checked reference transcripts, with word error rate and a checklist for specialist terms.
3. Platform. Operation on the AI Cube in your own network or on a managed GPU server in Germany, with an OpenAI-compatible interface via Speaches or vLLM.
4. Integration. Speaker separation, summarisation by a local language model, hand-over to business software or filing, retention periods and access rights according to your data protection concept.
5. Operations. Updates, monitoring and regression testing before model changes. Support, consulting and implementation by WZ-IT.

Audio processing is a separate integration and is implemented after a professional and technical review. The framework for ongoing operations is described in Managed AI.

Further guides

• Local AI for law firms, professional secrecy, client data and operation on your own hardware.
• Local AI for medical practices, health data, documentation and local models.
• vLLM, Ollama or llama.cpp, which inference software fits which use.
• Processing documents with AI, how transcripts and documents flow into processes.
• Sizing a local AI server, users, tasks and hardware.
• AI solutions from WZ-IT, the hub with all offerings for local AI.

Transcription without a cloud service? We test Whisper, Voxtral and Parakeet with your recordings and run speech recognition on the AI Cube or a managed GPU server. Book a meeting

Sources

• OpenAI, Whisper repository
• OpenAI, Whisper large-v3 model card
• OpenAI, Whisper large-v3-turbo model card
• OpenAI, Whisper large-v3 announcement
• OpenAI, Whisper large-v3-turbo announcement
• Mistral AI, Voxtral
• Mistral AI, Voxtral Transcribe 2
• Mistral AI, Voxtral Mini 3B model card
• Mistral AI, Voxtral Small 24B model card
• Mistral AI, Voxtral Mini 4B Realtime model card
• NVIDIA, Parakeet TDT 0.6B v3 model card
• NVIDIA, Canary-1B-v2 model card
• pyannote, speaker-diarization-community-1
• SYSTRAN, faster-whisper
• WhisperX
• Speaches
• vLLM, online serving
• Koenecke et al., Careless Whisper: Speech-to-Text Hallucination Harms
• GDPR, Art. 4
• GDPR, Art. 9
• GDPR, Art. 35
• Section 201 StGB, violation of the confidentiality of the spoken word (German)
• Section 203 StGB, violation of private secrets (German)
• Section 87 BetrVG, co-determination rights (German)
• EU AI Act, Article 5