Local AI for Tax Firms: §57/§62a StBerG, §203 and the RAG Path
Editorial note: The information in this article was compiled to the best of our knowledge at the time of publication. Technical details, prices, versions, licensing terms, and external content may change. Please verify the information provided independently, particularly before making business-critical or security-related decisions. This article does not replace individual professional, legal, or tax advice.
AI in the tax firm without risking confidentiality? We build local, §203-compliant AI on your hardware - see AI for tax advisors and AI for confidentiality professionals, or book an initial consultation.
Capturing receipts, pre-qualifying client emails, researching in tax-authority letters - AI can save enormous time in a tax firm. The problem is not the AI but where the data flows. Anyone entering client data into ChatGPT or Copilot transmits it to third-party servers. That can breach tax-advisor confidentiality, with consequences under both criminal and professional law.
The German Federal Chamber of Tax Advisors (BStBK) took a clear position on this in early 2026: the question is not whether firms may use AI, but how to do so in compliance. This article frames the legal situation, shows the permitted operating models, and explains how we build a RAG pipeline on your client knowledge - locally, so the data never leaves the firm.
Table of contents
- The legal situation in 2026: §57 and §62a StBerG
- Why cloud AI becomes a risk for tax firms
- Operating models for firm AI
- The RAG pipeline: components step by step
- Connecting multiple knowledge sources securely
- How we work at WZ-IT
- Further guides
The legal situation in 2026: §57 and §62a StBerG
The tax advisor's confidentiality is anchored in §57 StBerG and criminally sanctioned via §203 StGB. Importantly, this duty is broader than data protection. The BStBK stresses that §57 StBerG protects not only personal data but every mandate-related piece of information - even the fact that a mandate relationship exists (BStBK FAQ catalogue on AI, as of January 2026).
For the use of external IT, the profession-specific norm §62a StBerG applies. It permits engaging service providers but ties this to clear conditions:
- Careful selection of the provider and termination in case of violations.
- Contract in text form (§126b BGB) with a confidentiality obligation and a warning about the criminal consequences, plus a restriction to the information required.
- Under §62a (5) StBerG the requirements also apply to directly mandate-related services - here with the client's consent.
In its FAQ catalogue the BStBK explicitly distinguishes three levels: the freely accessible browser version (unsuitable for client data), a business tier with a DPA, and a dedicated platform with a §203 confidentiality contract. Client data regularly does not belong in freely accessible AI services; for sensitive data, prior consent must be obtained.
On top of this comes the EU AI Act: from 2 August 2026, transparency obligations for deployers of AI systems apply. Typical firm applications such as receipt triage or drafting correspondence are generally not high-risk systems; governance structures should nevertheless be in place early (as of June 2026, European Commission). This article is general information and not legal advice.
Why cloud AI becomes a risk for tax firms
Cloud AI is not banned per se - but with US providers it is hard to map cleanly in practice. Three points stand in the way:
- The subprocessor chain. An AI provider itself uses subprocessors (Azure, AWS, and others). Each would have to be bound in text form under §62a StBerG. With a long, opaque chain this is not practically feasible.
- The CLOUD Act. US providers are subject to the US CLOUD Act, which enables access by US authorities. An EU region does not protect against this as long as a US parent company stands behind it.
- Training on your data. Many consumer AI services process inputs further - for mandate-related data a clear breach of §57 StBerG.
The data processing agreement does not solve this: it governs data protection under Art. 28 GDPR, not the criminally sanctioned confidentiality. The DPA and the §62a/§203 obligation are separate layers - both must be satisfied.
With local AI on your hardware most of this chain disappears: the data does not leave the firm. Because we retain remote-maintenance and admin access, we remain a participating person - which is why we supply the §62a/§203 contract package for the build and maintenance phase.
Operating models for firm AI
Not every firm needs the same setup. We implement three models - with a clear link to our services:
- On-premise appliance (AI Cube). A turnkey AI box in your firm. Open WebUI, Ollama/vLLM and local models are pre-installed; ideal for small and mid-sized firms. The data stays in-house.
- Dedicated GPU server or LLM hosting. For larger firms with more users and throughput. Operated on your own hardware or in a German data centre, with an OpenAI-compatible API for integration into DATEV or your firm software.
- Managed fallback. If you do not (yet) want to operate your own hardware, you can have AI run in our EU infrastructure - with a short, controllable provider chain instead of a US hyperscaler.
In all models only open-source building blocks are used (no vendor lock-in), and the §62a/§203 contract package is standard. More on the cross-industry framework under AI for confidentiality professionals.
The RAG pipeline: components step by step
The real value comes not from a bare language model but from Retrieval-Augmented Generation (RAG): the AI answers questions from your own documents instead of from generic training knowledge - with source references. This is how we build the pipeline (more under Custom RAG):
- Ingestion. Documents are read in from the source systems (PDF, receipts, emails, scans with OCR) and normalised.
- Chunking. Content is split into meaningful sections - for tax texts oriented around sections, marginal numbers and outline levels.
- Embeddings + vector database. Each section is translated into a vector and stored in a vector database (Qdrant) - with access and client-separation metadata.
- Retrieval and re-ranking. For a query, the most relevant sections are retrieved (semantic plus keyword) and sorted by re-ranking.
- Generation. A local model (Ollama or vLLM, e.g. Mistral) formulates the answer - based solely on the retrieved passages, with source references.
- Observability and quality. With Langfuse we measure quality, latency and cost and make answers traceable - important for audits.
We run exactly this stack ourselves: our AI offer finder on wz-it.com is a production RAG system on Qdrant, LiteLLM/Mistral and Langfuse.
Connecting multiple knowledge sources securely
A firm AI is only as good as the sources it can access. We connect multiple knowledge sources into the same RAG pipeline - with enforced access rights:
- Client files and the DMS. Connection to your document management and (via integration) to DATEV.
- Receipts and accounting data. Receipt recognition, classification and pre-accounting as a basis.
- BMF letters and case law. External professional sources, where permitted by licence, for research.
- Internal templates and firm know-how. Annual-report cover letters, checklists and processes searchable from one place.
The decisive factor is client separation: through access rights and payload filters in the vector database, every query only sees the sources it is allowed to see. This preserves confidentiality even within the firm - a requirement generic cloud tools do not meet.
How we work at WZ-IT
We deliver firm AI as a lifecycle, not as a device purchase:
- Advise and design. Workshop, sizing, data classification and the §62a/§203 contract framework - before any hardware arrives.
- Build and integrate. On-premise setup, RAG on your documents with access control, integration into DATEV or your firm software, confidentiality obligation plus DPA.
- Operate and maintain (optional). Updates, monitoring, model upgrades and RAG curation as a contract - or you operate it fully yourself after handover and training.
The operational contract texts are drafted by qualified professionals; this article does not replace case-specific legal advice.
Further guides
- AI for tax advisors - the solution page with use cases and hardware options.
- AI for confidentiality professionals - the cross-industry §203 framework (also for lawyers, doctors and notaries).
- Custom RAG - how we build RAG pipelines end to end.
- AI hub - local AI infrastructure, LLM hosting and RAG at a glance.
Ready for AI that protects your clients? We build it locally, §62a/§203-compliant, and operate it on request. Book your initial consultation now.
Sources
Frequently Asked Questions
Answers to important questions about this topic
Yes. The German Federal Chamber of Tax Advisors (BStBK) makes clear in its FAQ catalogue (as of January 2026): the question is not whether but how to use AI in compliance. §62a StBerG permits engaging IT service providers if they are bound to confidentiality in text form and warned about the criminal consequences. Local AI is the cleanest option because client data never leaves the firm.
No. The data processing agreement under Art. 28 GDPR only governs data protection. Confidentiality under §57 StBerG is broader: it protects every mandate-related piece of information, even the fact that a mandate exists. It is criminally sanctioned via §203 StGB and governed for providers by §62a StBerG - separate layers in addition to the DPA.
Into the freely accessible version, regularly not. The BStBK distinguishes between the free browser version, a business tier with a DPA, and a dedicated platform with a §203 confidentiality contract. For mandate-related data the dedicated, ideally local solution is the clean path; for sensitive data, prior client consent must be obtained.
Yes, through integration into your existing firm software. DATEV's own AI functions are usable within the existing client model; a local AI adds RAG over your own documents without data leaving the building.
Retrieval-Augmented Generation combines a language model with a searchable knowledge base. The AI draws answers from your own documents - BMF letters, client history, internal templates - with source references and access control. This reduces hallucinations and keeps client knowledge local.
Client files and the DMS, receipts and accounting data, BMF letters and case law, internal templates and firm know-how, plus DATEV documentation. Through access rights and client separation, every query only sees the sources it is allowed to see.
Typical applications such as receipt triage, drafting correspondence or research are generally not high-risk systems. From 2 August 2026, however, transparency obligations for deployers apply, and governance structures should be in place. We set up operations so these requirements stay achievable.
Entry runs via the AI Cube at a fixed price; an integrated build with RAG and DATEV connection is quoted per project. Recurring costs only arise with optional maintenance - no cloud subscription. The initial consultation is free.
Written by
Timo Wevelsiep
Co-Founder & CEO
Co-Founder of WZ-IT. Specialized in cloud infrastructure, open-source platforms and managed services for SMEs and enterprise clients worldwide.
LinkedInLet's Talk About Your Idea
Whether a specific IT challenge or just an idea - we look forward to the exchange. In a brief conversation, we'll evaluate together if and how your project fits with WZ-IT.
Leading companies trust WZ-IT
Timo Wevelsiep & Robin Zins
Managing Directors of WZ-IT