“Couldn’t we just use ChatGPT for this?”
We hear this question almost every time we talk to a construction company about AI. And it is a reasonable one. ChatGPT can summarize documents, answer questions and work with large amounts of text.
Things become more complicated when real project data enters the picture: contracts, invoices, payment schedules, customer names and addresses.
At that point, a good chatbot is no longer enough. You need to know where the data is stored, who can access it and what information the AI is actually using to produce an answer.
For a German property developer, we built an internal AI platform designed around exactly these requirements. It searches construction law and project documents, answers internal questions and keeps sensitive customer data inside the company’s own infrastructure.
Here is how it works.
The real problem: the information exists, but finding it takes time
A single construction project can easily generate hundreds of documents: contracts, payment schedules, progress invoices, change orders, measurements, delivery notes, site reports and email attachments.
Then there is the regulatory layer: the German Civil Code, VOB/B, GEG, MaBV, state building regulations and funding requirements from institutions such as KfW and BAFA.
The information is already there. It is simply scattered across too many places.
When someone needs to know which payment is due next or what was agreed in the latest change order, they start searching. If they cannot find it, they ask a colleague, who then starts searching too.
This is exactly the kind of problem AI can solve well.
A public AI chatbot, however, only solves part of it. It does not know the company’s internal project data. It can produce convincing but incorrect answers to legal questions. And copying contracts or personal customer information into an external cloud service creates an obvious data protection problem.
So we designed the architecture differently.
Privacy is part of the architecture
We did not want data protection to depend on every employee remembering which information they are allowed to paste into which AI tool.
Instead, the platform technically separates two different types of knowledge.
Public knowledge includes laws, regulations, funding guidelines and technical documentation. It contains no customer data and can be processed using powerful models running within a controlled EU cloud environment.
Internal project data includes contracts, invoices, payment schedules, construction-site images and other sensitive information. This data stays on the company’s own infrastructure and is processed by a locally hosted open-source language model.
The separation happens automatically.
Employees do not have to decide which model should handle a particular question. The system routes the request to the appropriate environment. Sensitive project data cannot accidentally be sent to the cloud model.
That distinction matters to us: privacy becomes a technical property of the system rather than a rule written in an employee handbook.
How does the AI find the right information?
A language model should not be expected to remember project details or guess legal provisions from its training data.
Instead, our system first searches for the relevant information and then gives those sources to the language model together with the user’s question.
This approach is known as Retrieval-Augmented Generation, or RAG.
The language model is only part of the equation. The more important question is often whether the system found the right information before asking the model to answer.
That is why our retrieval system combines several different techniques.
Exact search meets semantic search
Traditional full-text search is very good at finding things such as “§ 650m BGB”, invoice numbers and specific technical terminology.
Semantic search solves a different problem. It can recognize that two differently worded sentences may be talking about the same thing.
Someone searching for “termination by the client”, for example, can still find relevant documents that use different legal terminology.
We run both types of search and combine their results.
Understanding what the user actually means
People rarely phrase questions the way a legal document does.
“The contractor isn’t showing up” may actually relate to legal concepts such as delay, deadlines or specific provisions of the German Civil Code.
The system therefore expands everyday language into relevant technical and legal terms before searching.
This makes it much more likely that the correct source will be found.
Checking the results again
The initial search produces a list of potentially relevant passages.
A separate reranking model then evaluates those passages against the original question and determines which ones are most relevant.
Only the strongest results are passed to the language model.
It sounds like a minor technical detail. In practice, it makes a substantial difference to the quality of the final answer.
Splitting documents where it makes sense
How documents are prepared also matters.
We do not split legislation into arbitrary blocks of 500 words or tokens. Whenever possible, a legal paragraph remains a complete unit.
It is a simple decision, but it prevents important legal context from being cut in half before the AI ever sees it.
The assistant is not really a chatbot
To the user, the system looks like a normal chat interface inside the project dashboard.
Behind that interface, however, it behaves more like a router.
The platform first determines what kind of question is being asked.
For a question such as “What is the amount of the fourth payment?”, the system queries structured project data directly. There is no reason to let a language model guess a number that already exists in a database.
For questions such as “What did we agree in the structural work change order?”, the system searches the project archive and uses the local model to summarize the relevant passages.
For legal questions, it searches the legal knowledge base and generates an answer based on the retrieved sources, with references to the underlying material.
And if a request could trigger an actual business action, such as approving a milestone or initiating an invoice, a chat response is not enough. A human must explicitly confirm the action.
This is intentional.
For these workflows, we prefer controlled and deterministic processes with carefully defined AI steps over highly autonomous agents.
When contracts, payments and project decisions are involved, traceability matters more than impressive autonomy.
Relevant answers can be traced back to their sources. Actions are recorded in an audit log.
Trust comes from testing, not from a good demo
AI demos are easy to make impressive. Reliable production systems are considerably harder.
That is why we evaluate retrieval and answer quality separately.
We use real questions from construction practice and compare the system’s responses against answers that have already been reviewed. We also ask the same question in different ways: the way a homeowner, site manager or lawyer might phrase it.
Different wording should still lead the system to the same relevant source.
New retrieval components are first tested alongside the existing system. They only become part of the production pipeline once we can demonstrate that they perform at least as reliably as the current setup.
There is another capability we consider particularly important:
The system needs to know when it does not have an answer.
If it cannot find a reliable source, it should say so. In construction law, a confident-sounding guess can be far more dangerous than an honest “I don’t have enough verified information to answer that.”
What we learned
Search quality often matters more than model size.
A smaller language model with the right context can produce much better answers than a larger model working with poor information.
AI should not be creative with money.
Financial amounts are stored and calculated as exact cent values. Payment schedules and invoices belong in deterministic software, not in probabilistic language-model calculations.
Local models are becoming genuinely useful.
A well-configured model running on dedicated GPU hardware can answer project questions, analyze documents, extract information from tables and process voice notes without sending sensitive project data to an external AI provider.
Document preparation is at least as important as the AI itself.
Before a document becomes searchable, a surprising amount of work has to happen. PDFs need to be parsed, tables detected, addresses normalized, duplicates removed and broken OCR text identified.
If the underlying data is poor, a better language model will not rescue the system.
Conclusion
An internal AI assistant for a construction company is ultimately much more than a chatbot.
It combines document processing, intelligent search, language models, data protection and controlled business workflows.
Getting an AI model to produce an answer is no longer the difficult part.
The difficult part is making sure it finds the right information, protects sensitive data, shows where its answers came from and knows when it should not answer at all.
That is the difference between an impressive AI demo and a system people can actually rely on in their daily work.
If you would like to explore what a platform like this could look like for your company, contact us at info@bergstone.de.