In the Solvency 360° platform built during my end-of-studies internship, a bank agent has to understand why a client received a given score before granting a checkbook, a BNPL credit or an aval. Confidentiality ruled out any cloud API, so every LLM call stays on a local Ollama instance running a small model. The goal was a readable explanation in natural language, with one non-negotiable rule: the model must never change a number.
It took four iterations to get there. In the first version the LLM rewrote the scoring factors freely, and it corrupted a figure: a ratio of 3030% became 30.3%. In the second, the LLM inserted placeholders into its own sentence, and a placeholder was treated as a bare number, which produced broken grammar. In the third, the sentence was built entirely in Python, with the source details copied verbatim and the LLM only choosing a connector from a closed list. It was reliable, but too flat and robotic.
The final version builds the body of the explanation in Python from figures computed by the scoring engine, and only asks the LLM for a short analytical conclusion. That conclusion is validated before being shown: no digits, no decision vocabulary, a length limit, and risk wording consistent with the real score tier. If it fails validation, a deterministic sentence takes over. The feature is also non-blocking: if the LLM or the scoring service is down, scoring and decisions work exactly as before. The lesson I kept is simple: use the LLM for tone, never for facts, and validate every output.