• 151

    Projects submitted

  • 109

    Published projects

  • 47

    Comments left

  • 66

    Number of votes

  • 52

    days until voting ends

← Back to list

LLM Document Library: natural-language search of audit evidence linked

  • Project manager on customer side

    Nthabiseng Mosupye

    Rand Water

    CTIO

  • Category

  • Nomination

  • Goals

    Rand Water's Digital Technology and Information Management team needed one place to find, track and question the audit record. Audit reports, management action plans, policies and closure evidence were held in separate repositories, so answering an audit committee question about the history or status of a finding meant searching by hand. The project set five objectives: track every audit finding, its owner, due date and status on one dashboard; build an indexed library of audit reports, working papers, action plans, policies and supporting evidence; let users ask questions in plain English and receive answers that cite the source document; reduce the time spent preparing audit and risk committee packs; and show that generative AI can run inside a regulated state-owned entity with access control, version control, audit trails and human sign-off in place from day one.

  • Timeline

    march, 2026 — june, 2026
  • Project completion year

    2026

  • Project scale

    2000 human-hours
  • Results

    Results

    Users find audit and governance information faster and spend less time searching across repositories. Answers are drawn from the approved version of each document, so different users get consistent responses. Related information is linked across documents, which makes repeat findings and overdue actions easier to see and gives the audit and risk committees source-backed evidence for their decisions.

    Next steps

    The same pattern will extend to compliance and risk, policy and procedure look-ups, management reporting and technical support across the utility.

  • Project uniqueness

    The solution joins two things that are usually kept apart: a structured tracker of audit findings and a language model that reads the documents behind them. A user opens a finding on the dashboard, goes straight to the report, action plan and evidence that support it, and then questions the full audit history in natural language. Typical questions are 'Which audit findings remain open beyond their due dates?' and 'Summarise all repeat findings over the last three years.' Answers come only from the organisation's own approved documents and cite the source, so an auditor can check each one in seconds. Governance controls form part of the design: role-based access that mirrors existing permissions, document version control and retention, activity logging, and a mandatory human check before any AI-generated output enters a committee paper. The controls map to the King IV governance code, POPIA and internal ICT security and records management policies.

  • Software used

    Microsoft Power BI: Audit Tracker dashboards for status, ageing, ownership and overdue actions.

    Large language model with retrieval-augmented generation (RAG): answers questions using only retrieved document content, with source references.

    Indexed document repository: stores audit reports, working papers, combined assurance reports, action plans, policies and procedures with metadata (document type, owner, date, version).

    Microsoft SharePoint: hosts the document library, with user access controlled through central identity and access management (IAM).

    Security and governance services: role-based access control, activity logging and audit trail, document version control and retention.

    End-user access: standard corporate workstations through a web interface; no additional equipment was required.

  • Solution from the Global CIO catalog

    The project does not use solutions from the Global CIO catalog

  • Project implementation complexity

    Most of the effort went into the content. Audit reports, working papers, action plans and policies came in mixed formats, with inconsistent naming and several versions of the same document. Each item had to be cleaned, tagged with an owner, date and version, and indexed, or the model would return out-of-date answers. Findings in the tracker then had to be linked to the right source documents so users could move between the dashboard and the evidence. Audit material is sensitive, so retrieval had to respect existing permissions: a user can only retrieve what they are already entitled to see. Every answer had to be traceable to a source so reviewers could confirm it, and no AI-generated summary may enter a committee paper without human verification. All of this had to satisfy POPIA, King IV and internal ICT security and records policies, inside a public-sector governance and procurement environment.

  • Description

    Context

    Rand Water is a state-owned bulk water utility. Internal audit, external auditors and other assurance providers raise findings every year, and each finding produces an action plan, an owner, a due date and evidence of closure. Over several years this built up a large body of reports, working papers, combined assurance reports, action plans and policies held in different places. Management could see a list of findings, but tracing a finding to its history and evidence took manual effort, and repeat findings were hard to spot.

    What was built

    Rand Water's DT/IT Applications team built the solution in-house, with no external supplier, between March and June 2026. It has two connected parts. The Audit Tracker, built in Power BI, is a structured register of audit findings with dashboards for status, ageing, ownership and overdue actions. The LLM Document Library is a central, indexed repository of audit reports, working papers, combined assurance reports, audit action plans, policies and procedures. On ingestion each document is processed and stored with its document type, owner, date and version. A large language model queries the library through retrieval-augmented generation (RAG).

    How it works

    A user asks a question in plain English. The system searches the library, retrieves the relevant sections with their metadata and source references, and the model writes an answer based only on that content. Each answer lists the documents it used so the user can open and check the source. From the tracker, a user can select a finding, open its supporting evidence and continue the enquiry in natural language. Example questions: 'Which audit findings remain open beyond their due dates?', 'Summarise all repeat findings over the last three years', 'Show all findings related to supply chain management from the last three years' and 'What does the ICT Security Policy say about password management?'

    Governance and security

    The design applies King IV, the Protection of Personal Information Act (POPIA) and Rand Water's ICT security and records management policies. Controls include role-based access, privacy and confidentiality controls, an audit trail of user activity, document version control and retention rules. Content stays in Rand Water's environment and is not used to train the model. A person must verify every AI-generated output before it is used in a report or committee paper.

  • Project geography

    South Africa. Delivered at Rand Water head office, Johannesburg, Gauteng, for use by the utility's DTIM team for audit, risk and governance functions.

  • Additional presentations

Comment 1

Only authorized users can comment.
We suggest you or register.

  • Lebohang Moloi

    Lebohang Moloi

    Rand Water

    Manager Project and Portfolio Management

    From a governance and PMO perspective, this is an excellent initiative. Audit evidence is often distributed across large volumes of documentation, making retrieval time-consuming and dependent on individuals knowing exactly where information is stored. The LLM Document Library changed the narrative by allowing users to search evidence using natural language, strengthening accessibility, institutional knowledge, audit preparedness, and operational efficiency. This initiative deserves my vote.
    Reply
  • Customer

    Rand Water

We use cookies for analytical purposes and to deliver you the best experience with our website. Continuing to the site, you agree to the Cookie Policy.