AI for public good: the future is brighter with RAG

Using the RAG approach could deliver a major boost to government ambitions to adopt AI effectively, argues writes Peter Dutton at Elastic

In the UK government’s drive to improve public services, high hopes are pinned on the potential of artificial intelligence (AI) technologies to deliver meaningful change.

The AI Opportunities Action Plan, announced in late July 2024, is a good example of how policymakers are thinking about the topic. According to Chancellor of the Exchequer Rachel Reeves, a key focus of this plan is identifying how AI technology can be used to “deliver the public services that people deserve whilst saving taxpayers money.” 

This is by no means the UK government’s first foray into AI, but it does reflect a growing confidence around the potential efficiencies and cost savings that AI is expected to deliver. As the previous government identified in the Autumn Statement 2023, “the potential productivity benefits from applying AI to routine tasks across the public sector are estimated to be worth billions.” 

To achieve its ambitions, the government faces the tricky task of maximising AI’s benefits while mitigating its risks. Concerns around data security and privacy are the most common hindrance to AI adoption in the public sector. Public sector officials are responsible for guaranteeing that sensitive data remains protected and are understandably reluctant to implement generative AI without assurances of data privacy and responsible use. 

Meanwhile, a March 2024 report from the National Audit Office points to other barriers, including limited access to good quality data and the need to modernise legacy systems. As the report’s authors state: “Updating legacy systems and improving data quality and access is fundamental to exploiting AI opportunities but will take time to implement.”

Overcoming AI implementation barriers

But what if there was a way to overcome these barriers, speed up AI adoption and enable government agencies and the people they serve to experience the positive impact of AI sooner? I believe that there is—and that it lies in an AI approach known as retrieval augmented generation, or RAG.  

Put simply, RAG is a technique for enriching the output of large learning models (LLMs), the AI programmes trained on vast data sets to understand questions, contextualise responses and generate accurate output.

The problem with off-the-shelf LLMs is that they are typically trained on public data sets that age quickly and may lack context from a specific organisation’s own private information sources. This can result in out-of-date, inaccurate or irrelevant responses. 

What RAG brings to this picture is the ability to enable an LLM to reference authoritative information sources that lie beyond its ‘official’ training data – from a trusted third-party provider, for example, or an organisation’s own data stores. The result is AI output that is more up-to-date, accurate and relevant. 

More specifically, RAG offers solid answers to the misgivings that many public officials have about AI. For example, data security and privacy: RAG can maintain data and security safeguards that prevent unintended data exposure. Users can implement role-based security to control access to data within a data stream and restrict security permissions to particular documents. 

In addition, RAG supports the attribution of data sources, providing an important audit trail that explains how an AI response was generated and including citations or references to the sources used along the way. In terms of data and decision-making transparency, this will be a big deal to both government officials and citizens wary of falling victim to AI ‘hallucinations’. 

Let’s now look at limited access to good-quality data and the impact of legacy systems. These become less of an issue when RAG is used because, to fortify LLM responses, RAG can access a wide range of relevant external resources, such as real-time dynamic information drawn from the web or data that provides vital organisational context. Of course, this can include legacy systems.

A data mesh approach

In effect, I’m describing a ‘data mesh’ approach that enables public sector organisations to get more utility and insights from the data they already have, regardless of the format or system in which it is stored. Within a data mesh, RAG essentially serves as a context layer, sending only the most relevant data to an AI application, generating more trustworthy and traceable responses. 

In my mind, this could be what it takes to win the confidence of UK public sector bodies; but the financial advantages are pretty convincing, too. Using RAG is far more cost-effective than constantly training and retraining LLMs, requiring fewer computing and storage resources but leveraging broadly the same skills. Training LLMs is a resource-intensive and time-consuming process, especially when you consider the volume of new business data and public information generated each day.

The scope of the opportunity is enormous, and there is no time to waste. Research published by the Alan Turing Institute makes that abundantly clear: its researchers estimated that around 84% of repetitive transactions across 200 government services could be easily automated by AI, resulting in services that are more “responsive, efficient and fair” for citizens and that cost less money for the government to run.

I firmly hope that the AI Opportunities Action Plan—the findings of which will be presented to Science Secretary Peter Kyle in September—is equally successful in uncovering new opportunities to give public sector services a much-needed makeover. 

Peter Dutton is AVP, Head of Public Sector, EMEA, at Elastic

Main image courtesy of iStockPhoto.com and Thinkhubstudio