Small language models: a big idea for the public sector?
Under the surface of these solutions you'll find many have at least one thing in common; They are essentially user interfaces for large language models (LLMs) owned by OpenAI, Amazon, Google, or Meta.
We've seen a lot about scaling up the use of AI in the public sector recently, and this letter from the Department of Science, Innovation, and Technology states that the Government calculated their efficiency targets using the assumption that 100% of routine tasks could be automated.
Whether that's achievable is a question for another time, but even if half of that target was reached, that's 50% of UK public sector tasks essentially outsourced to US tech giants.
A Small Language Model (SLM) is a compact, efficient version of a LLM, designed to perform well with fewer computational resources.
Unlike LLM such as OpenAI's GPT-4 (with trillion-scale parameters), SLMs typically range from a few million to around 10 billion parameters, making them faster, cheaper, and easier to deploy especially for specialised tasks.
Most interaction with the public sector is for a single simple task; renewing a passport, ordering a new bin, making a doctor's appointment, and so on. I know this is true in local government having looked at councils' website analytics. There will always be users who need help from multiple teams or organisations, but that sort of requirement wouldn't be a routine task.
So here's my notion: small language models could save the public sector.
Lets take planning for example. Generative AI for planning doesn't need a LLM that's been trained on the collective works of Shakespeare, the laws of thermodynamics, and the synopsis of every episode of Friends, it just needs a SLM that's been trained on UK planning law, planning application formats, planning report formats and so on.
Using a LLM is like trying to do your weekly shop in a monster truck. Sure if you can make it road legal (or in AI terms, get it through a governance process) it's probably possible, but you'll take up much more space and use far more fuel that you actually need.
SLMs could be built, or commissioned, hosted, and owned by the UK Government, the NHS, or local government, therefore retaining control here, and could be updated quickly when UK legislation changes.
I would love to know if this approach is already being thought of, or even adopted in the UK, or anywhere else in the world, and as always, I welcome your comments below.