User satisfaction in 2024 is lower than it was in 2019 - what are we going to do about this?

Back in 2019 when I wrote my first Manifesto article, what prompted me to do so was an observation that over the previous few years, the Govmetric user satisfaction ratings for the top 10 councils month by month appeared to have been falling. I went on to look closely at the data and found that my gut feeling was in fact correct - the top council for January 2015 had an aggregate score of just over 0.4, and the top council for March 2019 had a score of just over 0.3, and Excel could plot a clear downward trend line between the two ends of the data set.

By way of reminder, the Govmetric satisfaction index works by using three faces on every page of a participating website - a happy face, a neutral face, and a sad face. There's some more complicated maths than the simplification which comes next has it, but the simplified explanation is if the user clicks a happy face it ends up in the index with a score of 1, if the user clicks a sad it's logged as a -1, and a neutral face scores a 0, and all those face-clicks are averaged into the overall index.

This consistent fall in satisfaction was despite the fact that by the beginning of that period the whole LocalGovDigital sector had pretty much pivoted to designing home pages and landing pages according to the Top Tasks Methodology, coined by Gerry McGovern and promoted by the annual SocITM Better Connected survey of council websites. The theory went that if you heavily promote the top 10 tasks for your website, as determined by your analytics, you'll make the site super-easy for the majority of your citizens to use. There were conflicting ideas about whether a task was purely about submitting a form - a transaction, in the lingo - or whether finding out information also constituted a task, but either way, that was the theory. 

My manifesto challenged that notion, on the basis of me considering Top Tasks to be a flawed application of the data - ie, just because the highest individual page views might be on the School Term Dates page and the Pay Council Tax form, it does not follow that most of the people coming to the site are doing so to check the school term dates or pay their council tax. And I used real Google Analytics data to demonstrate that point.

By 2019 another idea had taken hold — ‘Fix The Plumbing’. The meaning of that phrase was that it was an indulgence for us to look at nice things or innovation in our services if people had difficulty doing the basics, so we needed to drop the clever stuff entirely and just concentrate on those basics. Similarly I challenged Fix The Plumbing on the basis that if we limited our ambition only to making it easier to pay council tax online, and we can’t even get that right, then we really were done for.

A personal reflection

Before I go on with this article, it's worth an aside to explain the background on why I'm writing this stuff. Contrary to what a lot of people think - including, alarmingly, many people in the organisation - I've never actually been the person 'in charge of' my council's website. There was a period when I was quite influential in its development, though - I wrote our first content strategy, I led on two major site redesigns, and I initially led from the Webteam end on the procurement process to exit us off a much-hated content management system and onto the CMS that we'd originally chosen before we had the hated one foisted upon us. We weren't tracking satisfaction back then so it's impossible to know what our users thought of mine and my Webteam colleagues’ work, but it is worth noting that when we received complaints about the site, those complaints overwhelmingly were about an aspect of it we had no control over - the pages and forms related to the online account we had at the time - rather than our own work.

Unfortunately for me a restructure part way through that CMS procurement process took me away from the core of the Webteam, and successive restructures have taken me further and further away from it, and thus further and further away from a position of influence. So I wrote the 2019 Manifesto in the hope that somebody in some council somewhere would pick up the ideas shared, even if I wasn't able to steer them myself. Then Covid happened and nobody was doing anything other than mitigating the effects of the pandemic, and I started to feel more and more fraudulent writing articles about how to manage a council website when I wasn't actually managing one myself - so I put my blogging on hiatus.

Further time and restructures passed, and I came to a point where I thought no, I'm not going to have my ideas silenced, and I realised that I could build my own site in my own instance of LocalGovDrupal in which to start to demonstrate some of my ideas. So whence I picked up my blogging again, and whilst this site design right now is somewhat over-busy and over-coloured whilst I’ve been experimenting, and I know I need to tone it down a bit, I'm reasonably pleased with what I've rustled up with no training or previous background in Drupal.

Satisfaction by September 2024

That indulgence aside, the sector carried on without me to influence it. Whilst the literal interpretation of the Top Tasks Methodology doesn't hold the sway it had in the 2010s, its clutches can still be seen across most sites. We've seen another trend rise since 2019 - the influence of GDS thinking on local government.

Oh yes, and five years later we’re still talking about fixing the plumbing. That’s one mother of a fatberg we must be rocking.

So you think with all the good work which council digital teams up and down the land have been doing the last five years, that downward trend I noted between 2015 and 2019 would have taken a turn upwards, yes?

Oh.

Well at least we can say there's no continuing falling trend there; the best we can say is that the top three sites opened at around 0.2 in January 2023, and by August 2024 the 12 month average of the top three sites is around 0.2; we can also see that the overall average satisfaction of all participating council websites is around -0.1, peaks and troughs notwithstanding.

It's important to add a note of caution here that I was remiss in not noting in 2019 - this does not include all council websites, just the ones using the Govmetric smiley face satisfaction widget. But it is a broad spectrum data source and thus can reasonably be taken as representative, so if you know of any other satisfaction aggregators which are telling a different story then feel free to let me know.

So what we can see is that the users of the top three sites are less satisfied with them by 2024 than they were in 2019, and we can see that on average users are slightly dissatisfied with their council's website.

Are we pleased with that?

Or more to the point, were you aware of that before reading this article?

The self-serving explanation

Often over the years when I've had this discussion about satisfaction with colleagues and peers, folks have tried to explain away the fact that our users hate our work by saying 'oh, but people are used to using Facebook and Amazon and the BBC these days, so they're comparing our sites with those big megabudget sites, and we can't possibly be expected to compete with those'.

Which - and I say this with all due respect, srsly - is complete testicles. Amazon is the main comparator site I hear people refer to, which has one job - to sell you stuff - and it does that one job quite badly. You want to buy a spanner, you're overwhelmed with hundreds of identical-looking spanners for a wide range of prices and no means of choosing between them but for a wide variety of reviews that you can't necessarily trust because you don't know which reviews are written by paid shills and which reviews are written by people who don't know how to use a spanner properly, so you plump for one roughly in the middle of the price range.

Amazon's high overall level of customer satisfaction is not driven by its website, but by the overall customer experience, which they pursue like a bloodhound; once you've made your spanner choice the ease of then buying it, the speed it'll arrive through your letterbox (or be left in your designated safe space if you're not going to be in), and the ease of return if the spanner turns out to not meet your needs is what people like about Amazon. Whilst Amazon asks you why you’re returning a product as part of the feedback process, it’s essentially a no questions asked return process; Amazon takes the simple fact of you changing your mind as a valid reason to return the item and is content to take a profit hit on that if you come away from the overall experience with a positive view of it. Compare that with some of the hoops some shops make you jump through to actually return actually faulty goods.

With Amazon, users have a middling opinion about the website but they love the customer service. With local government, they hate how long it takes us to collect their missed bin collections, and they hate the hoops we make them jump through to report that the bins weren't collected.

We don't have the budget of Amazon to give everybody free doughnuts (real deep-fried sweet batter with a hole in the middle like what you get at the seaside, not the fake 'instant' doughnuts from the company that uses a K where standard English uses a C, thank you very much) when we don't collect their bins, but we can make it easier for people to report the missed collection, and it would take no budget at all to remove the page full of text before the button that basically says it's their fault we didn't collect.

How do we turn this around?

I have no easy answers to that question. In 2019 and earlier, and over the last couple of years when I started blogging again I've shared various ideas on things which I think would make for improved online services, but in the absence of them being fully tested, they're just that - ideas; ideas written by somebody whose influence doesn't match their ambition.

To be clear, now, as in then, I don’t consider top tasks to be completely irrelevant. If we’re delivering basic core services badly we should indeed fix them as a priority. And my antipathy towards GDS form design principles aside, the design thinking and promotion of user research which GDS has introduced is to be embraced.

I'm planning on writing a part two of this article after I've looked at some solid Govmetric data to see if there are any trends I can pick out which might be applicable across the whole sector, but the first step in solving a problem involves actually acknowledging the problem exists.

So what I'm asking for in the first instance is for leaders in the sector to take a long hard look at yourselves. Are you leading effectively? Are your ideas making an impact in making our online services better? Are you falling back on old, or even new, ideas which have been demonstrated to be ineffective in improving our services? Are you blaming outside forces for our collective poor performance? Are saying ‘we’re trying our best’ or are you genuinely acknowledging that our approach right now is still failing right now?

Or are you up for the challenge of turning this boat around to steer the user satisfaction for our online services to look more like the satisfaction for the telephone channel?