For many people, six months into a new job is about the time you start to feel fully on top of things. You’ve figured out how the organisation works and your place in it. You’ve met all of your colleagues and got to know your way around the office. The job makes sense, everything’s under control. But then, a pandemic hits! What do you do? What is going through your head?
That’s a question we put to Professor Sir Ian Diamond, UK national statistician, who was only six months into the job when Covid-19 upended everything. He said: “My overall sense at all times was one of, ‘What needs to be done? What role can we play in helping to do it? And how do we make sure that we are doing things at pace?’”
There was no “flapping,” he said, but there was a real risk of exhaustion. “I could see pretty quickly that this was going to be a marathon, not a sprint, and while we had a lot to do,” he explained, “the last thing on earth we needed was for people to start burning out.”
Almost four years have passed since that time, but the effects of the pandemic continue to be felt – not least within the Office for National Statistics (ONS), the organisation Sir Ian leads. The Covid experience helped shape his thinking about how the ONS would operate post-pandemic, as he explains in this interview.
When we spoke with Sir Ian, he was six months into a second term as national statistician. By the end of that term, in 2028, what kind of organisation will the ONS be? Read on to find out.
What was your experience of the Covid pandemic, being only six months into the role of national statistician at the time?
It was all-consuming and required an enormous amount of focus. At the beginning of the pandemic, huge amounts of data were flying in every different direction. I felt we were in a data deluge, and we needed to move to [delivering] insight, and really working hard early on to change the agenda towards a situation where we were asking questions – really serious and sensible questions – and working out if we had the data, or how we answered those questions.
On the whole, ONS and the Government Statistical Service were praised for the way they responded to Covid. Did the pandemic experience inform your thinking about how the statistical system should operate once we moved out of that crisis situation?
Yes, in a number of areas. One was that we should not be completely dependent on data collected traditionally. For example, as we went into the pandemic, our ways of calculating inflation were pretty much dependent on people with clipboards going into supermarkets and shops and writing down the prices of things. We already had a project which was starting to think long term of using scanner data. But actually, being able to pivot very quickly to using web scraping to get data was incredibly important. We were also able to use web scraping early on to understand the availability of various goods in what we might call “adaptive purchasing,” or some would call “stockpiling.” And so, identifying that there were new ways of doing things and new data sources, I was pretty clear in my mind that we were into a no-going-back situation.
The second thing we demonstrated was that we could set things up very agilely and very quickly. And one final thing that I thought we absolutely have to continue with all the time is improved communication. You may recall that there were press conferences every day [during the early part of the pandemic], and I think during the start of those press conferences, the graphs and the slides were not always as brilliant as I would want them to be. We embedded a team into the Government Communication Service to work on the slides, and I thought that team did a great job. Improving the communication of statistics was incredibly important, because one of the things to come out of this dreadful pandemic was that people across the country became more data literate, and more demanding of data, and more able to interpret data. That was a good thing which I wanted to make sure we continued.
In terms of embedding the lessons or the learnings of the pandemic into the ONS going forward, how much of it is culture change? How much is about rethinking the systems and the processes?
Was it culture? Was it improved processes? Was it better methods? All of the above. As an organisation, our main role in life is to measure the economy and society, and if you take that as your starting point, and then you ask the question, “In your lifetime, has the economy ever stood still? Has society ever stood still?” In my lifetime, I would argue, no. Therefore, we have to be an organisation which is constantly changing in order to reflect what is going on in the economy and in society. We have to change how we do things, and to ask questions about whether there are better ways of doing things, and that, I think, has been a really important reflection for us over the period both during and since the pandemic. We’ve learned a lot about the use of, for example, reproducible analytic pipelines to really improve the quality of our data at large, to improve the quality of our processes, and to enable us to do things more efficiently and effectively. We’ve learned a lot about new data sources, and we’ve really built on the opportunities and the skills so that you can now link data to be able to address questions that I could only have dreamt of 20 years ago.
And so, I do think we have changed the culture, changed our techniques, and changed our data. But does that mean that we’ve metaphorically thrown away the baby with the bathwater? Absolutely not. What we have now are appropriate methodologies to answer appropriate questions. Do we still use qualitative data? A hundred percent, when it is necessary to do so. Do we still use surveys? Yes, we’ve got some of the best surveys in the world. But equally, we also use digital data, administrative data, and we use very modern techniques of analysing those data. And we use data science in its broadest sense as often as we can.
So, I do think it’s been a major change in what we do, and that will continue. But underlying it all is a total commitment to quality, a total commitment to making sure that we have the best data to answer the question that we are trying to answer, and that all the time we are using the best approach to answer the questions.
We have changed the culture, our techniques, and our data. What we have now are appropriate methodologies to answer appropriate questions. Do we still use qualitative data? A hundred percent, when it is necessary to do so. Do we still use surveys? Yes, we’ve got some of the best surveys in the world. But equally, we also use digital data, administrative data, and we use very modern techniques of analysing those data.
As well as changing the culture and the processes within the ONS itself, has some of your work also been about trying to bring the user community with you? When I first started reporting on official statistics 20 years ago, there was a sense that users of the data valued consistency in methodology, because that meant they could go back and look at the time series. But now, with this emphasis on innovation and looking at different ways of producing insight from different sources of data, has there been a tension, if you like, between these two cultures?
That whole question of “we’ve always done it this way” against “we can now do it better” is a super important one. And the length of time series is also important. When we change the ways of doing things, we need to take our user community with us, and we do. At the same time, we also need a very strong narrative (a) about why what we are doing gives us better data, and (b) about what the changes in the time series mean. So, just this year, with regard to prices and inflation, we’ve been able to bring in much better data than we had previously on rail ticket prices by using electronic data. It’s really super exciting. But we didn’t just bring them in and say, “Hey, we’ve got this new way of doing train prices and we’re planning to do the same again next year with used car prices using electronic data!” What we do is we dual run, and we work with our prices advisory committee to ensure that we understand what the implications of this change are, and we understand how to communicate them. But, you’ve got to be measuring the economy in the very, very best way that you can. We should not shy away from improving what we do.
To what extent can the changes in thinking, the changes of approach, be credited to the experimentation and innovation work that is coming out of the ONS Data Science Campus?
The Data Science Campus has been absolutely brilliant. But at the same time, innovation does not only take place in the Data Science Campus. What we’ve built is a culture of innovation right across the organisation. Is that culture of innovation driven by the Data Science Campus? Not so much driven, but certainly helped, and certainly in partnership, and the fact that it is there encourages that culture of innovation.
I do think it is important to recognise, as I say to my colleagues many times, that we are not a blue-sky research institute, we are a national statistics institute, and our job is to produce economic and social statistics. Therefore, we need to be in the business of not just research but research and development – and thinking through how the research on new data that we do will enable improved economic measurement is, for us, incredibly exciting.
I’ll give you an example. We’ve [recently] signed a contract to get telephony data – a few years historically, and then regular data going forward. Now, this is entirely anonymised, but it will enable us to understand much more about, for example, commuting. It means that we will now need to do research on how to use those mobility data, and we’ll be really pushing that forward very quickly. But, at the same time, it’s not just about what can we do that’s interesting in this area; we need to have a very clear vision of what success looks like and the measurements, the economic measurements, that we are going to improve.
How do you see that innovation mindset rolling out across government as a whole?
It’s worth saying two things. Firstly, my job is not only national statistician, I also have an extra couple of hats: one is head of the Government Analysis Function, and one is head of the Government Digital Service. I take those roles very seriously because I do think we need to propagate good practice and innovation right across government departments. It’s no use if it just sits in ONS.
We try really, really hard to have innovation meetings and innovation months, and I try to speak at as many as I’m invited to. And I think it is incredibly important that we really see ourselves – right across the Government Statistical Service, right across the Government Analysis Function – as seeking to propagate good practice.
You mentioned there about the Government Statistical Service and the Government Analysis Function. Is there scope one day for a Data Science Service within government?
The answer is yes. Under both the leadership of Laura Gilbert, who is head of 10 Downing Street’s data science, called 10DS, and Osama Rahman, who heads the Data Science Campus, we recently held a town hall for data scientists right across government to discuss, fundamentally, the question of what data scientists want from the analysis function, but equally [what they want] from a community of data scientists. Part of that is a question as to whether there should be, in government, a data science profession. I stress we haven’t come to the conclusion for that yet, but I would have to say it was a very successful town hall – many, many people attended, we had a really good discussion, and Laura and Osama will be taking forward that discussion over the next couple of months.
You gave a keynote address at the RSS Conference in September, and one of the things you mentioned that I was particularly excited about was “Stat Chat.” I understand this is in the very early stages, and I have a very rudimentary understanding that it might well be a large language model trained on the ONS website and the data resources available, as a way to query the website. Can you tell me a bit more about the project?
We’re in private beta, and so there’s a whole set of agendas there. But we started it as a much better way to enable people to interrogate a pretty complex website with an enormous amount of data on, and to not only get to the datasets but to get to the existing metadata that are attached to them. What we wanted to do was to use open-source models, where the underlying data and the research behind them are made publicly available, so there’s nothing secret about this at the moment, and it’s very early stage. But we see the potential, as we move forward, as being able to really make it much easier for people to interrogate the data that we own and the data that we have published. If we can actually use large language models to enable people to be able to ask questions and then to get an authoritative answer from publicly available data, that seems to me to be a good place to be.
I imagine, though, that when you’re dealing with something like national statistics, you need to be very alive to the danger of the hallucination problem in large language models; that if you’re querying something, it doesn’t throw up an invented statistic?
I couldn’t agree more. And that’s why we’re in private beta, working very hard to make sure that (a) it is working properly, (b) that it’s got the right security around it, and (c) that it is actually really useful.
The hallucination issue in LLMs leads us onto the topic of trust in information, and obviously the ONS is very keen to ensure that there is trust in official statistics and in the data that is produced. This is a wider problem than anything the ONS can hope to address by itself, but what are the kinds of conversations you’re having internally about distrust in official sources of information?
This is something I say to my colleagues a lot: We should not expect people to trust us. We have to demonstrate to people that we are trustworthy. That’s incredibly important. A lot of it is about transparency. A lot of it is about absolute openness, showing your working, explaining where your data came from, and explaining your motivation for doing something. People say to me, “You might write a really strong methodological piece, but not many people read it.” Yes, but it’s there. And I’m a huge believer in research integrity, and in open data, and enabling data to be available for secondary analysis. And I think the more you are transparent, the more you work with people, the better.
A critical part, also, of demonstrating that you are trustworthy is engaging with the public. And that’s not telling the public; it’s engaging with the public. We put a lot of time into working with the public to say, “Well, what if we did this? What if we did that?” and getting their input. I don’t have any kind of switch to make people feel that we are trustworthy. It’s a continuous process of transparency and openness, where people feel that they have everything they need [to know] about what we do and about our data.
We also are absolutely passionate about explaining uncertainty. Don’t tell me the answer is 62% – the answer has some uncertainty about it, and we need to really think about how we display that uncertainty. And I have to say, I think some of the techniques now to display uncertainty are just so beautiful – unbelievably beautiful – and we need to do that, not in a gimmicky way, but in a way that really explains the uncertainty in any data that we present.
Final question: by the end of your second term as national statistician, where do you think ONS will be as an organisation?
I hope it will be an innovative, agile organisation which is using evermore diverse types of information, but doing so in a transparent and open and rigorous way to improve economic and social statistics which can impact positively on the lives of our fellow citizens.
- Copyright and licence
- © 2023 Royal Statistical Society
This article is licensed under a Creative Commons Attribution 4.0 (CC BY 4.0) International licence. Photo of Professor Sir Ian Diamond is not included in this licence.
- How to cite
- Tarran, Brian. 2023. “‘I was pretty clear in my mind that we were into a no-going-back situation.’” Real World Data Science, December 15, 2023. URL