Watch the discussion

Takeaways at a glance

• AI is built on statistical thinking – even when it’s not labelled that way.
• Job titles change; core skills don’t.
• Statistics sometimes undersells itself by focusing on mechanics over impact.
• Communication and visualisation are central, not peripheral, to modern statistical work.
• Kindness, collaboration, and trust are professional assets.
• The future belongs to skill-based identities, not title-based ones.

Key themes and analysis

The “rebranding” meme

The panel opens with a familiar joke: take statistics, put a new frame around it, call it machine learning or AI, and suddenly everyone pays attention. It’s humorous—but revealing.

Many roles advertised today as “AI” or “data science” positions are deeply statistical at heart. They involve modelling uncertainty, validating assumptions, managing bias, evaluating performance, and interpreting results. In other words: core statistical competencies.

Rather than resisting this relabelling, the panel suggests recognising it as part of the natural evolution of fields. The key question becomes not “What should we call ourselves?” but “What value are we delivering?”

Identity versus skills

One of the strongest messages from the discussion is this: don’t over-identify with a job title.

“Statistician,” “data scientist,” “AI specialist” are all potentially transient labels, whereas the skills underpinning them remain the same:

• Framing problems carefully
• Questioning assumptions (“Are you sure? Are you sure-sure-sure?”)
• Quantifying uncertainty
• Designing analyses that are robust and defensible

The panel suggests that the healthiest professional stance is to focus less on identity and more on what you can do and what you care about.

The communication gap: loving the sausage-making

Statisticians, the panel observes, sometimes make things harder than they need to be—at least in how they explain their work.

“We’re too interested in the mechanics,” one panellist notes. “Nobody cares how you made the sausage.”

This doesn’t mean rigour is unimportant. It means that impact must lead the narrative. Instead of focusing first on models, methods, and diagnostics, statisticians might begin with:

• What problem was solved?
• How did this make life easier, safer, or better?
• What decision did this enable?

AI has been marketed effectively because it is framed in terms of transformation and possibility. Statistics can claim that space too, without sacrificing integrity.

Visualisation and bringing data to life

Visualisation is a key bridge between statistical thinking and real-world impact. Good visualisation:

• Makes uncertainty legible
• Builds trust
• Enables decision-making
• Tells stories grounded in evidence

In a world flooded with dashboards and generative outputs, the ability to present data clearly and responsibly is not a soft skill. It is core infrastructure.

Trust, collaboration, and professional culture

People want to work with statisticians they trust, which flows not only from technical competence but from clarity, openness, and collaboration.

As AI systems become more powerful—and more controversial—the professionals who can explain, contextualise, and responsibly deploy them will be in

From background discipline to visible foundation

If AI continues to evolve—as it surely will—so too will the labels attached to those who work in it. But uncertainty, inference, modelling, and critical thinking aren’t going anywhere.

We would love to receive contributions to the site that tackle this issue.

Is statistics undervalued in the AI conversation, or quietly thriving?

Where, in your experience, does statistical thinking most visibly shape AI work?

And where is it least acknowledged?

Have you seen statistical work rebranded as AI in your organisation?

We are actively seeking submissions on these topics so, if you would like to be part of the conversation, get in touch.

Watch the discussion

Takeaways at a glance

• Innovation and regulation are not opposites – both are essential, but difficult to balance.
• Responsible progress requires proportionality – not all AI applications carry the same level of risk.
• Transparency enables better governance – open dialogue between developers and regulators is key.
• Risk-based frameworks provide structure – distinguishing low-, high-, and unacceptable-risk uses helps focus oversight.
• Global disparities complicate regulation – some regions are regulating advanced AI systems, while others are still building foundational capacity.
• Innovation needs protected space – experimentation, iteration, and even failure are critical before formal standardisation.

Key themes and analysis

Proportional regulation through risk

Not all AI systems pose the same level of harm. A risk-based approach — distinguishing low-, high-, and unacceptable-risk uses — offers a practical middle ground. It avoids blanket restrictions while ensuring stronger oversight where impact is greatest. The debate becomes less about whether to regulate, and more about how proportionate that regulation should be.

Transparency as common ground

Openness can bridge the gap between technologists and regulators. Clear communication about capabilities, limitations and risks enables more informed policy decisions. When innovation happens transparently and in dialogue with regulators, governance can evolve alongside technology rather than lagging behind it.

The global unevenness of AI governance

AI regulation is developing unevenly across regions. While parts of the West are formalising frameworks, many countries are still building foundational AI capacity. This raises difficult questions about sequencing: should regulation lead innovation, or follow it? A one-size-fits-all model may not reflect global realities.

Protecting space to experiment

Innovation requires room to test, iterate and occasionally fail. Early experimentation should not be overburdened with rigid controls — but successful, scalable systems must eventually transition into more standardised and regulated environments. The challenge is designing pathways that support both creativity and accountability.

Looking ahead

As AI continues to evolve, the balance between innovation and regulation will remain dynamic — and contested. This conversation opens up important questions, and we would love to hear our readers’ thoughts about how we move some of the principles mentioned in the video into practice.

• How do we facilitate transparent channels of communication between those developing AI and those designing the regulatory frameworks that will govern it?
• What should determine whether an AI system is low, high, or unacceptable risk?
• How do we define a “safe speed” for AI development — and who gets to decide?

We are actively seeking submissions on these topics so, if you would like to be part of the conversation, get in touch.

From Governance Principles to Technical Work

Colorado’s law followed a pattern increasingly visible in global AI governance, particularly the European Union’s AI Act. These frameworks share a risk-based logic in that systems that influence consequential decisions face higher expectations for transparency, oversight, and accountability.

At a high level, these expectations (fairness, consumer protection, responsible use) sound abstract. In practice, they translate directly into technical work:

• Clear documentation of model purpose, training data, and limitations
• Records showing where data comes from and how it changes over time
• Reproducible experiments and versioned artifacts
• Ongoing monitoring for performance drift and unintended impacts
• Defined processes for human review and intervention

None of this lives in legislation. It lives in scripts, workflows, dashboards, deployment systems, and operational infrastructure.

Colorado’s stalled implementation of AI policy surfaced a familiar pattern, as many organizations are well equipped to optimize model performance, but far less prepared to operationalize accountability at scale. The friction emerged not because governance goals were controversial, but because the supporting technical infrastructure was uneven.

Why Uncertainty Becomes a Design Risk

One challenge Colorado encountered was definitional ambiguity. For example, what qualifies as “high risk”, what safeguards are sufficient, and how should harms should be assessed? These questions are not merely legal - they are technical and context dependent.

Different data sources, deployment approaches, and users lead to different answers. For teams building data systems today, that uncertainty creates risk. When teams cannot easily see how data moves through a system, how models change over time, or how decisions are produced, adapting later becomes costly and disruptive.

Recent federal signals add another layer of complexity. President Trump’s executive order discouraging state-level AI regulation aims to reduce fragmented policy on AI, but it does not replace state experimentation with a concrete national policy. Teams now operate in a moving landscape shaped by state initiatives, evolving federal priorities, and international regimes like the EU AI Act. In this environment, aiming for minimal compliance is risky. Teams are better served by designing systems that are flexible and easy to observe from the start.

Responsibility Does Not End at Deployment

A lesson emerging from both policy debates and practice is that accountability does not stop when a model goes live. Responsibility shifts across teams over time, from data scientists to engineers, product owners, operators, and decision-makers.

This challenge is the focus of the Responsible Handover of AI framework developed by Sense about Science, which emphasizes the need for clear transitions of responsibility as AI systems move from development into real-world use. Rather than treating deployment as a handoff to “the business”, the framework highlights the risks that arise when assumptions, limitations, and responsibilities are not carried forward with the system.

For practitioners, this framing maps governance concerns onto familiar operational questions, such as who monitors systems after deployment, which development assumptions still matter in production, how limitations are communicated to users, and what happens when systems are updated or handed over to new teams.

Without explicit handover practices, accountability gaps emerge because responsibility becomes diffuse as systems evolve. From this perspective, many regulatory requirements are not adding entirely new work, rather they formalize practices teams already rely on. This includes documentation that travels with systems, monitoring in production, and clear escalation paths when something goes wrong.

Practical Steps Teams Can Take Now

Regardless of how U.S. and international regulation ultimately settles, many investments pay off immediately while reducing future risk, including:

• Standardizing documentation. Ensure model summaries and data descriptions travel with systems as they move between teams
• Build end-to-end visibility. Version datasets, features, models, and configurations so results can be reproduced
• Instrument monitoring early. Track input drift, unstable predictions, performance decay, and downstream impacts once systems are in production
• Clarify governance workflows. Define who approves releases, who monitors systems, and how responsibility shifts over time
• Translate risk for leadership. Gaps in documentation and visibility tend to come back later as messy, expensive fixes; addressing them early saves time and pain

These practices are not limited to machine learning. Any system that informs decisions can create similar accountability challenges.

Governance Lives in the Data Stack

There’s still no settled agreement on how AI should be governed. But for people building real-world data systems, its implications are already concrete. Accountability increasingly lives in the data stack in how workflows are instrumented, how models are monitored, and how decisions can be examined after the fact.

This is not simply about regulatory compliance. It is about building systems that are transparent, resilient, and trustworthy at scale. Organizations that treat governance as a core technical problem (rather than an external policy constraint imposed later) will be best positioned to navigate whatever regulatory balance ultimately emerges.

Video: What Are the Key Challenges Facing Data Scientists Today?

The Patterns Behind the Problems

Although the challenges raised span technical, organisational, and personal domains, they are connected by a small number of deeper themes that shape modern data science.

Looking Ahead

Taken together, these themes suggest that the biggest challenges in data science are not isolated problems to be solved individually. They are interconnected tensions between speed and rigour, access and expertise, innovation and organisational inertia.

Addressing them requires interdisciplinary, systems-level thinking.

Which of these challenges resonates most with your own experience in data science? How can practitioners use these tensions as inflection points to actively shape the field, rather than simply react to it?

Our Audience:

People working in data science looking for practical insights, methodological rigour and thought-leadership that informs their work and decision-making.

Our Voice:

Authoritative, Trustworthy, Cutting Edge

Our Editorial Sections:

Real World Data Science has four editorial sections. Please read through and consider where your piece would fit best. Each piece we publish needs to be tailored towards the focus of one of these sections.

THE PULSE

News, updates and real time commentary.

Purpose: To respond to current events, trends and debates in the data science world with rigour, insight and relevance.

Content Types: Articles that speak directly to current events/trends/launches

Example Call To Action: Invite readers to share your commentary with their networks as a trusted voice in the space. Invite engagement, discussion and debate over the topics.

APPLIED INSIGHTS
How data science is used to solve real-world problems in business, public policy and beyond.

Purpose: To showcase real-world applications of data science, including hands-on tutorials, project walk-throughs, and case studies from industry, academia, or public service.

Content Types:

• High-quality step-by-step tutorials with code
  
• Case studies detailing a problem, approach, and outcome
  
• Lessons learned from real-world deployments

Example Call To Action: Readers should walk away with something to try.

FOUNDATIONS & FRONTIERS
The ideas behind the impact: the concepts, tools and methods that make data science possible.

Purpose: To deepen understanding of the theoretical and ethical foundations of data science, and to spotlight thought leadership and emerging ideas.

Content Types:

• Think-piece style articles with an engaging angle on methodology, ethics and standards
  
• Interviews with thought-leaders
  
• Data Science Bites - our handy summaries/explainers of academic papers

Example Call To Action: Invite discussion and engagement – pose questions and challenges to the reader.

PEOPLE & PATHS
Strategic reflections on careers, leadership and professional evolution in data science.

Purpose: To explore the evolving nature of data science careers through the lens of experience, leadership, and long-term impact. This section highlights how professionals shape and are shaped by the field—through roles, decisions, and philosophies.

Content Types:

• Profiles of/interviews with senior professionals reflecting on career philosophy and leadership
  
• Roundtables with experts on hiring, mentoring, or organisational design
• Commentary on career-defining trends, such as the rise of AI governance or the shift toward interdisciplinary teams

Example Call To Action: Encourage readers to share our strategic insights with their community.

Use of AI in Submissions

We recognise that LLMs and other generative AI tools are increasingly part of the data science workflow, from code generation and data cleaning to drafting documentation and shaping analysis. We welcome a transparent approach in submissions that have made use of these tools, and ask that authors include a declaration outlining where and how AI was used in the development of their submission. This helps us maintain transparency, uphold standards of reproducibility, and better understand the evolving role of AI in real-world data science practice.

To make your submission, please review our contributor guidelines and email us at rwds@rss.org.uk

Meet the journal’s editors-in-chief and editorial board

View the full editorial board here: Editorial Board | RSS Data Science | Oxford Academic (oup.com)

Consultation response

News of the funding for regulators came as part of a long-awaited response by the government to the consultation on its AI regulation white paper. The response essentially confirmed that the government was proceeding with its principles- and context-based approach to regulating AI, having received “strong support from stakeholders across society.”

This approach is right for today, the government said, “as it allows us to keep pace with rapid and uncertain advances in AI.” However, it acknowledged that “the challenges posed by AI technologies will ultimately require legislative action in every country once understanding of risk has matured.”

“Highly capable general-purpose AI systems” would, for example, present a particular challenge to the government’s current approach. It explained: “Even though some regulators can enforce existing laws against the developers of the most capable general-purpose systems within their current remits, the wide range of potential uses means that general-purpose systems do not currently fit neatly within the remit of any one regulator, potentially leaving risks without effective mitigations.”

As a next step in delivering on the white paper approach, the government is asking key regulators to publish an update on their strategic approach to AI by the end of April. This was welcomed by Royal Statistical Society (RSS) president Andrew Garrett, who said:

“Urgency is certainly warranted, and the directive for key regulators to disclose their approach in the coming months is a positive development. Ensuring consistency and coherence not only among key regulators but also those who follow is crucial.”

Garrett also reiterated the need for government to engage with statisticians and data scientists, particularly through its new AI Safety Institute (AISI). In the white paper consultation response, AISI is billed as being “fundamental to informing the UK’s regulatory framework”: it will “advance the world’s knowledge of AI safety by carefully examining, evaluating, and testing new frontier AI systems” and will also “research new techniques for understanding and mitigating AI risk.” Garrett said:

“As always, fostering diversity of representation within government and regulatory bodies remains paramount; it cannot solely rely on input from major tech companies. It is especially important that the AI Safety Institute engages with a diverse array of voices, including statisticians and data scientists who play a pivotal role in both the development of AI systems and novel evaluation methodologies.”

Risks and opportunities

Calls for a “diversity of representation within government and regulatory bodies” certainly chime with a warning bell sounded by the Lords Communications and Digital Committee last week, in the February 2 release of its inquiry report into large language models and generative AI. “Regulatory capture” by big commercial interests was highlighted as a danger to be avoided, amid concern that “the AI safety debate is being dominated by views narrowly focused on catastrophic risk, often coming from those who developed such models in the first place” and that “this distracts from more immediate issues like copyright infringement, bias and reliability.”¹

The committee called for enhanced governance and transparency measures in DSIT and AISI to guard against regulatory capture, and for a rebalancing away from a “narrow focus on high-stakes AI safety” toward a “more positive vision for the opportunities [of AI] and a more deliberate focus on near-term risks” including cyber security and disinformation.

It also wants to see greater action by the government in support of copyright. “Some tech firms are using copyrighted material without permission, reaping vast financial rewards,” reads the report. “The legalities of this are complex but the principles remain clear. The point of copyright is to reward creators for their efforts, prevent others from using works without permission, and incentivise innovation. The current legal framework is failing to ensure these outcomes occur and the Government has a duty to act. It cannot sit on its hands for the next decade and hope the courts will provide an answer.”

Again, here’s RSS president Andrew Garrett’s take on the Lords committee report:

Snowy sky

We begin by loading in the following packages, adding a couple extra to the ones Nicola uses:

library(ggplot2)
library(ggforce)
library(sf)
library(png)
library(patchwork) 

Then we add the sky, now recoloured in RWDS purple using fill and color:

s1 <- ggplot() +
  theme_void() +
  theme(
    plot.background = element_rect(fill = "#939bc9", color = "#939bc9")
  )
s1

We use the same code as Nicola to create the snowflakes, but we do this step first before adding snow on the ground, as we’re using the RWDS site background colour, hex code #f0eeb, to represent our settled snow:

# add snowflakes
set.seed(20231225)
n <- 100
snowflakes <- data.frame(
  x = runif(n),
  y = runif(n)
)
s2 <- s1 +
  geom_point(
    data = snowflakes,
    mapping = aes(
      x = x,
      y = y
    ),
    colour = "white",
    pch = 8
  )
s2

# snow on ground
s3 <- s2 +
  annotate(
    geom = "rect",
    xmin = 0, xmax = 1,
    ymin = 0, ymax = 0.2,
    fill = "#f0eeeb", colour = "#f0eeeb"
  ) +
  xlim(0, 1) +
  ylim(0, 1) +
  coord_fixed(expand = FALSE)
s3

Oh, Christmas tree

To build her snowman, Nicola created a series of circles that were stacked and overlaid. A simple Christmas tree, though, requires a series of triangles. So, taking Nicola’s snowman’s nose (also a triangle) as our starting point, we coded three sets of coordinates – tree_pts1, tree_pts2, and tree_pts3 – for three triangles of decreasing size that would sit on top of one another.

# coordinates for tree base
tree_pts1 <- matrix(
  c(
    0.2, 0.3,
    0.5, 0.6,
    0.8, 0.3,
    0.2, 0.3
  ),
  ncol = 2,
  byrow = TRUE
)

# coordinates for tree middle
tree_pts2 <- matrix(
  c(
    0.3, 0.5,
    0.5, 0.7,
    0.7, 0.5,
    0.3, 0.5
  ),
  ncol = 2,
  byrow = TRUE
)

# coordinates for tree top
tree_pts3 <- matrix(
  c(
    0.4, 0.65,
    0.5, 0.75,
    0.6, 0.65,
    0.4, 0.65
  ),
  ncol = 2,
  byrow = TRUE
)

# put tree together
tree <- st_multipolygon(list(list(tree_pts1),
                             list(tree_pts2),
                             list(tree_pts3)))
s4 <- s3 +
  geom_sf(
    data = tree,
    fill = "chartreuse4",
    colour = "chartreuse4"
  ) +
  coord_sf(expand = FALSE)
s4

A tree also requires a trunk, so we borrowed one of the rectangles from Nicola’s snowman’s hat for this purpose:

s5 <- s4+
  annotate(
    geom = "rect",
    xmin = 0.45,
    xmax = 0.55,
    ymin = 0.2,
    ymax = 0.3,
    fill = "brown"
  )
s5

And, of course, no Christmas tree is complete without decorations. The “rocks” that formed the buttons and eyes on Nicola’s snowman were updated to become gold and red baubles for our tree:

# add gold baubles
s6 <- s5 +
  geom_point(colour = "gold",
             data = data.frame(
               x = c(0.3, 0.4, 0.5, 0.6, 0.57, 0.62, 0.45, 0.5),
               y = c(0.325, 0.4, 0.45, 0.35, 0.57, 0.52, 0.6, 0.7),
               size = runif(8, 2, 4.5)
             ),
             mapping = aes(x = x, y = y, size = size)
  ) +
  scale_size_identity()
s6

# add red baubles
s7 <- s6 +
  geom_point(colour = "red3",
             data = data.frame(
               x = c(0.7, 0.6, 0.5, 0.525, 0.43, 0.38, 0.55, 0.5),
               y = c(0.375, 0.4, 0.55, 0.65, 0.43, 0.48, 0.5, 0.375),
               size = runif(8, 2, 4.5)
             ),
             mapping = aes(x = x, y = y, size = size)
  ) +
  scale_size_identity()
s7

Season’s greetings

The final step was to add text to the top of the image, wishing you all a Merry Christmas, and our logo to the bottom, so you know who the card is from:

# add text
s8 <- s7 +
  annotate(
    geom = "text",
    x = 0.5,
    y = 0.875,
    label = "Merry Christmas",
    colour = "red3",
    fontface = "bold",
    size = 18
  )
s8

# add logo 
path <- "images/rwds-logo-150px.png"
img <- readPNG(path, native = TRUE) 
s9 <- s8 +                   
  inset_element(p = img, 
                left = 0.3265, 
                bottom = 0.0, 
                right = 0.6735, 
                top = 0.2
  ) 
s9

I hope you like the Christmas card! From all of us at Real World Data Science, thank you for your support throughout 2023. Merry Christmas, happy holidays, and best wishes for 2024!

Regulatory approaches

The UK government’s AI Safety Summit has been criticised for focusing too much on the hypothetical existential risks of AI. But, on regulation at least, there was broad agreement that the UK’s principles- and sector-based approach, outlined in a March 2023 white paper, is the right one. That’s not to say it’s perfect: discussions were had about whether regulatory bodies would be adequately funded to regulate the use of AI in their sectors, while Hetan Shah of the British Academy wondered “where was the golden thread” linking the AI white paper to the AI Safety Summit and its various pronouncements, including plans for an AI Safety Institute. (On the Safety Institute in particular, Lord Tim Clement-Jones was sceptical of yet another body being drafted in to debate these issues – a point made by panellists at the RSS’s recent AI debate.)

Delegates also got to hear from the UK’s Information Commissioner directly. John Edwards delivered a keynote address in which he acknowledged the huge excitement surrounding the benefits AI promises to bring, while cautioning that deployment and use of AI must be done in accordance with existing rules on data protection and privacy. The technology may be new, he said, but the same old data rules apply: “Our legislation is founded on technology-neutral principles of general application. They are capable of adapting to numerous new technologies, as they have over the last 30 years and will continue to do.”

He warned that noncompliance with data protection rules and regulations “will not be profitable,” and that persistent misuse of AI and personal data for competitive advantage would be punished. Edwards concluded by saying that AI is built on the data of human individuals and should therefore be used to improve their lives, and not put them or their personal data at risk.

Elections in an era of generative AI

One major looming risk is the use of generative AI to create mis- and disinformation during election campaigns. Hans-Petter Dalen of IBM suggested that next year is perhaps the biggest year for elections in the history of mankind, with votes due in the UK, US, and India, to name but a few. Generative AI represents not a new threat, he said, but an “amplified” one – a point further developed by Henry Parker of Logically.ai. Parker spoke of the risk of large-scale breakdown in trust due to mis- or disinformation campaigns. Thanks to AI tools, he said, we are now seeing the “democratisation of disinformation.” What once might have cost millions of dollars and required a team of hundreds of people can now be done much more cheaply and with fewer human resources. As the Royal Society’s Areeq Chowdhury said, the challenge of disinformation has only become harder.

Asked how to counter this, Dalen said that if he were a politician, “I would certainly get my own blockchain and all my content would have been digitally watermarked from source – that’s what the blockchain does.” But digital watermarking is only part of the answer, added Parker. Identifying mis- and disinformation is both a question of provenance and of dissemination. Logically.ai is using AI as a tool to analyse behaviours around the circulation of mis- and disinformation, Parker said – positioning AI as but one solution to a problem it has helped exacerbate.

AI safety, and AI risks

Andrew Garrett: For those who were listening to the commentary last week, the PM [prime minister] made a very interesting speech. Rishi Sunak announced the creation of the world’s first AI Safety Institute in the UK, to examine, evaluate and test new types of AI. He also stated that he pushed hard to agree the first ever international statement about the risks of AI because, in his view, there wasn’t a shared understanding of the risks that we face. He used the example of the IPCC, the Intergovernmental Panel on Climate Change, to establish a truly global panel to publish a “state of AI science” report. And he also announced an investment in raw computing power, so around a billion pounds in a supercomputer, and £2.5 billion in quantum computers, making them available for researchers and businesses as well as government.

The RSS provided two responses this year to prominent [AI policy] reviews. The first was in June on the AI white paper, and the second was on the House of Lords Select Committee inquiry into large language models back in September. How do they relate to what the PM said? There’s some good news here, and maybe not quite so good news.

First, the RSS had requested investments in AI evaluation and a risk-based approach. And you could argue, by stating that there will be a safety institute, that that certainly ticks one of the boxes. We also recommended investment in open source, in computing power, and in data access. In terms of computing power, that was certainly in the [PM’s] speech. We spoke about strengthening leadership, and in particular including practitioners in the [AI safety] debate. A lot of academics and maybe a lot of the big tech companies have been involved in the debate, but we want to get practitioners – those close to the coalface – involved in the debate. I’m not sure we’ve seen too much of that. We recommended that strategic direction was provided, because it’s such a fast-moving area, and the fact that the Bletchley Park Summit is happening tomorrow, I think, is good for that. And we also recommended that data science capability was built amongst the regulators. I don’t think there was any mention of that.

That’s the context [for the RSS event today]. What I’m going to do now is ask each of the panellists to give an introductory statement around the AI summit, focusing on the safety aspects. What do they see as the biggest risk? And how would they mitigate or manage this risk?

Detlef Nauck: I work at BT and run the AI and data science research programme. We’ve been looking at the safety, reliability, and responsibility of AI for quite a number of years already. Five years ago, we put up a responsible AI framework in the company, and this is now very much tied into our data governance and risk management frameworks.

Looking at the AI summit, they’re focusing on what they call “frontier models,” and they’re missing a trick here because I don’t think we need to worry about all-powerful AI; we need to worry about inadequate AI that is being used in the wrong context. For me, AI is programming with data, and that means I need to know what sort of data has been used to build the model, and I need AI vendors to be upfront about it and to tell me: What is the data that they have used to build it, how have they built it, or if they’ve tested for bias? And there are no protocols around this. So, therefore, I’m very much in favour of AI evaluation. But I don’t want to wait for an institute for AI evaluation. I want the academic research that needs to be done around this, which hasn’t been done. I want everybody who builds AI systems to take this responsibility and document properly what they’re doing.

Mihaela van der Schaar: I am an AI researcher building AI and machine learning technology. Before talking about the risks, I also would like to say that I see tremendous potential for good. Many of these machine learning AI models can transform for the better areas that I find extremely important – healthcare and education. That being said, there are substantial risks, and we need to be very careful about that. First, if not designed well, AI can be both unsafe as well as biased, and that could lead to tremendous impact, especially in medicine and education. I completely agree with all the points that the Royal Statistical Society has made not only about open source but also about data access. This AI technology cannot be built unless you have access to high quality data, and what I see a lot happening, especially in industry, is people have data sources that they’ll keep private, build second-rate or third-rate technology on them, and then turn that into commercialised products that are sold to us for a lot of money. If data is made widely available, the best as well as the safest AI can be produced, rather than monopolised.

Another area of risk that I’m especially worried about is human marginalisation. I hear more and more a lot of companies talking about AI general intelligence, and how AI is going to take over the world, and I’m tremendously concerned as an AI researcher about this. There is an opportunity to build AI that is human empowering, that keeps us strong, able, capable, intelligent, and can support us in all our human capabilities.

Martin Goodson: The AI Safety Summit is starting tomorrow. But, unfortunately, I think the government are focusing on the wrong risks. There are lots of risks to do with AI, and if you look at the scoping document for the summit, it says that what they’re interested in is misuse risk and the risk of loss of control. Misuse risk is that bad actors will gain access to information that they shouldn’t have and build chemical weapons and things like that. And the loss of control risk is that we will have this super intelligence which is going to take over and we should see, as is actually mentioned, the risk of the extinction of the human race, which I think is a bit overblown.

Both of these risks – the misuse risk and the loss of control risk – are potential risks. But we don’t really know how likely they are. We don’t even know whether they’re possible. But there are lots of risks that we do know are possible, like loss of jobs, and reductions in salary, particularly of white-collar jobs – that seems inevitable. There’s another risk, which is really important, which is the risk of monopolistic control by the small number of very powerful AI companies. These are the risks which are not just likely but are actually happening now – people are losing their jobs right now because of AI – and in terms of monopolistic control, OpenAI is the only company that has anything like a large language model as powerful as GPT-4. Even the mighty Google can’t really compete. This is a huge risk, I think, because we have no control over pricing: they could raise the prices if they wanted to; they could constrain access; they could only give access to certain people that they want to give access to. We don’t have any control over these systems.

Mark Levene: I work in NPL as a principal scientist in the data science department. I’m also emeritus professor in Birkbeck, University of London. I have a long-standing expertise in machine learning and focus in NPL on trustworthy AI and uncertainty quantification. I believe that measurement is a key component in locking-in AI safety. Trustworthy AI and safe AI both have similar goals but different emphases. We strive to demonstrate the trustworthiness of an AI system so that we can have confidence in the technology making what we perceive as responsible decisions. Safe AI puts the emphasis on the prevention of harmful consequences. The risk [of AI] is significant, and it could potentially be catastrophic if we think of nuclear power plants, or weapons, and so on. I think one of the problems here is, who is actually going to take responsibility? This is a big issue, and not necessarily an issue for the scientist to decide. Also, who is accountable? For instance, the developers of large language models: are they the ones that are accountable? Or is it the people who deploy the large language models and are fine-tuning them for their use cases?

The other thing I want to emphasise is the socio-technical characteristics [of the AI problem]. We need to get an interdisciplinary team of people to actually try and tackle these issues.

Do we need an AI Safety Institute?

Andrew Garrett: Do we need to have an AI Safety Institute, as Rishi Sunak has said? And if we don’t need one, why not?

Detlef Nauck: I’m more in favour of encouraging academic research in the field and funding the kind of research projects that can look into how to build AI safely, [and] how to evaluate what it does. One of the key features of this technology is it has not come out of academic research; it has been built by large tech companies. And so, I think we have to do a bit of catch up in scientific research and in understanding how are we building these models, what can they do, and how do we control them?

Mihaela van der Schaar: This technology has a life of its own now, and we are using it for all sorts of things that maybe initially was not even intended. So, shall we create an AI [safety] institute? We can, but we need to realise first that testing AI and showing that it’s safe in all sorts of ways is complicated. I would dare say that doing that well is a big research challenge by itself. I don’t think just one institute will solve it. And I feel the industry needs to bear some of the responsibility. I was very impressed by Professor [Geoffrey] Hinton, who came to Cambridge and said, “I think that some of these companies should invest as much money in making safe AI as developing AI.” I resonated quite a lot with that.

Also, let’s not forget, many academic researchers have two hats nowadays: they are professors, and they are working for big tech [companies] for a lot of money. So, if we take this academic, we put them in this AI tech safety institute, we have potential for corruption. I’m not saying that this will happen. But one needs to be very aware, and there needs to be a very big separation between who develops [AI technology] and who tests it. And finally, we need to realise that we may require an enormous amount of computation to be able to validate and test correctly, and very few academic or governmental organisations may have [that].

Martin Goodson: Can I disagree with this idea of an evaluation institute? I think it’s a really, really bad idea, for two reasons. The first is an argument about fairness. If you look at drug regulation, who pays for clinical trials? It’s not the government. It’s the pharmaceutical companies. They spend billions on clinical trials. So, why do we want to do this testing for free for the big tech companies? We’re just doing product development for them. It’s insane! They should be paying to show that their products are safe.

The other reason is, I think it’s an insult to the UK’s scientific legacy that we’re reduced to testing software that has been made by US companies. I think it’s pathetic. We were one of the main leaders of the Human Genome Project, and we really pushed it – the Wellcome Trust and scientists in the UK pushed the Human Genome Project because we didn’t want companies to have monopolistic control over the human genome. People were idealistic, there was a moral purpose. But now, we’re so reduced that all we can do is test some APIs that have been produced by Silicon Valley companies. We have huge talents in this country. Why aren’t we using that talent to actually build something instead of testing something that someone else has made?

Mark Levene: Personally, I don’t see any problem in having an AI institute for safety or any other AI institutes. I think what’s important in terms of taxpayers’ money is that whatever institute or forum is invested in, it’s inclusive. One thing that the government should do is, we should have a panel of experts, and this panel should be interdisciplinary. And what this panel can do is it can advise government of the state of play in AI, and advise the regulators. And this panel doesn’t have to be static, it doesn’t have to be the same people all the time.

Andrew Garrett: To evaluate something, whichever way you chose to do it, you need to have an inventory of those systems. So, with the current proposal, how would this AI Safety Institute have an inventory of what anyone was doing? How would it even work in practice?

Martin Goodson: Unless we voluntarily go to them and say, “Can you test out our stuff?” then they wouldn’t. That’s the third reason why it’s a terrible idea. You’d need a licencing regime, like for drugs. You’d need to licence AI systems. But teenagers in their bedrooms are creating AI systems, so that’s impossible.

Let’s do reality-centric AI!

Andrew Garrett: What are your thoughts about Rishi Sunak wanting the UK to be an AI powerhouse?

Martin Goodson: It’s not going to be a powerhouse. This stuff about us being world leading in AI, it’s just a fiction. It’s a fairy tale. There are no real supercomputers in the UK. There are moves to build something, like you mentioned in your introduction, Andrew. But what are they going do with it? If they’re just going to build a supercomputer and carry on doing the same kinds of stuff that they’ve been doing for years, they’re not going to get anywhere. There needs to be a big project with an aim. You can build as many computers as you want. But if you haven’t got a plan for what to do with them, what’s the point?

Mihaela van der Schaar: I really would agree with that. What about solving some real problem: trying to solve cancer; trying to solve our crisis in healthcare, where we don’t have enough infrastructure and doctors to take care of us? What about solving the climate change problem, or even traffic control, or preventing the next financial crisis? I wrote a little bit about that, and I call it “let’s do reality-centric AI.” Let’s have some goal that’s human empowering, take a problem that we have – energy, climate, cancer, Alzheimer’s, better education for children, and more diverse education for children – and let us solve these big challenges, and in the process we will build AI that’s hopefully more human empowering, rather than just saying, “Oh, we are going to solve everything if we have general AI.” Right now, I hear too much about AI for the sake of AI. I’m not sure, despite all the technology we build, that we have advanced in solving some real-world problems that are important for humanity – and imminently important.

Martin Goodson: So, healthcare– I tried to make an appointment with my GP last week, and they couldn’t get me an appointment for four weeks. In the US you have this United States Medical Licencing Examination, and in order to practice medicine you need to pass all three components, you need to pass them by about 60%. They are really hard tests. GPT-4 for gets over 80% in all three of those. So, it’s perfectly plausible, I think, that an AI could do at least some of the role of the GP. But, you’re right, there is no mission to do that, there is no ambition to do that.

Mihaela van der Schaar: Forget about replacing the doctors with ChatGPT, which I’m less sure is such a good idea. But, building AI to do the planning of healthcare, to say, “[Patient A], based on what we have found out about you, you’re not as high risk, maybe you can come in four weeks. But [patient B], you need to come tomorrow, because something is worrisome.”

Martin Goodson: We can get into the details, but I think we are agreeing that a big mission to solve real problems would be a step forward, rather than worrying about these risks of superintelligences taking over everything, which is what the government is doing right now.

Managing misinformation

Andrew Garrett: We have some important elections coming up in 2024 and 2025. We haven’t talked much about misinformation, and then disinformation. So, I’m interested to get your views here. How much is that a problem?

Detlef Nauck: There’s a problem in figuring out when it happens, and that’s something we need to get our heads around. One thing that we’re looking at is, how do we make communication safe from bad actors? How do you know that you’re talking to the person you see on the camera and it’s not a deep fake? Detection mechanisms don’t really work, and they can be circumvented. So, it seems like what we need is new standards for communication systems, like watermarks and encryption built into devices. A camera should be able to say, “I’ve produced this picture, and I have watermarked it and it’s encrypted to a certain level,” and if you don’t see that, you can’t trust that what you see comes from a genuine camera, and it’s not artificially created. It’s more difficult around text and language – you can’t really watermark text.

Mark Levene: Misinformation is not just a derivative of AI. It’s a derivative of social networks and lots of other things.

Mihaela van der Schaar: I would agree that this is not only a problem with AI. We need to emphasise the role of education, and lifelong education. This is key to being able to comprehend, to judge for ourselves, to be trained to judge for ourselves. And maybe we need to teach different methods – from young kids to adults that are already working – to really exercise our own judgement. And that brings me to this AI for human empowerment. Can we build AI that is training us to become smarter, to become more able, more capable, more thoughtful, in addition to providing sources of information that are reliable and trustworthy?

Andrew Garrett: So, empower people to be able to evaluate AI themselves?

Mihaela van der Schaar: Yes, but not only AI – all information that is given to us.

Martin Goodson: On misinformation, I think this is really an important topic, because large language models are extremely persuasive. I asked ChatGPT a puzzle question, and it calculated all of this stuff and gave me paragraphs of explanations, and the answer was [wrong]. But it was so convincing I was almost convinced that it was right. The problem is, these things have been trained on the internet and the internet is full of marketing – it’s trillions of words of extremely persuasive writing. So, these things are really persuasive, and when you put that into a political debate or an election campaign, that’s when it becomes really, really dangerous. And that is extremely worrying and needs to be regulated.

Mark Levene: You need ways to detect it. Even that is a big challenge. I don’t know if it’s impossible, because, if there’s regulation, for example, there should be traceability of data. So, at the moment, if you type something into ChatGPT and you ask for references, half of them will be made up. We know that, and also OpenAI knows that. But it could be that, if there’s regulation that things are traceable, you should be able to ask, “How did this information come about? Where did it come from?” But I agree that if you just look at an image or some text, and you don’t know where it came from, it’s easy to believe. Humans are easily fooled, because we’re just the product of what we know and what we’re used to, and if we see something that we recognise, we don’t question it.

Audience Q&A

Closing thoughts: What would you say to the AI Safety Summit?

Andrew Garrett: If you were at the AI Safety Summit and you could make one point very succinctly, what would it be?

Martin Goodson: You’re focusing on the wrong things.

Mark Levene: What’s important is to have an interdisciplinary team that will advise the government, rather than to build these institutes, and that this team should be independent and a team which will change over time, and it needs to be inclusive.

Mihaela van der Schaar: AI safety is complex, and we need to realise that people need to have the right expertise to be able to really understand the risks. And there is risk, as I mentioned before, of potential collusion, where people are both building the AI and saying it’s safe, and we need to separate these two worlds.

Detlef Nauck: Focus on the data, not the models. That’s what’s important to build AI.

About the panelists

Andrew Garrett (chair) is president of the Royal Statistical Society. He is executive vice president of scientific operations at the clinical research organisation ICON plc, where he is responsible for the strategic direction and operational delivery of a range of clinical trial services. Having worked extensively in the area of rare diseases, he has held various biostatistics managerial positions in the pharmaceutical industry, including vice president of biostatistics, medical writing and regulatory affairs at Quintiles (now IQVIA).

Peter Wells is a technologist, who accidentally started a second career in public policy. He has both worked on AI policy and helped design AI-enabled services. After 20 years in the telecoms industry, he found himself spending 2014 developing digital government policy for the Labour Party. Since then he has worked with multiple governments and organisations including the Open Data Institute, Projects by IF, Google, Meta and the Government Digital Service.

Maxine Setiawan is a data scientist specialising in AI and data risk and trusted AI in EY UK&I. She works to help clients from various industries assess and manage risks from analytics and AI systems, and implement AI governance to ensure AI systems are implemented with fair, accountable, and trustworthy principles. She combines her socio-technical background with an MSc in Social Data Science from the University of Oxford, and her experience working in data science within consulting firms.

Sophie Carr is chair of the Real World Data Science editorial board and is the founder and owner of Bays Consulting, a data science company. Having trained as an aeronautical engineer, Sophie completed her PhD in Bayesian analysis part time whilst she worked and, following redundancy, founded her own company. She is the VP for education and statistical literacy at the RSS and sits on the executive committees of the Academy for Mathematical Sciences and the International Centre for Mathematical Sciences. She is also currently the world’s most interesting mathematician.

Chris Nemeth is a professor of statistics at Lancaster University. His primary research area is in probabilistic machine learning and computational statistics. He holds an EPSRC-funded Turing AI fellowship on Probabilistic Algorithms for Scalable and Computable Approaches to Learning (PASCAL), and through his fellowship he works closely with partners including Shell, Tesco, Elsevier, Microsoft Research and The Alan Turing Institute. He is chair of the Royal Statistical Society Section on Computational Statistics and Machine Learning.

Karen Tingay is a principal statistical methodologist at the Office for National Statistics where she specialises in natural language processing and in managing complex survey imputation. She established and heads up the Text Data Subcommunity, a large network of public sector analysts to build capability and best practice guidance in managing and analysing unstructured text data, on behalf of the Government Data Science Community. She sits on several cross-government and international working groups on responsible use of generative AI.