The “Tag and Batch” Heuristic

Rather than reacting to every notification, Piers advocated for a “batch processing” approach to professional development.

• The System: Piers uses raindrop.io to store relevant bookmarks in a monthly repository for the newsletter. Each bookmark is added to a specific monthly collection, and also includes a tag for the newsletter topic it is relevant for. The bookmarks are then pulled together into a coherent narrative once, at the end of the month.

• Curation: Once a week, he performs quick scans of various trusted sources and newsletters he has built up over the years—such as TLDR (AI and Data) for technical updates and The Batch for curated commentary—saving interesting links without immediately diving deep. He advocates for a brief high level assessment at this stage to avoid going down rabbit holes- the risk is spending an unsustainable amount of time in this stage.

• Assimilation: Then, once a month, he calls up the saved bookmarks and compiles them into a coherent story. This can be time consuming but is very worthwhile in terms of understanding. He starts with the first topic, opening up all the relevant links, and thinks through how they are related, how important they are and how best to position them from a narrative perspective. Once the first topic is done, he continues through the rest of the topics, finally pulling out the “top 5 must reads” for the headline links at the top of the newsletter.

• The Benefit: By tagging articles by section (e.g., Ethics, Engineering, Big Picture) at the point of discovery, the final synthesis becomes a task of finding themes rather than hunting for links. And the deeper dive learning happens in a consolidated period once a month rather than spread in an expanding amount of time throughout the month.

Evals and Breadth

The evening concluded with a spirited Q&A that touched on the future of the role of a data scientist. When asked how early-career data scientists should prepare for a world where AI automates code, Piers’ advice was twofold: evals and breadth.

With AI making output generation increasingly easy (whether it is applications or analytical output), the critical question becomes whether or not the output is any good. A crucial capability all data scientists need is the ability to evaluate the output. How do we measure how good it is? How do we build processes that allow us to keep track of this quality measurement? How do we understand whether or not changes we make, cause the system to actually improve?

Separately, AI allows individual engineers, data scientists (and product managers and designers) to do more things. So exploring and expanding your capabilities across different areas - giving yourself more breadth - allows you to stand out if there is increasing downward pressure on team sizes and hiring.

Source Toolkit

Workflow Tool: Raindrop.io for bookmarking and tagging.

The Evolution: Compare our Feb 2020 debut with our April 2026 edition.

Reading List:

• TLDR (AI and Data newsletters);

• The Batch from Andrew Ng;

• Data Science Weekly;

• Import AI from Jack Clark;

• One Useful Thing from Ethan Mollick;

• Benedict Evans newsletter

• Exponential View from Azeem Azhar.

If you have found this helpful, make sure to check out the newsletter here.

The Case of the Month

According to the US Centers for Disease Control (CDC), roughly half of adults in the US have hypertension (high blood pressure). Roughly 60% of these take medication to keep their blood pressure under control.

However, measuring blood pressure turns out to be much more complicated than is widely acknowledged. This leads to the question of how many “false positives” there might be; that is, how many people diagnosed with high blood pressure don’t actually have it.

Blood pressure medication, like all medication, has side effects, and people who don’t actually have high blood pressure shouldn’t be taking these medications. There may also be false negatives – people who are told they have normal blood pressure but are actually hypertensive, which can have extreme and well documented implications. This is why obtaining high-quality data prior to diagnosis and treatment is so critical in addressing this common medical problem.

What Actually Happened?

As noted by Myers and Kaczorowski, different ways of measuring blood pressure produce different results. In particular, the automated devices commonly used in clinical settings, as well as those bought for at-home measurement, are known to be highly variable. For example, one of the current authors experienced an automated reading in a clinic of 138/92, but when measured manually five minutes later, the reading was 123/82.

Manual readings are also subject to tester-to-tester variation. Again, one of the authors experienced a 152/96 manual reading from a nurse, followed by a 122/78 reading by a doctor a few minutes later.

These are not the only issues. In addition to ensuring that the patient is at rest, the American Medical Association recommends seven tips to ensure accurate and consistent blood pressure measurement:

1. The patient’s arm should be held at the same height as the heart.
2. The cuff should be placed on the bare arm, with no clothing between the cuff and arm.
3. The correct size cuff should be used, based on the diameter of the patient’s arm.
4. The patient should not speak, or be spoken to, during measurement.
5. The patient should have an empty bladder.
6. The back and feet of the patient should be supported (e.g., feet not “dangling” from the examination table).
7. The patient’s legs should not be crossed.

Research has been carried out that shows how important cuff sizing is in accurately diagnosing hypertension, especially in overweight adults. The research showed that too-small cuff size resulted in a mistaken diagnosis of high blood pressure in 39% of patients, while 22% of those with hypertension were given false negatives by the automated monitors.

And this is only 1 of the 7 guidelines. In our collective experience, medical personnel frequently do not follow a number of the steps necessary for accurate readings. In fact, in a statistical analysis of stock images in which the subject is measuring blood pressure, including images disseminated by prestigious medical and educational institutions like Harvard Publishing and Oxford University, only 1 in 7 images aligned with all the recommended clinical guidelines. So it is hardly surprising that inaccurate and highly variable measurements are common.

Disaster or Near-Miss?

Taking medication unnecessarily, or not taking medication when it is needed, can have serious consequences, including stroke and heart attack. Modelling data from Canada, Leung et al. found that, “If both systolic and diastolic BP were overestimated by 10 mm Hg, the prevalence of hypertension would falsely increase by 50% to 63%, potentially leading to overtreatment of approximately 3.5 million Canadians.” We suspect that the implications, both for misdiagnosis as well as under-diagnosis, are very serious, and this is at least a potential disaster.

But there is also another angle to consider. The impact this may have on trust in medical results, as well as the implications for population statistics and national and international health interventions, are unknown. As good data are essential for improving health outcomes, especially in more deprived areas (see for example, the Public Health England report ’Tackling high blood pressure From evidence into action’), this prevalence of poor data is especially concerning.

Why This Matters Now

The differences noted previously are of clinical significance. In addition to deciding whether someone should or should not be on medication, there are decisions as to which medications to prescribe, and at what dosages, with one of the authors’relatives having been on three different blood pressure medications at the same time, with serious side effects. Regardless of the expertise of physicians and nurses, inaccurate data will prevent them from making the proper decisions for their patients.

This is also relevant for research – and decisions made based on that research – that aggregates data to inform population-level health programming and initiatives that target hypertension across the globe, like the WHO’s Global report on hypertension 2025: high stakes: turning evidence into action.

The Practitioner Takeaways

For patients, the advice is simple enough: when having your blood pressure taken, insist on medical staff following the recommended protocols. Don’t rely solely on the automated measurement.

But there is also the question of how inaccurate readings impact population data, resource allocation and health outcome measurement on a national and international level. For the data scientist, the key lesson is to make sure data are never taken at “face value,” and to always delve into the “data pedigree,” the details of how, when, where, etc. the data were collected.

In the meantime, medical professionals taking care to follow the guidelines, recognise the issues with BP readings and communicate them clearly to patients, and not giving too much credence to any one measurement, could go a long way to combatting over-treatment and undertreatment, as well as mistrust in results.

The Data Quality Pattern

The fundamental issue here is that clinical decisions are being made based on poor and inaccurate data. But the cascading consequences of poor measurement, both for patient trust and population level outcomes, could be leading to what we at Data Quality Under the Lens call the Metric Mirage: a measure that appears to show success, failure, or progress, but is misleading because it is poorly defined, incomplete, or taken out of context.

With an underlying instability involving questionable measurement accuracy and precision, the pivotal questions around health outcomes and the reliability of the data that backs decisions become paramount. Are the data being used to make and measure medical interventions fit for purpose, or are choices being made based on only the mirage of evidenced understanding?

Data that come from an automated device, or from a trained and experienced healthcare provider, are often assumed to be accurate. In the case of blood pressure measurement, they frequently are not.

Introduction: Navigating the Data Deluge

The greatest challenge in modern genomics is no longer generating data, but knowing which data can actually answer your question.

Genomic data has become a fundamental pillar of biomedical discovery, fuelling advancements from Genome-Wide Association Studies (GWAS) to Polygenic Risk Scores (PRS) and rare variant analyses, including those informing novel therapeutic approaches like CRISPR-based gene therapies [3]. This wealth of data holds immense promise for elucidating the genetic architecture of disease, personalising therapeutic strategies, and accelerating drug development.

However, for researchers new to genomics, the initial challenge is often not the complexity of the analysis, but the complexity of the data landscape itself; a challenge well-documented in the literature. This includes the sheer scale of data, where individual genomes can exceed 100 gigabytes of information [4], the significant ethical and logistical hurdles in managing it [5, 6], and the persistent issue of non-representative data that limits equitable progress [2, 7]. The pivotal question, “Which genomic dataset is right for my research?”, can be daunting. Navigating a maze of biobanks, consortia, and public repositories, researchers are confronted with a bewildering array of technical jargon and specifications. Selecting an unsuitable dataset can lead to months of wasted effort, irreproducible results, and fundamental biases that undermine scientific validity.

This article enriches existing approaches by providing a solution: a clear, structured framework designed to guide researchers in systematically evaluating and selecting genomic datasets. We move beyond simple catalogues of resources to dissect the critical decision-making process. By interrogating any potential dataset across five core pillars, you can ensure the data you source is robust, fit for purpose, and capable of driving reliable science, whether for GWAS, PRS, exome analysis, or other downstream applications.

Researchers often turn to existing genomic datasets rather than generating new data due to several key motivations: Cost efficiency, as sequencing and genotyping are resource-intensive; Access to larger sample sizes than may be feasible for individual studies; The ability to leverage well-phenotyped cohorts with longitudinal or linked data; As well as accelerating research by building on previously harmonized and QC-ed data. Common sources for such data include public repositories such as dbGaP, EGA, the NCI Genomic Data Commons (GDC), and databanks such as UK Biobank, FinnGen, and the UK’s National Genomics Research Library (NGRL).

The Core Framework: The Five Pillars of Genomic Data Selection

Before downloading data, a researcher must interrogate it across five key dimensions. These pillars form a checklist of non-negotiable considerations that determine technical compatibility, scientific utility, and practical feasibility. Importantly, most of these questions can, and should, be answered before you apply for access or begin data transfer. Doing so can prevent months of downstream re-analysis or worse, discovering too late that a dataset is fundamentally unsuited to your research question.

Synthesising the Framework: From Evaluation to Selection

The Road Ahead: Federation and Multi-Modal Data Integration

While our five-pillar framework provides a robust foundation for selecting a dataset, the future of genomic discovery lies in what comes next: the ability to connect and integrate data across multiple sources. This is where the vision of the Global Alliance for Genomics and Health (GA4GH) and the principles of data federation become tangible.

A single dataset, no matter how well-curated, is ultimately limited by its sample size, population diversity, and the breadth of its phenotypic measurements. The next frontier involves securely combining information from multiple datasets to ask questions that no single cohort could answer alone. This might involve:

• Federated Analysis: Instead of pooling data into a single location (which raises significant ethical and governance challenges), federated approaches allow researchers to bring analysis algorithms to disparate datasets. The algorithm travels to the data, learns from it, and returns only the aggregated results, never the individual-level data. This approach, championed by GA4GH, is already being used to build more powerful polygenic risk scores and to discover rare variant associations by effectively and securely increasing sample sizes across international borders.
• Multi-Modal Data Integration: The real power of platforms like DPUK, UK Biobank, and Terra is their ability to link genomic data with other data modalities; brain imaging, proteomics, metabolomics, electronic health records, and wearable device data. For the researcher, this integration opens up entirely new classes of questions. You can move beyond simple genetic association to study molecular mechanisms, gene-environment interactions, and the real-world progression of disease over decades. Understanding how genetics influences protein levels (via pQTLs), or how a polygenic risk interacts with air pollution to trigger asthma, requires this integrated, multi-modal perspective.

As you apply the five-pillar framework, consider not just the dataset in isolation, but its potential to connect with others. Does it use common data standards? Is it part of a federated network? Does its platform support linkage to other data types? The datasets that will drive the next generation of discovery are those designed to be nodes in a larger, interconnected, and responsible data ecosystem.

Conclusion

Selecting the right genomic dataset is the foundational step that determines the validity, reproducibility, and ethical integrity of your research. By systematically applying this enhanced five-pillar framework, interrogating the Discovery, Access & Governance, Technology, Content, and Quality & Harmonization of potential datasets, you transform a daunting maze into a navigable landscape.

In the next article of this series, “From Dataset to Discovery,” we will put this framework into practice, demonstrating how to apply these pillars to real-world study designs including GWAS, PRS validation, and rare variant analysis.

The Case of the Month

On 15th April the UK government published the Independent Review into Releases in Error, authored by Dame Lynne Owens (the former head of the National Crime Agency). Between April 2025 and March 2026 179 prisoners were freed “in error” in England and Wales, according to the Ministry of Justice. Some of these releases resulted from misplaced warrants for imprisonment or remand, sentence miscalculations, or the result of mistakes by courts or other authorities, but most were due, essentially, to poor data quality.

The consequences – individuals released into the community earlier than intended – highlight the potential public safety implications of data quality.

What Actually Happened?

In the last few years there has been a sharp increase in the number of prisoners released in error.

These include:

• A convicted sex offender, whose release, caused by paper notes being misinterpreted by poorly trained staff, led to the government’s independent report.
• A high-profile fraudster released due to the court incorrectly recording his 45-month jail term as a suspended sentence.
• A man who had committed serious violent offences and robbery, released due to a clerk failing to properly aggregate multiple sentences.

Although it appears that there has been a downturn in error since the peak of 2024, the report acknowledges that “some releases in error incidents are only identified when a prisoner returns to custody at a later date. Consequently, figures for more recent years may increase over time.” In her report, Dame Lynn states “The fact that the Department’s understanding of the true extent of releases in error is the minimum of what is likely to be the case was not my only point of alarm. The immediate concern I had was that the Ministry of Justice and HMPPS seemed to explicitly accept that the data is not representative of the ‘true’ picture of releases in error and yet had no means by which they were regularly quantifying the level of inaccuracy and therefore the level of extant risk of harm to the public. Without an established level of ‘accepted’ inaccuracy and risk, nor a regular means by which to check this, my concern thus became that the Government has historically and ongoingly accepted an outstanding and unquantifiable risk to the public.” In short, the crisis was two-fold: not only were dangerous prisoners being released in error, but the data intended to track these failures was so fundamentally flawed that the true risk to public safety remained hidden and unquantifiable

Disaster or Near-Miss?

Some of the prisoners who were released in error handed themselves back in. But, in some cases, the error of their release was only identified as a result of their committing a new offence. There are likely more unlawfully released prisoners that we don’t yet know about.

Why This Matters Now

This case reveals failures at two levels. At the operational level,low staffing, inadequate training and poor morale, within what has been described as a “highly challenging operational environment,” are leading to errors in how information is recorded or interpreted. At the systemic level, there is insufficient visibility into how and when these failures occur. Together, these issues show that a digital-first strategy cannot succeed on a foundation of “fragmented information”. Without skilled human oversight to verify facts, there is a risk that systemic problems will be automated rather than resolved.

The Practitioner Takeaways

The report highlights understaffing and low morale as key contributors to failure, yet the government’s response – only £8m in funds allocated for additional staff, but £82m allocated for a digital overhaul – prioritises digital investment over workforce capacity. That imbalance matters: if previous moves toward computerisation have coincided with declining system integrity, it suggests the issue is not simply tooling, but the capacity of people to manage and validate the data those tools depend on.

For practitioners, there are some clear takeaways to avert similar disasters:

• Don’t just track outputs; measure uncertainty—missing data, overrides, corrections, and inconsistencies. If you can’t see where errors arise, you can’t manage them.
• Design for verification, not just speed. Build checkpoints that prevent unvalidated data from flowing downstream.
• Ensure staff have the time and training to exercise judgement, not just act as a procedural step.
• Regularly trace records back to their origin, especially where legacy or paper systems are involved.
• Make data quality visible. Surface error rates and data gaps alongside performance metrics so trade-offs are explicit.

The Data Quality Pattern

This case is an example of what we at Real World Data Science refer to as a Metric Mirage. Leadership accept and rely on data they know is unrepresentative because they have no means of quantifying the true scale of failure. This happens when fragmented, often paper-based, records are treated as reliable digital data without being properly verified. As systems become more complex, human judgement and professional curiosity are increasingly displaced by the pursuit of speed and automation. This doesn’t fix the underlying errors; it simply hides them, allowing a Silent Drift in accuracy to continue unchecked until a high-profile tragedy forces a review.

Our prisons remain in crisis, and there is no quick fix.

It is therefore right that the government has accepted Dame Lynne’s findings and is moving swiftly to strengthen training, support and oversight for frontline staff. But lasting progress will depend on whether prisons are given sufficient, experienced staff with the time, tools and confidence to calculate sentences accurately. New systems and simpler rules will help but, when there is underinvestment in the people responsible for data quality, no amount of digital transformation will fix the problem— failures will simply be scaled more efficiently.

Digital Sovereignty, The Five Paradoxes of AI Adoption and the Pilot Trap

Brown begins by setting out the conditions that underpin what he terms ‘digital sovereignty’. Rather than framing sovereignty in narrow political terms, he defines it practically, arguing that any nation seeking to benefit from AI must secure three core levers: sufficient compute capacity, access to appropriate data, and effective systems of oversight. This framing is helpful both for its clarity and for grounding often abstract debates about AI leadership in tangible capabilities. In the same early chapters, Brown provides accessible definitions of the main categories of AI, striking a careful balance between technical accuracy and readability for those without deep specialist knowledge.

One of the most compelling contributions of the book emerges early on, when Brown introduces the five paradoxes confronting UK leaders attempting to adopt AI at scale. Among these, the tension between centralisation and local adoption stands out. The balance between national consistency and local accountability - often caricatured as the choice between coherence and the ‘postcode lottery’ - is a familiar problem in public policy, and Brown articulates it with clarity. These paradoxes recur throughout the book, acting as a lens rather than a checklist, and they help explain why well‑intentioned initiatives so often struggle to move beyond initial success.

Brown’s exploration of the UK’s digital past reinforces this point. Drawing on a range of examples, he highlights the persistence of what he calls the ‘pilot trap’: the tendency to celebrate high‑profile demonstrators while failing to deliver equivalent capability at scale. The lesson is not that pilots are misguided, but that scaling is a fundamentally different challenge, requiring institutional commitment, procurement maturity, and sustained leadership. This section is particularly effective in drawing practical lessons rather than offering retrospective criticism.

Making AI Work: Institutions, Skills and Governance

Turning to the present, Brown shifts his focus to what must change if AI is to deliver real value now. He groups the core challenges into three interconnected themes: institutional change, workforce upskilling, and governance. Importantly, ‘institutional’ here extends beyond government. Brown treats business, industry, and civil society as active participants in the national AI ecosystem, each needing to adapt if progress is to be collective rather than fragmented.

The discussion of workforce upskilling is especially strong. Using concrete examples from the NHS, financial services, the creative industries and government, Brown shows how AI skills requirements differ by sector while sharing common obstacles. These case studies exemplify one of the book’s key strengths: its grounding in real‑world experience rather than abstract aspiration. Brown does not shy away from the barriers to upskilling, including structural inequalities related to gender, ethnicity and geography, and he situates skills development firmly within a broader transformation strategy rather than treating it as a standalone intervention.

Governance and ethics are addressed with similar pragmatism. Rather than rehearsing familiar warnings, Brown draws on practical experience to suggest specific actions that align oversight with innovation. The result is an approach that recognises risk without allowing it to become an excuse for inertia.

A Strategy for What Comes Next

The final section, ‘What comes next’, is framed not as speculation but a proposal, even an exhortation. Brown compares the UK’s position with that of other countries - supported by a useful appendix - but repeatedly stresses that imitation is not a strategy. The UK, he argues, must develop an approach that reflects its own institutional strengths and constraints. He then sets out a phased programme covering governance, procurement, infrastructure, skills, and international positioning, before returning to the human dimension by examining what leaders themselves must do differently.

Who Should Read This Book?

I found the book both accessible and grounded in hard‑won experience. It would serve well as a primer for leaders, advisers and informed members of the public who want to move beyond slogans and understand how AI transformation happens. In bringing together clear analysis, original insights and practical recommendations, Brown offers a persuasive and constructive roadmap for making AI work for Britain, and I commend it to anyone concerned with how the nation will navigate its AI future.

Watch the discussion

Takeaways at a glance

• Reusability – rather than just reproducibility – of code and methods is the next frontier.
• Digital twins have huge potential but are still in an early, exploratory phase.
• New tools (especially coding tools) are lowering barriers to entry and improving communication of complex ideas.
• The rise of “vibe coding” and similar approaches is dramatically increasing development speed.

Key themes and analysis

From reproducible to reusable data science

Rhian highlights an important shift: reproducibility (being able to rerun someone else’s work) is becoming baseline, while reusability (adapting and building on it) is where the real value lies.

This reflects a maturation of the field when you consider that reproducibility supports verification and confidence (“can we trust this result?), where reusability supports acceleration and scale (can we take this and build on it?) The implication is that tooling, standards, and culture should increasingly prioritize modular, well-documented, interoperable outputs. “Clever but opaque” solutions become less valuable; simplicity and interface design become more important and standardisation becomes a force multiplier.

Acceleration through tooling, but not automation without expertise

Janet is excited about the speed gains from modern tools, including AI-assisted coding (“vibe coding”), but is careful to frame this as augmentation of experts rather than unregulated democratisation. In the hands of experts, new tools can lead to exponential productivity. In the hands of non-experts, they create a real risk of misuse.

This reinforces a key tension in modern data science, where an influx of new tools are making it easier to build solutions, but not necessarily easier to build correct solutions.

Lowering barriers to communication

Beyond technical productivity, tools are also improving how data science is communicated, making complex ideas more accessible, supporting scientific communication and bridging the gap to general audiences

Digital twins as an emerging frontier

Edith mentions digital twins — a high-potential but not yet fully realized concept that would allow for replicating real-world systems digitally, and enabling simulation, testing, and prediction at scale.

A digital twin is a continuously updated simulation of something real — a machine, a city, a supply chain, or even a human system — fed by real-world data. Moving far beyond a typical static model, a digital twin provides a live, evolving digital model of a real system that you can test against, experiment on, and optimise without touching the real thing. This means that instead of asking “what is likely to happen?” you can ask “what happens if I change this in the real system?”, shifting from prediction to experimentation.

Applications of the digital twin concept are already emerging in manufacturing and industrial systems (simulating production changes before implementing them) and urban planning (simulating effects of new housing developments or managing congestion manually). At a more advanced frontier, digital twins could be used in a healthcare context to simulate how a patient responds to treatment, test the effects of drugs before prescribing them, or even to plan surgeries using personalised anatomical models.

For now, the concept is promising but the practical frameworks, standards, and widespread adoption are still evolving.

Real-time responsiveness as a new capability

A key implication of faster data science workflows is the move toward real-time or near real-time responsiveness. Where analysis once took days or weeks, improved tooling and reusable components now allow teams to identify and respond to new patterns much more quickly. For example, emerging fraud patterns can be detected and addressed without building solutions from scratch.

As a result, latency — the delay between observing and acting — becomes increasingly important, and iteration cycles become more continuous. Rather than producing insights at fixed intervals, data science is more closely embedded in ongoing decision-making.

Overall, this reflects a shift from batch-style analysis to more responsive systems, where value depends on how quickly insights can be turned into action.

An inflection point

Hearing what our panel is most excited about at the moment makes clear that data science is at an inflection point. Technically, the field is accelerating rapidly, driven by advances in tooling and AI. In practical terms, there is a clear shift toward reusability, greater speed, and more real-time application of methods and models. At the same time, there are increasingly urgent social considerations, particularly around ethics, communication, and governance.

Taken together, these trends point to a broader change in where value lies. The most significant gains are unlikely to come solely from developing better algorithms, but from how effectively those algorithms and methods are reused, communicated, and deployed in a responsible way.

We want to hear what you’re most excited about, so get in touch.

Interview

Key Takeaways at a Glance

Join the conversation

This conversation touches on a theme we often explore here at Real World Data Science: the idea that the future of data science and AI will not be defined by technical capability alone, but by how well we integrate human judgment, context, and responsibility into our systems. The position paper—and RSS: Data Science and AI more broadly—is an open invitation to engage with these questions. Whether through research, case studies, or reflections from practice, there is a clear call for contributions that connect technical work with real-world impact.

As Neil suggests, the answers are unlikely to come from any single discipline or organisation. They will emerge from a broader conversation across the data science community.

Now is the time to be part of that conversation: answer RSS: Data Science and AI’s call for submissions.

Watch the discussion

Takeaways at a glance

• AI is built on statistical thinking – even when it’s not labelled that way.
• Job titles change; core skills don’t.
• Statistics sometimes undersells itself by focusing on mechanics over impact.
• Communication and visualisation are central, not peripheral, to modern statistical work.
• Kindness, collaboration, and trust are professional assets.
• The future belongs to skill-based identities, not title-based ones.

Key themes and analysis

The “rebranding” meme

The panel opens with a familiar joke: take statistics, put a new frame around it, call it machine learning or AI, and suddenly everyone pays attention. It’s humorous—but revealing.

Many roles advertised today as “AI” or “data science” positions are deeply statistical at heart. They involve modelling uncertainty, validating assumptions, managing bias, evaluating performance, and interpreting results. In other words: core statistical competencies.

Rather than resisting this relabelling, the panel suggests recognising it as part of the natural evolution of fields. The key question becomes not “What should we call ourselves?” but “What value are we delivering?”

Identity versus skills

One of the strongest messages from the discussion is this: don’t over-identify with a job title.

“Statistician,” “data scientist,” “AI specialist” are all potentially transient labels, whereas the skills underpinning them remain the same:

• Framing problems carefully
• Questioning assumptions (“Are you sure? Are you sure-sure-sure?”)
• Quantifying uncertainty
• Designing analyses that are robust and defensible

The panel suggests that the healthiest professional stance is to focus less on identity and more on what you can do and what you care about.

The communication gap: loving the sausage-making

Statisticians, the panel observes, sometimes make things harder than they need to be—at least in how they explain their work.

“We’re too interested in the mechanics,” one panellist notes. “Nobody cares how you made the sausage.”

This doesn’t mean rigour is unimportant. It means that impact must lead the narrative. Instead of focusing first on models, methods, and diagnostics, statisticians might begin with:

• What problem was solved?
• How did this make life easier, safer, or better?
• What decision did this enable?

AI has been marketed effectively because it is framed in terms of transformation and possibility. Statistics can claim that space too, without sacrificing integrity.

Visualisation and bringing data to life

Visualisation is a key bridge between statistical thinking and real-world impact. Good visualisation:

• Makes uncertainty legible
• Builds trust
• Enables decision-making
• Tells stories grounded in evidence

In a world flooded with dashboards and generative outputs, the ability to present data clearly and responsibly is not a soft skill. It is core infrastructure.

Trust, collaboration, and professional culture

People want to work with statisticians they trust, which flows not only from technical competence but from clarity, openness, and collaboration.

As AI systems become more powerful—and more controversial—the professionals who can explain, contextualise, and responsibly deploy them will be in

From background discipline to visible foundation

If AI continues to evolve—as it surely will—so too will the labels attached to those who work in it. But uncertainty, inference, modelling, and critical thinking aren’t going anywhere.

We would love to receive contributions to the site that tackle this issue.

Is statistics undervalued in the AI conversation, or quietly thriving?

Where, in your experience, does statistical thinking most visibly shape AI work?

And where is it least acknowledged?

Have you seen statistical work rebranded as AI in your organisation?

We are actively seeking submissions on these topics so, if you would like to be part of the conversation, get in touch.

Start with What We Are Trying to Achieve

Too often, data science projects begin with vague aims such as “build a model” or “forecast sales.” These are activities, not outcomes. What matters is the result the organisation is striving for—for example, increasing upsell revenue by £2M this quarter or preventing 500 contract cancellations per month through timely intervention. Asking the right questions early is essential for designing solutions that can actually be implemented. For instance, a retention model might flag 1,000 customers at high risk of leaving, but if capacity allows only 50 calls per week, the key question becomes: which 50 should be prioritised, and does contacting them actually improve retention compared to a control group?

Before writing a single line of code, it is essential to gather as much context as possible:

• What problems is the business actually solving?
• How does the model fit into the wider business process?
• Who will use the outputs, and what actions will follow?
• How will success be measured—commercially, operationally, behaviourally?
• What trade-offs are acceptable in cost, complexity, or speed?
• How will performance be monitored over time?
• What are the operational constraints?

Once the essentials are understood (to the extent they can be), the vision for the project and the success metrics must be agreed collectively. All key stakeholders—technical, operational, financial, and strategic—need to be involved in defining what success looks like. Without this shared vision, each group risks optimising for its own priorities rather than the organisation’s overall goals. Crucially, the vision should extend beyond performance metrics: it should tell the story of the problem being solved and what success will mean in practice. This shared narrative becomes the project’s guiding star. To keep it on course, data science and AI teams, working with stakeholders, must guard against scope creep and shifting success criteria, ensuring that any new requests fit within the agreed scope. Flexibility still has a place—experimentation and design changes are healthy—but only when they remain consistent with the original vision and are aligning to achieving the success metrics.

The Power of Test-and-Learn

Evaluation and monitoring must be built in from the beginning. Doing so ensures that systems are designed to capture the right metrics for monitoring, rather than scrambling to measure impact after the fact. This means defining not only technical performance measures but also organisational impact measures, all aligned to clear, measurable success metrics. These metrics should be developed collaboratively with stakeholders, and while data scientists may not set them alone, they play a critical role in shaping and challenging them where needed.

A test-and-learn approach is particularly powerful because it generates direct evidence of what works under real-world conditions. For example, a simple test-and-control design splitting customers into two groups, one acted on and one left as business-as-usual, provides incremental evidence of benefit that is far more persuasive than retrospective accuracy scores. Unlike abstract metrics, this method shows whether interventions truly drive the desired outcomes, and it allows organisations to learn, adapt, and refine strategies over time.

Ultimately, evaluation is about measuring decision performance in practice, while monitoring ensures that impact remains robust as circumstances evolve.

Map the current business process end to end, noting all user interactions and data collection points. Then overlay where the model will integrate into that process— the inputs into the model pipeline, who receives the model outputs, how they are acted on, and how outcomes flow back into the system. This makes clear both the operational impact of the model and what changes are needed for it to deliver value.

Design for Value, Not Novelty

Data science is not about building impressive models for their own sake. It is about solving valuable problems in ways that make business sense.

If a model improves accuracy by two percent but costs ten times more to run, is it worth it? The answer depends on whether those extra points translate into measurable financial impact.

Ask:

• Could a simpler model deliver “good enough” accuracy at lower cost?
• What is the marginal value of added complexity?
• Does the design reflect operational constraints?

Here, the product mindset for data science and AI practitioners becomes critical. Treating an AI solution as a product reframes the goal from “building a model” to “delivering value.” Like any product, an AI system has costs to design, build, deploy, and maintain. Its worth lies not in technical elegance but in whether the return justifies those costs. That means asking early: is the investment worth it?

One practical way to answer that question is by forecasting scenarios. Before scaling, estimate the expected impact under different conditions: a base case, a best case, and a worst case. For example, in a retention project, you might forecast incremental revenue by combining churn rates, average customer value, intervention costs, and expected uplift. This makes assumptions explicit and gives decision-makers a clear view of risk and upside. A solution is rarely a guaranteed win, but scenario planning allows stakeholders to judge whether the likely outcomes justify the investment.

Build for Adoption

Adoption must be planned from the start. Trust develops gradually, and regular check-ins with stakeholders help sustain it by keeping the project aligned with its agreed vision. These sessions are not box-ticking exercises but chances to test assumptions, surface blockers, have continuous feedback and make timely adjustments. Ultimately, a model succeeds only if people use it — so adoption depends on seamless integration into existing processes while delivering something users can see a clear benefit from.

Instead of starting with purely technical questions—such as “will I need to export this to a CSV?”—it is often more effective to begin by considering the user journey. For example, if the end goal is for users to view the results in a dashboard, that should frame the discussion from the outset. Once the user’s needs are clear, the practitioner can then work with the data engineering team to determine the most appropriate technical solution, such as the optimal data format or storage approach.

Hence it is important to ask early:

• Where will predictions appear (CRM, dashboard, alert)?
• Will outputs be delivered in tools people already use?
• What training or support is required?
• How will impact be made visible to leadership?
• How best should the outputs of the model be presented to ensure they are usable and actionable for the next stage of the business process.

Thinking about these questions early prevents the familiar fate of a technically brilliant model that sits idle.

Adoption is strongest when development is iterative. Rather than disappearing into a three-month build, teams should work in cycles: release a minimum viable product (MVP), test it with users, gather feedback, and refine. The first iteration of the MVP should be the simplest form of the product while testing the core principle of what is trying to be achieved. An MVP could be as simple as a weekly spreadsheet with a risk score; if it proves valuable, the team can then invest in automation, dashboards, or more advanced models. This staged approach reduces risk, delivers value early, and builds trust among stakeholders. Crucially, reaching an MVP quickly lets both technical and business teams see what works—and what doesn’t—in practice, instead of relying on endless planning meetings where edge cases are difficult to anticipate.

Communication is critical. Just as one study on doctor–patient interactions found that 91% of patients preferred doctors who avoided jargon [1], stakeholders respond more positively when practitioners present results in plain language. Clear explanations build understanding, and understanding builds trust. It is also important to explain, in accessible terms, how a model or tool works “under the hood,” so users can better grasp how decisions are being made. Adoption can be further strengthened by having champions within the business—trusted and respected leaders in the business area who engage end users, promote new tools, and support day-to-day use through training and guidance.

The Bottom Line

Great models rarely fail because of poor algorithms; they fail because they are disconnected from the goals, workflows, strategies, and people they are meant to serve.

To avoid the fate of the Vanishing Model, projects must begin with a clear vision — one that is co-created with stakeholders and sustained through regular check-ins. Frame every project around measurable business outcomes and define success before writing a single line of code.

Prove value under real-world conditions with well designed and measurable evaluation plans such as test-and-control approaches. Weigh technical ambition against practical trade-offs—cost, complexity, deployment speed, and maintainability. Translate precision, recall, and ROC curves into outcomes the business understands: contracts retained, revenue gained, costs reduced. And above all, plan for adoption from day one, so that predictions are not just accurate but usable, trusted, and embedded in daily decisions.

In the end, the mark of a great model is not the elegance of its algorithm but its ability to have a positive impact.

For a broader, strategic view of why organisations struggle to realise value from AI—and how leadership and structure can change the odds—check out “Why 95% of AI Projects Fail.”

Sources: [1]Allen, K. A., Charpentier, V., Hendrickson, M. A., Kessler, M., Gotlieb, R., Marmet, J., Hause, E., Praska, C., Lunos, S., & Pitt, M. B. (2023). Jargon Be Gone – Patient Preference in Doctor Communication. Journal of Patient Experience, 10, Article 23743735231158942. DOI: 10.1177/23743735231158942.

Watch the discussion

Takeaways at a glance

• Innovation and regulation are not opposites – both are essential, but difficult to balance.
• Responsible progress requires proportionality – not all AI applications carry the same level of risk.
• Transparency enables better governance – open dialogue between developers and regulators is key.
• Risk-based frameworks provide structure – distinguishing low-, high-, and unacceptable-risk uses helps focus oversight.
• Global disparities complicate regulation – some regions are regulating advanced AI systems, while others are still building foundational capacity.
• Innovation needs protected space – experimentation, iteration, and even failure are critical before formal standardisation.

Key themes and analysis

Proportional regulation through risk

Not all AI systems pose the same level of harm. A risk-based approach — distinguishing low-, high-, and unacceptable-risk uses — offers a practical middle ground. It avoids blanket restrictions while ensuring stronger oversight where impact is greatest. The debate becomes less about whether to regulate, and more about how proportionate that regulation should be.

Transparency as common ground

Openness can bridge the gap between technologists and regulators. Clear communication about capabilities, limitations and risks enables more informed policy decisions. When innovation happens transparently and in dialogue with regulators, governance can evolve alongside technology rather than lagging behind it.

The global unevenness of AI governance

AI regulation is developing unevenly across regions. While parts of the West are formalising frameworks, many countries are still building foundational AI capacity. This raises difficult questions about sequencing: should regulation lead innovation, or follow it? A one-size-fits-all model may not reflect global realities.

Protecting space to experiment

Innovation requires room to test, iterate and occasionally fail. Early experimentation should not be overburdened with rigid controls — but successful, scalable systems must eventually transition into more standardised and regulated environments. The challenge is designing pathways that support both creativity and accountability.

Looking ahead

As AI continues to evolve, the balance between innovation and regulation will remain dynamic — and contested. This conversation opens up important questions, and we would love to hear our readers’ thoughts about how we move some of the principles mentioned in the video into practice.

• How do we facilitate transparent channels of communication between those developing AI and those designing the regulatory frameworks that will govern it?
• What should determine whether an AI system is low, high, or unacceptable risk?
• How do we define a “safe speed” for AI development — and who gets to decide?

We are actively seeking submissions on these topics so, if you would like to be part of the conversation, get in touch.

The Critical Issue of Algorithmic Bias in Credit Scoring Models

One of the most concerning aspects of algorithmic bias is the limited recourse available to those negatively impacted, leaving them vulnerable to opaque decision-making processes. This challenge underscores the need for increased transparency in how credit scoring models are developed and deployed. Ultimately, proactive mitigation strategies are essential to ensure fairness and equity in financial outcomes.

One of the biggest issues surrounding algorithmic bias in credit scoring is that adversely affected parties usually have little or no recourse for appealing unfavorable decisions. This problem happens because most widely used algorithms still can’t explain how they reached specific decisions, leaving people in the dark and forcing them to trust the technology, even as it potentially ruins lives.

A November 2025 academic review revealed numerous flaws in financial algorithms and confirmed various impacts. They included systematic disadvantages for minority groups and miscalibrated credit scores for individual borrowers. The researchers also discovered that these issues appeared despite the financial technology industry’s promises of superior efficiency.

One of the cited studies consistently showed that female applicants received credit scores six to eight points lower than their male counterparts. The researchers determined that the associated effects diminished economic welfare and that the ramifications continued for multiple borrowing cycles. Another investigation revealed persistent disparities across minority groups, despite the applicant’s chosen lender type.

Elsewhere, researchers examined the effects of using large language models to evaluate applicants’ loan data. This approach regularly recommended charging higher interest rates to Black applicants or denying their applications. It did not make the same suggestions for identical white applicants. These examples demonstrate why data science professionals must remain constantly aware of the potential for bias and uphold fairness by reducing the issue whenever possible.

Practical Tips for Bias Detection and Mitigation

Sources of bias in credit scoring data and algorithms are more common than you might think. They can include:

One straightforward way to identify bias in training data is to be aware of the most common types and build algorithms to be less reliant on them when possible. Regularly reviewing the training data is similarly effective because it can catch biases before they have real-life effects.

You can also perform a disparate impact analysis to mitigate bias, which compares aggregate measurements attributed to fewer privileges or less representation to their counterparts. Dividing the proportion of one group that received an adverse outcome and comparing it with that of the other group identifies bias.

Creating a set of fairness metrics is another practical mitigation approach, as it encourages data scientists to understand the impacts of various aspects of a person’s background that they can and cannot control. The examined attributes could include someone’s employment history, income and debt, but also the extent to which they experienced equal opportunities.

Improvements in explainable artificial intelligence will further both bias detection and mitigation. Developers then see the heavily weighted but irrelevant factors that could lead to unfair outcomes and correct issues early.

Considerations to Reduce Algorithmic Bias in Credit Scoring

Reducing algorithmic bias in credit scoring requires cooperation across roles and departments and includes those who will use the tools containing the algorithms. Committing to specific postprocessing steps empowers people to become more familiar with an algorithm’s functionality and potential shortcomings rather than automatically trusting the results. Those developing the algorithms should prioritize transparency by designing explainable models when possible and making them accessible enough for the expected audience.

Staying abreast of recent research shapes data scientists’ efforts by helping them understand the possibilities. In a 2024 case study, MIT researchers created a new technique that identifies and eliminates the specific attributes of training data that are the strongest contributors to a model’s biases about minority subgroups. This approach also preserves overall accuracy because it preserves more of the data compared to other options.

The developers confirmed that the technique can find hidden bias sources and training datasets with unlabeled information. This capability is significant because the data used by many applications lacks labels. They envision combining their technique with other approaches to improve fairness in high-stakes situations. This detail makes it well-suited for the financial industry because many of the associated decisions alter people’s lives and opportunities.

Maintaining responsible data science practices requires equipping professionals with the skills to detect and mitigate bias in an evolving technological landscape. A 2025 study involved a biased dataset that contained a disproportionately high number of white people with happy faces. This issue caused the AI algorithm to correlate race with emotional expressions.

The results of three experiments with human participants showed that most individuals did not notice the bias. This result shows why data scientists need ongoing education to spot less-obvious examples.

Stay Vigilant to Maintain Fairness

Your work on algorithms for credit scoring could adversely affect people’s lives and leave them with no way to contest unfavorable outcomes. Being a responsible data scientist means understanding the numerous risk factors and the controllable factors to minimize harm. Remaining aware of emerging AI applications in the financial industry and regularly meeting with colleagues to discuss ways forward increases fairness for everyone.

From Governance Principles to Technical Work

Colorado’s law followed a pattern increasingly visible in global AI governance, particularly the European Union’s AI Act. These frameworks share a risk-based logic in that systems that influence consequential decisions face higher expectations for transparency, oversight, and accountability.

At a high level, these expectations (fairness, consumer protection, responsible use) sound abstract. In practice, they translate directly into technical work:

• Clear documentation of model purpose, training data, and limitations
• Records showing where data comes from and how it changes over time
• Reproducible experiments and versioned artifacts
• Ongoing monitoring for performance drift and unintended impacts
• Defined processes for human review and intervention

None of this lives in legislation. It lives in scripts, workflows, dashboards, deployment systems, and operational infrastructure.

Colorado’s stalled implementation of AI policy surfaced a familiar pattern, as many organizations are well equipped to optimize model performance, but far less prepared to operationalize accountability at scale. The friction emerged not because governance goals were controversial, but because the supporting technical infrastructure was uneven.

Why Uncertainty Becomes a Design Risk

One challenge Colorado encountered was definitional ambiguity. For example, what qualifies as “high risk”, what safeguards are sufficient, and how should harms should be assessed? These questions are not merely legal - they are technical and context dependent.

Different data sources, deployment approaches, and users lead to different answers. For teams building data systems today, that uncertainty creates risk. When teams cannot easily see how data moves through a system, how models change over time, or how decisions are produced, adapting later becomes costly and disruptive.

Recent federal signals add another layer of complexity. President Trump’s executive order discouraging state-level AI regulation aims to reduce fragmented policy on AI, but it does not replace state experimentation with a concrete national policy. Teams now operate in a moving landscape shaped by state initiatives, evolving federal priorities, and international regimes like the EU AI Act. In this environment, aiming for minimal compliance is risky. Teams are better served by designing systems that are flexible and easy to observe from the start.

Responsibility Does Not End at Deployment

A lesson emerging from both policy debates and practice is that accountability does not stop when a model goes live. Responsibility shifts across teams over time, from data scientists to engineers, product owners, operators, and decision-makers.

This challenge is the focus of the Responsible Handover of AI framework developed by Sense about Science, which emphasizes the need for clear transitions of responsibility as AI systems move from development into real-world use. Rather than treating deployment as a handoff to “the business”, the framework highlights the risks that arise when assumptions, limitations, and responsibilities are not carried forward with the system.

For practitioners, this framing maps governance concerns onto familiar operational questions, such as who monitors systems after deployment, which development assumptions still matter in production, how limitations are communicated to users, and what happens when systems are updated or handed over to new teams.

Without explicit handover practices, accountability gaps emerge because responsibility becomes diffuse as systems evolve. From this perspective, many regulatory requirements are not adding entirely new work, rather they formalize practices teams already rely on. This includes documentation that travels with systems, monitoring in production, and clear escalation paths when something goes wrong.

Practical Steps Teams Can Take Now

Regardless of how U.S. and international regulation ultimately settles, many investments pay off immediately while reducing future risk, including:

• Standardizing documentation. Ensure model summaries and data descriptions travel with systems as they move between teams
• Build end-to-end visibility. Version datasets, features, models, and configurations so results can be reproduced
• Instrument monitoring early. Track input drift, unstable predictions, performance decay, and downstream impacts once systems are in production
• Clarify governance workflows. Define who approves releases, who monitors systems, and how responsibility shifts over time
• Translate risk for leadership. Gaps in documentation and visibility tend to come back later as messy, expensive fixes; addressing them early saves time and pain

These practices are not limited to machine learning. Any system that informs decisions can create similar accountability challenges.

Governance Lives in the Data Stack

There’s still no settled agreement on how AI should be governed. But for people building real-world data systems, its implications are already concrete. Accountability increasingly lives in the data stack in how workflows are instrumented, how models are monitored, and how decisions can be examined after the fact.

This is not simply about regulatory compliance. It is about building systems that are transparent, resilient, and trustworthy at scale. Organizations that treat governance as a core technical problem (rather than an external policy constraint imposed later) will be best positioned to navigate whatever regulatory balance ultimately emerges.

Interview: Beyond Quantification and Uncertainty in AI

Key Takeaways at a Glance

Join the conversation

RSS: Data Science and Artificial Intelligence has an open call for submissions responding to the paper.

Sylvie Delacroix’s work is a call to action for data scientists, designers, and professionals alike. We have a window of opportunity to shape AI systems that encourage humans to keep re-articulating the values they care about.

We want to hear from you. As AI tools become more integrated into high-stakes professions, how can we ensure that systems support human judgment in all its facets rather than simply optimising for efficiency?

Read the full paper here, or our accessible digest here, and join the conversation about building AI tools that truly serve people, not just processes.

Current State of Play with Generative AI

While data analyses based on text data such as NLP (natural language processing) have been around for a while, a new type of approach has quickly become widespread with the rapid rise of generative AI methods. These models, known as LLMs (large language models), have entered the everyday vernacular as all organizations grapple with how to use these tools. With the debut of ChatGPT in 2023, these LLMs have become increasingly sophisticated, trained on a wider variety of data sources and optimized for a variety of different scenarios.

The accuracy of these models depends on the quality of the training data used. While LLMs are known to hallucinate or give inaccurate answers, they tend to perform better in situations where the training data is precise and exact, with less nuance than language often carries. As a result, LLMs can produce large amounts of computer code based on large amounts of training data based on accurate computer code. Some specialized LLMs (e.g. Claude Code) have been specifically designed to generate accurate code that can access, clean, combine, analyze and visualize data. While LLMs are not perfect in code generation, they can increase the efficiency of an experienced coder.

As a result of these LLMs, less knowledge is required to analyze and work with large datasets. A user can provide a specific prompt on the business question or research objective, upload the relevant data and have an LLM provide a relevant data analysis, complete with the underlying code used to generate that analysis.

The “New” Data Scientist

As LLMs improve in their ability to quickly generate vast amounts of accurate code, what does that mean for a discipline which has prided itself on its data wrangling skills? As Davenport and Patil noted “data scientists’ most basic, universal skill is the ability to write code” (1). Data science has seen coding as a viable career path.

When a skill becomes accessible to a wider variety of people and can be automated, how does one distinguish themselves in an organization? When coding can be done by AI tools, what happens to those who are known for their coding abilities? For a discipline to be recognized as a discipline, it must have some distinguishing characteristic that defines it as different from other disciplines. In addition, disciplines become more valuable and prominent as their contribution to society grows.

So, what are the skills that data scientists have that can’t be done well by AI? While AI capabilities are rapidly increasing, we do believe there are things that may be beyond the reach of AI for the time being.

We believe that the greatest factor limiting AI is data quality. There seems to be a growing consensus of the importance of data quality as noted in Davenport, Hoerl, and Redman (3), Davenport and Tiwari (4), and Redman (5) — as well as recent Real World Data Science pieces. As the adage goes, “garbage in, garbage out”. With poor, inaccurate data used in training, the resulting AI output will also be poor and inaccurate. A related fear is that, as AI models start to use AI generated content as a data source, a recursive loop happens that degrades the quality of any AI output (6).

Discussions of data quality are limited in books, university courses and training programs. When they do occur, they are restricted to the question of “are the data right?” and discussions of data cleaning. Data cleaning is often focused on eliminating outliers or invalid points. While there is often a reason to remove invalid points, those outliers can sometimes be the source of valuable insights.

However, data quality is much more than data cleaning or checking to make sure the data are accurate. There is an element of contextual understanding and process knowledge that enables the data scientist to properly prepare the data for analysis. We are skeptical of AI’s ability to fully understand context and the nuances of assumptions that go into data analysis. In an earlier piece, Jensen provided some examples of the limitations of AI when it comes to proper data cleaning (7). For any set of data, subject matter knowledge of how the data were collected and what they represent is crucial to a proper analysis.

This creates an opportunity for data scientists to become more valuable. By employing probing questions to better understand the context of the data, they will be in a better position to identify data quality issues and ways to improve the data quality, thus leading to better model output.

Conclusion

As coding becomes easier in an AI-enabled world - where anyone can code and analyze data - the skill set of a data scientist becomes less unique. Data scientists were once in high demand because they set themselves apart as coding wizards who could wrangle large datasets and extract insights. To remain successful and continue to deliver value, data scientists must now pivot their skillset. The real limiting factor in successful data science is data quality. A renewed focus on owning, improving and governing data quality will not only strengthen outcomes but also provide future job security and increase the value data scientists bring to organisations.

Note that this article is based on the following paper and contains some of the same ideas: Hoerl, Roger W. 2025. “The Future of Statistics in an AI Era.” Quality Engineering, published September 10, 2025.

You can find out more about the Real World Data Science stance on data quality from our article ‘Why We Should All Be Data Quality Detectives’.

Video: What Are the Key Challenges Facing Data Scientists Today?

The Patterns Behind the Problems

Although the challenges raised span technical, organisational, and personal domains, they are connected by a small number of deeper themes that shape modern data science.

Looking Ahead

Taken together, these themes suggest that the biggest challenges in data science are not isolated problems to be solved individually. They are interconnected tensions between speed and rigour, access and expertise, innovation and organisational inertia.

Addressing them requires interdisciplinary, systems-level thinking.

Which of these challenges resonates most with your own experience in data science? How can practitioners use these tensions as inflection points to actively shape the field, rather than simply react to it?

Three Reasons for Failure

There are three important reasons why so many projects fail that are often overlooked in the literature. First, the problem is often mis-specified: it is framed by technologists or vendors rather than co-owned by the domain experts who understand the decision and bear the consequences. Second, leadership expectations are frequently misaligned, short time horizons and demands for certainty collide with a technology that improves through iteration and organisational learning. Third, many deployments are brittle: they assume stability in a domain defined by rapid model change and rising user expectations, when what is needed is an engineered system designed to adapt.

This article draws on two decades of work building AI systems for medicine discovery at GSK (7) and at Tangram (8) to argue that success rests on three principles, in roughly this order:

• integrated subject matter expertise;
• patient and informed executive leadership;
• and building AI systems that learn with the organisation.

The sections that follow develop each element in the context of drug discovery and show how an AI platform can help move real projects into the small minority that deliver value.

Essential Contexts Outside Our Focus

Before turning to the three principles developed below, it is worth acknowledging four adjacent domains that I will not treat in detail here, each of which now has a substantial literature of its own. First, organisational scholars have long shown that new information systems reshape power, status and discretion, making the politics of implementation as important as the technology itself (9); (10). Second, governance, regulation and responsible AI practice which includes everything from model documentation and auditability to privacy, robustness and safety, have become central determinants of what can be deployed in practice, especially in regulated sectors (11); (12); (13). Third, there is an emerging body of work on workforce transformation: how AI complements or displaces skills, how hybrid human–AI roles are designed, and how training, trust and professional bodies mediate adoption (2); (3); (4); (5). Finally, the question of how to measure value and learn at portfolio scale with existing legacy IT systems (through experimentation, counterfactuals and disciplined comparisons between use cases) is itself a rich field that extends well beyond any single organisation. (3); (4); (5); (6); (14). Each of these strands is critical to understanding why AI succeeds, stalls or remains at a proof-of-concept level.

To maintain focus, I concentrate on three overlooked questions arising from direct experience: how to organise subject matter expertise such that the enterprise owns its AI; how to cultivate genuine leadership ownership; and how to engineer systems that learn and adapt rather than remain isolated demonstrations.

Principle 1: The importance of building with subject matter experts

The hardest part of building an AI platform is not the models or the engineers but assembling the subject matter experts (SMEs) who will frame and judge the work. Most commentary treats SMEs as validators, brought in at the end to bless a prototype. It rarely explains how to organise a molecular biologist, a clinician and a chemist so that they can state, in plain terms, what counts as an acceptable outcome.

Most commentators are not operators. They observe patterns across organisations but do not live with the consequences of poor SME integration. This distance between writing and practice shows up in the surveys. Foundry’s 2024 State of the CIO, summarised in MIT Sloan Management Review, reports that 85% of IT leaders see the CIO role as a driver of change, yet only 28% list leading transformation as their top priority (15)]. The people commenting on AI often sit with strategy decks rather than with the unglamorous work of managing technological change and cross-functional coordination.

Drug discovery starkly exposes the gap. The relevant team is wide and requires exceptional coordination. Business and portfolio leaders understand how projects absorb capital and create value whereas molecular biologists and geneticists judge whether a gene is plausibly causal for a disease. Clinicians think through trial design and patient risk. Chemists know what can be made and delivered. Statisticians, AI engineers and data scientists understand models, data pipelines, experimental design and evaluation. This diversity is a strength but requires a lot more from leaders of such teams. When these groups work as separate silos, the result is a generic set of tools whose outputs are not trusted by the users and whose inputs are irrelevant. When these experts can operate as a single team, the conversation starts with a simple set of questions. Which decisions are we trying to improve? How will we know if we have succeeded? What data and statistical methods count as acceptable evidence? Which risks are we prepared to take and which are not negotiable?

At Tangram, that joint framing often collapses into one critical choice: which disease do we want to target for drug development, and which gene is driving it? That decision already embeds genetics, hepatocyte biology, chemistry, clinical feasibility and commercial context. The role of AI and engineering is then precise. It is to help the group search the vast hypothesis space, structure the evidence and quantify uncertainty, while leaving the final judgement with experts who feel they own the AI platform, can see why the AI has come to the conclusions it has, and also own the consequences.

Principle 2: Patient and strategic executive leadership.

The second element is executive patience. Research from MIT Sloan shows that firms gaining value from AI tend to run more projects, over more years, with a sustained focus on learning how people and AI work together. (3) Leaders in these organisations accept that early returns are small and uneven. They invest in a pipeline of use cases rather than a single bet. They resist what researchers have called the “last mile problem”: AI projects that reach technical proof of concept but never change how work is done. (14) In life sciences this is acute. Discovery timelines are long, data are messy and early signals are faint. Leaders who expect quick, clean returns tend to cycle through pilots without ever building an asset that scientists trust.

Patience does not mean passivity; actually, the opposite is true. It means choosing a small number of important decisions, funding cross-functional teams to attack them, and holding the bar for quality high. It requires senior sponsorship to unblock data access, align incentives across discovery, clinical and commercial groups, and shield long term work from quarterly fashion cycles. When those conditions are in place, AI stops being a sequence of demonstrations and starts to become part of how the organisation thinks: the informed leader knows the difference.

Principle 3: Building AI Systems that learn with the organisation

The third element is the AI engineering. The MIT findings on the 95 percent figure are instructive: most generative AI projects fail not because the models are weak but because the systems around them are brittle. (1); (11) Foundation models are dropped into existing workflows with minimal adaptation. There is limited monitoring. Data quality is assumed rather than measured. When something breaks (as it inevitably will in non-deterministic systems) teams revert to manual work.

Modern AI engineering starts from the opposite assumption. Models and tools will change quickly. The surrounding stack must absorb that change without being rebuilt each time. The strategy must be built so that, when the product director hears a new technology is built or an LLM improved, this is always a good day.

Build only what you must.

The sensible principle is to only build the components where your domain expertise creates defensible value. Everything else should be bought. But buying is not effortless. Integration, monitoring, and vendor management drains teams unless you are staffed for it. Organisations that succeed at scale partner with vendors offering systems that learn and adapt; they focus on workflow integration; they deploy tools where process alignment is easiest.

So assuming you have the right staff who are working together, supportive leaders and good vendor relationships the next problem is how to create a platform itself.

A useful pattern for a modern AI platform has four parts:

• First, the data stack. Discovery teams need a small number of trusted stores for data with clear provenance and at least basic quality checks. For early target selection this means human genetics, expression data, interaction networks, preclinical phenotypes, clinical outcomes and internal experiments. Traceability and reproducibility is critical here. Any claim a model makes about a gene and disease pair should be traceable back to specific pieces of evidence.
• Second, the platform needs to be built on modular services rather than monoliths. Each service has a single responsibility and can be swapped when a better service appears. This keeps the cost of change low and allows teams to combine external tools with internal components in a controlled way.
• Third, the system needs to have continuous evaluation. Every component that answers questions is tested on held-out tasks, with simple metrics for accuracy, faithfulness, and recall, and monitored continually. There should be repeat measures and other tests of robustness. (16) There is no reason not to report error bars in AI and yet they are rarely part of AI publications. Where this matters most is at the interface with non-determinism, inherent in large language models. A good medical AI assistant should give consistent answers even when questions are phrased differently. It should also say it does not know when the information is unclear or incomplete. (16); (17); (18)
• Fourth, include memory and reinforcement learning so that the system learns. This is the most difficult component to implement and the one most often deferred. A system that cannot learn from use will make the same mistake repeatedly. Even the most patient users will lose trust and patience. But building memory into production systems, where the model retains context across sessions and improves from feedback, requires specialist expertise in reinforcement learning, retrieval-augmented generation with persistent stores, and the infrastructure to support online learning without catastrophic forgetting (19). These skills are in high demand and short supply. The alternative is a system that feels potentially useful in demonstrations but frustrates users in daily work.

For this to work, engineering teams need to stay in constant contact with biologists, geneticists, clinicians, chemists and portfolio managers. Together they decide what error rate is acceptable for a triage tool, what form of uncertainty estimate a portfolio board will respect and where human review is mandatory, for example before a new target enters serious preclinical work. Work on human–AI interaction design reinforces this point: systems should explain what they can and cannot do, expose their confidence and make it easy for users to correct them. The hardest part is that the first version is almost never right. Cross-functional teams need patience and ownership. They contribute real examples, refine prompts and evaluation sets, and expect the system to learn from its mistakes. The AI platform must be useful enough, early enough, that experts are willing to spend scarce attention improving it.

A worked example: target-indication pairing in siRNA

At Tangram we built LLibra OS, an internal system designed to surface and assess new small interfering RNA targets (8). siRNA refers to short double-stranded RNA molecules that can silence a specific gene via the RNA interference pathway. The purpose is narrow: the AI needs to help scientists identify medicines worth taking forward.

In early discovery, hypothesis generation often reduces to a single question: which target and disease pair should we move into the siRNA pipeline? The question sounds simple. It is not.

A purely data-driven approach can surface millions of candidates. Genome-wide association, expression atlases, and protein interaction networks will produce statistical associations at scale. But association is not mechanism. Two things can correlate because they share upstream causes, because they sit in the same pathway without being rate-limiting, or because of confounding in the data.

Plausibility requires a different kind of evidence. If we modulate this target, what functional change should we observe at the cellular or tissue level? Does that functional phenotype connect credibly to the disease we care about? For our purposes, the chain of reasoning must pass through liver biology: does knockdown of this gene alter a measurable secretory or metabolic function, and does that function relate to the clinical phenotype we wish to treat? Is there a real unmet need for your research for patients? (20)

The AI assists in answering these questions. It helps the team hold multiple threads of conditional evidence in view simultaneously. It retrieves, reasons and summarises over tens of millions of papers in the literature, joins and flags inconsistencies between thousands of data sources and quantifies uncertainty where the evidence is thin. Whilst the AI platform does the work of thousands of researchers and continually learns on the job, the expert judgement remains with the SMEs. The SME is central and is given all the reasons why and how the AI found a piece of evidence. The AI structures and expands the space in which that judgement operates. When it works well the AI platform uncovers the non-obvious connections that researchers may never have found.

Conclusions

Seen from this angle, the 95 percent figure is not a verdict on AI technology but a statistic about organisational design: how rarely good questions, high quality data, diverse experts and committed leaders are brought together at the same time. The systems described in this essay matter, but they are secondary. The primary determinant of value is whether biologists, clinicians, chemists, data scientists and portfolio leaders sit together, own the same objectives and are backed by executives willing to invest over years rather than quarters. Where that integrated team is absent, even elegant architectures will fail. Where they are present, imperfect tools still move the needle.

Much commentary on AI spends its energy on model choice, technical detail and tooling. This article has argued that the more important work is organisational: deciding which decisions to improve, agreeing what counts as acceptable evidence, and creating cross-functional teams that can live with the consequences. The few organisations that succeed treat AI as an experiment in decision making rather than a procurement exercise. They expect the stack to change and the vendors to turn over, but they hold fast to the team, the questions and the discipline.

For Real World Data Science readers, the implication is direct. AI projects fail when nobody owns the estimand, the counterfactual and the error bars. AI doesn’t need to be perfect, but it needs to be good enough. As the statistician George E. P. Box famously observed: “All models are wrong, but some are useful”. Usefulness here depends on design, discipline and humility as much as model choice. Statisticians, data scientists and methodologists can reclaim the narrative by insisting not only that every AI project begins with a clear question, a credible experiment and a plan to learn, but also that these are held collectively by an integrated team with visible executive backing. That is how more organisations move into the 5 percent.

1. Start With the Simplest Possible Use Case

One of Knowbot’s core design principles was minimal friction. MHF looked for a “gateway use case”: a low-risk, easy-to-understand tool that organisations could immediately see value in and adopt quickly.

Practitioner takeaway: Don’t begin with the most ambitious AI project your organisation can imagine. Begin with an easy, low risk project that still delivers value and treat it as a learning experience..

2. Culture Matters More Than Budget

Hudson notes that AI readiness among nonprofits varies widely and isn’t correlated with organisational size. Some large charities are slow to innovate due to bureaucracy; some small ones are enthusiastic but unlikely to benefit.

Practitioner takeaway: When planning an LLM deployment, assess cultural readiness, not just technical readiness. Ask:

• Who are the internal champions?

• How much AI literacy exists?

• How cautious is the organisation by default?

This will drive adoption far more than infrastructure.

3. Build for Trust First, Then Functionality

The biggest obstacle MHF faced wasn’t the model, the infrastructure, or the code. It was accessing the right decision-makers and establishing trust with LLMs in general and Knowbot in particular.

Practitioner takeaway: AI deployments in nonprofits are trust projects as much as technical ones. Practitioners should:

• Engage early with leadership.

• Be explicit about risks and mitigations.

• Provide clear, responsible documentation.

• Avoid overclaiming what the model can do.

The more transparent the process, the smoother the adoption.

4. Where Appropriate, Restrict the Model’s Knowledge Domain to Reduce Risk

Knowbot deliberately confines itself to the content on the host organisation’s website(s), plus general internal LLM knowledge. It doesn’t trawl the open internet. This dramatically limits opportunities for hallucinations, unsafe advice, or reputational risk.

Practitioner takeaway: Whenever possible, design LLM answer engine systems that operate on curated, organisation-owned content. Domain restriction is one of the most effective forms of practical AI safety.

5. Expect Surprising User Behaviour — And Design for It

One of the unexpected patterns in early usage: people asked Knowbot, “Who are you?” This required the team to add a new prompt component and require every partner to host a “What is Knowbot?” page.

Practitioner takeaway: Build processes for:

• Unexpected inputs

• Prompt evolution

• Iterative refinement

LLM deployment is never “set and forget.”

6. Technological Timing Matters — And Keeps Improving

Hudson emphasised that many capabilities now considered standard (e.g. long context length, ring fenced access to specific types of knowledge, server deployment ease) would have been impossible even a year earlier. The tools needed to fulfil a nonprofit’s evolving needs often appear in the LLM ecosystem soon after the nonprofit requests new functionality - making the decision whether to ‘build custom’ or ‘wait’ a tricky one.

Practitioner takeaway: Stay current. Model capabilities, guardrails, and hosting options evolve at high speed. What was impossible last quarter may be trivial today.

7. Value Impact Over Volume

Knowbot’s LLM processing costs MHF money, and soKnowbot’s team evaluates success not just by the number of questions answered but by the relevance of those questions to valuable decision-making. A tool that helps a policymaker or researcher retrieve something critical can have outsized impact.

Practitioner takeaway:When measuring impact, develop metrics that capture qualitative value, not just quantitative usage. For example you might consider:

• Complexity of queries

• Decision relevance

• Equity of access

• Whether the tool reduces burden on staff

8. In Fast-Moving Environments, Admit What You Don’t Know

Both Knowbot and TestRAMP were built in contexts where knowledge was changing daily. Hudson emphasises the importance of asking “naïve” questions, learning quickly, and not pretending expertise where there is none.

Practitioner takeaway: Cultivate humility. Curiosity and fast learning beats early certainty. Pair technical exploration with organisational openness about unknowns.

9. Relationships and Partnerships Are Everything

Across both initiatives, success depended less on algorithms and more on building new human relationships.

Practitioner takeaway: AI for public good is a team sport. Map stakeholders. Share progress transparently. Community buy-in creates technical resilience.

10. The Next Frontier: Agentic AI

Hudson argues we’re at a turning point where AI will expand from being “retrieval engines” to becoming “agentic systems that can do things.” With that shift comes both opportunity and new categories of risk.

Practitioner takeaway: Prepare now for agentic systems. Start with controlled automation, clear constraints, auditable logs, and robust governance. Retrieval is only the beginning.

Read our full conversation with Mike Hudson here.

Find out more about the Mike Hudson Foundation here.

Mike Hudson is an entrepreneur in technology & electronic markets. He now uses his expertise to help solve social problems. Mike founded TestRAMP, a pandemic nonprofit social market described as a “major contribution to Covid PCR testing & genomic sequencing” & donated its £2.4mn profits for charity. Mike is a Fellow of ZSL & adviser to its CEO. He is an honorary Research Fellow at City, University of London. Mike is a member of the Responsible AI Institute. He is a Foundation Fellow at St Antony’s College, University of Oxford.

History and Background: Defining MAS

There has been more than 30 years of research into intelligent agents. Traditionally, Multi-Agent Systems (MAS) referred to collections of autonomous software agents that could communicate, coordinate, and collaborate to solve complex tasks, often in domains like robotics, logistics, or distributed control systems.

The MAS + LLM Shift

What’s new today is the emergence of LLM-powered agents, where each agent is a Large Language Model (or a wrapper around one) capable of generating language, and calling external tools. This shift marks a new phase: MAS + LLM, where agents are not just rule-based or symbolic, but generative and language-driven.

​This distinction is crucial:

• Traditional MAS Example (Rule-Based): A fleet of warehouse robots coordinate to move packages using pre-programmed rules and message-passing protocols.

• MAS + LLM Example (Generative & Tool-Enabled): That same warehouse might now use a set of LLM agents to plan a delivery route, query traffic data via APIs, and negotiate with each other in natural language to optimise timing, all while calling tools like maps, databases, and calculators.

This new architecture introduces challenges and opportunities: LLM agents can be more flexible and adaptive, but also more prone to errors like hallucinations or inconsistent tool usage.

Data Scientists and MAS: A New Mandate

​In traditional ML workflows, Data Scientists might train models and deploy them behind APIs for consumption by other services. In MAS + LLM setups, those models become tools that LLM agents can call as part of a broader reasoning process. Data Scientists may now be involved in designing these tools, defining agent roles, and testing how agents interact.

​Imagine, for example, that a Data Scientist is training a sentiment analysis model. Instead of embedding it in a web app, they expose it as a tool. A “Customer Feedback Agent” (LLM) calls this tool to analyse reviews, then passes results to a “Product Strategy Agent” to decide next steps. This new mandate means Data Scientists are uniquely positioned to ensure the reliability and responsible deployment of agentic systems by rigorously testing the individual tools and the complex communication logic.

​When using MAS + LLM, it is essential that Data Scientists ask the right questions to assess how well a system performs on a given task. Below is a proposed framework, based on established software testing hierarchies, adapted for MAS + LLM architectures:

​A Four-Level Testing Approach

Summary

MAS + LLM is becoming an increasingly essential part of a Data Scientist’s toolkit. With the numerous agentic orchestration frameworks available (LangGraph, Autogen, CrewAI, etc.) and that number increasing over time, understanding how to assess MAS and actions to improve them is necessary for their development. I encourage you to use this framework as a starting point to establish robust testing pipelines and governance standards within your teams.