In this interview with Real World Data Science, Thomas reflects on his career journey so far, from hands-on coding work to team leadership and management. He also argues for the importance of communication and storytelling as part of the data science skill set.

Members of the committee nominated two individuals to be featured as part of our career profile series, and so we are pleased to bring you interviews with Claire Morton (below) and Albert Lee.

Read on to discover more about Claire’s data science career (so far).

Hi, Claire. Thank you for sharing your career story with Real World Data Science. Please tell us a little about yourself and your role in data science.
My name is Claire Morton, and I’m an undergraduate student studying mathematical and computational science and environmental justice at Stanford University. I’m particularly interested in using statistics and data science to work with community-based organizations and advance evidence-based environmental justice policy. I have conducted quantitative research on tools to classify disadvantaged communities, oil wells, climate resilience, housing justice, and the connections between soil and health.

What drew you to study statistics and data science?
I really enjoyed my math, statistics, and coding classes in school. I was also inspired by the power that numerical data have to tell stories and promote policy change.

What do you think is your most important skill as a data scientist?
Listening to others. Listening allows me to learn new statistics skills from my mentors and to learn about how best I can work with community partners on their priorities in my research.

How does your gender and/or sexual identity factor into your career?
I am a lesbian, and, at the start of college, I didn’t have any mentors who shared my identity. I’ve now found several through the ASA and queer communities at my university, and I’m continually inspired by the achievements of queer statisticians, mathematicians, and computer scientists. My research hasn’t explicitly connected statistics and queerness yet, but I’m interested in working on projects involving hard-to-reach populations, such as queer people, in the future.

How did you get into data science?
In high school, I worked in a cell biology lab. As part of that work, I learned to model cellular processes and analyze data from my experiments. I realized that those elements were my favorite parts of the science I was doing, so I decided to study math, computer science, and/or statistics in college. I had always been interested in environmental issues, and so I got involved in quantitative research about environmental justice. I realized that this type of research allowed me to connect the skills I have to my passions, so I’ve kept working in these areas ever since.

What, or who, first inspired you to pursue this career path?
My mom! She’s also a statistician, and she has encouraged and mentored me throughout my academic journey. I’m inspired by her success as a woman in statistics.

What hurdles or challenges have you faced in your studies?
My classes can be tough, which makes it hard to stay motivated sometimes. I also struggled to maintain a healthy work-life balance at the start of college. Finally, it has been tough to learn some of the ins and outs of the research process and publications – how best to engage with research mentors, what it looks like to write and submit a paper, and some of the nuances of working in academia. I think my next big challenge is deciding what to do after college, though I’ve been trying to reframe the question as an opportunity rather than a hurdle. I’m excited to continue doing research at the intersection of statistics and public policy in the future.

What was your first job in data science, and how does it compare to your current role?
My first job in data science was as a researcher, working at a non-profit called Physicians, Scientists, and Engineers for Healthy Energy (PSE). As part of this job, I worked with community-based organizations to code quantitative optimization models to locate climate resilience hubs in California that took community priorities into account. I’m currently a student, which involves less research but gives me the chance to focus on learning new skills for my next job. I hope to be able to work in a role like my job at PSE, combining statistics/data science, community engagement, and policy impact, in the future.

What was the most important thing you learned in your first year on the job?
The importance of being adaptable and self-directed. Research projects shift and change as you uncover new information, and it’s useful to be able to shift with them. It’s important to be able to take initiative to learn skills, talk to people, and solve problems as they come up – but it’s also critical to not be afraid of asking for help when you need it.

What have been your career highlights so far?
One was publishing research about the demographics of people living near oil wells in California, which informed policymakers about racial and socioeconomic differences in exposure to oil wells and is part of a long-standing effort from activists and researchers to protect the health of Californians. I’ve also gotten to work directly with organizers on several mapping projects, which was deeply fulfilling. Finally, I loved getting to present my research at the Joint Statistical Meetings last year, and I look forward to presenting my undergraduate thesis this year.

What three things are at the top of your reading/study list?
Some statistical areas I’m hoping to learn more about are spatial statistics and survey methods. Some books I’m excited to read are The Color of Law, Data Feminism, and Thicker Than Blood: How Racial Statistics Lie. 

What advice would you give for anyone wanting to study statistics and data science?
Find mentors that inspire you, support your career goals, and challenge you to learn and grow as a statistician.

What new ideas or developments in the field are you most excited about or intrigued by?
I’m really interested in combining quantitative research with community-based research, so I think that cross-disciplinary developments are exciting. I’m intrigued by AI tools, and I’m interested to see how these tools change what the day-to-day of being a statistician looks like and the skills that are most sought after in a statistician.

And what do you think will be the main challenges facing the profession over the next few years?
The main challenges will be related to statistical literacy, both for the people consuming and doing statistics. While statistical methods and data becoming more accessible is a positive development, it has meant that more analyses are done incorrectly and that more misleading results are publicized (and absorbed) as truth. It’s getting much easier to twist numbers to support whatever we want them to say, and I think this will continue to challenge both statisticians and non-statisticians in the future.

Members of the committee nominated two individuals to be featured as part of our career profile series, and so we’re pleased to bring you interviews with Albert Lee (below) and Claire Morton.

Read on to discover more about Albert’s data science career (so far).

Hi, Albert. Thank you for sharing your career story with Real World Data Science. Please tell us a little about yourself and your role in data science.
My name is Albert Lee. I’m the founding partner at Summit Consulting, a quantitative and financial consulting firm in Washington, DC. Summit delivers data-driven solutions to help make government effective and society just. I started Summit in 2003, and we recently celebrated our 20th anniversary.

I received my PhD in economics from UCLA in 1999. My professional practice is focused on econometrics – an academic specialty that blends economic theory with statistical practices – and statistical sampling.

What does your job involve?
A large portion of my time is spent running Summit and making decisions about management, personnel, and business development. That said, I am still pretty active in technical topics. I am a testifying expert in econometrics and statistical sampling. Recently, I have been leading a team of data scientists who are reformulating the edit and imputation algorithms for the US Department of Agriculture’s National Agriculture Statistical Service, which collects survey data from US agriculture sectors.

What do you think is your most important skill as a data scientist?
Explaining technical concepts is a big part of my job, and it requires the ability to consume the technical literature and know the concepts well enough that I can explain them to a lay audience (such as lawyers, judges, and program staff).

How has your gender and/or sexual identity factored into your career?
My gender and identity have given me important perspective as a data scientist and an entrepreneur. Living my identity takes courage. It is the same courage necessary to start a new business. From a young age, my identity has conditioned me to be comfortable with differences.

My identity has also taught me to see similarity among differences. Empathy is essential in client services and especially in quantitative consulting, where some of my clients feel disempowered by the complex subject matter.

How did you get into data science?
Although I studied mathematics and economics as an undergraduate student and economics as a graduate student, my academic training was very theoretical. I didn’t work with data and computers extensively until my first job outside of academia in the early 2000s. Little did I know that it was the advent of the “big data” revolution.

At Summit we serve mostly federal agencies, who are sitting on decades of administrative data – information they collected as part of their mission but not of research quality. These agencies want to use their administrative data to automate their routine tasks (like predicting which loans will default first) and evaluate program efficacy (determining whether a training program reached its goals). Extracting and analyzing administrative data has been a big part of my career.

When I founded Summit, data science was not a recognized discipline. But as the datasets get larger, decisions about hardware setup, software programs, estimation algorithms, and data virtualization have become increasingly intertwined and interdependent. This really was my first taste of data science as we know it today.

What, or who, first inspired you to become a data scientist?
There are too many people to mention by name. I owe a lot of my career to my first two managers at KPMG, Alan Salzberg and Rick Holt. They taught me how to code and reason quantitatively. And Rob Gould at UCLA has patiently converted a theorist to an empiricist. Once a convert, now a zealot.

What were the hurdles or challenges that you needed to overcome on your route into the profession?
I am an immigrant and a first-generation college graduate. My journey was full of unknowns. Figuring out my academic and professional career has taken a lot of exploration. In this regard, the same exploration that guided my identity also guided my academic and professional journey.

And what are the challenges that you face now that you are working in data science?
The data science field is moving very fast. Every day brings a new algorithm, software program, and hardware innovation. Since data science is a multidisciplinary field, keeping up with it has been challenging. As I progress along my professional journey, striking the right balance between management, hands-on practice, and learning has been difficult as well.

What was your first job in data science, and how does it compare to your current role?
As an entrepreneur, I was given a lot of professional freedom to actualize my career. To a large extent, I have the career that I envisioned. To me, data science lives in the intersection of methods, software, and hardware. I have spent a large part of my career in this intersection.

Of course there are many things that were not part of the original vision, such as running a 100-person organization. My approach has always been intention with openness. By this metric, my current role is not far off from my original vision.

What was the most important thing you learned in your first year on the job?
The ability and the love of learning constantly, regardless of the topic.

What have been your career highlights so far?
The biggest highlight was that on June 15, 2023, Summit celebrated its 20th anniversary! Reformulating the National Agricultural Statistics Service’s edit and imputation systems is also a big deal. And being a testifying expert in some of the most consequential legal cases in the United States was a highlight as well.

What three things are at the top of your current reading/study list?
In recent years, I have been binge-reading Stoic philosophy. I have read most books by Ryan Holiday. His most recent book was Ego Is the Enemy. In between the Stoics, you will find me reading Buddhist meditation literature, including Thich Nhat Hanh’s The Heart of the Buddha’s Teaching. David McCullough’s Truman is also by my bedside.

What advice would you give for anyone wanting to be a data scientist?
Be open and multidisciplinary. Many good ideas in statistics come from other fields, such as economics, medicine, sociology, and education. Computer science enables computational statistics. Having the openness to these topics is key.

What new ideas or developments in the field are you personally most excited about or intrigued by?
Machine learning has transformed statistics both as a consumer and a contributor. It consumes statistics in that it requires cutting-edge statistical techniques and algorithms for its estimation. Machine learning has important applications in many of the statistical sciences.

And what do you think will be the main challenges facing the profession over the next few years?
The proper use of statistics or statistical ethics is an important societal challenge. Machine learning is becoming increasingly sophisticated, and its applications are more broad and pervasive. Machine learning algorithms are making more and more decisions in society, including mortgage loan approvals, residential home prices, and which prisoners receive parole. These are important and weighty decisions. How do we know that these decisions are unbiased and fair?

I am also the founder of beautifulplaces.ai, which is a continuation of my University of Warwick and Alan Turing Institute PhD work where I provided the first large-scale evidence that beautiful places are connected to our wellbeing.

What does your job involve?
My role at Zoopla involves managing data scientists both for Zoopla and Hometrack. At Zoopla, we use data science to create an engaging experience for users who want to buy, sell and rent properties. At Hometrack, the data scientists mainly work on an automated valuation model that provides property valuations to most of the major mortgage lenders in the UK.

As a leader in data science, my role primarily involves helping stakeholders across the business to best leverage data science to reach our business goals, as well as ensuring my data science team has all the support and mentoring they need to design, develop and maintain high performing data science algorithms.

What does “data science” mean to you?
That is a really good question! One thing I have noticed is that people who aren’t familiar with the field often confuse data science and data analytics. There are indeed many similarities – both require quite a bit of knowledge to be able to leverage the correct insights from both structured and unstructured data (data hidden within images, for example). However, data science is essential when you need to make inferences. For instance, you are not only analysing data to see what certain consumers may prefer, but you are also predicting what similar consumers might prefer. Thus, having a strong grounding in statistics is really important for anyone working in this field.

What do you think is your most important skill as a data scientist?
Aside from a good grounding in the technical aspects of data science (which is possible for really anyone to pick up from the many good courses that are available), the most important skill is how you can leverage data science to create products that actually create value for the business. I find this to be the most challenging journey that junior data scientists find themselves having to navigate. They are really excited about the technology, and get carried away with wanting to perfect their algorithms. But when you are building commercial products, what is really important is to constantly engage with stakeholders to make sure you are building something that actually has a tangible business benefit. Early release of a model for user testing is also essential, as models only really get better once you have real user input.

How did you get into data science?
It was somewhat by accident. I previously had a long career in digital and decided to take a career break to return to university and study economics. When I was working on my Master’s degree in behavioral economic science at the University of Warwick, I saw an ad for a PhD to “use online data to understand human behaviour” and I thought this was perfect, as it combined my prior knowledge with a new area I was increasingly becoming drawn to. I quickly taught myself how to program in Python and convinced my supervisors to take me on, and from there on, I came to love data science!

What, or who, first inspired you to become a data scientist?
It was more about realising what you could do with data science. In my PhD, I was quantifying the connection between beautiful places and our wellbeing. While this has long been an intuitive connection, we were not able to test this on a large scale due to lack of data. Being able to use data science to start predicting the beauty of outdoor images was fascinating as it opened up a whole new method for potential research combining beauty with various wellbeing ratings. Once I started to see what was possible with data science, there was no going back.

What were the hurdles or challenges that you needed to overcome on your route into the profession?
For me personally it was the challenge of moving sideways into a leadership role after my PhD and not having to start all the way from the bottom. I would have loved to continue in academia and expand my research even further, but starting from the bottom earning a tiny salary after I had taken quite a large career break to do my PhD wasn’t an option for me. So I decided to go back into the commercial world and look for a senior role from the get go and luckily Channel 4 agreed to give me my first commercial stab at data science.

What are the challenges that you face now, as a working data scientist?
Trying to keep up with everything that is constantly changing in the world of data science. I love the rapid change but it can also be quite time consuming to make sure you are on top of it and giving the right advice to people.

What was your first job in data science, and how does it compare to your current role?
My first job was working as a senior data scientist at Channel 4. As a senior data scientist, even though you have additional goals to help run the team and be a mentor to more junior data scientists, you still get a great deal of time to do coding and develop your own projects. When you move more into a management role, the time you have to develop data science models diminishes. People also expect you to give in-depth guidance when you haven’t actually had much time to deep dive into a project. So, I am often trying hard to make sure I am on top of what is going on even when I have limited time, and really focus on building a strong team that can support each other and collaborate often to create better data science products. Learning to delegate is key!

What was the most important thing you learned in your first year on the job?
How hard it is to actually get organisational buy-in to use data science at scale. It is really easy to get approval to build a proof of concept (POC). However, if you do not use the time when developing your POC to also make sure to get the right stakeholder on board, your project is dead before it even starts. So, in data science, getting the model right is not enough, and working with people across the business to make sure the model can be integrated into the business processes is essential.

What have been your career highlights so far?
It has been great being able to give talks about my research, and data science in general, all around the world. I have actually come to love public speaking, and I hope that as I continue to be recognised for my expertise, I can encourage and aid potential data scientists with their careers – especially minority women, as I think that diversity in the field is very important. This is a role that is fit for people from all kinds of backgrounds and I hope I am exemplifying this.

Have there been any mistakes or regrets along the way?
In smaller companies, it can easily happen that the founders don’t fully understand data science and often use data science as a buzzword to get investors on board. Whenever you take on a new job, and data science is just getting established, make sure the founders or leaders are actually fully onboard with integrating data science into the product and understand what this means. See if they know how tricky data science can be when first integrating into a product and are actually willing to overcome the challenges with you to eventually reap the huge benefits data science can bring.

How do you think your role will evolve over the rest of your career?
I see my role evolving into being more strategic and less about the data science day-to-day modelling. It is more about being able to advise companies on how to make use of data science as a strategy and helping them figure out where in the product or process to inject it to get the most out of it for the business as a whole.

If you were starting out in data science now, what would you put at the top of your reading or study list?
Practise how you would apply using data science for a real life problem. Seek a placement, as this will pay dividends in being able to speed up your learning.

If you don’t understand the statistics involved in data science, make sure to upskill in that area before starting your first role. A lot of junior data scientists focus on learning how to code or get carried away with the modelling without first learning the importance of preparing the data in the correct way so that your predictions can work well in a real life setting.

What personal or professional advice would you give for anyone wanting to be a data scientist now?
Try to find ways to stand out from the average data scientist. When we open up applications for data science jobs, I get hundreds of applications for each one. I am looking for people who can not only do data science but who also have other stand-out qualities that they can bring to the business. This can be something along the lines of effective stakeholder engagement to deep expertise in a certain domain or technology.

What new ideas or developments in the field of data science are you personally most excited about or intrigued by?
I am really interested to see where generative AI will take us – particularly about how it can help us improve the speed of our own performance. It feels like generative AI can be a technology that can help everyone – even the everyday person – as it can help speed up so many processes, from coding to writing to ideating. While the technology is still in its early days, it is progressing rapidly and I am very curious to see where this will lead in the next year!

What do you think will be the main challenges facing data science as a field in the next few years?
Generative AI’s latest breakthroughs have made AI capabilities accessible to the broader public, but it has also stoked fears around the use of AI. The headline-grabbing narratives around AI and existential threat is distracting from other conversations that are really important. There are some very real issues we do need to solve – from biases in AI to the impact generative AI can have on wages and workforce – but this needs to be approached in a constructive and thoughtful way.

We need to find a way to engage with the public in a more meaningful way – rather than scaremongering – to have public debates on issues that actually matter.

These issues – of verification, explainability and interpretability – are of particular interest to data scientists like Anjali Mazumder, whose work focuses on the impact AI technologies could have, and are having, on society and individuals.

In part 2 of our Q&A about ChatGPT and other LLM-powered advances, and what all of this might mean for data science, Mazumder kicks off the conversation by setting out her perspective.

Our full list of interviewees, in order of appearance, are:

• Anjali Mazumder, AI and Justice & Human Rights Theme Lead at the Alan Turing Institute, and DS&AI committee member.

• Detlef Nauck, head of AI and data science research at BT, and editorial board member, Real World Data Science.

• Martin Goodson, CEO and chief scientist at Evolution AI, and DS&AI committee member.

• Louisa Nolan, head of public health data science, Public Health Wales, and DS&AI secretary.

• Piers Stobbs, VP science at Deliveroo, and DS&AI committee member.

• Trevor Duguid Farrant, senior principal statistician at Mondelez International, and DS&AI committee member.

• Giles Pavey, global director for data science at Unilever, and DS&AI vice-chair.

• Adam Davison, head of data science at the Advertising Standards Authority, and DS&AI committee member.

Anjali Mazumder: I work in research, but I also sit in the crux of government, industry, and civil society, looking at how they’re using these technologies. For me, it’s about knowing what the opportunities are but also understanding the limitations, the risks and the harms, and how we balance those and put in place oversight mechanisms that act as safeguards to ensure that these technologies don’t cause harm. We’re taking a very socio-technical approach that requires an interdisciplinary team to understand these issues and what should be done. Part of this is about not only the outcomes and the impact but also the upstream side of it – recognising that these models have been built on the work of people who have done the labelling, and that this also has implications – to say nothing of the associated environmental issues or energy issues!

Detlef Nauck: I think the regulators really have to look at this. It has come completely out of left field for them. All the regulators that we are monitoring, they regulate the space as it was three years ago – they are mainly concerned about predictive models and bias. But if you look at, say, what Microsoft wants to do – putting GPT into Office 365 and into Bing – that will completely change how people interact with and consume information. I think the large tech companies really have a responsibility here, when they make this public, to make sure that people understand what this technology actually is, and how it can be used and has to be used.

Also, they need to open up about how these things have been built. There are a lot of stories around how OpenAI used cheap labour in order to do the labelling and reinforcement learning for ChatGPT, and these things have to become public knowledge; they need to become part of some kind of Kitemark for these models: “Ethically built, properly checked, hallucinate only a little bit. Whatever you do, don’t take it for granted. Check it!” That’s the kind of disclaimer they need to put on these models.

Regulatory principles always seem to stress that AI systems should be understandable, and we should be able to explain how we get particular outputs. But a lot of our conversation has focused on how we don’t really know how these models work. So, is that, in itself, a problem, and is it something that the data science community can help with – to dig into how these things work and try and get that across – to help meet these principles of explainability and interpretability?

DN: That’s a very specialist job, I would say – specialist research into how these mathematical structures work. It’s not something I could do, and I’ve not seen any significant work there. One thing that we are interested in is whether we can do knowledge embedding, so that you can “teach” concepts that these models can then use to communicate, and that would lead to smaller systems where you have some understanding of what’s inside. But all of this kind of work, I think, is very much just beginning.

Martin Goodson: Do we actually need this? There’s sort of a big assumption there that you need to understand how LLMs work in order to build in the kinds of things that we care about as a society. But we don’t understand how humans think. Of course, we can ask a human, “Why did you make that decision?” You can’t understand the cause of that decision – that’s a complex neuroscience question. But you can ask what the reason is for making a decision, and you can ask an LLM what its reasoning is as well. I think a lot of these questions about explainability are stuck in the past, when you’re trying to explain how a linear model works. It’s really not the same thing when you’ve got an LLM where you can just say, “Why did you make that decision?”

Louisa Nolan: I was going to say something very similar. Most people don’t know how most things work…

DN: My point was, these things are largely still like the Improbability Drive in the Hitchhiker’s Guide to the Galaxy. You press a button, and you don’t really know what comes out, and that’s the problem we need to get our heads around.

LN: But people don’t know what percentages are, and yet we still use them for decision making. I don’t think people need to understand the maths behind LLMs, and I think it would be a hopeless job to expect everybody to do that. What we do need to understand is what LLMs can and can’t do. What’s the body of work that they are drawing from? What isn’t in there? What are the things that you need to check? So, for some things, it’ll be brilliant: if you’ve written something and you want it rewritten for a nine-year-old; if you want to summarise a paper; if you want to write code, as long as you already know how to code – these could be real labour-saving tasks. If you’re using ChatGPT to write a thesis about something that you haven’t looked at, that’s where the danger is. It’s this kind of simple understanding that people need to get in their heads – and the maths, except for the people who care about it, is beside the point, and probably detrimental, because it means that people won’t engage with it.

DN: I agree, but I wasn’t talking about the general public. I meant, the people who build these things – they should know what they’re doing.

We talked there about communication. There was a webinar recently by the Royal Statistical Society’s Glasgow Local Group, and the presenter, Hannah Rose Kirk, showed how you can take tabular data and statistical results and ask ChatGPT to produce a nice paragraph or two that explains the numbers. Is this the sort of thing that any of you have experimented with? Have you had any successes at using ChatGPT to translate data into readable English that decision makers can act on?

Piers Stobbs: I have an interesting use case. We had a basic survey: 200-odd responses, multiple languages, and we just said, “Please summarise the results of this survey contained in this CSV file.” And it came up with five or six relevant bullet points. What was amazing was that we could then interrogate it. For example, “Please compare the results that were in English versus in French, and describe the differences.” Again, it did it, but then you have the issue of, was it all correct? Well, the bulk of it was. Now I am intrigued by whether you can ask it to do correlations and some actual statistical things on a dataset, and does it get that right? I don’t know. We’ve not really got to that. But, to go back to one of the earlier discussion points around productivity, that initial survey work could have easily taken a week of someone’s time if we didn’t run it through ChatGPT.

Trevor Duguid Farrant: Piers, in this case you’re interested in checking and seeing if it’s right. If you’d asked a group of people to do that survey for you and get the results, you’d have just accepted whatever they gave you back. You wouldn’t have questioned it.

PS: That’s true. And the results were plausible, certainly.

AM: I think one of the challenges is that the results could seem like they’re plausible, right – whether that’s a statistical output or a text output. This was not a proper experiment, but I asked ChatGPT about colleagues and friends who are quite prominent researchers, asking, “Who is so-and-so?”, and it produced biographies that were quite plausibly them, but it wasn’t them. It might have listed the correct PhD, say, but the date was off by a year, or the date was correct but it was from the wrong institution. So, depending on what the issue is, these seemingly plausible results could have more serious implications.

LN: So, just to join those two things together: for me, the question is not, “Do we understand how ChatGPT works?” As Martin says, we don’t understand how humans work, and surely we’re trying to develop something that enhances human thinking in some way. The more important question for me is, “How do we know that what is produced is useful?”

Giles, you mentioned previously that you’re doing some work at Unilever around how to minimise hallucination. I don’t know how much you can say on what direction that’s going in, and how successful you expect that to be, but that’s obviously going to be a really important part of refining these models to be more widely usable.

Giles Pavey: I’m by no means an expert, but there’s quite a lot you can do with both the architecture of it and also the pre-prompts that you put in – more or less saying, “Quote what the source is, and if you’re not sure, then tell me you’re not sure.” I think what’s interesting is the question of whether we’ll have to rely on OpenAI or Microsoft to do that work, and it will be just another thing that we have to trust them for. Or, will it be something that people within an organisation can put in themselves?

MG: I think it’s absolutely critical that open-source models are developed that can compete with these tech companies, otherwise there’s going to be a huge transfer of power to these companies.

GP: Arguably, the single biggest issue is, who elected Sam Altman (no-one) and are we as society happy with him having so much power over our future?

To close us out, I’d like to return to a question Trevor posed earlier, which is: How might organisations like the Royal Statistical Society help companies to embrace LLMs and start using them, so that everyone can benefit from the technology?

Adam Davison: My instinct is that there’s some great parallel here with the stuff that the Data Science and AI Section have been doing in general, where we’ve said, “OK, there’s lots of good advice out there on how to do things in data science, but how do you make it practical? How do you make it real? How do you apply those ethical principles? How do you make sure you have people with the right technical understanding in charge of projects to get value?” If, five years ago, the hype around data science was leading organisations to hire 100 data scientists in the hope that something innovative would happen, well then, we don’t want those same organisations now thinking that they need to hire 100 prompt engineers and keep their fingers crossed for something special. Our focus has been on “industrial strength data science”, so I think we can extend that to show what “industrial strength LLM usage” looks like in practice.

Data scientists certainly have been thinking along these lines, and to find out more about current use cases, proofs of concepts and potential applications, Real World Data Science got together with members of the Royal Statistical Society’s Data Science and AI Section (DS&AI) for a group discussion.

Our interviewees, in order of appearance, are:

• Piers Stobbs, VP science at Deliveroo, and DS&AI committee member.
• Detlef Nauck, head of AI and data science research at BT, and editorial board member, Real World Data Science.
• Adam Davison, head of data science at the Advertising Standards Authority, and DS&AI committee member.
• Trevor Duguid Farrant, senior principal statistician at Mondelez International, and DS&AI committee member.
• Giles Pavey, global director for data science at Unilever, and DS&AI vice-chair.
• Martin Goodson, CEO and chief scientist at Evolution AI, and DS&AI committee member.

The first part of our discussion focuses on how large language models are becoming part of the data science toolkit, and what this new development means for data science teams and skillsets. Stay tuned for part two, which we’ll be publishing soon!

(UPDATE: Part two is now published: “Large language models: Do we need to understand the maths, or simply recognise the limitations?”)

As data scientists, how has ChatGPT – and other tools built on large language models (LLMs) – changed your working lives?

Piers Stobbs: Up to about a year ago, although I was really impressed with the developments in deep learning and the improvements in computer vision and natural language models, it felt in line with general improvements in machine learning. And then, probably about six months ago, with things like DALL·E and ChatGPT, it felt like something changed – properly ground-breaking capabilities. And I still can’t quite get my head around the fact that you can basically have a model that tries to predict the next token, and it comes up with outputs that really feel quite sensible and human-like – if prone to hallucination.

The way I think about it is, this feels like a brand-new capability that we’ve just not really had before. It’s almost like an interface with unstructured information. Historically, you sort of have to turn text into something, and then turn something back into text, if you want to have this interface with humans. Now, we’ve got this really quite elegant way of plugging the gaps, which feels full of opportunities.

I’m having great fun playing around with the code co-pilots – GitHub’s Copilot is amazing and, productivity wise, is helping me a lot. I am now a much faster coder because there’s all those Stack Exchange lookups that I don’t have to do anymore. Again, from a personal productivity perspective, I’m using [ChatGPT] for initial drafts of documents and other things. And then I use it almost for validating things. For instance, I had a random discussion the other night with ChatGPT about logistic algorithms. It’s not going to solve problems for you, but I asked it to give some pointers of things I could be thinking about – some of which I had, some of which I hadn’t. So, it’s almost like a brainstorming helper, somehow.

But probably the thing I’m most excited about is the knowledge sharing side of it – plugging it into, or on top of, private information, and surfacing all that knowledge that is locked away in documents and intranet pages.

Detlef Nauck: We’re looking into running proof of concepts to see whether LLMs do bring value. Software engineering is the most obvious one, and easiest to set up and run. And then we want to look at making use of internal documents – so, either summarization or creation of internal documents in appropriate language. The latter use cases are trickier to evaluate. We want to know whether the outputs produced are any good. With software engineering you can track GitHub statistics, for example. But if you give ChatGPT to somebody to write marketing material, or to get information out of a document, how do you know that the results are good? We need to get our head around metrics for evaluation.

Adam Davison: I’ve been using it for basically anything where I don’t remember the API very well or it’s a bit confusing. Pandas is the key, right? We all use pandas, but you don’t really remember how to do some complicated thing with apply(), say, so you just ask GPT-4 to give you the answer, and it saves you that hassle. Also, I read some insightful tweet that said these chat systems are really good for things where generating the solution is hard, but verifying it is easy. And I think that’s true for some of these things. You know, you get a short piece of Python code, you can basically look at that and you can tell if it’s right.

In data science, you’re a bit of a jack-of-all-trades. You need to do little bits of everything, but you’re not a specialist in anything. And so, I think for software development, it’s been really helpful. For example, right now, I’m doing a bit of frontend development in a project to visualise something. I’m never going to be a professional frontend developer, but GPT-4 can help deal with the oddities of JavaScript much more easily than it would be for me to trawl through Stack Overflow posts.

But the thing that we’re using it for, practically, is natural language processing (NLP) and classification. We have this particular problem at the Advertising Standards Authority (ASA) where we are running lots of different models that are completely unconnected to each other because every project is a different topic. So, one week we’re looking at, “Do these gambling ads appeal to young people?” and then the week after it’s, “Are these cryptocurrency ads being clear about the risks involved?” It’s very disparate, we don’t have a lot of time to iterate on models, and we don’t have huge amounts of training data. Ten years ago, when you were doing sentiment classification, you were on Mechanical Turk getting 10,000 examples, and even then it didn’t work very well after these really complicated models. Now, you’ve got a couple of hundred examples and with the embeddings [from LLMs] you can get to a pretty decent classifier quite quickly. We’re also starting to experiment with using OpenAI’s fine-tuning tools, and the performance that we’ve seen from that is very impressive, to the extent that it’s making us rethink whether we bother doing anything else in some of our classifiers.

Trevor Duguid Farrant: My organisation is not as far forward as the rest of you. I’ve introduced it to the leadership team, and the digital services team – what was IT – are looking to make a decision on whether we can use it or not. I think there’ll be so much pressure they’ll have to use it, but there’s still a feeling of discomfort with it, whereas I think it’s really good and have started using it. Everyone else on the call seems to have started using it. So, can organisations like the Royal Statistical Society help companies to embrace this and start using it, and then everyone can benefit from it?

Giles Pavey: I wish I could be with Piers and Adam – actually using it – but my life has been taken over as the guy who goes and explains it to the business. Unilever is a massive business, and we are concerned about privacy, confidentiality and trustworthiness. We’ve now built an initial GPT instance on Azure and fed it with some of our own documents, and a lot of my time has been working with legal to convince ourselves that that’s okay. Now we are really trying to work out just how we manage the amazing demand for proofs of concepts and use cases – and what we’re just about to uncover, I think, is the unknown but potentially massive expense of running it.

In pure proofs of concepts, departments that have large knowledge banks are using it: research and development, and marketing, for example. And one of the big technical things that we’re working on – and, because of the size that we are, we’re doing a lot of work with OpenAI and Microsoft on this – is how to stop the models from hallucinating.

Have your experiences with ChatGPT and other tools changed your thinking about the skillsets required of data scientists and data science teams?

AD: A little bit. As someone at a small organisation, I think it’s quite exciting because, five years ago, maybe you were in a world where if you had a sophisticated problem involving text or images, you’d need a big research team with a big budget to tackle it. But increasingly we find, like many other people, that you can take models off the shelf and repurpose them for quite diverse tasks. So, I think it’s becoming increasingly viable to have a small team of people who are implementers, who aren’t necessarily backed up by a big research organisation, doing increasingly sophisticated stuff.

I don’t think it does away with the sort of things that we always bang on about in the Data Science and AI Section, like the need for an understanding of statistics and how the underlying systems really work, because I think you still need to understand what you’re doing with LLMs, as with any other machine learning technique. But, if I had to guess, what we’re going to be seeing now is that for a lot of problems, you’re going to have more of a division – so, you’re either in one of a small number of very large labs doing research on very cutting-edge big models, or you can be an implementer who is taking things off the shelf and applying them. And maybe that space in between is going to get a little bit squeezed – that would be my guess, but obviously it’s very unpredictable.

PS: That’s exactly my view. When I first started hiring data scientists, a long time ago, you basically had to write stuff from scratch, and you needed PhDs – people who really understood, at a deep level, how the maths all works. But I think there’s been a steady progression towards valuing software engineering skills, and I think, in some ways, this is another step along that path. If I think now about implementing a chatbot over your own knowledge base, it’s basically like plugging APIs together with some Python. Adam’s point is still hugely important, though, because I think we still need the background knowledge about what’s actually going on – OK, I’m creating embeddings here, and that’s allowing this search to work so I can surface the right docs – that whole process, which an average software engineer is maybe not going to know. But I think it’s definitely blurring the lines.

Martin Goodson: It’s just as important now to understand how to evaluate performance. The difference is, it used to be that you were trying to figure out whether it’s 80% accurate, or 85%. Now, it’s like 99.9%. But you still need to figure it out. You still need to understand what the failure modes are, what caused it; how is it actually working, and is it doing what you need it to do for the product? Is it actually satisfying our needs as users or as customers of the products.

DN: I think in the future, the skills we will need are people who can run and build these models. Giles made the point about how expensive it is to run these things. Right now, you have two options: subscribe to an API, and then you are limited in how you can modify these models; or build your own – take an open-source LLM and modify it. But then you need people who know how to build a high-performance computing environment and operate this efficiently. You need to know how to actually train the models, how to curate the data, how to set the model parameters. And I always think there’s too much alchemy still going on in this field, right? It’s not proper science. People build these things and then are surprised at what they can do; they didn’t know such things would be possible. A lot of these capabilities only emerge when you make the models really, really big and, essentially, you also have no control over them – you can’t stop them hallucinating. So, these are the kinds of issues we need to get under control if we want to get any value out of them.

Prompt engineering is another one – you really need to understand how these models work and how to prompt them. If you want to give them to, say, a marketer to generate copywriting, they may not have the right ideas of how to prompt the machine. So, I could see roles developing out of data science that understand how to influence these models and make them do what we want them to do.

MG: The other angle to this is junior engineers. Now, the bar for being a useful junior engineer is that you’re better than GitHub Copilot. Why do you need a really junior person if you can just use a large language model to be the junior developer?

DN: I’m not thinking about the data science person who needs to write some code for a project here, but if you have a large software team in an organisation that produces production code, they will become more efficient by using these tools. But still, with all this overhead of testing and putting it all together, there will be a lot of manual work that needs to be done. But the teams will get more efficient and junior people will get up to speed quicker. That’s probably another advantage.

PS: I think Detlef’s point about understanding is an interesting one. It definitely feels like there’s been this sort of continuum from, you know, “OK, it’s a linear regression, we know what’s going on” to complex models to ensemble models where, again, you’re combining these things you can individually understand. Even with big ImageNet architectures, billions of parameters, at least conceptually you can understand how these work and build out tools where you can understand the layers. To me, what’s different now is you’ve got this reinforcement learning layer on top, or diffusion layer, or some other additional approach – this combination of really complicated things. I honestly don’t know where to start with trying to understand why a specific output is generated, and I think that is a proper concern. That’s definitely an area of research, because I think we need to understand this.

GP: I think there’s also a question in large companies of just who owns these things. Up until this point, everybody’s been happy that AI is the realm of data science. And, suddenly, generative AI looks like it might be the realm of the IT team – that it’s a service that you get off the shelf. It’s going to be interesting to see how that plays out. I really liked the point that Martin was making about being able to tell what the systems are actually doing, what they are supposed to do and how to check them, because if you don’t have a background in that area, you might just assume they work. Now, nobody knows exactly how these things work – not even the people who build them. But having a background in how you test things, for potential causes for things not working, is actually going to be incredibly powerful or useful.

TDF: Will experts like us actually be able to check it because of the speed that new versions are coming out and the developments that are happening? Is it going to take us six months to check that GPT-3.5 works? Well, too late, a month later GPT-4’s out! I just think that pace is going to keep accelerating.

What does your job involve?
My role is about harnessing the power of data to drive extraordinary outcomes. Leading a skilled team, we empower our company to leverage data and cutting-edge technologies for informed decisions and automation. I help shape our organisation’s capabilities in data science, analytics, machine learning, data management, and strategy.

What does “data science” mean to you?
Data science, to me, is a captivating fusion of modern data technologies and computational statistics that tackles business challenges, crafts intelligent automation, and generates insightful revelations.

What do you think is your most important skill as a data scientist?
Active listening is key. A data scientist must be surgical and precise in developing models, analysis, and tools that reinforce the company’s bottom line and operations. Data science exists to create value using data.

How did you get into data science?
I began with a psychology degree, which led to working as business psychologist where I discovered psychometric data analysis. After a master’s in countering organised crime and terrorism and a few short jobs in counter terrorism/intelligence, I decided that it wasn’t for me. I embraced my love for statistics and research, I dove into data science, learning Python through online platforms, and secured my first data scientist role at a WPP agency called Essence.

What, or who, first inspired you to become a data scientist?
I always thought someone like me couldn’t work in data, let alone data science. Dr Suzy Moat’s fascinating talk on machine learning’s application to human behaviour and psychology showed me that a psychologist could make a significant impact in this field, inspiring my aspiration to try to have a data science career.

What were the hurdles or challenges that you needed to overcome on your route into the profession?
Breaking into data science without a typical background in maths/computer science/physics was daunting. Building a Kaggle portfolio and coding models for fun prepared me for interviews. Another challenge was learning to harmonise my “data brain” and “business brain” to solve problems efficiently. Understanding how data solutions impact business problems will always propel you forward. 

And what are the challenges that you face now, as a working data scientist?
As I’ve transitioned into management, maintaining my coding prowess is an ongoing challenge. I stay sharp by doing data science and infrastructure development for fun, leveraging tools like ChatGPT and AirOps where I’m rusty. I’m currently building my own cloud data platform and running a lot of image neural networks on it.

What was your first job in data science, and how does it compare to your current role?
As an analytics executive at WPP agency Essence, I tackled data science, cloud engineering, and analytics problems for clients. They were a lot more singular and tactical in nature. Now, as head of data engineering and data platform at Penguin Random House, I focus on shaping data and technology strategy to align with the company’s broader vision.

What was the most important thing you learned in your first year on the job?
To always consider the bigger picture: how your work integrates with the organisation/client’s objectives, delivers value, and aligns with the aspirations of other stakeholders. Actionable insights and value is the most important thing.

What have been your career highlights so far?
Two shining moments include being the first of three of HSBC UK fraud data science leaders, where each of our departments tackled a different type of crime and protected our customers, and developing data strategies and capabilities for analytics, science, and business intelligence at Penguin Random House.

Have there been any mistakes or regrets along the way?
I regret not delving deeper into natural language processing (NLP) or spatial data science, which are now more accessible and growing fields within data science. I reckon the NLP methodologies would’ve been extremely useful seeing as I’m at a publishing company now!

How do you think your role will evolve over the rest of your career?
As data technologies become more accessible, I anticipate data roles will transform. I envision a future where data professionals focus on general AI, quantum machine learning, and multi-dimensional data analytics as traditional specialisms become democratised.

If you were starting out in data science now, what three things would you put at the top of your reading/study list?
I’d recommend Skin in the Game by Nassim Nicholas Taleb, Calling Bullshit: The Art of Scepticism in a Data-Driven World by Jevin West and Carl Bergstrom,  and Artificial Intelligence: How Machine Learning Will Shape the Next Decade by Matthew Burgess.

What personal or professional advice would you give for anyone wanting to be a data scientist now?
Success in data science hinges on understanding how it can transform organisations and engaging with business stakeholders. My advice: never stop listening to the business – the stakeholders are your biggest allies. I would also try to find your niche that sets you apart from everyone else. Mine when I first started in the field was my expertise on computational psychology and behavioural machine learning. 

What new ideas or developments in the field of data science are you personally most excited about or intrigued by?
Transfer learning excites me most, as numerous large technology companies now offer pre-trained models based on billions/trillions of parameters. This will revolutionise industries worldwide, as it will be easier to build more performant models even if a company has less data.

What do you think will be the main challenges facing data science as a field in the next few years?
The challenge lies in staying relevant amidst the democratisation of data science. Through large language models, low-code, and transfer learning, advanced data science methods will become easier for non-specialists to do and use. Innovation and keeping up with modern data technologies will be crucial.

How should these cuts be distributed among grant holders? CJD had no meaningful performance data on which to base its decisions, and I would know: I was hired by the agency just a few months before to analyze grant performance. Still, decisions needed to be made. We had to come up with a plan of action, and the clock was ticking…

This is a story of making opportunity out of crisis, of the interaction between not just theory of change and technical implementation, but the “political” process of negotiating these changes with stakeholders in a manner that led to better decisions. Through careful outreach and continuous communication, we developed a data collection and performance assessment process that enabled us to allocate budget cuts in a manner widely accepted.

The story ends on a bittersweet note. But, along the way, there are lessons to be learned about how to find common ground, manage expectations, forge productive working partnerships, and sustain a data science project longer term.

What does your job involve?
My role involves using the connected car and AI to make our products and customer experiences better, whilst leading within our wide data science team too. The data science team in Manchester, UK, originated with myself and one of my teammates – it’s since grown to nearly 40 (cross-sites and countries) and developed into a high-performing, advanced data science team.

What makes us stand out is the nature of our work – we mostly use vehicle data (of participating customers), which is different to a lot of other commercial businesses or teams who’ll focus more on transactional or web data. The data we use lends itself to some pretty interesting projects, and a general futuristic feel here.

I’m particularly interested and active in enabling a more electric and modern luxury future from the use of vehicle data.

What does “data science” mean to you?
The realisation of value! Whether that is added revenue, saved costs or improved growth, I’m led by what data science can do for the business and its customers. The use of data science can open up many exciting, value-adding opportunities.

What do you think is your most important skill as a data scientist?
I’ve always presented myself as a technically astute data scientist, even when entering leadership. But my niche is my ever-growing commercial awareness and passion about our products, customers and business. These aren’t new qualities, but they now align with my professional interests, as well as personal (I’ve been a fan of the Jaguar brand since childhood)!

How did you get into data science?
I did a maths degree at the University of Manchester, where I specialised in statistics. I didn’t do any post-graduate education and this was fine for me.

After graduating, I joined a digital fashion retailer (with a financial services proposition) as an analyst initially. I learned a lot about real-life data and analytics itself, whilst developing a rounded understanding about the business and how to deal with stakeholders cross-functionally. I must have served a few hundred at least(!) and left most of the ‘fancy’ stuff I learned at university aside, whilst getting to grips with so many aspects of commercial analytics. A great way for me to set solid foundations for what followed, and I personally feel this gives me a lens that others who dive straight into data science don’t have.

I was soon attracted to data science because it tapped into what I learned at university and challenges you to keep learning; there’ll always be things to learn, and new advances in the field.

What, or who, first inspired you to become a data scientist?
I have a twin sister, we’ve always been together throughout education. Even before we graduated together, she secured her first role as an analyst. This opened my eyes to data, and data science followed for us both!

What were the hurdles or challenges that you needed to overcome on your route into the profession?
I had a few people tell me I couldn’t do data science, possibly because I didn’t fit the typical data scientist stereotype in several ways. I think attitudes in the field have changed over time though and on a personal level, it’s motivated me to give it everything, and I can’t regret that.

And what are the challenges that you face now, as a working data scientist?
I need to manage my diary well to ensure effectiveness and work-life balance. I’m overseeing people, other projects, doing public speaking and trying to remain hands on. I sometimes block out chunks of time in my diary – I need some meeting-free time to produce quality technical work. I try to finish on time and enjoy a very busy social life with my family and friends. A flexible attitude to how we work helps to keep me happy and energised whilst I’m delivering from various angles.

Thinking back to your earlier roles in data science, how do they compare to your current role?
My current role is very different to my previous roles. I’m continually learning and adapting how I can be a good leader, providing support to a breadth of colleagues (in and outside the team) whilst delivering myself. I’m actively involved in setting and refining our team’s strategy and I’m enjoying leading projects which either deliver high financial impact or help set the path in terms of new tech and/or machine learning capability. There is much more responsibility but it’s easy to stay energised when working on cars and for a business I’ve long admired.

What was the most important thing you learned in your first year on the job?
I should have had more confidence in myself, but this grew – as I adjusted to the new environment I became much more assertive. My domain knowledge and data science expertise combined help to build my self-confidence, credibility and reputation.

What have been your career highlights so far?
I’m most proud of my recent promotion from senior to lead data scientist. Also it was exciting for my family and I when I gained an offer to join Jaguar Land Rover.

Have there been any mistakes or regrets along the way?
No, what’s meant to be will be!

How do you think your role will evolve over the rest of your career?
My progression has been relatively rapid, and I hope I’ve got many, many years ahead of me in my career. It’s hard to say how my role will evolve, I have a blend of responsibilities in my role which combined provide great fulfilment for me at the moment.

If you were starting out in data science now, what would you put at the top of your reading/study list?
A good understanding of analytics and the domain you’re in are my recommended prerequisites to doing data science.

Analytics is an important part of the data science lifecycle, being able to get the data yourself and communicate results with influence, for example, are just a few aspects of analytics which underpin successful data science projects.

Also, without awareness of the business and industry you’re working in, you can become very dependent on others. Data science itself can be quite challenging, so it’s great to have a solid foundation before starting out.

What personal or professional advice would you give for anyone wanting to be a data scientist now?
With the level of continuous learning required to just simply keep up, it can be more of a lifestyle and not a job, so this is something to consider!

What do you think will be the main challenges facing data science as a field in the next few years?
I still expect to see a skills gap in the field. There are more routes to getting into data science nowadays, but it’s important to not lose sight of fundamentals such as statistics and mathematics. A lot of people can code-up models but it’s fair to say that only a portion of them appreciate how to do this responsibly, understanding samples versus populations, statistical testing, which type of regularisation to use in a neural network, et cetera.

I also think there’s a challenge of questionable data science products reaching high levels of popularity and usage amongst the public… Some recent developments in this space have been extremely intelligent but raise ethical concerns. Just because something can be done with AI doesn’t mean it should, and my preferences are towards AI being ethical and (ideally) explainable.