Stefaan Verhulst, PhD, Co-Founder and Chief Research and Development Officer at the GovLab at New York University

Stefaan (00:00):

Everyone assumes that they can formulate a question because yeah, question is kind of natural, but that does not mean we are good at it, and especially does mean if we are making decisions that we know what are the questions, if answered, would make a difference in how I make decisions. And so that's when I started working on what I call a new science of questions. We have data science, but we don't have question signs. And so, I started looking into how would we develop that? And here I started with an initiative called the One Hundred Questions Initiative, which is, what are the 100 questions that matter, that if answered, we could make a big difference? And most likely we probably have the data for doing so. But we never used the data because we didn't have the question that mattered.

Jim (00:45):

Welcome to What If? So What? The podcast where we explore what's possible with digital and discover how to make it real in your business. I'm your host, Jim Hertzfeld, and we get s**t done by asking digital leaders the right questions. What if, so what, and most importantly, now what?

I'm really happy to welcome Stefaan Verhulst to What If? So What? He is a research professor at New York University and a co-founder of the Governance Laboratory, also known as the GovLab, which focuses on decision-making using science and technology, of course, AI. He's widely published, he's well-traveled. He joins us, I think, from Brussels today. Great to talk to you again, Stefaan.

Stefaan (01:21):

Pleasure. Thanks for having me, Jim.

Jim (01:22):

Thanks for taking the time here. I don't think a lot of people know about the GovLab. Certainly, very relevant I think in, in today's AI way, but can you give, share a little bit more about the GovLab and, and why you started it, what sort of problems and opportunities you're exploring?

Stefaan (01:37):

Sure. And again, thanks for having me, Jim. So yeah, so the GovLab is an action research center. We founded it about 15 years ago now. And before I was at GovLab, I was head of research over private foundation where we looked into how we address a whole range of societal problems. And I came to the realization that part of the reason why we are not making progress on a few societal problems is because the decision process that we have in place is not up to the game. And so, we really need to change the way we go about making decisions, the way we go about designing public policies, the way we go about designing public services in some, the way we go about solving problems. And so that's really the impetus of the GovLab because we felt that at that time, and especially now in 2025, we really can change, and we really can improve the way we go about making decisions.

And so that's what we hope to do with the governance lab. And so, by governance, we really focus on decision making, not government per se because it can also be within the private sector or the nonprofit sector. And we believe, and that's what we are testing out at the GovLab, that there are a new set of tools and methods and technologies that we can leverage in order to upgrade the current decision toolbox. And specifically, we focus on two important, what we believe are assets that we can leverage in order to change the way we make decisions on the one hand people. And so here we actually try to solve the problem. Who in society knows what and how do we bring to the table when decisions are made? And it turns out that as a society, we are actually very poorly organized to identify expertise and then connect that with the decision cycle. And so, this is the work we do around collective intelligence, around smarter crowdsourcing, and really trying to tap into the supply of expertise, of skills, and of talent in order to really make a difference. And then the other asset, in addition to people or collective intelligence, is, of course, data. And so here we've done a lot of work around how do we leverage data and create data intelligence in order to change the way we make decisions. And more recently, of course, that has led us also to explore artificial intelligence.

Jim (04:16):

That's fascinating, and I love this. These are, as I said, highly relevant, highly pressure problems. As you described this, I'm thinking of a range of societal and economic problems, from climate to sustainability to just fundamentals of even democracy or healthcare, and how we decide on nudge people to do the right thing, as we might say. As you described that, I'm thinking of sort of the ultimate decision making, which is, you know, casting a vote, and what drives people to do things and motivate them. So I imagine you're uncovering, and I hope you're uncovering a number of things, but you know, the focus on data, I think, is the most fascinating because we've talked about, I think we all know that the, the digital revolution, if I can call it that, has created more data points.

I think this is an old statistic, but I would, I like it. But you know, the amount of data that is just in video form that, that YouTube was creating is sort of doubling every six months or was at one point. And so, I thought that was a really tangible way for people to understand just the vast amount of data, and so that we have there available to us. And then the question is, what do we do with it? You know, you mentioned that we are in an AI summer, we just entered fall. So, I don't know if that's metaphorically relevant, but you know, we've been in an AI summer, but you are maybe a data winter, and I've run into this with some of my clients as well. Is AI exposing a data problem that already exists? Or is decision making and governance creating a data problem or, or exposing it? Or is it creating a data problem? Is this proliferation of data going in the sort of wrong direction? How do you sort of see that juxtaposition, and what does that mean for decision-making and...

Stefaan (05:59):

Governance? Just to quickly answer your question and then have a longer <laugh> answer perhaps, is that yes, AI clearly has changed the data ecosystem but definitely has create growing awareness about the import of data and especially large volumes of data, but also high-quality data. And I think without a high-quality and sufficient amount of data, it's very hard to train models or to augment models for, for instance, rank applications. But also, AI has also pointed to some of the challenges and has created challenges as it relates to data. And I will go into detail in a second, but then the more positive note, perhaps, Jim, is that AI can also be used to actually solve some of those problems as well. And so, you refer to the data winter that indeed have announced as kind of an emerging development.

And that was mainly the result of the work that we've been doing around how do we make sure we have access to the relevant data to make better decisions? Because as I said, our work and one line of work is specifically focused on how do we leverage data for decision making. And I always say, in order to use data for decision making, one of the real challenges is one needs data. And that turns out to be a key challenge, which is that on the one hand, we have massive amount of data. Ation and Jim, as you refer to, to digitalization, has led to the data ation, meaning that every transaction or every device we use that has some kind of a digital component leaves a digital trail. And that ultimately is data. And so, we have a massive amount of data, but unfortunately, we quite often don't have access to that data.

And that turns out to be a real societal challenge because those that need data, especially the ones that are working on some of the problems you mentioned, Jim, whether it's understanding climate patterns or understanding how to best adapt to some of the climate change phenomenon that we are currently experiencing, whether it's about preventive healthcare, whether it's about mapping jobs with those that have skills but don't have degrees perhaps, but are still employable. And as a result, you would actually increase the potential of many people in society, and so on. So, I can give you more examples, but in order to do all of that, one needs, of course, data, and that resides typically elsewhere. Now, we've worked on trying to have access to that data for quite a while. We've anyway coined to concept of data collaboratives and investing in data stewardship in order to do so...

But what we are witnessing now as a result of AI, we are actually witnessing a decline in having access to data. And that has a variety of reasons. First one is exactly that many that had data and that would provide it in the past have to a large extent become convinced that they should hold it and then use it for their own internal purposes. And we see this especially in a kind of social platform context, where, for instance, in the past, you could access Twitter feeds or X feeds. Same thing for Meta, that has now been closed or is only available at high cost. What we also have seen is that open science data, which typically was provided by scientists at no cost, has also been closing down and is now becoming proprietary data because people are now willing to pay for it.

And I think that's not a bad thing because ultimately, you need to create sustainability. But if it's a public interest topic that you seek to address, then you obviously need to have access to that. And then the last thing I would say is that because we've seen this massive extraction from data for training a few models in the current ones that have the largest market share, we've seen a really uncomfortable reaction by many. And that has led to creating new devices like Robo Text and so on, to actually close down access to data, which, anyway, is typically unstructured data. And that also has led to kind of a new data scarcity for both the large models, but also for the ones that are seeking to use the data for public interest purposes. And so, yes, so we are entering a data winter at a time of an AI summer. And from my point of view, the AI summer will actually be impacted if we cannot figure out how to really deal and come up with new models, which we, anyway, one model that we've worked on is data comments, new models to provide access to data in less extractive manners, but also in manners that are more systematic than currently the kind of piecemeal approach. And in most cases, a declining access context as well.

Jim (11:23):

A whole range of problems. I wasn't thinking about those problems when I woke up this morning <laugh>. So, you know, and also imagining this FOMO of data, sort of the fear of missing out, like, well, I kind of know that's there and I'm thinking of how you know, work a lot with manufacturer consumer goods manufacturers who are working with their retailers, and those retailers have all this data around the customer, so who are using their products, right? And there was always tension between those manufacturers and those retailers better understanding their customers. So one of the things that, that you've written about that I was really inspired by is that let assuming you, you, you have the data and you we're, we're building the right models and the access and the incentives correctly. You, you, you've talked about heavily having the right questions and I think a lot of people, a lot of organizations or teams will run out there and they, they'll say, look, I just, I need all the data.

Just give it to me. I'll figure it out. Give me, you know, and in fact, I'm experiencing this with a client right now who gave us a lot of data <laugh>, and we're, we're scratching our heads and many of it on much, much of it unstructured. How are we gonna get through this? And sometimes, you know, you're looking for, you'll find a pattern, you'll find some inspiration. So, it seems like I wanna know the questions before I go get the data, but I kind of want to be inspired and discover something. Is there a tension sometimes between this sort of intentional focus and this need for some sort of innovation? Tell us a little bit more about sort of the need to write them, ask the right questions, and how you balance that with the need for innovation.

Stefaan (13:01):

Thanks for the question <laugh>, Jim. And by the way, I'm very much inspired by the title of this podcast as well, which is definitely aligned with my effort to have more questions drive the conversation as opposed to statements. But your question, anyway, refers that there is it, that there might be a tension. And, and I would argue that doesn't have to be, it's not an either-or from my perspective, right? I think it's both. But too often we start with kind of the experimental or at least we believe that we will find something in a big hole, big pile of data. And I'm not sure that is always the case, nor I'm not sure that is the most efficient and effective way, especially at a time when you have data scarcity, right?

And so and so yeah, so I started working on questions because I became kinda sick and tired of sitting on panels and getting calls from multiple parties in the world who all were advocating for the need for more data and advocating for having access to data. And at a certain point in time, I said, yeah, I, I understand you need more data, but can you explain what the question is so that we are at least make sure that you have the right data and that you are not a kind of investing in piles of data or having access to data that might not make a difference. And then it turns out that most of the time, people could not answer me with regard to what the question is. And so, I started looking into then the whole issue of questioning.

And I also, of course started looking into, well, how can you measure the value of data? And it turns out the value of data depends on the value of the question. And it turns out that as a society, we're actually really badly organized in terms of formulating questions, and then also priority prioritization of questions. Everyone assumes that they can formulate a question because, yeah, the question is kind of natural, but that does not mean we are good at it, and especially doesn't mean if we are making decisions, that we know what are the questions if answered, would make a difference in how I make decisions. And so that's when I started working on what I call a new science of questions. We have data science, but we don't have question science. And so, I started looking into how would we develop that?

And here I started with an initiative called the Hundred Questions Initiative, which is, what are the a hundred questions that matter that if answered, we could make a big difference, and most likely we probably have the data for doing so, but we never used the data because we didn't have the question that mattered. And so, we've done this now on a variety of topics whether it's air quality, whether it's urban planning, whether it's climate, of course, whether it's also mental health among adolescents. And last thing that we are currently working on is actually what are the top priority questions for women's health as it relates to how can we innovate in that space? And so, by doing this, we've developed a methodology which starts with kind of a topic mapping, but then also by bringing a cohort of what we call bilinguals.

And by bilinguals, we mean people that are sophisticated with regard to the domain but also have a certain sophistication in terms of data or in terms of research, or in terms of course AI now as well. And so, typically, we bring about a hundred bilinguals to the table, and then we source them. Now, when we started sourcing them, then we came also to the realization that actually there is need for a taxonomy of questions, which is, anyway, anyone working in this space, this is not gonna be shocking. But quite often people don't realize that, for instance, if they wanna start developing a diagnostic analysis where they try to understand causes or try to understand kind of the drivers behind the phenomenon, then obviously you first need to understand the current phenomenon. IE need to have a baseline, which we call descriptive kind of questions.

And so that's what we are working on now, is to source those bilinguals around it a variety of topics, but then using a taxonomy of a descriptive, diagnostic, predictive and prescriptive questions, and then also helping them understand that in order to address a particular kind of challenge, you need to understand the, the sequence of those questions. Because what is interesting, Jim, about questions is the first part of the term IE it's a quest. And how do we architect the quest is equally important as too, anyway, what's the question? Right? And I think that's what we are working on right now. And I hope that we will, as a society, become far more sophisticated in questioning that we will focus on questions, literacy within organizations, not just data literacy. Because data literacy, without question, literacy from my perspective doesn't really get you the value, doesn't get you the answers you need to make better decisions.

Jim (18:42):

Stefaan, I love this approach, and I think just, you know, sometimes I think all of us feel like, you know, we, we've sort of accumulated the approaches and the knowledge as we've mastered the approaches that already there, and they're still not producing the results we want, right? Einstein said, right, the definition of insanity is doing the same thing over and over again, expecting different results. So, I love this approach. I love the quest. I'm, you reminded me of a, a client I had years ago, and this is a very traditional corporate setting, and you can, and with a lot of meetings, and you can imagine the sort of, the sort of Kafkaesque, you know, feeling every day going into meetings. And he was a guy that would sit in a meeting for about the first five minutes, and then he would say, what problem are we solving here today? And it was a provocative question. It was definitely stirring the pot. And Dan, if you're out there listening, thank you, you, I think you did a great job <laugh> at that, and I, I've taken that forward. So, I think I think this notion, this notion is very powerful for people.

Stefaan (19:48):

To quote Einstein. Although no one really can confirm whether he said it, but apparently, he's, apparently that's true. Apparently he said that if I would be tasked to solve or to save the world in one hour, I would spend 55 minutes on formulating the question or defining the problem, and then five minutes on I three addressing it, because I would go automatically. And so, yes. So, problem definition, question formulation, super important and underappreciated. May I still remember when we started GovLab, we had the GovLab Academy, which was an academy, kind of an executive course for the innovation labs within government. As you know, at a certain point in time, every government across the world needed to develop an innovation lab. Then it turns out that many of those innovations labs got stuck.

Stefaan (20:46):

And, and then of course they needed to get unstuck. And so, quite often they came to the GovLab Academy, and the first question that we asked in order to be accepted by the GovLab Academy was, can you formulate the problem we seek to solve? And most of the time, IE 99% of people came to us with the solution they had in mind, but not with a problem. And that turns out to be common. And that's why also, in addition to the question literacy, we also have a problem definition methodology, and quite often do camps for teams to actually really have them focus what the problem is, right?

Jim (21:29):

<Laugh>? No, I think that's critically important. And you know, I, I try to encourage my clients to, to do the same thing. They're often in a hurry. There's a lot of pressure for time to market and that sort of thing. One of the selling points that we put on the table is, well, if you define the problem, you may find that you only need to do half of the things you think you do, and that will save you a lot of money. And then you get their attention. So, you know, the, the different ways to sort of sell that, that, that change in thinking. Well, speaking of questions, Stefaan, that, you know, this is, there are, there's a, there's a third part to the title of this podcast, so it's what if, so what, the last question that comes out in the episode is now what, and so as a, just a kind of a final thought here. If you were advising some of our listeners who I think are often in Fortune 500, Fortune 1000 companies, maybe sitting in a meeting, wondering what problem they're solving, what <laugh>, what steps would you recommend to them today to sort of take on some of this thinking?

Stefaan (22:29):

There's a lot to be done, right? But I will focus on one specifically that I feel quite often doesn't get enough attention, which is that a lot of the organizations in space work on the technical infrastructure, which is very important. They work on the kind of policy infrastructure, which is very important. They work on kind of the business case, which is very important. But of course, without human infrastructure, none of this will happen. And so from our point of view, a missing profession in the current environment is actually data stewards people or teams that have a clear understanding about a, what is the question or can help formulate the questions, have a clear understanding what is then the data you need, and what is the potential value of their data they already have, but who else they might need to have to work with in order to start answering the questions that matters to their organization, and how to then get the data and how to then make sure it is used in ways that matches the demand.

And I think this is kind of a function. It doesn't have to be just one position, but it's quite often a function that gets ignored because there's a lot of investment in data science. Now, of course, in AI capabilities, there's a lot of investment quite often in data governance, data protection, which is, anyway, super important. I'm not gonna argue against that, but who, and who is actually working on making sure you have the data and making sure that is done in ways that aligns with the demand. And I think that kind of function, which we call data stewards, gets underrecognized. And as a result, quite often no one really has the function or the, the, the authority to really act upon that being during COVID, for instance. And that's problematic within an organization.

But it's also problematic if you wanna start working around your data for public value creation and still remember when COVID hit, we got asked all the time with regard to how can we set up collaborations with important data holders? And those data holders quite often were large private sector companies. And the question they typically asked was, who should we talk to? And most of the time, I have to say, good luck finding out because there is no position, no function, quite often within organizations that really have the authority to think about the data for other purposes than initially collected. And I think that's a real kind of missing profession. It's a me missing function. And I would say to anyone listening, who in your organization has that data stewardship function, it's probably not just one individual, it's probably a team or a few, but having that clearly figured out and also having investment in upskilling those functions, especially now at a time of AI would a would be a wise thing to do.

Jim (25:52):

I think that's great advice. And I think you're right, it's often overlooked, and I'm certainly seeing that in some of the AI use cases that we're helping to address. It feels like sometimes just a few minutes later, we're discovering the, the data issue, you know, even, even if we've discovered the right questions, <laugh>.

Stefaan (26:07):

Right? Yeah. Yeah. But look, we cannot stop at questions, right? Then you need to answer it. But, but yeah, I've made my point is that in order to know what data you actually need and in order to also, anyway, have a minimum viable data point approach you actually do need to know what's the question, and then you need to know where does the data reside. Right.

Jim (26:29):

Well, Stefaan, thank you so much for the time giving us a really, a new, for me, a fresh perspective. I hope this is helpful to folks. I hope you enjoy the rest of your trip in Brussels.

Stefaan (26:39):

Thanks so much, Jim. Much appreciated.

Jim (26:41):

Take care.

Joe (26:42):

You've been listening to What If? So What? a digital strategy podcast from Perficient with Jim Hertzfeld, we want to thank our Perficient colleagues, JD Norman and Rick Bauer for our music. Subscribe to the podcast and don't miss a single episode. You can find this season, along with show notes, at perficient.com. Thanks for listening.