CIO PODCAST: THE AI ADVANTAGE | EPISODE 1

Barbara Call 00:10

The AI era has arrived, but there's no guarantee of success. Industry estimates say anywhere from 70 to 95% of AI pilots fail to get off the ground. The question is, how can you move beyond POCs prototypes and feasibility studies to real ROI and measurable business outcomes? What does successful AI adoption look like, and what are the steps to move from potential to pay back?

Barbara Call 00:38

Welcome everyone. I'm Barbara Call Global, Director of Content Strategy at CIO.com and this is "The AI Advantage, Navigating Risk, Reward and Real World Deployment" created in collaboration with CIO.com and Vertesia.

Barbara Call 00:54

AI is not just a new tool or a collection of siloed projects, it's a fundamental shift in your company's operating model, but success will depend on whether you're treating it like a standalone product or a scalable, integrated business model. In today's first episode, we'll be talking with leading minds at MIT about how can forward thinking CIO s move past siloed projects and endless pilots to create enterprise wide AI adoption. What does a strategic blueprint for integrating AI across your business look like? And the elephant in the room, what does this AI first approach mean for your employees and teams?

Barbara Call 01:36

But first, let's introduce today's speakers. First up is Dr. Abel Sanchez, research scientist at MIT. Welcome, Dr. Sanchez, tell us a little bit about yourself and your work at MIT.

Abel Sanchez 01:49

Thank you, Barbara. I think, like many of us in the academic field, we have long and boring bios, but I will shorten it to say that I've spent a couple of decades doing applied computation, so much like those in statistics, because of what I do, I get to play in many people's backyards. And I also like the applied aspect of computation, meaning I like to see things actually be tried in corporations, and see it fail, as you go from the lab to the corporation. And I like that real aspect to it. So I think the one characteristic to note is that I've spent the last 20 years applying technology in the organizational context.

Barbara Call 02:33

All right, great to have you. And next up is Professor John Williams, Professor of Information Engineering at MIT. Welcome Professor Williams, tell us a little bit about yourself and your work at MIT.

John Williams 02:46

Thank you. Barbara, basically, my research has been in computation, and Abel and I have been hanging out together for probably the last 20 years. We spent quite a bit of time in the auto ID lab, where the Internet of Things (IoT) was invented. And it was there that we learned that getting the technology right is just one part, and that's not necessarily the important part. And I think that's playing out as well in the AI area that as we all see, as we go further on in this discussion, that the technology often is working, but getting it deployed and scaled is another issue, and it's a real challenge for companies, and so I suspect we'll be talking a little bit more about that in the following questions.

Barbara Call 03:44

And our third speaker is Keith Schlosser, multi-time CIO with 35+ years of experience in the insurance industry. Welcome Keith. Tell us a little bit about yourself.

Keith Schlosser 03:55

Hi, thanks, Barbara. I'm really appreciative to be on the podcast because I, like Abel and John, think that the the human part of this is really the important part. And, in my 35 years experience, I've been helping companies grow, but transform as well, and that takes everybody rowing in the same direction. So currently, I'm working as an advisor in AI and cyber security and look forward to the conversation.

Barbara Call 04:24

All right, great. Nice to have you, Keith. All right. So let's kick things off. My first question, I'm going to start with Abel. What we're seeing and reading is that many of the first AI projects were set out to solve a singular problem, so often pigeonholing the investment to solve just one or a few problems. Are you seeing this in the conversations and research that you're doing?

Abel Sanchez 04:49

So when it comes to the success that companies are pursuing, there's a tremendous amount to unpack here. Many of us have seen the results when it comes to the pile.

And the tremendous amount of failures and the effort that many companies are undergoing. I think one of the things to leverage here, and here's where the advantage of having gray hair and having been around for more than 20 years helps, is to think of the parallels to cloud. What did companies go through as they went on that journey? And of course, there are companies that are still embarking on that transformation, but there are a number of different alternatives and different strategies that can be used. You can, of course, build bare metal with high talent and low cost. When it comes to the software that you're using, if you use open source, this is out of reach for most companies, because of the talent, because of the amount of infrastructure that you need to replicate. There's also a hybrid model, where you might leverage some from the cloud and some you might build yourself, or you can go completely, 100% reliant on a product. And so I think as we see these approaches and these different ways in which companies are approaching implementations, we do get to see that one vertical that's being addressed. We do get to see the deployment of thousands of agents in other scenarios and these mixtures of approaches, and ultimately, when it comes down to this, can you have a vision? Can you have lined up technology? Can you have change management? Can you have culture? Can you have HR careers? Can you have governance? And this is not a simple thing to do, and because of it, we're seeing the tremendous challenge in the in the corporation. John?

John Williams 06:45

I was thinking, that we're from the software area, and in software, you never get it right first time. And if you look at software projects, a lot of them fail, and you learn from those failures. And I'm thinking here of Amazon's paper about their 60 failures of projects. And these were not minor projects. The Fire Phone was one of them, and it just didn't go anywhere. And so my sense is, everyone's worried about this MIT study that said there were all these failures. And actually, as far as most projects go, I think those numbers are to be expected, that you're going to learn from the things that you get right, and then you're going to learn from the things that you get wrong, and to actually deploy something and make it a scalable product. The first thing is, you've got to be posing the right question. The next thing is, you need to have data that's good data. That's often a problem. You don't have the data that you really need to answer the question. The modeling piece often is, I'm thinking here the large language models now, and the agents, they're actually performing pretty well often. But, then trying to scale those systems is very difficult indeed. I think the human side is, is a real challenge. What we're seeing in MIT is we're not looking at the real problem that this is going to revolutionize education, and we can come back and discuss that more, because that's going to be scary for academia, that we're not going to be teaching classes of 20 or 30. We'll have a personal agent that will teach us, and that's going to just change education radically. And I think this is the thing that companies need to address. They need to get up to speed, get into this game, because it may be actually wiping them out. It's going to change things very radically, as you were saying, Barbara, this is not a moderate technology. This is something that we haven't seen before, it's certainly at the level of the Industrial Revolution, and that may be a good model, that 80% of the people were in farming before the Industrial Revolution. Afterwards, it's something like 2% and I think we're going to see similar issues with AI.

Keith Schlosser 09:25

From a CIO perspective, I think when we see new technology hitting the enterprise, we often see a rush to solve a singular problem, to try the technology, to get it up and running, to prove to the board in the C-suite that it can be done. The technology team, the operations team, and the business teams, are in the forefront of this technology. So that's what we're seeing. I talk to a lot of colleagues, CIOs, CTOs and others who have done that and with limited success. But I don't think it's all bad. I think that they're learning along the way, and they're learning what they need to do to prepare for an enterprise-class solution, a scalable, secure solution. They need to experience what the outcome of that singular use case is, and then put up proper guardrails and so on. So, from my perspective, get that behind you as quick as you can and start looking at a more robust platform that can handle front office to back office. Pick your tool and go and probably bring it up later as well. But it's all about the data. It's been said, getting the data right is going to increase your chances of success exponentially. These singular tries, if you will, will help you identify where your gaps are.

Barbara Call 10:54

That's great. Thank you, Keith, let's double click on little bit of that advice for CIOs. Abel, what's your advice for CIOs and CEOs for creating an AI-first enterprise, versus scratching the current itch?

Abel Sanchez 11:10

So I wanted to follow up in answering your question, Barbara, with what Keith said. He talked about people trying to do it themselves, and in the first days of cloud, you saw that a lot, right? We're going to build our own data center. We're going to create our own solutions. If you remember, in the first days of the cloud, financial institutions wouldn't go there, health care institutions wouldn't go there, government wouldn't go there. And they all gave the same excuse. We can't trust it, right? And of course, a decade later, when it comes to cloud spend, this is the number one spend for IT departments the world over, approaching a trillion dollars. So when it comes to this, and we think about what can be done and what's possible, we're still seeing that ramp up as well. There are many internal groups that say, "Hey, we can go open source, and we can build it" and this is what I was referring to before, where you can home-grow it completely. And you're starting to see that maturation curve as well. There are many that are saying, "Hey, we can't trust Company X, we're going to do it ourselves," and you also are seeing some hybrid solutions and some that are handing over completely. For example, if you think about Sierra, the customer service agents company, they will do it all for you, and you can just sort of hand over, right? So there's a number of perspectives here and approaches and think about that. And especially as you go into the C-suite and somebody pounds on the table and says, if we go to cloud, we're going to spend 100x more, right? Or if we if we start using OpenAI, all of our data is going to fly out the window. And so you're starting to see those parallels. And the last thing that I'll mention is that when it comes to implementations in the ones that I've seen with the least friction, and that are bottom up in many ways, there are those that are adopting agent construction in the non-technical sense. And I think this is a part of the workforce that will be empowered the most. I call them the power users. These are not the technical part of the organization. These are going to be the people that are supported by AI and can now build agents. And so you see companies like Moderna, you see companies such as Philips, that are creating agents in the thousands. And much like in cloud, from all of those, you can you find a subset, perhaps 100 or so, that provide more value. This is equivalent to the centers of excellence in the past when it came to cloud, and you're starting to see a similar type of journey, but this is with a different segment of the corporation, self serve it. And I can see John smiling, so I know he has different opinion. John?

John Williams 14:00

I'm not sure I'd let everyone create agents. The first thing you find is that you bump into security. And security is a major issue in the cloud, you can't get anywhere with a message unless you've got a security token. Machines will just not accept messages if they don't have that security. And so I'm chuckling, because we've both built websites with Lovable which is a great, great tool. They will generate a website for you very quickly indeed. The interesting thing is the back end, I looked at how many prompts I was using. So for the front end, you could get a decent website up with about five prompts. For the back end, it was over 100 at least. Because you're hitting security issues there that they go with super base, and it's you just cannot get away from security as soon as you build any kind of system. So I agree with you, you can just talk to the app and it'll create an agent for you. But I don't think that's the problem. It's when you deploy it, and if it's talking to other things. I think at the moment, you need your data engineers involved. Am I overstating that? I think Abel to agree with that, right?

Abel Sanchez 15:43

I'd say there are two types of agents. There are the ones that software engineers build, and then there are those that I would say are power users. These are people who do not write code and are building very helpful, help desk-type, applications that they can use and are simplifying their lives, but are not the ones that are rolling in security teams and are rolling in software engineers and need data persistence in a very different way. And so there's a very different, I think, use case to these very different groups.

John Williams 16:23

Yeah, would it be fair to say there are different kinds of agents?

Abel Sanchez 16:26

Yeah, absolutely.

John Williams 16:28

Simple agents and complex agents. We shouldn't get into it. But yeah.

Barbara Call 16:34

Okay, I love it. All right. My next question. In efforts that are not delivering or maybe that have failed to deliver anything, are you hearing that it's more technical issues or organizational readiness? And what can CIO do to solve both? John, let's start with you.

John Williams 16:53

Okay, so if you look at an organization here, I'm basically following a guy called Jeffrey Moore that wrote about a company having different zones and so say, if you look at your finance department, they're not going to be moving at the same clock tick as your R&D. And there's very different cultures within the same company. Now, Amazon kind of hit this when they went into cloud computing, they actually spun off a different organization, AWS, that has a very different culture to Amazon. And so this is the problem that most companies, I'd say, are hitting, is they're R&D people, they'll pick up this AI quickly. But the other part of the company that's actually say, still running and selling your product and has to be kept going has a very different clock tick and a very different culture. And so I think this is the problem. Main problem for corporations is it's fine while your R and D people are doing toy problems, but when they try to scale up now, they're going to start entering the main business of the company and taking resources away from other parts of the company. And that's where I think is the major challenge that these AI systems, once you start scaling them, they're going to be bumping up against your traditional business. The company needs to be ambidextrous. Part of it needs to be running at high speed, breaking things and failing, but getting but learning fast about AI. The other part is still maintaining your business. And I think that's the essential problem. You've got different cultures within the same corporation, and the CIO is caught in the middle, so to speak, they're not necessarily on one side or another. Then that's my my thought about it's a human problem. It's not it's not so much the AI.

Abel Sanchez 19:04

Here's the surprising thing, the AI works, and it really does surprisingly so beyond, I think, the understanding of most people it is freaking awesome. Now, when it comes to how do you leverage it in the organization, the numbers are right. The failures are off the chart. Now, if you look at small businesses, over 60% of them are having success, up 40% from in 2025 from 2024 and if you look at it compared to 2023 you have doubling and so they are rocking it. They are using it as ;an accounting team. They're using it as a communications team. What's happening in the organization, what's happening in the large corporations. I'm working right now in a proposal with a company in. Oil and Gas, and we selected a pretty low hanging fruit type of project to sort of get going. And as the proposal hardened, this simple thing that we were going to address surfaced the question of data. Keith brought up that it's all about the data, right? I've spent 20 years ;and sometimes in nations where there's a king, and the king said, You will do this, and there's still, there's data failures, right? And so in this scenario, as this hardened, it started to touch it. It started to touch HR, it started to touch surface. The question, so, what was going to be that interface? Who was going to maintain, who was going to look after it, what teams were going to support it? And on and on. And it starts to explode throughout the organization. Then the question comes to me back, what is the ROI I'm asking them? What is your vision? Who are you trying to be? What are you trying to achieve? And so all of these tensions that John is mentioning start to take place. So this is not a small business of two people where they're having a conversation across the table and they say, Yeah, let's use this to answer our email. No, this is existential. Who are we going to be? What are we going to use? How do we get there? Right?

Keith Schlosser 21:25

As we think about how a CIO can influence a project that may be off the rails, maybe just starting, it's all about the organizational structure around that project. Having done it for as long as I have a lot of projects are just handed over the fence to it. It builds it, they hand it back. Everybody celebrates and they move on. You can't have that anymore. You need everyone accountable at the same level, whether it's HR, the legal team, the IT team, the business operations and so on. You have to think about structuring the company, and in particular, an AI project team a little differently than what we've done in the past. Otherwise it will fail.

Abel Sanchez 22:12

Just a quick follow up on what Keith said, because in these conversations that I just mentioned, we were talking about six months, and then we were talking about a year. And then the IT representative, the CIO was saying it might be a year and a half. And so who is going to do that translation to the domain of the company? And that, how should I say Town Hall tied support to try to bring that understanding. Because, as Keith mentioned, and one of the the the friends that we have is Mark Schwartz, and he's written a great number of books on that many CIO have read. There's still that sort of mindset in many organizations. Says we will make the important decisions, and then we will have the IT department build it for us, right? And that, of course, no longer works, and how are we going to transition to that new type of partnership? Is one of the big questions.

Barbara Call 23:11

Really interesting. How does this all parlay into what it means to shift your company's operating model to a scalable, integrated business approach. John, what are your thoughts there?

John Williams 23:24

I just wrote a paper on this. Now, the upside is that we could monitor things pretty well now that so we can understand our internal processes. We've got slack and we've got very various technologies that we've the communication so we can monitor ourselves and see what we're really doing. The problem is the outside world that is changing as well. And so again, Keith comes up with this, the problem of data. Abel and I typically look at ourselves in academia, we've got terrible data about our students. We don't, I mean, we don't have it really. We have A's and B grades, and they vary. What does that mean? It's very difficult. And so I think corporations have to master their data so they understand what's going on. They have to have visibility of how these things are performing. That if you get a multi agent system, it's very difficult to monitor and to understand which of those agents is making what decision, what's happening to the context window between them are you? Do they all have the same context? Or do the different agents have different context? It's not an easy decision there. So things get complex very quickly, I think so what we're seeing is we've got some of the technology is evolving, but also I think that we've got. Business structure problem, the organizational structure needs to adapt. And that's not easy. Abel, I could see you as Jumping, yeah, straining, yeah, jump in.

Abel Sanchez 25:13

I'll mention what I mentioned before, that it needs to start off with a vision. And John and I were talking earlier today, and we were saying, "Are we getting that from our leadership?" How often do we see it? And I was speaking with a former student that's consulting in this space, and he was talking about 100 different companies that I spoken to, and one had a crystal clear vision. Their goal was, this is a chocolate company that nobody's going to touch. No human hands are going to touch human the chocolate production by 2030 This is a clear direction of where we're going when it comes to academia. The example I was given to John is, can we have the digital professor that can impart education at a fraction of the cost that it exists today, again, a clear direction of where we're going. Then when it comes to it, I would bring it back to what John was mentioning. We have failed to democratize data, and it's not for a lack of trying, right? The cycle of discovery, procurement, integration and eventually abandonment for my most data projects, is one that is pervasive in company, and it's not because we haven't tried the data warehouse, the lakes and the modern data stack and so on. The sponsor of the program, I give them heads up. The question of being able to ask questions about your own data, it's a big deal that hasn't been achieved, but this is an example of the state of data in organizations. And so as you propagate this through when it comes to change management, yes, consulting companies have made a fortune on this alone. But this is changing. For example, the new lobbying in the United States is shifting to influencers, so as opposed to the traditional model, and how is this measured, for example, in a corporation? Now, how do we bring in AI tools and start to gather that those things that are taking place internally, so that we can do smarter interventions, not to mention culture, right? And, and when it comes to careers, how are we going to level up? And we're talking here about one hour of instruction that could enable a big part of the corporation. We're not talking about a two years masters, right? And how is HR going to play on that? And what type of legislation or governance are we going to have? And so this is the big picture going forward by what can we do today? For example, for the CIO, the data pictures is a great one. There's a tremendous amount to do there. And whether you're doing cyber security investigations or you're trying to do analytics, this is at the heart of it. And so I think it's easier now, because there's more awareness right? Within the C-suite, for sure, but this is, this is what I would flag now.

Barbara Call 28:45

So Abel, if we can agree that AI is a needed capability in any modern organization, how should CIO prepare their teams to fully leverage the value of AI, while also at the same time retaining company culture and the importance of human interaction and knowledge in their business. I know it's a long question.

Abel Sanchez 29:11

I love this question. It's the question of, well, we have a job in the future, right? And here, I'll remind the audience, perhaps most of us already know that IBM built a machine at the end of the 90s to play the chess grand master Kasparov. The machine was called Blue Jean, and they faced off, and surprisingly, blue jean won, and it was written about and hyped about the world over. Now what people don't talk about often is that the following year, they had a follow up competition where you were allowed to bring your own machine to be able to compete against blue jean. So Kasparov returns with its own chess program. Handily beats blue jean, but the person that beats Kasparov in the entire field, is not a chess grandmaster or chess expert, it is an amateur with a couple of laptops, right? And so the takeaway here is that human plus machine is better than machine or human. And one of the things that I often say is, you will not be replaced by an AI. You will be replaced by a human using an AI. And this new class of workers that I'm referring to the power users. They are not the software profile people. They are the person working in finance that's figure out how to run a script to be able to summarize spreadsheets that had millions of rows and that before were crashing, as anybody's ever tried it, we're tracking We're crashing, and it was taking days to be able to carry out this simple task, and can Now do it supported by an LLM. And so this, I think, is a tremendous opportunity, right? And how do we enable this? At the very outset, there are four steps that companies tend to be following. The very first one is to identify this opportunity is the second one is to give that one hour so of instruction. There's five or six things that you need to know in order to be able to carry out a lot of these tasks and goals. And then, there's the identifying of in building and mapping those workflows, and ultimately by the organization, trying to understand which of those are high ROI. And so there's an opportunity here for those classes of people. And in some ways, you can think of these power users through four layers. The very first one are just consuming what exists. And if you think that is a simple task, simply being able to know what tool to use is actually quite a difficult problem, because the industry is innovating tremendously. Then there are those that are improving on that by doing some light programming. And then there are those that, of course, are the ones that John was referring to before, that are going to create their own agents and build beyond but I think within those first two classes, the ones that are simply going to be knowledgeable of what exists, plus those that are going to do light customizations, like I mentioned about people in finance, we're going to have capabilities that we haven't had before. So let me leave it there and pass it to John.

John Williams 32:35

Yeah, I think you're right, Abel. I agree that I think the human actually has some things that these systems don't have that these systems are trained on data. That's historical, right? Because the future doesn't yet exist, and that's a problem if the future has different characteristics or different statistics to the present, and the future is actually changing very rapidly. I'd say, look at society, and in the last 20 years, it's radically changed. And so there are weaknesses for these AI systems. But I do take your point about, the human plus the machine. I don't think that applies necessarily in all areas, because I was thinking here of the Microsoft's, diagnostic orchestrator for doctors that will diagnose better than the doctors. And what they found was, was that the doctors were biasing the machines. They they were saying, no, no, you're not right. And actually, the machine itself was better than Doctor plus machine. So I think there are cases where you've got to be a little bit careful. But coming back to that, I think the real challenge we've got is, what should the machine be doing and what should humans be doing. There's a great book. It's called Primal intelligence, and it points out, as humans, we've been honed, over the million years or so, that we can make decisions in low data situations where there isn't that much data around. Now, it turns out that if you've got, say, a war situation, that humans can outperform the machines that still in these war games, humans and humans plus machine. But having the human in the loop actually helps you win. So I think there are areas where humans outperform these machines. We have a lot of capabilities that sometimes we don't realize. We get we get fascinated with problems, and we work on problems for years, and we get passionate the machine. Machines, don't I mean, the machines just don't have that capability. We get anxious. It's a good sign that our plans are not working out, and we change our plans because our model isn't working.

Keith Schlosser 35:19

As we think about preparing the teams, I go back to my experience as a CIO, and I will tell you that it's my feeling that CIO is are in the people business. They're not banging on a keyboard, writing code and managing projects. They're managing people, and they're managing complex organizations. So I would stress the importance of the team's role in the transformation. To AI explain that their knowledge and their expertise is valued and they need to adapt. And I think having those straight and candid conversations with them is going to help prepare the team for what's coming. Give them access to education and tools that maybe you haven't been doing before, maybe they didn't value because they were so busy doing other things. I think it's time for a pivot there, and make sure that they are improving themselves and keeping up to speed, making sure that they understand the way things are going to get done in the future, not the way things the way they've been doing things up until now. I'll hit again. Prepare the data. If you want your team to be successful, you want the organization to be successful, you got to focus on the data. If you don't, you're going to frustrate a lot of people. You're going to frustrate the end users. You're going to frustrate the people actually doing the work and getting the AI and the agents Ready, set up guardrails, security first and transparency. If you don't do that, you're going to have distractions, hitting the team, hitting the organization. That one, you don't want to go through, and two, are just really going to derail any forward progress. Lastly, although it's a little off topic, I think I can bring it back around and stress why it's important to prepare. Do this. To prepare the teams is get your vendors who use AI to support you to follow your standards, because if you don't, you're going to distract the team. They are going to be presented with problems that maybe they aren't prepared to handle. So it's best to get your vendors who are supporting you, third party data providers, vendors that are providing services to you, make sure that they have their house in order when it comes to AI in the support of your organization and your team, in my opinion, that's what a CIO can start to do. There's a lot of other things as well. In the interest of time, I won't get into them, but those are pretty important ones, in my view.

Barbara Call 37:53

So to wrap things up, I want to ask you about the future. What's your prediction for the next big change in AI, and how can it and business leaders prepare for that change. Abel, let's start with you.

Abel Sanchez 38:05

So legislation is coming to some degree. The EU AI Act already broke ground, and although that's in flux as we speak, this is something that everybody needs to watch out for. I think about it personally, a little bit like cyber security. It's here. You need to pay attention to it. You need to know what your industry is doing, and you need to prepare for it. The same thing is going to happen with the AI. Everything else that we've said about doubling down on data, when it comes to cyber security, when it comes to compliance, everyone is going to be impacted, especially if you are operating in multiple multiple geographies.

John Williams 38:44

So I think Jensen Wang, the CEO of Nvidia, has an interesting way of looking at things. He says, look, these LLMs are pushing out tokens. Their responses are tokens that are intelligence and basically the hyper scalers are building these massive data centers that are going to be pushing out trillions of tokens. We're talking about two gigawatt power stations being built, four gigawatt power stations. Meta's proposing a massive one. And that, these are spitting out tokens that have intelligence and have value. Now, at the moment, there's a shortage of compute, and there's going to be a shortage of energy. And, you know, I think most companies of a decent size need to be thinking ahead. Where are you going to get your compute from? Because you're going to be short of compute, and there's going to be a widget bidding war for that compute you. And the power stations can't be built overnight, you know, to put up a decent power station, you're talking five years, and it's going to be, probably be nuclear, you know, model and nuclear reactors of some kind, maybe gas. But it's, it's clear that as a country, we're going to be short of energy, and that the infrastructure we have at the moment will not be sufficient to provide that. Now, that same problem is echoing through other countries. So Europe has doubled the cost for energy of the US, and we have doubled the cost of China. So the game, the real game, at the global scale, is going to be about energy, that you're going to have to have cheap energy. So that's the main thing I'm thinking about in the future, that AI needs energy.