Data Analytics Chat

The Rise of AI Agents

• Ben Parker

Share episode

In this episode of Data Analytics Chat, host Ben welcomes Ilya Meizin, SVP Head of AI Solutions at Dun & Bradstreet. Ilya shares his career journey from management consulting to leading AI solutions, highlighting the importance of implementation, continuous learning, and cross-domain expertise. He discusses the evolution of AI agents, their capabilities, and the shift from linear models to stateful, autonomous agents. The conversation delves into complexities of scaling AI, embedding agents in enterprise architecture, and balancing technical innovation with business value. Ilya also emphasizes the necessity of high-quality data and the role of collaboration across departments for successful AI deployment.

00:00 Introduction and Initial Insights
01:14 Guest Introduction and Career Overview
02:45 Transition from Strategy to Implementation
04:15 Learning and Adapting in Consulting
05:35 Defining Career Moments
10:35 Handling Challenges and Stress
13:03 Leadership and Team Dynamics
14:45 Balancing Technical and Business Skills
20:01 Focus and Credibility in Leadership
23:18 Introduction to AI Agents
23:39 Capabilities of AI Agents
24:22 Technical Aspects of AI Agents
25:39 Complex Problem Solving with AI Agents
28:40 High-Value Use Cases for AI Agents
32:14 Team Collaboration in AI Deployment
33:19 Challenges in Scaling AI Agents
37:21 Data Quality and AI Agents
45:12 Future of AI Agents in Enterprises
48:17 Conclusion and Final Thoughts

Thank you for listening!

ilya meizin:

I realized that I was a very good strategy consultant, but I had no idea how to implement my strategies. Just make reasonable assumptions, be transparent about your assumptions, and just get to a directionally correct answer. And directionally correct is, is much, much better than a blank page. So the fastest path forward is just find people who do. And the most important thing is kind of like. Just check your ego, right? And then once you're done checking your ego, check it again. You can be absolutely brilliant, but to do big things, you need a lot of good people. If you're always the smartest guy or the smartest gal in the room. You're in the wrong room. So just deliver, deliver, deliver, deliver. Because I think. If you, if you, if you get, if you gain a reputation of someone who can take on difficult challenges and just get things done influence will come, like, and, and progression will come as well.

ben parker:

Welcome to Data Analytics Chat, the podcast where we discuss the world of data AI and the careers shaping it. My guest today is Ilya Ma, who is the SVP head of AI Solutions at Dun and Bradstreet. In this episode, we will explore his journey, the challenges and insights he has learned during his career and the data topic. We'll look at the rise on AI agents. Ilia, welcome to the podcast.

ilya meizin:

Thank you very much, Ben. Thank you for having me on your show.

ben parker:

No, it's a problem. Obviously the topic is a popular one and everyone's, yeah. Getting involved with this. Simon, do you wanna start off with Yeah. Introducing yourself and sharing your career journey? I.

ilya meizin:

Yeah, sure. So like you said I'm the head of AI Solutions at Dun and Bradstreet. Obviously we are doing a lot with AI and agents and all things LLM. My career path has been anything but linear. So I started out in management consulting, then moved into strategy. Worked with private equity shops a lot, large corporates. Kind of, like never specialized in any industry vertical and did everything under the sun from automotive to. Aviation, metals, retail, consumer products which is it was great for me. It suits my personality. I really like learning and actually I get bored pretty quickly. So then I, at some point I realized that I was a good strategy professional, but I didn't know anything about what it. Takes to actually implement strategies. So I started looking to transition more into implementation roles, so to speak. Then came to Dun Bradstreet. About 12 years ago, started in strategic business planning and then did a lot of really interesting things across the company from technology finance, and eventually found my home and data science and ai, which again, just like it's a perfect field for someone who really likes variety and gets bored pretty quickly because ai, it's not really a discipline, it's just a collection of. Tools and solutions that can be applied to an infinite range of problems really. So I keep learning every day and, we're incredibly busy with all the cool AI and agent stuff, so it's been really great. I,

ben parker:

Yeah, I think it's definitely one you gotta keep on your toes and keep learning this at the moment, with yourself, you, me, you mentioned you are one that's sort open to working in many different industries. Do you think that's just more down to your,'cause obviously a lot of people, businesses want people that have got the domain expertise and I guess there's always that element of can someone learn a new industry? Was that just something you picked up as a sort of in your sort of consulting days?

ilya meizin:

Yeah, I think it was just a combination of things. Um. The group that, the consulting group that I was working for was relatively small, so everybody kind of expected to be able to do everything. And then once I got into the mode of doing that, I discovered that I really enjoy it.'cause you know, like it can be very stressful'cause you're constantly learning new things and you're basically, you're never doing anything twice. So there is a little bit of stress and the learning curve is always very steep. But it's just like, uh, it's, it's a lot of fun.'cause like I said, you keep learning and you're developing a very broad perspective on things. And then you can start, at some point you realize that you can draw some higher level inferences about, you know, like all these seemingly disparate industries, but you kind of like, you start seeing patterns. And that really helps. You know, and I've kind of, you know, once I discovered that. I really enjoy that and I'm okay, like, putting up with a little bit of stress that comes with, uh, you know, like I have to get up to speed on this thing in, in two, three weeks. I, I realized that I really, really like it and I've always tried to, to sort of like look for roles that allow me to do that.

ben parker:

Okay. Brilliant. And then so if you had to pinpoint one or two defining moments that's changed your career, what would they be?

ilya meizin:

So, you know, the pivot from, from strategy to implementation. So, you know, I mentioned, I, I realized that I was a very good strategy consultant, but I had no idea how to implement my strategies. And I actually, you know, I got that feedback from a very senior. Person in the firm I was working for and you know, in the moment, you know, I was taken aback, like walk out of the meeting and you mumble all kinds of words under your breath as like, what does, you know, of course I know how to implement. But then, you know, like I took a hard look at myself. I was like, well, yeah, like I've always been a, a consultant, so what do I know? And that's, that, that was really the point when. I started looking for roles outside of consulting and really like looking for roles that were not even on the strategy side, that were much more on implementation side. So that was a very, and, and, you know, you know, with 12 or or so years later or 13 years later I'm really happy that I did this. Um. Pivot. Like obviously, you know, like the, the, what I discovered is the world is infinitely messier and complicated, more complicated. And you know, like things that look great on a piece of paper, they just don't work, right. So it's, it's kind of like, you know, I, before I switched to industry as consultants say, I didn't truly realize that. I didn't see the, I didn't have the full picture of the world. So that was one very important moment. Another one I would say like learning to to be comfortable with ambiguity and uncertainty. And there's one very clear moment that like I always go back to in my mind. Back in my consulting days, I was on, on a project where I had to build. What consultants call a supply curve. Basically, it's a, it's an analysis and, uh, a framework of which producers in a specific industry can profit. At different price levels and who's gonna be essentially like, priced out of production. And I had to do that for ethanol. Which is, you know, like ethanol is a really complex industry. Like you can make ethanol from pretty much anything, right? Like you could, you could make ethanol from like stale beer. And then, you know, like the, the ethanol prices are linked to multiple commodities, oil and corn and sugar, and. There's regulations and subsidies all over the place. Capacity was surging so you know, it, it was an impossible thing to seemingly to analyze and I complained to my mentor, um, mark Chenko, who is now a senior partner at KPMG. That it was an impossible task, and basically like nobody on Earth knew the answer, right? And Mark's response was, that's right. That's exactly why the client is paying us. And then he just he kind of explained to me that everybody knows that you don't have a magic wand. Like nobody expects you to give a perfect answer. Just make reasonable assumptions, be transparent about your assumptions, and just get to a directionally correct answer. And directionally correct is, is much, much better than a blank page. And so that, that advice that I got from him just completely reframed how I think about really difficult problems. And, it probably has helped me more in my career than just about anything else, right? So I kind of like, I've always had to operate in environments where, like I said, it's the first time I'm doing something. Things are very much in flux. There's never enough information to make decisions. And actually most of the time I don't even know if I have enough information to make a decision, right? So that ability to just like separate things into what's knowable, what's not knowable, understand the risks of not knowing stuff, it's just finding a way to move forward and, and starting to do things. Like without that masterclass from Mark, like honestly I don't think. I would've been nearly as successful in my career.

ben parker:

Amazing. It's, I think, yeah, like you said, it's something getting right, starting something. It gets your brain going, doesn't it? The creativity sparks and I just. Things just start, you think your brain starts getting creative and then new ideas pop up and it just, things happen, doesn't it?

ilya meizin:

Exactly. Yeah. And then like you get feedback from your environments. Like you try things, some things work, some things don't work. Like the most important thing is just to get going. So

ben parker:

And

ilya meizin:

avoid the, like another, yeah. Another favorite consulting term, like just avoid analysis paralysis. Right.

ben parker:

Yeah. And that I said that sometimes starting you get so worked up, don't you? And then you end

ilya meizin:

Yes.

ben parker:

yeah. You just need to get something written out. Brilliant. So what's been the tough challenges you faced along the way, and I guess how have you overcome them?

ilya meizin:

So I mean, you know, it's, it's like I mentioned, right? Like, I make it sound like a lot of fun, like constantly learning and doing new things. But there is that element of. Of stress and sort of like constant discomfort because you're always doing things that you've never done before. And you know, like sometimes you have weeks or even days to just learn something completely new. Like back in consulting was a new industry or like in ai, like the, the pace of change is so quick that Yeah. Something that was state of the art a few months ago is outdated. Today. And, uh, you know, I think, I, I think the way I've always gone about it is just like, figure out who the real experts are, right? So when you're tackling something new, you don't even know what you don't know. So the fastest path forward is just find people who do. And the most important thing is kind of like. Just check your ego, right? And then once you're done checking your ego, check it again. So I literally call people every day and say, forgive me, but I'm about to ask you a lot of really dumb questions. Please bear with me. Um, and that, that, that mindset has served me incredibly well. Right? So you have three, four conversations with the right people. And you sort of like start seeing the contours of the problem space, right? So suddenly you understand what there is to know what you need to know and what you still don't know. And that's a huge win, right? When you can finally map out the landscape. That's a huge win. And you can. Go deeper, ask better questions, start connecting the dots, et cetera. Right? So I think, you know, like that's ultimately, this is how I've, I've handled, um, all these big challenges, just like staying curious, being patient and surrounding myself with people who know much more than I do and just being honest with myself and with the environment. It's like, this is something that I don't know anything about. Help me get up to speed.

ben parker:

Okay good. And then, so obviously you mentioned learning's been quite key for you,

ilya meizin:

Yeah.

ben parker:

Obviously you've been successful to get to leadership and there's many people out there that wanna do it, but maybe uncertain or feel they've lack of skills. How have you, how has your career progressed into leadership? Was it, has there been any particular, have you had mentors, training? Has it been self-study? How have you gone about it?

ilya meizin:

Yeah I've always had an incredible fortune of working for very smart, very successful people who I learned I've learned from and tremendous amount. And but also I've always had the great fortune of being surrounded by incredible team members.'Cause at the end of the day, I, I don't, like I, if someone junior is, or, like aspiring leaders are listening to this the most important thing to understand that it's all about people, right? Sometimes we get so wrapped up and we're solving this analytic problem, or. This is the thing that we're building. It's all about people. And as you progress, your success singularly depends on who you work with and your team members and the collaboration that you build with them and, and the relationships. So it's, it's, you know, more than anything, it's like it, people are people, right? Like people give their best when they feel understood and they feel appreciated and see how their work is creating genuine impact. So I think it's, it's always it's critical to remember that, like you can be absolutely brilliant, but to do big things, you need a lot of good people. To support you and, and drive that thing alongside with you. And I guess, you know, so that's one thing. And another thing would be what has worked really well for me is the ability to play in multiple worlds, right? My, my degree actually is in philosophy, but I have enough physics and math background to. To, uh, hold my own with very quantitative PhDs. And then I have my strategy background. You know, like, so at my core, I am quote unquote a business person, right? But obviously I can operate, I'm very comfortable operating in the, you know, AI data science world and leading these incredibly capable. More technical resources. So that ability kind of like to be on the intersection of the business and the AI has been incredibly helpful in my career. Just like, like blending technical insights, analytical insights, strategic insights, and just like going in and out of both those worlds, right? And they're like, there's so many, like AI is creating so many really interesting opportunities for, for people to do that, to play in both worlds. Like one example that like I kind of like talking about, so without getting too technical, that you know, you can take the way large language models, one of the things that they do, you can take a lot of unstructured text and you can turn that. Unstructured text into what they call like vector embeddings. Right? Which is basically like numeric representation of that text. A lot of ones and zeros, right? So there's this interesting question in my head. So when somebody does that, when you take, let's say a document and then you create a vector embedded representation of that document, is that the same data but in a different format? Or is that a completely new artifact? And the reason it's, it's important is because like, you know, like when you're talking about the legal aspect of ai, like contractual aspect of ai, so that's a really important question. Right? When, let's say your customer takes your data and creates some beddings out of it, is that a derivative output or is it basically the same data just in a different format? So I think Eli, so that's an example like lawyers. Who will be able to engage with these issues that didn't exist. Like even, you know, like two, three years ago. They will be, they'll have a lot of work and like, I think it's gonna be really interesting work. I think, like I said, like AI is creating a lot of these opportunities to be. Play in in, in multiple domains. And I think it's, you know, I think it's fun and, uh, there's not that many people who can do that. So, you know, that that's a pretty good way to, to get going with or accelerate your career rather.

ben parker:

Yeah, I definitely think now the, the skill set is. Changing, isn't it? You need the blender skills now. I think you need obviously the technical skills. If you can talk, obviously there's so many different roles as well but you need, technical skills are getting less technical now'cause you've got the heavy lifting being done. So you need to now have that business knowledge strategy. It's some, so it's just evolving, isn't it? Now I think even if you're a data scientist now you and you're pure techie, I think you need to start learning them business skills.'cause you're gonna need to be able to communicate with the business and add value or you skill you. Roll's just gonna get void.

ilya meizin:

Yeah, exactly. Exactly. And like, you know, the AI is blending so much, you know, all this age agentic all these age agentic capabilities, and, and in a few years, if you're a business leader who, who doesn't really understand these tools or these technologies. I think you're really, um, locking yourself out of a huge part of, like the problems that you can credibly try to solve. Yeah, I completely agree with you.

ben parker:

And you men, oh, you mentioned you've got, you've got a blend of good skills. So is there, is there anything else that sort of helped you progress into leadership?

ilya meizin:

I mean, like I said, like I, you know, I, I've been very, very lucky. I've, I've always worked for and with incredibly good people and the way I sort of like think about it if you're always the smartest guy or the smartest gal in the room. You're in the wrong room. You know, I, I, I think every person on teams that I've been, that I've had the privilege to lead is much better than me at many things. And that's great. That's how it should be if you ask me, like, so I don't, you know, I don't try to. To kind of like pretend that this is not the case. I lean into it. I want people to be better than me and learn thing, I mean, know things that I don't know and I can learn from them. I think that's been really critical for me. Then so the, the, the another thing is I'm pretty, uh. Pretty crazy about protecting my focus, right? There's just like so much stuff going on and, and the pace is just unrelenting, right? So even if days were 72 hour long, not 24 hours, if I engaged with everything that I can engage, I wouldn't get anything done. And that's one of the skills that I learned in consulting.'cause sometimes we would do. Projects where we had to help a private equity shop make a decision about buying a company and not buying a company. And we had like three weeks to to, learn about a target company and completely new space. So if that's, if that's the timelines that you're operating with, then you learn very quickly to separate things into those that are critical to understand in this context. And things that are nice to have to understand, right? So it's sort of like I've been applying the same mindset also to problems, right? Like, so I'm trying to do whatever I'm trying to do what is critical, what is not critical. And it's just like, I don't I don't, I don't have the, the luxury of engaging things that are not critical. And that's actually really helpful, right? So you end up, you end up doing more and you end up doing. Bigger things that drive bigger value for your company or your organization? Yeah, I think, I think the, you know, we talked about the ability to play in, in multiple worlds, and I guess, you know, like the last, but very much, not least, maybe it is the most important thing is build credibility before, before you seek influence, right? So just deliver, deliver, deliver, deliver. Because I think. If you, if you, if you get, if you gain a reputation of someone who can take on difficult challenges and just get things done influence will come, like, and, and progression will come as well. So that's probably the main thing.

ben parker:

Yeah, I like comment about the focus.'cause I think that is key, isn't it? I mean, you need to, there's so much going on and you need to, what's gonna move the needle basically isn't what's gonna, what's the big thing. And again, that's the same with business. I think everyone's got bit loss, regenerative ai, haven't they? Again, tr there's so many projects you wanna do, you need to, what's gonna impact the business right now?'Cause there'll be so much you wanna do in the business now. There's so much opportunity now, but. What's the current challenge, isn't it really? You need to focus on what's gonna be the big impact for you right now.

ilya meizin:

That's right. That's right.

ben parker:

Okay. So if say, I wanted, I was an aspiring leader, what advice would you give myself? Give.

ilya meizin:

Advice to give yourself? Um, that's a great question. I think, you know, ev everything, so you, you get, you're in a really interesting position because you get to talk to people from many different walks of life and different roles. And many different industries. So you kind of, you know, like to do successfully what you do on this podcast. So you must be curious and you must be willing to engage with things that you maybe don't know much about. And, sounds like you're doing really, really well with that.

ben parker:

I hope so. That's fine. Okay, so let's tap into the question then on AI agents. Obviously this is, yeah, massive next evolution, obviously brings many opportunities and I guess there's obviously challenges and risks in the business environment, so they're obviously pushed as the next layer on top of LLMs. In your view, what capabilities do they unlock that base models cannot achieve on their own right now?

ilya meizin:

This is an awesome question. This is really like truly foundational to understand. So there's a couple of things, right? So first of all, LLMs understand and agents do, right? So there, there's there's a huge progression from LLMs that are essentially like passive interlocutors or assistance. Where whereas agents are active problem solvers, you know, like a digital coworker that can reason, plan and execute things. So just to get like a tad more technical agents add state. Planning memory and tool usage, right? So like I planning memory and tool usage. I think that's pretty self-explanatory. Let me talk a little bit about state. So LLMs are, are awesome at understanding and, and generating language, but they're not inherently stateful, right? They don't retain goals, they don't retain memory or, or continuity beyond. The current context, right? And agents add that missing layer of planning and, and memory and action. They can decide what to do next. They can call external tools. They can coordinate steps over time. So. And, and, and all those things is just again, like, just to bring it back like LLMs talk. LLMs can converse with you, but agents can actually do things and like pretty, pretty complex things at that. So, and speaking of complexity, so the other thing is agents, I apologize, lost my voice. And so sorry agents can decompose large, large, complex problems into smaller tasks and orchestrate how those tasks are solved. Something that even smartest models, smartest LMS really struggle with, right? So you can, like, if you throw, if you ask even the smartest LLM. In the world to interact with petabytes of diverse data and ask it to, to derive conclusions from that data. It's too much, right? Like it's just too much data and there's too much variety and there's too much complexity. So LLMs in these situations, hallucinate like crazy. They make, they just make things up. Like their answers sound very confident, but factually. There are a lot of inaccuracies. So agents on the other hand work very differently. They kind of, they embrace tool use. They can call APIs, they can run database queries, they can delegate to specialists, sub-agents. And the beauty is, so those tools or, or subagents or, or APIs. You, they can be specialists like in just one thing, right? So you can have an agent tool that is only good at retrieving, like in, in our world, for example, data about ESG, right? And then you'll have ESG data, and then you'll have another tool that is exceptionally good at. Retrieving data about I don't know, like news about a company, right? And all those specialist tools, they'll be extremely good. They will use deterministic mechanisms of finding that information. Meaning, you know, like they, they will convert fluid conversational language from the user into whatever, like SQL calls or, or whatever, right? And by breaking the problem down into a bunch of smaller pieces for each piece, you have a specialist that is really, really good at doing that particular thing. So suddenly you can handle like enormous amounts of data and very complex, multifaceted user queries. And you can do that. Very successfully. Right. And, and that's just huge, right? Like you it's, even today, even like the lo the, the biggest, smartest lms, they hallucinate quite a bit. So it's like I said, like you can't just unleash them directly in the business context. Like, like you can't afford. You know, your tolerance for hallucination is extremely small. Like you can't even afford, like 5% of hallucinations. So that in those environments, like you really need to bring in agents that are exceptionally good at breaking those enormous problems down and just solving them one step at a time.

ben parker:

And so what would be like a telltale sign of a high value use case for agents then?

ilya meizin:

So I think there's a couple of things, right? So obviously, you know, everything, everything that you do, like it, it needs to have, clear value, right? Like, like if you're deploying an agent, you need to understand why you're doing it. What's the value that, that you want to drive, be it. Time savings or, or, or new revenue or efficiencies, or you can just, in some cases you can do something they were not able to do previously. So the first thing is like, what's the actual tangible impact, right? So that's sort of a given, right? That specifically for agents, I think repeatability is a big thing. If there is a workflow or a task that happens. Often enough, and it's sort of like more or less the same thing that is being done repeatedly over time. That's a really good signal that this is something where an agent can be quite impactful. Also complexity is actually like, I would say that complexity is, and by complexity I mean, a task may require. Reasoning and context, right? So you're not just like, not a single deterministic answer because like we said a little bit earlier, like, agents can actually break down problems, figure out what path to choose, which tools to use and which sequence. So if there's a little bit of complexity that's, that's also really good. And I guess, you know, like there's, there's also a dose of realism. Agents are very, very powerful tools, but it's not like it's a magic wand. So the sort of like the way to think about it is if you drop the brilliant person into the same task and ask them to do it, would they say, yep, I've got everything that I need to do this task. Or would they say, you didn't give me the data or the right tools to execute this task. If it's the latter, then agents are not going to be successful to execute again, like super, super powerful, but they can't create something out of nothing. It means that if that's the answer, then it means that you have to do some pre-work before deploying agents into the space. And of course, you know, in, in. In enterprise context, of course you have to be very mindful about regulatory compliance and sort of like government context. There are some use cases that inherently riskier than other use cases, and you just need to be, you need to understand very well the risk profiles of different use cases and judge accordingly. If this is something that. Um, you're very comfortable deploying agents into, or something that maybe you want to wait for a few months to get more clarity. So I would say just see, just to kind of like recap, impact repeatability, um, some, you know, like some complexity, some, like just realistic realism, right? And just remembering to think of regulatory and compliance stuff.

ben parker:

So does that mean, oh, so this is a new, new thing for businesses now? So is this mean you need to get, when you're getting involved with this? This has gotta be bringing in multiple different departments within the business to cover all bases. So it's like not more team effort now.

ilya meizin:

yeah, exactly right. So I I always say that Gen AI or agents, it's a team sport it like it truly is right. If you have a bunch of super smart people doing something in the basement I don't think it's gonna be I don't think it's going to scale. I don't think it's going to succeed. So you really need to have, you know, like different people call it differently. Like I've heard a name, gen AI pod like you, you have to have your product people, you have to have your technology people. You need to have legal, cybersecurity, um, you know, like all, all the, the folks who commercialize things. So all, all the sort of like, how do we price it, how do we bill it, how do we manage, uh, usage? So you gotta have, and I'm probably forgetting a bunch of important functions, right? So if you don't have all that. As a cohesive group or cohesive team, it's gonna be pretty hard to to, to deploy at scale.

ben parker:

Okay. And then obviously many pilots fail to scale. So what are the hidden blockers that organizations overlook when deploying agents?

ilya meizin:

So, you know, it's interesting. Complexity doesn't scale linearly. It kind of scales exponentially. I'm so sorry, I'm still having my voice issues. So, if you prototype an agent that works on a small, clean sliver of data for a pretty narrow use case, you can't assume that it will behave the same way. When it's exposed to the full scope of, of your data or workflows, right? So you, you, you may, something that works beautifully at small scale, you may discover that when you scaled up, you're running into memory limits context, window constraints, retrieval, quality orchestration load, and it's like all the things that don't show up in, in your demos. But they will show up in production. So that's one. Second, you gotta have agent ready data, right? Like agents can only reason as well as the data that they can see and interact with. So you need to make sure that you have the right data for the agents to work with. Meaning your data has to be trusted, timely, contextualized. If it's sort of like locked in silos or poorly mapped where you can just tr you can't trust your data, um, that that's, it's gonna be pretty hard to, to do anything at scale. And then there's this, this interesting concept of like signal to noise ratio, right? So let's say you do have agent ready data. So what's really important is to not overwhelm the agent. With data that it doesn't really need to solve the problems that you want it to solve. So it's sort of like, you know, if I go back to my, you know, like if you drop the really smart person into the problem, would they be able to do it? You know how like they say that most people can remember and repeat back only about seven to 10 digits if you tried to make people memorize. 20 digits or 30 digits, almost everyone would fail, right? And there comes a point, you know, like you get to 50 digits, like 99.9% of people would fail. And it's the same thing with agents, right? Like the more data you throw at them, the higher the probability that they will get confused and will start hallucinating. So and so, the challenge is not to stuff more data in the context window. Like you really gotta know. What is the right data and the most impactful data that you want to bring into the problem. So this is not, you know, like the more, the better. Like you need to have, like this is the most impactful data that I can bring into this problem. So, and I think it's interesting that this concept of context engineering, it's really starting to evolve. It's, so, it's definitely an area to watch as you're scaling. And of course, you know, like I, it goes without saying, you gotta have the right infrastructure. Because I, you know, like some pilots often run on sort of like highly bes, bespoke setups, which don't translate into production, right? Because, you know, like, just because something runs. I, you know what? Well enough for, for your pilot, you may very quickly discover that that setup that you have to run the pilot is way too expensive to run in production, right? And like I said, you know, like some things just don't scale. So the question is not when you do your, your pilot, the question is not like, Hey, can I get it to work? The question really should be, can I get it to work 10,000 times a day reliably? At enterprise level. So I think these are the big things.

ben parker:

I definitely hearing a lot around data quality is getting so key now, isn't it?

ilya meizin:

Yeah. Absolutely. Yeah, exactly. It's sometimes what's really interesting, LLMs can do amazing things and then you throw agents into the mix and it's I look at what my team creates and I'm like. Wow, this is science fiction, right? I'm in it 24 7, but I still have those moments. Like I look at the stuff that they're building, I'm like, wow. Like I, I literally can't believe what I'm seeing. But so sometimes like we get really anchored on that, that we forget, like agents are still software, right? Like the old, garbage in, garbage out. As smart and capable as agents are like, you put bad data in, you're gonna get that stuff out. So yeah, having the right data, trusted, timely, contextualized well, accurate obviously. That is, you're going to fail if you don't have all that.

ben parker:

A lot of people have said especially with lms, like building LM is easy, the easy part, easier part, the back, like the getting data is the hardest bit. Is that the same with building agents?

ilya meizin:

So it, it's an interesting question, right? Um, building good agents is. Pretty hard, right? It's, it takes a lot of very specialized skills ability to innovate very, very quickly. So I think, I would say it's both right? Like you do need excellent teams and excellent infrastructure and like, just excellent sort of like, innovation process. But yeah it's, you can't do, even if you, you're exceptionally good at building stuff, you're not gonna be successful if you don't have great data. So it's, you really need both.

ben parker:

Okay, perfect. And how do you separate orchestration complexity from genuine intelligence?

ilya meizin:

See that, that's, that's an interesting, that's a really interesting question. Like intelligence is. Intelligence is a very loaded term. And, and I have, I have a degree in philosophy. Um, during my time at Yale, actually, I took a seminar with Nick Bostrom, who is one of the leading thinkers in, philosophy of ai. So I can debate intelligence and what that means and whether we have it or not until the cows come home. I think honestly, like I, I will punt a little bit on the intelligence thing, and I instead, I will say that in practice, the real metric is not intelligence, but value, right? So to me, the real question is, are agents creating value beyond, you know, like your traditional workflows or, you know, what, what automation could deliver? And to me the answer is an unequivocal yes. I think, you know, I, I think. So it's interesting, like sometimes when people say agents, they think of like early day agents that didn't really have agency or autonomy. They were really just like very sophisticated workflows wrapped around an LLM like executing static plans with, with style and flair, right? And that's sort of like that, that linear. Chains of things that agents were asked to ex execute. That's where, like the term chain of thought came from. There's just linearly, linear chains of, of execution. But now the agents that, that we have today with memory, like really good memory and planning and tool usage, they truly can, can act autonomously and decide how to approach a problem. Instead of following a fixed script. Right. So, and I mentioned agents are stateful. Um, they don't start from scratch with every request. Like they remember prior steps and they can track goals. They can track their execution against the goals, and they can adjust their next move based on. On outcomes, right? So they don't have to move linearly through predefined paths anymore, right? They can navigate dynamically through almost like a graph of actions that they can take. You know, like and then they can choose the most relevant path in real time. They have the autonomy to loop back, skip steps, like change course when new information comes in. It's, it's really like pretty similar to how humans solve problems, right? So I think, you know, like the agents that we have today absolutely add tremendous value on top of what those early day agents we're bringing to the table. And it's important to remember, like. It feels like there's so much has happened and the cha the pace of innovation so quick that it feels like these things have been around forever. We're still very early days of these, of this agentic revolution. We'll get to a place where these systems can learn and they can adapt. They can continuously improve outcomes. You know, I, I, I know that I totally punted on, on, uh, intelligence, but. Yeah. To me the value is unquestionably there. I see it every day many times. And the beauty is we barely scratch the surface.

ben parker:

Yeah. Yeah, true. Like it's still early days, isn't it? Like it's.

ilya meizin:

Yeah

ben parker:

again, this is where I guess obvious you've got the technical knowledge, but also if you've got the business domain expertise here, like how can you actually, again, add value to the business? It's gonna, you're gonna add some small power to what you are trying to achieve.

ilya meizin:

Yeah, exactly. Exactly.

ben parker:

So you see like a big risk in over reliance agents for businesses.

ilya meizin:

Um, I think you know it's a little bit of what I said earlier, unbelievably powerful tools, frameworks, um, when well designed, they can do amazing things. But agents are software, albeit software with autonomy to make very complex decisions and based on complex inputs, all that stuff, right? But you know, like it's really important not to get complacent and just like stay on top of how those environments evolve and how inputs shift over time. Um. So I would say, you know, like today, agents should augment human judgment. But you shouldn't just say, all right, like, we have an agent running, like we can go fishing for the rest of the month, right? Like, maybe we'll happen very, very quickly as models get better and these frameworks get better. Like maybe we'll be able to, to rely on these things more. But again, like unbelievably powerful tools. But it's important to remember that today they're tools. So yeah, I, I think it's really important to stay on top of, like to monitor what goes in monitor, what goes out track performance really, really closely. And there's some, you know, like really powerful tools that allow you to do that today, right? Like that, that can see all the like prompts, for example. Into an agent system and see all the outputs and do some like, extremely intelligent things like analyzing analyzing outputs from the quality perspective and like even governance, right? Like so I think, it, the, the risk is forgetting that we still need people very closely involved with these powerful tools.

ben parker:

It.

ilya meizin:

Yeah, definitely.

ben parker:

Cool. So then if we look, say two to three years ahead, do you see agents becoming embedded into the enterprise architecture or is it gonna be more of a like a niche tactical tool for businesses? I.

ilya meizin:

I think, um, I think it's definitely in the former, I think agents will become very embedded. And you know, like in a way they'll simply be how enterprise systems work, right? So you'll see agent capabilities built directly into CRMs, ERPs master data management platforms, often behind the scenes, sort of like similar to how APIs operates today. And I think, you know, the key enabler of that, of that embeddedness. It's going to be standardization, right? So if you take things like model context, protocol, MCP they will make a lot of data. Like MCP will make a lot of really, really good data sets. Enterprise, sorry. It will make really good data sets like agent ready. Um, they will make real good data sets more accessible, uh, well described, easier for agents to consume and understand. Um, like, so MCP will give agents a common language to, to use for communications across systems, right? So that's gonna be a very, very important driver of that embeddedness. The other one is going to be a to a, or, you know, like the protocol for agent to agent collaboration. I think, you know, like we're, I, maybe it hasn't happened at scale yet, but I think very quickly we're gonna get to a point where agents communicate with other agents across different enterprises, across different systems. So you kind of, you know, that, that will allow enterprises to move from having isolated. AI tools or agents to really this, these like connected collaborative ecosystems that reason and act together. So I I, I, I definitely think that the, the some of this embeddedness is happening, is happening already, right? So for example, like DDMV has pretty incredible MCP, servers that we already exposed to customers, we have a, to a capabilities. So we're starting to see that embeddedness already, and I think it's, it's only going to get much more pronounced very quickly. So I, I think like the vision is sort of like a lot of these interconnected agents powering workflows, powering important decisions, powering analytics behind the scenes, and sort of like seamlessly integrated through protocols like MCP and coordination layers, like eight to a again, I think this is still pretty early stages of that transformation, but I think we're starting to see the curve starting to bend up pretty quickly.

ben parker:

Brilliant. Cool. Here we go. It's been a pleasure having you on shared some great insights and yeah, so obviously a hot topic. So thanks for joining us.

ilya meizin:

Thank you very much. Thank you again for having me on your show. Like really interesting questions. There's a lot going on in the space and things are evolving very quickly, so it's definitely an area to watch and to learn about and to be excited about.

ben parker:

Yep. I agree. And to our listeners, if you enjoyed today's episode Yep. If you could give us a follow, leave, a quick review, just really helps others discover the show. Thank you.