Enterprise AI, Why Data Matters and Mosaic AI with Nick Eayrs, Simon Fassot and Craig Wiley

Fresh out of the studio, we sat down with three leaders driving enterprise AI transformation across Asia-Pacific — Nick Eayrs and Craig Wiley from Databricks and Simon Fassot from Hafnia — to explore how data intelligence and generative AI are reshaping industries. Nick Eayrs, Vice President at Databricks, unpacked why governance, security, and lifecycle management are foundational to building trustworthy AI. Drawing parallels to commodity markets and Asia’s rising compute economy, he showed how transparent data catalogs and end-to-end control enable enterprises to safely deploy multi-agent frameworks at scale. Simon Fassot, Head of Data Analytics and AI at Hafnia, shared how one of the world’s largest tanker operators is reinventing itself with Databricks. From breaking data silos via a “data supermarket” to deploying the in-house Gen AI copilot “Marvis AI”. Simon showed how shipping — a traditionally conservative industry — can accelerate ROI, streamline documentation, and empower domain experts to build their own dashboards and AI use cases. Finally, Craig Wiley, Senior Director of Product Management at Databricks, revealed the thinking behind Mosaic AI and Agent Bricks. He explained how Databricks is addressing the toughest problems in LLM operations — evaluation, model flexibility, and enterprise governance — while giving customers the ability to A/B test models, orchestrate agents, and drive accuracy without sacrificing speed. Together, these conversations show that clean, well-governed data plus rigorous evaluation loops are becoming the decisive factors for success. Across industries and regions, enterprise AI is moving from experimentation to production — and the companies that build on solid data foundations and flexible model strategies will define the next wave of innovation in Asia-Pacific.

Part 1: Nick Eayrs from Databricks and Simon Fassot from Hafnia

Part 2: Craig Wiley from Databricks

"I think the biggest trap to potentially fall into is, "Hey, it's moving so fast, so much is changing. Let's just wait it out." Completely the wrong approach. You just gotta get started." - Nick Eayrs from Databricks

"As tech people within the shipping industry, how do we explain, how do we make it accessible to all our users? So that's where we came up with the idea of a data supermarket, with in mind really the target of enabling self-service for our business. So by giving the analogy of a supermarket, it was much easier at the beginning to explain our business." - Simon Fassot from Hafnia

"85% of AI use cases are being evaluated by the engineer who built it saying, 'yep, seemed to work pretty well.' If you're gonna build a system that's going to be critical to the business, that's going to be important that it gets it right, then you can't do that without evaluations." - Craig Wiley

Part 1: Nick Eayrs from Databricks and Simon Fassot from Hafnia

Part 2: Craig Wiley from Databricks

Profile:

Nick Eayrs, Vice President of Field Engineering, Asia Pacific & Japan at Databricks (LinkedIn)

Simon Fassot, General Manager and Head of Global Data and Analytics at Hafnia (LinkedIn)

Craig Wiley, Senior Director of Product Management at Databricks and Mosaic AI (LinkedIn)

Here is the edited transcript of our conversation:

Part 1: Nick Eayrs

Bernard Leong: Welcome to Analyse Asia, the premier podcast dedicated to dissecting the pulse of business, technology and media in Asia. I'm Bernard Leong. I'm at the Data + AI World Tour by Databricks today in Singapore, and with me today is Nick Eayrs, Vice President of Field Engineering, Asia Pacific and Japan at Databricks. As we are going to dive into the evolving landscape of AI in Asia, from infrastructure to intelligence and why data intelligence may be more important than general intelligence for enterprises. So Nick is going to discuss how companies can build the foundation for responsible and scalable AI in the region. So, Nick, welcome to the show. Thank you very much for having me. Like every other guest on Analyse Asia, we want to hear your origin story. So how did you start your career and what got you into the area of data and AI?

Nick Eayrs: Well first things first, it's been a journey, a 25-year journey in analytics. To be honest, it's all I've ever done. Not in a bad way. I absolutely love the space. I'm super passionate about the space. I wouldn't be doing anything else. About 25 years ago, I started working in Sub-Saharan Africa, South Africa. For a management information systems company. I was actually developing an early version of what you would call an intelligence platform.

Yes. Specifically for financial services customers in Sub-Saharan Africa. We had great success with our product. I was the developer. I was the engineer from a pre-sales post-sales perspective. I was technical support, I was marketing, I was everything. But it was a wonderful experience building that product, taking that product to market, and getting really good success and traction in financial services.

From there, that product, obviously all of the contracts and the work that we had done, got transitioned to a company known as Computer Sciences Corporation. Yes. Now DXC. I established myself in DXC as a data and AI practitioner, and got spotted by a pretty prominent analytics company called SAS Statistical Analysis Software.

So I joined SAS based on my experience working in financial services, and actually built out an enterprise architecture division. For SAS in South Africa and eventually grew that to an international scale, running an international function that spanned Europe, Middle East, Africa, and Asia Pacific, and Japan, and actually relocated to the UK as a part of that process. From SAS, I saw the wave of open source innovation and I got really passionate about that space.

You know, SAS, very much closed source. I was a part of the open source movement and really saw a lot of value and a lot of future success in that model. So I opted to join a company called Hortonworks, one of the original open source creators around data and analytics. I took Hortonworks to the merger with Cloudera. So now today it's known as Cloudera, and, you know, during that merger phase. Obviously I sought other opportunities to take my knowledge and experience to the next level. Databricks, thankfully, was looking for someone with my skills and talents. So, I joined Databricks in EMEA, now I've been with the company for seven years, three years in EMEA, and four years in Asia Pacific and Japan based in Singapore.

Bernard Leong: Right. So reflecting on your journey, because it's pretty interesting in South Africa, EMEA and now Asia Pacific. What are the lessons that you can share with my audience? Yeah, from navigating, I guess very interesting career transformations across, but still within the same realm of engineering data and AI.

Nick Eayrs: Yeah, I think number one, it's really being customer obsessed in everything that you do. I don't mean to say that just to pay it lip service. I genuinely believe you've got to immerse yourself into the customer's world. You've really got to deeply appreciate the norms around their industry, their environment, their business context.

I think that's something that I've learned throughout my career and it's uniquely different in every single market, and especially in Asia Pacific and Japan. Let's be honest, Asia Pacific and Japan is, you know, I think by last count, 51 discrete countries. You cannot approach a customer in APJ the same way.

You've really got to meet them where they are and deeply, deeply understand their industry. The socioeconomic state that they're operating in. So I think having deep appreciation for customers and being customer obsessed and having great empathy for the journey they're on, and the journey they've been on is probably one of the most important lessons learned.

I think the other thing is, you know, over the years I've learned to be very good at challenging customers from an intellectually honest but humble sort of perspective. I think sometimes, you know, vendors can come across maybe a little bit arrogant and a little bit stubborn in their opinions.

I always try to side on the side of, you know, be humble, be intellectually curious, and challenge customers to think differently, but with real empathy for the position that they're in. I think that's a really good thing that I've learned over the years as well.

Bernard Leong: That's a great description of what customer obsession means.

Also, specifically, sometimes we have to say no to customer as well. So I want to get to the main subject of the day, which is about data intelligence and AI infrastructure in Asia Pacific and Japan. So maybe let's start baselining by defining some terms for the audience. Yeah. How would you define data intelligence and how is it different from what people typically think of as artificial general intelligence?

Nick Eayrs: Great question. Let's start with the latter first around describing general intelligence. I think everyone today will be familiar with these ChatGPT type interfaces, right? Yes. Very prevalent, especially in the consumer market. I think everyone at some point has leveraged these and experimented with them, be it to solve your son or daughter's math homework, or to plan a holiday for the family or to just, you know, get some basic quiz or general knowledge questions answered. The reason I bring that up is that's probably the best example to bring to life of general intelligence. So what is it? It's AI that's been applied, models that have been applied and built upon the corpus, the public corpus of knowledge. So everything that's out on the web, effectively the internet.

So effectively these models have been trained on a large corpus, a large footprint of knowledge in the public domain. Very good for general questions, general knowledge Q&A, planning your holiday solving general math problems. Not very good for solving your business problems because yeah, it's not trained on your business. It doesn't know your products and services, it doesn't know your policies and procedures, and it doesn't know your operational norms and taxonomy around how you describe your business.

So that's really what we're trying to solve is not the general problems, but the domain specific problems, the business specific problems. To do that, you don't want to train the AI on the general corpus of knowledge. You want to bring all of that AI capability and you want to train it and attach it and add it to your data because the competitive differentiator in enterprise is your data. So bring the model, bring the AI to your data and apply it to your world and allow it to learn your business, your business taxonomy, and about your products, services, and policies. That's what we call data intelligence.

Bernard Leong: So why is data intelligence now more important than general intelligence for enterprises today, especially in Asia?

Nick Eayrs: Well, again, what's going to uniquely differentiate you with how you engage with a citizen if you're a government agency, a patient, if you're in health and life sciences, or a consumer, if you're in a consumer orientated world, it's going to be your products and services. Really what you want to do is you want to leverage the information that you have.

Around your products, your services, what you take to market and your customers. The best way to do that in a differentiated way, again, is to leverage the insights you have internally, not the insights externally. So really that's the secret source, right? The moat for customers that we work with is their data being AI infused and AI enabled so they can uniquely differentiate the products and services that they take into market. The customers that do that, we're seeing massive growth in adoption.

Bernard Leong: So I remember a study that was done with the Economist and Databricks where they said that probably 15% of companies are actually ready for having the correct infrastructure for AI. What are the kind of unique challenges that companies in Asia Pacific and Japan face when building that infrastructure People don't appreciate that when everybody wants to do AI, but if you don't organize your data, everything is just garbage in, garbage out.

Nick Eayrs: You're spot on. You're absolutely right. We talk about having really good data and AI foundations. What does that mean? To your point, these models, and when you bring these models and you bring this AI capability to your data, they're only going to be as good as your data. So if the data is garbage, you're feeding garbage into the model, you're feeding garbage into these AI systems, these agents, you will get garbage out. So garbage in, garbage out. So the most foundational thing you can do to ensure that you're getting great insights, great outcomes is to make sure that you collate, store and curate high quality data and as much data as possible.

So, yes, the quality matters, but you also want to make sure the data's very rich and that you've got as much data as possible that represents your customers, your products and services. All combined in one place and in a quality form. So the most important thing, again, is good quality data and having lots of data that is good quality altogether.

So you can use that to derive insights and build AI agents, applications, and experiences. The other thing that I think is really important is connecting this to the why, because let's be honest, technology alone is never going to be the secret to success. Technology becomes incredibly impactful when it's connected to a higher mission and a higher purpose.

So understanding why this matters to the business, why this matters to the enterprise, how is it going to transform the way that we serve our customers? How is it going to transform lives in the way that we are serving patients, perhaps again, in healthcare, that really is really important because when you connect the people in the organization to the why. Then they can really understand, okay, now I know why we need this data. Now I know why we need to build these AI agents and models, and I understand the purpose and the mission, and I can connect to that and I can get behind that. I'm enthused by that.

Bernard Leong: One interesting thing is I know a lot of high growth companies in Southeast Asia: Grab and Zendit, they are all using Databricks. They're all publicly known references. So. One question I do really think about is how can companies in the region future proof their data and AI strategy to stay competitive in the next 5 to 10 years? Enterprise AI, I think we're really at the beginning because most of the current generative AI solutions tend to be point solutions, but not end to end. For example, like the way how Databricks does even combining with, say, Mosaic AI as well.

Nick Eayrs: Yes. Well, number one, you've got to start. I think the biggest trap to potentially fall into is, "Hey, it's moving so fast, so much is changing. Let's just wait it out." Completely the wrong approach. You just got to get started. The best way to get started is to leverage all of the best practices and all of the success stories that are already out there. It's okay to pick up, rinse and repeat and reuse the best practices that are out there, but make sure you are doing that with a vendor or a partner of choice. Databricks has a proven track record of success. What we are doing there as Databricks is that: we're bringing together a lot of PhD level research around the space. Let's be honest, the space is changing every single day. Like every single day there's a new model, there's a new technique, there's a new product surface area that's coming out. We are trying to put all of our researchers to work. To make sure that they're picking the best technological improvements around algorithms, models, techniques, packaging that up into really easy to consume, what we call Agent Bricks, then allowing our customers to consume and use those to solve domain specific problems. Rather than you figuring out which analytical technique to choose, which model to choose, you focus on the business problem. We've got that tried and tested, proven track record of success, of bringing the best techniques together, and we package it up in a very simple, easy to consume product surface area, all driven by natural language.

So literally our customers can interact and start to build AI agents and AI agent systems. Just using natural language. So the most important thing is get started now. Don't wait because it's going to move so fast. Work with a partner or vendor that has a proven track record of success in innovating in this space. Leading and defining the best practices and the technology evolution, which we are doing. That can really help abstract that complexity for you so you can focus on the business problem and not stitching together technology.

Bernard Leong: Can you share a real world example where Databricks enabled a customer, say, in Asia Pacific in Japan, to derive transformative value from this data intelligence.

Nick Eayrs: Absolutely, I've got tons that I can talk about probably for hours as much as you can. Maybe I'll start with some of the customers that were here this week up on stage, talking for us in various forms. You know, one of those customers is none other than Standard Chartered Bank. Standard Chartered Bank actually took a very unique horizontal use case around cybersecurity, and more specifically what we call SIEM. So, you know, really looking at security incidents and figuring out how do they triage those and how do they get ahead of cybersecurity threats and power AI driven analysis, threat detection, and prevention. So if you think about this world historically, you would have to go and license a lot of software, what we call SIEM software, to go and pull together logs from all over your enterprise landscape.

Bernard Leong: Is it like monitoring multiple threads of information for cybersecurity?

Nick Eayrs: Exactly. You're pulling information from routers, from firewalls, from applications, from, you know, human footfall in and out of the building. You're trying to connect the dots around. Is this activity on the app in the building through these systems and interfaces? Is that good behavior or bad behavior? Is it a bad actor? Is it a good actor? All of that data, if you think about the data there, it's all from different vendors. You might have one firewall vendor. You might have five firewall vendors across the estate that you have. So there's lots of different data formats. This data is typically high volume, high telemetry that's coming in super fast, and security is a problem space. You can't wait a week or two to figure out that someone's hacked your system. Like you need to intervene and react in real time and you need to prevent it before it even happens. So the problem is high volume, high complexity, and real time in nature. We worked obviously with the bank to solve that problem, and as a result, what they've been able to do is they've been able to take all of that telemetry from all of these different systems into Databricks. They've been able to build a world class cybersecurity platform on top of Databricks, and they are driving massive improvements. Like the TCO [total cost of ownership] savings were absolutely off the charts. But not only that, it's the time to value that has come down so significantly. So now instead of, you know, waiting weeks, months to kind of figure out a cybersecurity threat, they're down to days and hours. They will get down to minutes.

Bernard Leong: Do you have other customers stories you want to share too?

Nick Eayrs: Yeah, a hundred percent. Another one I can talk to is a customer called TechComBank. So one of the largest privately held bank in Vietnam. So we have a wonderful stakeholder and champion, Santhosh Mahendiran, who was up on stage telling the story. But the difference in this one is it's all about customer centricity and effectively what Santosh set out to do with the bank, and we've been very fortunate to partner with Santosh and the bank on this journey is to build a customer brain. So how can you intelligently derive insights about all of your customers and all the attributes about your customer in real time, to really hyper-personalize everything that you're doing with them in the moment? To make sure that you enrich their lives and they have a better experience with a bank, but ultimately they're getting better banking offers. All of that delivered through app experiences, through digital experiences in real time. So there, Databricks is pulling together, obviously all the information around the customer attributes that they have. They have, last time I checked over 12,000 attributes associated with the customer. So everything from demographics all the way to, you know, various other attributes. They use this with Databricks to derive a one-to-one personalized offer in real time, delivered in an experience that the customer knows and loves. So if they love the app. It's in the app. If they prefer email, it's in email. Not only that, it's delivered in the moment that matters to them. So, you know, they might check email in the evening, they'll get it in the evening. They might like their morning run and a coffee. It'll get delivered before they hit the Starbucks store. So really personalized, tailored one-to-one marketing through this customer brain that we've built with Santosh.

Bernard Leong: I'll keep the third one later, but we'll get into the subject of responsible AI, which has now become a priority for enterprises. How does Databricks ensure, say, AI products are deployed within safety, transparency, and ethical considerations in mind? I do bear in mind that just now for the two use cases, they were all financial services. So it comes with a lot of scrutiny and I probably will appreciate that Databricks must have done a lot of work to deal with all the regulations. Yes. How do you put systems in control as well?

Nick Eayrs: We have a number of other use cases in highly regulated industry across the board, not just financial services, government, and other areas as well. But I think it comes down to two things. Look, ultimately it is a shared responsibility model. There is a shared responsibility around the trusted relationship that we establish with our customers, because ultimately the customers own the data, right? So the shared responsibility side is making sure that the data has. The correct classification and is used for ethical, responsible, and sustainable purposes. From our side, we work really closely with our customers to do a couple of things. One, making sure they have the right organization model to ensure that AI is successful, sustainable, responsible, and ethical. What does that mean? We help them think through how do you marry the vision, mission, and purpose with real concrete strategies? A very executable plan that can be measured. Then how do we drive the literacy up so the entire organization is aligned behind that why, and everybody can contribute to that journey and understands their impact along the way in measurable terms.

So there's a lot that we do around organizing the company, the leadership, and the organizational entities for success. The other part then is everything that we do to work both with government. To work with industry around understanding and appreciating and shaping and informing a position on regulatory and compliance requirements in a market. This is complex, right? Again, as I said, APJ 51 discrete countries, all of these countries are looking at the regs and compliance, some of them in slightly different ways, but we work very closely with government. We work very closely with our customers. To interpret this changing landscape and to figure out what is best practice, what are the emerging best practices, and to ensure that they're thoughtful in their deployment of the technology.

In addition to that, there's all the technology aspects, right? Helping our customers make sure that when they stand up the data and AI foundations that the data is secure by default, that the right authorization and authentication mechanisms are in place, that they are looking at things and leveraging best practices, such as red teaming. Red teaming is an approach that you use on models. To check that the models are responding well and are giving great responses and can't be hacked or subject to prompt injection or other techniques. So there's a bunch of stuff that we do around ensuring that the technology is well architected, well implemented and tested and validated and is secure and is robust.

Bernard Leong: I suppose that it is the same in dealing with things like drift as well. Yes. Data concept drift within the models on that. So, do you help your customers to embed those kind of trust and governance in their AI life cycle right from the beginning?

Nick Eayrs: The foundation of everything in our data intelligence platform is this concept of governance, security, and control. What’s very unique to the way Databricks approaches this is we’ve always been a data and AI company. So when we think about this, we don’t think about it just from a tabular data perspective or an unstructured data perspective. We think about that entire life cycle — the agent being built, the model being built, the agent being served, and the multi-agent framework being stood up — because you have to traverse that entire life cycle. If you’re only focused on the data, you’re missing everything else that gets derived from the data. Ultimately this impacts consumers or citizens leveraging your products and services. So data governance is fundamental to the data intelligence platform. A key capability there is what we call a catalog. We’ve got this wonderfully rich capability called Unity Catalog. We’ve even created an open-source project off the back of that. So our customers have choice, interoperability, and a safer exit strategy should they choose to do so. Defining that catalog, setting up that catalog, and putting all the data under the management of that catalog is a foundational step. Then, once it’s in there, everything around how that data evolves into reports, dashboards, models, and models being served, inference being done, and agents being deployed into apps — all of that lifecycle management, all of that lineage, all of that traceability — comes with that platform capability.

Bernard Leong: Do you find that now in enterprise, just to dive deeper, is trust and governance a thing? Do you find that customers are now much more cognizant about the AI, the traces of what the AI is doing, and get some kind of an audit trail of how the agents are operating or maybe the AI models? Do you see that kind of deeper dive into the transparency of the models or interpretability or explainability of the models?

Nick Eayrs: Yes, it’s absolutely an area of concern, and increasingly as the regulations evolve, this is an area for everyone to stay on top of. Traceability, lineage, and auditability are really important — all key platform capabilities that we’re solving for. But the other part of that question is, is the problem the explainability and interpretability, or is the problem the quality of the output? More customers right now are facing problems around the quality of the output. How do you actually know when you build these things that you’re getting a good quality outcome?

That’s one of the hardest problems. It goes into a very complex area we call evaluation — model evaluation and agent evaluation. It’s quite a complex domain, and if you’re trying to tackle this yourself, it’s really hard to get set up for success. We’re trying to make that super easy.

If you go back to what I referenced earlier around Agent Bricks, the way Agent Bricks works is you define the problem you’re trying to solve — this domain-specific problem — and we will automatically evaluate the quality of it for you. We’ll automatically figure out, for this sort of domain problem, what sort of quality and metrics you probably want to look at.

Then, with human-based feedback in the loop, we start to raise the quality bar. The human feedback continues through continuous feedback, continuously improving the quality of the output. Most customers are actually really worried about that. They can get a model built quickly. They can get an agent built quickly. Getting it to be of quality is super hard.

Bernard Leong: That I think is art by itself. I have three questions that I want to get your thoughts before we go to the close. So my first question is, what's the one thing you know about data intelligence and AI and enterprise AI in Asia that very few do, but should?

Nick Eayrs: Well, I won't profess to be the only one that knows this. But what I think is super unique and what I love talking about is just the psychology is different in Asia Pacific and Japan. I'm not going to cast aspersions any which way, but when I talk to customers and when I talk to folks in Asia Pacific and Japan, it's always about the future. It's not about the past. It's not about bringing back the past or bringing back things historically, it's about the future, and I love that mentality around the future, future success. Future growth, future opportunity, securing the future for the next generation, securing a better future for all. I just love the fact that, you know, the culture and the psychology in Asia Pacific and Japan is all about the future as opposed to how do we bring back the part?

Bernard Leong: That's interesting. It means the mindset is really changing in the region. So what is the one question you wish more people would ask you about Databricks or building responsible and scalable AI systems, but they don't?

Nick Eayrs: I wish more people would ask us, "How do you get started?"

Bernard Leong: So, I will ask you now: How do you get started?

Nick Eayrs: Like, it sounds super simple, but I think sometimes we get so bogged down into the technical details and into the weeds of discussions around this model versus that model. This evaluation technique versus that evaluation technique. It goes back to what I said earlier, just get started. Just ask us how to get started. We just want to build with customers. Like we love building with customers. So for us, you know, it's ultimately around, I wish we could just build more with more customers and I would love that opportunity to do so. So all customers out there, all prospects out there, anyone who's interested, just come and ask us, how do you get started? We'd love to build with you.

Bernard Leong: So when the customers ask you: Do they actually first come to you with a set of problems or you have to try to figure out like which part of the platform can help you to get those quick wins?

Nick Eayrs: It’s a bit of both. There are a couple of ways you can slice that problem. Some customers come to us with very opinionated perspectives around industry-specific things they’re trying to solve. In that instance, we’ll work back from that specific business challenge and figure out what the pain points are and how that’s inhibiting success for the business.

What sort of value can we create for the customer by adopting our technology and then put forward a solution? Other customers are more interested in what we’re seeing with regards to their peers, what we’re seeing in the industry more broadly, and what our perspective is on which use cases are working and which are not.

For those customers, typically what we’ll do is more of a blue-sky thinking ideation session and show them the art of the possible. This is what we’re seeing. These are the use cases we think are valuable, high impact, and high feasibility of success if you have the requisite data in place.

Then we’ll showcase that to customers so that they can see it, believe it, and select it as a use case to tackle.

Bernard Leong: So they’re actually able to imagine what is the art of the possible. Then basically you can actually narrow down to those top two or three use cases. That may be able to help them and then start the building blocks moving.

Nick Eayrs: That’s right. Because you never want to boil the ocean. Again, some of that “why are we not getting started?” is inertia and fear of change — thinking “this is going to be another 12 to 18 months.”

Whereas if we can focus on something and distill it down to something business-critical, high impact, high feasibility, we can quickly get to business value and then build trust, excitement, and energy around tackling the next one.

Bernard Leong: I just want to get one little part of the getting started. How about the education for the customer?

Nick Eayrs: Huge. Absolutely huge. As I said, technology alone is not going to be the solution. We spoke a little around that AI organization piece — leadership alignment, aligning the vision, mission, and purpose — but you need to

Bernard Leong: enable people to use Databricks and how to exactly.

Nick Eayrs: use it well. Exactly. So part of that is thinking deeply around the talent you have, the talent you need to acquire, and the talent you need to develop. It’s never one of these things. You’re going to have to do all of those things. You’re going to have to figure out talent acquisition, development, and making sure all the talent in the organization is clear on what performance management looks like, what good looks like when you’re using data and AI.

So you’ve got to set objective measures around: we’re going to enable you, we’re going to give you these tools, we’re going to give you this learning and enablement, but we’re also going to measure you this way. If you’re successful with data and AI, this is how you know you’ve achieved success.

Bernard Leong: That would be what great looks like for me. Correct. So my traditional closing question: what does great look like for Databricks in Asia Pacific and Japan in a context of enabling data intelligence at scale?

Nick Eayrs: Ultimately we want to be the destination of choice. We want to be the first thing customers consider when they’re looking to solve any data and AI business problem. We really want everyone — whether a customer today, a prospect tomorrow, or academia — to consider Databricks as the destination of choice for building and solving your data and AI problems.

Bernard Leong: Wow. That's a great one. So we have in closing: two very quick questions. What any recommendations that have inspired you recently?

Nick Eayrs: Well, I definitely recommend to the learning and enablement point, you know, get out there and get busy. With Databricks, we've got this wonderful thing called Databricks free edition. So we recently ran a very cool hackathon. We called it the Smart Business Intelligence Insights Challenge. We basically set a bunch of people free on the platform to go and experiment and build crazy cool insights on top of Databricks. That was wonderful. It was absolutely wonderful. I must get the link from you then. Yes, I would like to go and try it out. There's some fantastic, again, customer examples of what they built. So, I would love for people to do that. Right? Get out there, get into Databricks free, start building your own apps, start building your own solutions as a completely free, full cut to Databricks and you can get started today.

Bernard Leong: So how can my audience find you and or stay relevant with what is going on with Databricks in terms of the field engineering and all the work with customers?

Nick Eayrs: Always check out the Databricks blog which is the company blog. There is an RSS feed on our documentation as well. If you're a live update nerd like I am, I follow religiously every single update to the documentation. Please connect and find me on LinkedIn, and follow me on LinkedIn as well, and I'd be happy to support our customers out there.

Bernard Leong: Of course everybody knows there's just one more thing here, so. We have been talking about customers. So one of the interesting thing that I got from Nick today is I'm going to interview a customer. He's Simon Fassot, general manager and Global Head of Data Analytics at Hafnia.

Nick Eayrs: So wonderful that we'll be introducing Simon. You know, a great example again of an organization that has very clear vision, mission, and purpose around how they're going to use data and AI in shipping. It's just been wonderful partnering with them.

Bernard Leong: Thank you very much Nick, and let's continue the conversation.

Part 2: Simon Fassot

Bernard Leong: As promised with me today, Simon Fassot. General manager and head of global data and analytics at Hafnia and we promised a customer story for Databricks today. Nick was talking very highly about the initiatives that are going on with your company. Can you first share your role and destination within Hafnia?

Simon Fassot: Of course. Hi everyone. I’m Simon, working in Hafnia for three years. Previously I was working in the corporate investment bank, a French bank. I’m French, I guess you get it. Yes. So my role in Hafnia is I’m leading the data analytics and AI team. We have a team of around 10 engineers. We are doing our best to produce value out of all the data that we have in a highly regulated industry, which is shipping.

Bernard Leong: Can you introduce Hafnia as a company business? You just said it is in the area of shipping. What does it do in the world of shipping itself?

Simon Fassot: Alright, so Hafnia is one of the leaders in the tanker industry. We have over 200 vessels. We have mainly two activities. We are ship owners — we own vessels — and we are also running an activity of pool management where we basically take care of the operations of the vessels from other owners. These are the two businesses that Hafnia is leading.

Bernard Leong: So it operates globally.

Simon Fassot: It’s a global company. Yes. We have offices in Singapore — we are the HQ. We also have another one in Copenhagen, another one in Houston, and one in Dubai.

Bernard Leong: Those are all the main ports. I know you have about 200 vessels and a probably fully integrated shipping platform. So where does data sit in to enable you to scale, to be able to know what’s going on across the world? Shipping is a very important business as we learned from the pandemic, especially on the supply chain side.

Simon Fassot: In terms of IT architecture, we worked hard internally after COVID to be fully cloud enabled, which really helped us to be very agile in what we’re delivering to the business. Most of our data now resides in Databricks. We are fully cloud oriented. We use a lot of applications for the business, for HR, for operations, for the commercial teams. We have dozens of applications, but all the data are getting centralized in Databricks to help make a lot of value out of all the data we’re collecting across those applications that are not always talking together. Thanks to Databricks developments we are deploying to our businesses, we are enabling applications to talk together and businesses to talk together to break silos.

Bernard Leong: What was the mental model behind choosing Databricks in the end? Is it because it’s easy to use, or was it very easy to deal with the use cases that you have and also to be able to approach with the data itself?

Simon Fassot: Yeah. Historically we were working on SQL Server with a classic legacy data warehouse. Now the management really wanted to break silos and enable self-service within the company. Of course, we have more than 300 employees onshore and more than 4,000 employees on the vessels. We like to have everyone have easy access to data. How to do it? Basically through enabling self-service. So what tool on the market, two or three years ago, what were the tools that were available? We had a few options. We really assessed Databricks and of course Snowflake, plus the Microsoft solutions. We came up with Databricks really because of all the different layers and all different maturity of our businesses. We have three different kinds of users. We have the users who are not interested in manipulating data — they want to conduct their daily operations.

We have those kinds of users who need data to take decisions. So they need to consume dashboards. They need to consume any kind of data. So they need business intelligence very quickly and they want to see. They don’t really want to do a lot of the work in order to put the data into different visualizations.

The third one are the, what we call the domain experts. This is what made us choose Databricks. It’s this capacity of having all kinds of users accessing the data. The domain experts are basically experts that we’ve trained within the different business teams on how to use Python, how to use SQL, how to themselves bring value to their colleagues within their business team in order to be basically the extension of the data team. So we needed to choose a tool that was allowing us to do that. Databricks was designated as the best candidate for it.

Bernard Leong: So they also enabled the team of 10 that is working with you to make sure that everything is in place so that you can build exactly with Databricks as well. So what is it that, how Databricks is transforming how your commercial, financial, even operational teams access and act on the data with the data intelligence platform itself?

Simon Fassot: Alright, so before focusing on the architecture we had to focus on building a setup with the different businesses and my team by splitting the activities but bringing the activities together without segregating or building silos.

So how did we do that? We basically chose to build three different squads. Within my team we have the commercial squad, the financial squad, and the technical squad. By building this setup we have squads dedicated to specific areas of the business and can really focus on each business value.

What we are focusing on now is projects with high ROI. We work hard with the business and those domain experts who are embedded within the business on selecting the projects with the highest ROI. Each squad will define a list of projects that will bring real value to our business. We will focus on developing those projects.

Bernard Leong: I see.

Simon Fassot: Whereas for the day-to-day activity, we are counting on the skills of our domain experts to develop the BI dashboards and the data team can focus on the more advanced developments. That’s how we’ve enabled and managed to really focus where the data can bring value.

Bernard Leong: So how do you make the data accessible, say to non-technical stakeholders? I guess, what’s the response from, say, business units when they work with Databricks — invisible to them — but they actually see all the data that they need?

Simon Fassot: Alright, so we have different levels of access. We can speak about Gen AI actually. Yeah. So you have the classic data engineering where Databricks can work as an IT system and will help applications to speak together. This is invisible for our businesses. We also have the classic BI where the business will consume data through the dashboards.

Bernard Leong: So you have also implemented retrieval augmented generation.

Simon Fassot: Yes. So it's the last layer. But before enabling AI, we had to ensure that our data platform and our data were pristine. We've tried in the past two years, right after ChatGPT came out to enable very quick wins. Very nice POCs. But we've noticed that without a solid data foundation, all those POCs were just not strong enough in terms of quality. So the quality was at the center of everything, where we have the nice demonstration for the different vendors for the different marketing campaigns, but in reality, what matters the most, it's good data and thanks to Databricks and all the innovation that we can put in place on this great platform, on the Unity Catalog platform. We were able to control the quality of the data we were providing to our Gen AI or to our AI use cases. I see.

Bernard Leong: This is the key to enable AI, right? So with the foundation model within the platform itself, so you are able to do questions and answers on that. Just walk me through like, how did the Gen AI enable that and maybe what kind of questions now can it answer?

Simon Fassot: So Gen AI for me, we have two different phases. The first one, it's how to use Gen AI to ingest data. We could not ingest in the past. I'm talking about unstructured documents mainly. Yeah, that's right. So we have contracts. We can have SOPs or the vessel manuals. Yeah. All those documents are complex and of different shapes.

At the beginning we've tried to find the magical solution, the generic solution that will help us to load all those data seamlessly and have a very nice output to enable Q&A and chatbots. But the reality is each kind of document, each kind of input have to have its own way of working on the data.

So to make sense out of those data. It's not so simple. It's a lot of work, a lot of trial and error. We need a solid platform that will help us to monitor the performance of what we're developing. So this is the first phase, how to ingest unstructured data to the data platform. Then the second phase is how to basically consume this data.

Yes, correct. Then we have the data that is coming from unstructured documents. We have the data coming from our database data lake. Basically we have our data coming from emails. From communications. Yeah. Data coming from the different applications that we're building on top of the Delta Lake platform.

So how, how does it work for each scenario? We will choose the right tool on the Databricks platform. Okay. So for instance, for the documents, we will use Vector database from Databricks. For the structured data. We will use Gen workspaces. Now with the enablement of the agentic solutions, yes, all those tools can speak together and I'm extremely excited about the future and how this will look like, where those tools could work one by one, but now that we can see applications that can bring those tools together to make sense out of all those data, it's going to be very, very powerful.

Bernard Leong: So you are one of those rare customers I know who actually got a retrieval augmented generation to work. I think in this case you achieved it with Databricks. Do you find that, let's say for example, questions and answers are more of art, like trying to figure out the correct data to get the correct answer for the Q&A from maybe different business units than, rather than the science itself.

Then you have to do a lot of different evaluations on the questions and answers.

Simon Fassot: Yes, exactly. So as I said, you have the beautiful POCs that work for demos. Then you have real life where in real life, you just need one bad answer from an LLM to lose the trust and confidence from our users and completely lose them.

This is really what we try to avoid. That's why we really take our time before releasing any solution in production. I can give a simple use case where we use Gen AI but not as a chatbot. I think by embedding Gen AI technology within the business process, that's where we will bring huge value compared to only using chatbots.

So I can give an example. So we have the SI reports, inspection reports on our vessels. Yes. Those inspectors will give some comments, some observations about things that could be improved onboard the vessels. Then this document is sent back to our superintendents onshore. Yes.

The superintendents have to analyze this document and answer to each one of the comments. In our company we have thousands of SOPs, and for each observation, we have a corresponding SOP where we could find the solution. So the objective of this application is basically for one observation. Do we have SOPs?

That will be able to answer to the observation or help at least our superintendents to come up with a very good answer or to shorten the time it takes. Here we're not using a chatbot, but we're using RAG within our workflow in the application to really build use cases that have an impact for the business.

Bernard Leong: So it actually follows the business process rather than being a separate chatbot by itself.

Simon Fassot: Exactly. This concept of having embedded Gen AI technology as a tool in the workflow, I think is key for process automation, or at least to assist our business users with better performance and higher quality.

Bernard Leong: So I’ve done some pre-research before this. I know there are two initiatives. One is Marvis AI. The other one is the data supermarket. Can you talk about what Marvis and Data Supermarket are supposed to do for Hafnia?

Simon Fassot: Of course. We don’t work in a tech industry. We work in the shipping industry. The shipping industry is different, so our users are focusing on the shipping industry and everything that goes with that. As tech people within the shipping industry, how do we explain and make it accessible to all our users? That’s where we came up with the idea of the data supermarket, with in mind the target of enabling self-service for our business.

I see. By giving the analogy of a supermarket, it was much easier at the beginning to explain to our business: if you want to get access to our data, you come to the supermarket, do your shopping, select the right data at the right place, go back home with a tool like Power BI and cook yourself and build your own dashboard. That’s how we’ve enabled self-service by building this data supermarket. At the beginning we were talking about the supermarket to help users understand, but the name stayed since then.

So, on top of this data supermarket, we spent two years working hard on building something strong that works very well, built on top of Databricks. Now we have a very well-organized data structure. As I said earlier in this podcast, without this solid structure, general use cases are really not even possible. Now that we’re confident this structure is solid, we wanted to enable generic use cases again. We wanted to find a way to personalize or basically give a persona to this Gen AI within the company and instead of saying, “this is a generic tool,” or “this is a generic process,” we call it Marvis.

For our users, when we speak about Gen AI, we speak about Marvis. It really helped us to enable different use cases and for the business to project themselves in some processes: “If I ask Marvis this question, then I could get a nice answer from the Gen system” — so they don’t have to understand the technicalities behind the scene.

Bernard Leong: So Marvis, what is it? So it’s a Gen AI tool. It’s a GPT kind of tool based on our company data. It’s kind of a copilot from the way you are explaining it to me.

Simon Fassot: It’s definitely a copilot connected to the company’s data. We have different ways of accessing those data as we spoke. We have the RAG, we have the structured data. We have also, as we did the speech today, a partnership with Neo4j, based on Databricks data loaded to Neo4j. It’s a Gen AI tool connected with our whole data ecosystem, and then we can use it for chatbot and also for our internal processes.

Bernard Leong: I think people do not appreciate this, but in the maritime industry you have a lot of complex documents, standard operating procedures and you need to manage these documents effectively. I suppose Marvis has been a great help in doing that.

Simon Fassot: Yes. Basically, as I said, Marvis helped us to integrate all those documents in our knowledge base and to consume those documents and by enabling all those use cases that were not possible two or three years ago. We have access to a huge amount of data compared to the only structured data world. I think the Gen AI use cases are a new chapter for us as a data engineering team, where we now need to really upskill, ensure that the internal team of developers we have within the company are constantly learning and keen to dig even more. That’s really with this kind of tool like Databricks that we can enable that without so much complexity, where the complexity is not in using the tool but more in finding the right use case.

Bernard Leong: Do you foresee a day where anybody in the shipping industry doesn’t really know the technical aspects but, because of generative AI and prompting in English or any language — French, etc. — they can just prompt and get whatever insights they want from your system?

Simon Fassot: Exactly. I think with the agentic AI systems that we see more and more, we will be able to build systems that are smart enough to handle any kind of use case. We have to also be careful with the hype. We have to stay focused on what works, and we have to maybe take our time, not push too quickly the features that we are not in control. But yes, definitely the agentic AI, by bringing all kinds of data from the industry and from the different departments together, will basically be a huge change.

Bernard Leong: What excites you most about where Hafnia is heading with AI and data?

Simon Fassot: Alright, so now that we've built our data supermarket. Yes. Data supermarket is well,

Bernard Leong: and you have Marvis as well.

Simon Fassot: Marvis as well is known within the company. It's used quite a lot. Marvis AI is becoming also a tool that people are using in their day-to-day activities. The next step is how do we break silos? How do we bring data to everyone in the business. How can someone from the commercial team have access to financial data, have access to operational data? How do we bring everyone together using data, using the same data? So this is our next initiative. Now we are building what we call DNA Port. So it's a platform built on top of Databricks, on top of Marvis AI, where we will bring all those data together in one single place.

I see. Instead of having scattered dashboards around application A, application B, we will bring everything under the same hood. Really break the silos. Marvis AI will be the links among all those processes where within this application, IMOS data, which is a very famous ERP system in shipping, can speak with our financial system.

Instead of having two disconnected systems, this DNA Port will be the platform where applications can finally talk together.

Bernard Leong: So what excites you with your partnership with Databricks and how are they going to support you for the data on the mission for Hafnia?

Simon Fassot: Databricks helps us to stay focused on business value, and this is the number one criteria for us. We don’t want to spend time like we used to do in the past on the operational side of IT. We really want to spend on business value. With the constant innovation that Databricks brings to the table, it enables us to develop very nice applications like DNA Port, built on the very new lakehouse-based technology from Databricks.

Before we had to consume different systems from different SaaS services. Now with this unified platform, it makes us extremely good with the velocity we can have and the development pace we are achieving. We can bring data, AI, and front end together under one single platform.

This is a game changer for us, for velocity but also for the skills within the team where everybody — all data engineers, software engineers, and AI engineers — are talking the same language, which is this unified platform.

Bernard Leong: So, Simon, many thanks for coming on and really telling me about shipping, what Hafnia is doing with Databricks, and also I think this is the first time I really see something that really works for retrieval augmented generation as well. So thank you for coming on.

Simon Fassot: Thank you. Thank you very much.

Interview 3: Craig Wiley

Bernard Leong: Welcome to Analyse Asia, the premier podcast dedicated to dissecting the powers of business, technology, and media in Asia. I’m Bernard Leong, and today we are diving into the forefront of enterprise AI from model development to deployment at scale. With me today is Craig Wiley, Senior Director of Product Management at Databricks, who leads Mosaic AI, the company’s unified platform for production generative AI. So, Craig, I’m a fan. Welcome to the show.

Craig Wiley: Thank you very much.

Bernard Leong: Yes. So very quickly, we always like to talk about origin stories. How did you start your career and what drew you into the world of AI and product leadership?

Craig Wiley: I was working at Amazon in the consumer business and found that, while the company was data-driven, there was still a lot of opportunity and we started diving deep on questions that were asked or on problems that we had. Very quickly I realized that the business analysts I worked with had questions they couldn’t answer. So we started hiring economists. Then we found there were some questions the economists couldn’t answer. So we started hiring this new, at the time, brand new class, which was data scientists and machine learning engineers. This was back in 2012, 2013. When those guys got started, they often weren’t moving nearly as quickly as the business needed them to.

So it really became a “what on that last project we worked on could have helped you go faster?” As soon as they would tell me, I’d grab a team of engineers, we’d see if we could help solve that problem and then drive the business faster. That’s how I got started. Then when AWS decided they wanted to get into machine learning, they came through to the consumer business to see who was doing this. It sounds very much like AWS and I was there, and they said, “Hey, do you want to come build SageMaker?” and literally at the time SageMaker was one sentence in a document.

Bernard Leong: One curious part of your career is that you have led AI product teams at both Google Cloud and Amazon Web Services. What are some of the pivotal lessons that you have learned building AI platforms at scale across multiple cloud providers? Because I’m a fan of your work because of ML operations — it’s the ability to deploy, build, train, and deploy models at scale. Can you talk about that experience?

Craig Wiley: I’ve had this amazing opportunity to follow and at times to help influence the industry. When we first started it was hard for people to train models. SageMaker went a long way to help people train models on large amounts of data. Then, when I was at Google, we really focused on model ops and how to help people build and deploy models. What we found there was that customers were having a lot of success getting their cycle time down and going much faster.

That’s when I realized that if I really wanted to influence the industry, the next opportunity was to figure out how to take advantage of this connection between the AI layer and the data layer. If we could take advantage of that connection, we could get another order of magnitude acceleration on those development cycles. Once I realized that, I started at Databricks six to eight weeks later.

Bernard Leong: So that’s the real reason for joining Databricks, because you want to have the intersection of data and AI together. So one thing for sure, Databricks acquired Mosaic AI two years back. Now you’re leading that team. What’s your current role and responsibility there and how has the scope of work changed after the acquisition, especially on the product side?

Craig Wiley: I joined Databricks back in early 2022, pre-ChatGPT, and I thought I was coming to the most performant tabular machine learning system in the world. I still think Databricks has that, but I was here to work on that problem and then Gen AI hit and everybody went crazy for this technology. All of a sudden I used to joke that I used to have meetings with the head of data science, then I got to have meetings with the C-suite because they needed to understand this technology.

We started helping them, built a vector database, and started building out all those components necessary, and then acquired Mosaic as part of that goal of trying to help our customers build and deploy any of these — at the time we didn’t use the word agents — we used the word Gen AI solutions. We acquired Mosaic to go after that problem. I’ve been leading product management both before and after the acquisition.

Bernard Leong: How is the transition, like before ML ops now to LLM ops? Are the problems similar or different? Because now we are dealing with structured and unstructured data all at the same time.

Craig Wiley: At some level they’re very similar. I’ve got data, I’ve got some models I need to interact with that data. I need to figure out how to deploy that in production in a stable, successful, and economical way. But in other ways they’re very different. With classical machine learning, we had statistics that would tell us precisely how accurate these systems were.

Whereas with Gen AI and agents, the vast majority of folks don’t have the ability to generate a statistically valid way of understanding the accuracy or performance of this system. So helping companies figure out not just the mechanics of deploying this, but also how can you build these systems such that you can trust them, has been one of the primary challenges. To be honest, it’s one of the challenges that I’m most excited about because I think we’re really leading the charge in the industry on that.

Bernard Leong: I would like to lay the groundwork here, right. What is Mosaic AI and how does it fit into, say, Databricks broader era of now going towards say, an AI platform strategy?

Craig Wiley: Yeah, so Mosaic AI is the brand name we use to describe all of our machine learning and AI products, right. Just to simplify it for customers. But fundamentally, you know, Databricks is an end-to-end platform and you'll hear from others that kind of, you know, hey, our AI platform is end-to-end.

But they often mean, you know, well, we start at training and we end at deployment for Databricks, we start at data ingestion. We go through the data ingestion, to data transformations, the analytics on top of that, the exploratory work that has to be done. Then also the kind of more technical machine learning or AI, you know, capabilities.

So Mosaic is really that set of capabilities around model training and vector database and you know, text to SQL and these kinds of things that give us the tools we need to be able to build and deploy the agents that the customers want to have.

Bernard Leong: So do you think that now, given that you also cover the ETL side — the extract, transform and load side — and then with the data ingestion coming into the data platform to do the model training all the way to model deployment, what does it mean in practice for customers thinking about Mosaic AI?

Craig Wiley: The best way to think about this is with a simpler kind of classical machine learning. You can imagine the biggest problem we have seen for years in classical machine learning is the fact that often the data someone trains their model on ends up being different. They train their model on a set of data warehouse tables, then they go deploy it on production data, and the data in production ends up looking much different than the data in the data warehouse tables.

At Databricks you go to that data warehouse table, click on the lineage tab, and immediately see the DAG that takes you back to production. All of a sudden this concept of training-serving skew is just gone because as a data scientist I can choose — do I want to use that downstream data warehouse table, or do I want to use that upstream table that might look more like production and might behave more like production? Whether that’s classical machine learning or Gen AI, having the ability to understand where that dataset came from and how that dataset was created is critically important in building machine learning and AI that you trust.

Bernard Leong: And also reduce overfitting, right?

Craig Wiley: That’s right, because you also want your training data and the production data aligned so that when the model is being trained, there’s no data drifting and then you can deploy as quickly as possible even if the data is from different pipelines. At most companies today, the model gets built by the data scientist and handed to the ML engineer who often has to rebuild the model for production. If we can help eliminate that step and allow the data scientists to ensure they’re using the right data and then just make it a couple of clicks to deploy, then we’re in a much better place with a much faster cycle time for these customers.

Bernard Leong: So how does Mosaic AI have to simplify that process now for enterprises that want to build and deploy large language models? Is the process slightly different? Because there is also the alignment, fine-tuning piece. That’s actually also making the process a little bit more art than science.

Craig Wiley: It is. Helping customers transform that from art to science has been one of our objectives. But helping companies figure out how to, not only effectively — for many of these models, they’re very large, it’s very complex to actually run them for model serving. Whether we help companies serve the models themselves or whether they use some of our pre-served models that they can just hit on a per-question or per-prompt basis, we want to make it just as easy as possible for them to do that so they can get those agents to market as quickly as possible.

Bernard Leong: Because I teach retrieval augmented generation and fine-tuning to all the engineers in Singapore that are working in the government, one of the questions I probably want to ask you is: between both approaches, where is the value and how should enterprises think about choosing one over the other?

Craig Wiley: In the last year we’ve really seen a change. A year ago it was really a split between whether fine-tuning was the right answer or whether using RAG or something like that was the right answer. In the last year or so, with these newer, much smaller, much more economical models, the need to fine-tune has become much less frequent. We see RAG used a lot more commonly for building agents, chatbots, or what have you.

Where we do continue to see fine-tuning is many times folks are building what I think of as industrial use cases or agents. It’s going to do the same task thousands or millions of times. If you’re trying to do that same task millions of times, then fine-tuning the model to do specifically that task can both have the advantage of increasing the accuracy and allow you to fine-tune a much smaller, much more economical model. So now instead of having to hit a big model to do this task millions of times, maybe you can hit a model that costs 10 cents on the dollar to go after that.

Bernard Leong: Do you see more customers thinking about doing more transfer learning given that you can now shrink the model size even further, and then specifically fine-tuning will actually become much more economical?

Craig Wiley: Certainly some of our sophisticated customers are doing those kinds of things — whether it’s transfer learning, fine-tuning, or quantization and distillation of models. But we also see that with some of the flash models or the mini models that are out there, a lot of those tasks that were so important for the economics of this in the past are just not always as critical. They’re important when you want to move from hundreds of thousands to tens of millions, but if you’re only in hundreds of thousands, maybe you could just go with the existing model that’s out there.

Bernard Leong: One question, because we’re talking about enterprise AI — it’s very important that the words security and governance come into play. How does Mosaic AI now help enterprises to deploy AI responsibly? I think it’s the appropriate controls, audit, and even the safety mechanisms that come in place, right?

Craig Wiley: This is huge. For a lot of our customers, this is one of their primary concerns, particularly in regulated industries. The great thing about Databricks is that Databricks is a unified platform from data ingestion all the way through to Gen AI. There’s a common governance scheme, and we govern not just tables — we govern unstructured data, embeddings, and the vector database. We also govern access to the model. When you’re building an agent on Databricks, at each step you have to explicitly give it permission to this specific piece of data or this specific tool.

Bernard Leong: Control up to that kind of level.

Craig Wiley: Absolutely. Because that’s the level of control our banks and our hospitals need in order to do this with confidence and responsibility.

Bernard Leong: That’s basically most enterprises. I think it’s on the top of their mind as well. Can you talk about, say, currently within the Databricks platform, what are some of the core innovations or unique edges that you think Mosaic AI can actually offer to the customers?

Craig Wiley: Certainly the thing I’m most excited about is the recent announcement we had earlier this summer around Agent Bricks. Agent Bricks is really designed as a way of helping folks build agents, but it turns this problem on its head. Most of the vendors out there who are giving you these more accessible or easier ways to build agents are really focused on simplicity.

We, while we wanted to make it simple, weren’t willing to make it simple at the cost of accuracy. The difference with Agent Bricks is it’s very evaluation-centric. The goal is to lure the customer into an evaluation loop so that they can continue to help drive the performance of the system.

You’re not just saying, “Hey, give me a RAG system on this corpus of data.” You’re saying, “Hey, give me a RAG system on this, and then let’s go through some examples and see whether or not those examples are good or up to expectation or not. If they’re not, we’ll learn why not, adjust the system, move forward, and continue to do that until we can get to a level of accuracy that the customer is comfortable with.”

Bernard Leong: So it’s a reinforcement learning mechanism for the enterprise.

Craig Wiley: Exactly. Some would say it’s been reinforcement learning all along, since the beginning. But certainly, this idea of using reinforcement learning as part of an evaluation loop is a way of really driving accuracy beyond what anyone is currently able to do.

Bernard Leong: As a practitioner, I myself look at all the different tools, so I know that Mosaic AI supports both open and proprietary models. I think how important now is model flexibility for enterprise customers from your perspective?

Craig Wiley: I think it’s critical — the more models you have access to, the better. The reason I say that is we so frequently see in this evaluation platform we’ve got, it’s very easy to swap out the model, build an agent, swap out the model, and see if the other model performs better or not. All too often what we see is we may have an intuition about which model is best, but until you test them, you don’t know which one is going to be best for this.

Having access to all of the models you possibly can is really a massive differentiator for both speed to market and accuracy. It’s one of the things I constantly advocate with customers: “I get that you may have a favorite — we all have a favorite model — but make sure you’re working and testing on all of the models.”

Bernard Leong: So is there a mechanism now in the Databricks AI platform that allows me to A/B test this particular use case, say with different models — maybe it’s a proprietary one, maybe it’s an open-source one — and we just want to know what the evaluation looks like?

Craig Wiley: Absolutely. If you take an agent and you run it through our evaluation system, you’ll get a score and you’ll see how many of the prompts and responses passed versus failed with the LLM judges we have set up. Then you put a new model into that agent, run it again, and you’ll see a diff: here are the ones that were green before and are red now, and here are the ones that were red before and are green now.

It’s funny — when new models get launched, we often hear about it, but if we don’t hear about it, we immediately see it in the usage of our evaluation because customers immediately come and start testing to see if this new model that’s been released is better than the one they’re using.

Bernard Leong: So, any interesting customer stories or real-world use cases globally where Mosaic AI is actually making a real tangible impact?

Craig Wiley: Companies like Suncorp use Mosaic AI and the ability to choose which models they want to provide a SunGPT experience to accelerate the productivity of their employees. On the other hand, we see companies like AstraZeneca — the pharmaceutical company — who had 400,000 documents describing all of their clinical trials and challenges. They were able to bring these in, parse them, make sense of them, and make it so they could chat with them and use them systematically, run queries and analysis over them in ways they never had been able to before. This allowed them to derive critical net new insight from unstructured data that had previously been locked away in PDFs and other formats.

Bernard Leong: Since you are now in town in the Asia Pacific region, what’s your sense of the current state of enterprise AI adoption?

Craig Wiley: It’s been really exciting to see locally just how both excited and, let’s be honest, aggressive the companies I’ve met with here are — whether it’s building out platforms for their employees to build and deploy more of these systems or building critical, business-critical use cases on top of this. It’s been exciting to see and hear some of those stories. It’s a real breath of fresh air to see that this is happening globally. This is happening in every corner of the world now.

Bernard Leong: The problems are not the same everywhere you go.

Craig Wiley: Right, exactly. The solutions also — this idea of, “Do I have the ability to run an evaluation that can really help me gain confidence?” The CFO of your company is never going to let you deploy an agent into a critical workflow unless they know that it’s going to work effectively. Being able to help customers here use those tools so they can start pushing these systems into production has been really exciting.

Bernard Leong: One interesting question — now a lot of companies are moving from experimentation to production. What kind of advice would you give them, thinking through the foundation model, the architecture for the right use case? It’s always tricky for the business owner to work out all these different choices and constraints.

Craig Wiley: It’s been a challenge. They say it takes 10,000 hours to become an expert in something, and we’ve only had agents for about a year and a half in their current form. So no one has 10,000 hours. Building these things, they have lots of knobs and dials. People think, “RAG — oh, just put some text in a vector database and I’m done.” But no — there are different search strategies you might use, different chunking strategies you might use, different embedding strategies, all of these different things.

This is why we’ve built Agent Bricks. The reason for Agent Bricks is so that you can come and describe the problem you’re trying to solve. Instead of having to be an expert in all these different chunking and parsing strategies, you can simply come describe the problem, bring in the data or point the system at the data, and we’ll take care of the rest and optimize it depending on your feedback over time.

Bernard Leong: In the next couple of questions, it’s going to be fast going, so I’m going to start off with the first question. What’s the one thing you know about building enterprise generative AI platforms that very few people do?

Craig Wiley: I hate to sound like a broken record here, but it’s eval, right? Eighty-five percent of companies today, or eighty-five percent of AI use cases, are being evaluated by the engineer who built it, saying, “Yep, seemed to work pretty well.”

Bernard Leong: That’s interesting you mentioned that because I always tell the business owners, if you get your engineers to build this, you better have the correct evaluation test exactly for them.

Craig Wiley: Right. Realizing the importance, I feel like my job is really educating folks on the importance of: “Hey, if you want to get this beyond…” Listen, if all you’re building is a system for you and your buddies to keep track of the documents you wrote last year, use Databricks, use anybody — we can help you do it, others can help you do it. But if you’re going to build a system that’s going to be critical to the business, that’s going to be important that it gets it right, then you can’t do that without evals.

That’s the thing. My hope is that if 2025 was the year of agents, the back half of 2025 and 2026 is the year of evals and evaluation quality.

Bernard Leong: Surprisingly, I get asked a lot on that question by the engineers that I teach on the course. So what’s the one question you wish more people would ask you about AI infrastructure, Mosaic AI, or anything related to building enterprise-grade AI architecture?

Craig Wiley: I think you asked it before and it was around models and model usage. The vast majority of enterprises I work with — you walk in and they’ll say, “Oh yeah, we’re a company that uses model A. We use Model A and we only use Model A,” whether that’s OpenAI or Gemini or Claude or Llama or what have you. They’ll tell you until they’re blue in the face why that’s the only model they can use.

I wish more of them would say, “Hey, what models should I be using?” because my answer would be easy: whichever models work best for the use case you’re building.

Bernard Leong: Or maybe have a selection of models so that you can switch it out with a better cost-benefit analysis as well.

Craig Wiley: That’s right. That’s exactly right.

Bernard Leong: So what does great look like for Mosaic AI within Databricks for the next three to five years?

Craig Wiley: We have our work cut out for us. I often joke that in this space right now, we often don’t even know the nouns of next year’s nouns. If you think back to last summer, if we had been having this conversation, we would’ve been talking about RAG, not agents. Now we’re talking about agents. Hopefully we’re talking about agents again next year, but if not, we’re going to be talking about something new and exciting.

Bernard Leong: But do you think now with what Mosaic is building with Agent Bricks, you can now do a lot of orchestration work that you previously couldn’t do before that?

Craig Wiley: Absolutely. Not only can you do it — you could do it before, you just had to build it yourself. You had to get out LangChain or the orchestrator of choice that you prefer. We support all the orchestration systems — I think we support 18 different orchestrators or something like this. You had to get out these orchestrators and build it yourself.

With Agent Bricks, the goal is really to take that complexity away, simplify it, but continue to keep accuracy as the primary objective. Regardless of whether it’s agents, solutions, or systems, our goal is going to continue to be, “How can we help companies steer these systems to exactly how they want them to behave?”

Bernard Leong: So, Craig, many thanks for coming on the show and giving me this valuable time with you so that I can ask you a lot about ML ops, everything else, and LLM as well. In closing, I have two quick questions. Any recommendations that have inspired you recently?

Craig Wiley: One of the things that’s inspired me recently — and it was a huge surprise — was actually a conversation with a customer who’s using our ability to throttle the number of tokens their employees can use in models as lifecycle management for their Gen AI production. They could say, “Everybody gets this many tokens or this many prompts. If you want to build an agent, come to us and we’ll give you a larger budget. If you want to move that agent to production, show us your evals and we’ll give you even more.”

The reason I say that is we didn’t build those governance capabilities for that. We built them for security, tracking, and monitoring. But I’m most inspired by the creativity of our customers when they use what we built to do something really exciting that we didn’t imagine they would use it for.

Bernard Leong: In building technology it’s always about the people there.

Craig Wiley: That’s right. That’s absolutely right.

Bernard Leong: How do my audience find you and how do they keep themselves up to date with Mosaic AI and your work at Databricks?

Craig Wiley: Certainly find me on LinkedIn — I’d love to connect with folks there. Pay attention to Databricks LinkedIn as well as Databricks on other socials. Certainly follow our CEO Ali Ghodsi — he always has a lot to say on the topic as well. Between those, they’ll have a broad source of great insights into how to build production agents.

Bernard Leong: So many thanks for coming on the show and I look forward to having another conversation.

Craig Wiley: I look forward to it as well. Thank you.

Part 1: Nick Eayrs from Databricks and Simon Fassot from Hafnia

Part 2: Craig Wiley from Databricks

Podcast Information: Bernard Leong (@bernardleong, Linkedin) hosts and produces the show. Proper credits for the intro and end music: "Energetic Sports Drive" and the episode is mixed & edited in both video and audio format by G. Thomas Craig (@gthomascraig, LinkedIn). Here are the links to watch or listen to our podcast.

Part 1: Nick Eayrs from Databricks and Simon Fassot from Hafnia

Part 2: Craig Wiley from Databricks

• Youtube: Part 1 and Part 2
• Apple Podcasts: Part 1 and Part 2
• Spotify: Part 1 and Part 2
• SoundCloud: Part 1 and Part 2
• Overcast FM: Part 1 and Part 2
• Libsyn: Part 1 and Part 2
• Amazon Music (US Only)
• RSS Feed
• Twitter
• Facebook Page
• Linkedin
• Mastodon
• BlueSky
• Threads
• RadioPublic
• Acast
• PodBean
• ListenNotes
• TuneIn