The Future of AI Hardware

Transcript generated automatically from the episode audio and lightly cleaned up (filler words removed, misheard names corrected where clear). Speaker labels were added afterward and may contain errors.

Ryan Eppley: Hello and welcome back. I'm Ryan Eppley, co-founder and CEO of RootAccess.

Anna Kirk: I'm Anna Kirk, sales lead at Thread AI. And we're your co-hosts of AI in New York City, the show where we cover all amazing things happening here in the best city in America and maybe the world, covering AI, startups, big technology news, and the people building all the amazing technology around it.

Ryan Eppley: We also have the ghost of Rob May in the corner over here. He's our normal co-host. Unfortunately couldn't make it today. But what an absolute pleasure it is to introduce our guest for the week, Waleed Atallah, CEO of Mako. Willie, do you want to give a quick background on yourself and your story?

Waleed Atallah: Yeah, thanks. First of all, thank you for having me. Really fun to be here with you guys. See, I you know, I grew up in California. I went to school in LA. I moved around a lot. I originally, got into semiconductors out of college. I was at Intel working on deep learning accelerators there. I was on the product planning team. It's where I would actually meet my eventual co-founder. After Intel I went to a startup called untether AI where again we were working on other deep learning accelerators chips specifically for AI inference. And you know from there I guess we kind of came across you know the challenges of actually writing code for these hardware.

Ryan Eppley: You skipped such a exciting moment though of like yeah I just got into Silicon, right? Like what did you study in school? Like how do you even trip yourself into such a important kind of piece of the stack? Like when did you really start saying like yeah I think I want to work on hardware.

Waleed Atallah: Yeah. So actually I it's this I'm glad you brought it up. I studied electrical engineering and I made a series of terrible financial decisions [laughter] that eventually let me land at Intel. The first one is a good one because I originally had a job offer from a really small startup right out of school. It was called Thor Trucks at the time. And I was going to be the first hire, the first engineering hire they made outside of the founders. But before my first day, I got the job offer from Intel and I was thinking, okay, you know, Intel, like come on, that's top five for electrical engineers. This company, I like it. I think the founders are great. I think the technology is cool, but got to play it safe. And then in 2021, that truck company got acquired for $2 billion.

Anna Kirk: Oh wow.

Waleed Atallah: So that was my first big miss.

Ryan Eppley: I know those stock options, you're like counting the money in your head, right?

Waleed Atallah: Meanwhile, I stayed at Intel which kind of you know did that for I don't know the entire time I [laughter] was there. For most of it, but they're on the comeback.

Ryan Eppley: All right. They're on the comeback. Coming soon for sure.

Waleed Atallah: And then when I left Intel for Unte, this was 2021, I had declined a job offer from Nvidia to go to Untether. So if you want to make some good money, just do the opposite of what I do up until now with obviously.

Anna Kirk: So maybe for those of us in the audience, those of us listening who aren't as familiar with how Mako works or kind of the chip ecosystem more broadly, could you walk us through, how make works, what it does, and why it's important, especially right now?

Waleed Atallah: Yeah. So, like GPUs are, you know, far and away the most important computer chip in the world today. And there's a lot we can say about like the transition from CPUs onto GPU hardware. But one of the really big challenges with moving from CPUs to GPUs is you have to rewrite what is like normal sequential code made for CPUs in a parallel fashion so that it can run efficiently on GPUs. And that's what Mako does. Mako makes AI tools that can automatically rewrite your software for GPUs. It writes what's called GPU kernels. These are small subprograms that can run on GPUs. It uses languages like CUDA which you've heard probably for Nvidia or other different kernel languages for different hardware platforms. So the point of the company is to automate GPU engineering.

Ryan Eppley: And can you maybe walk us through like the significance of GPUs and parallel processing and why that's significant especially right now.

Waleed Atallah: Yeah. So for the last 50 years the entire compute infrastructure of the world ran on CPUs which are you know sequential processors and for the last 50 years we were getting performance improvements by having CPUs that ran faster and you could do this reliably. This was called Moore's law that the density of the transistors would double or something every 18 months. And so you could just sit on your hands for for 18 months and your software would get twice as fast for 50 years. We had that. But lately for the last like decade, it was really starting to show that this scaling was slowing down.

Ryan Eppley: That whole physics thing, huh?

Waleed Atallah: Electrons, you have to deal with it eventually. And so instead of just getting faster CPUs, the world has started to turn towards GPUs which do a lot more parallel processing. So instead of faster sequential processing, let's do more like same speed but parallel processing. And so I think like these are real numbers from like a DOE report that 10 years ago 2015 something like 60 terowatt hours was the energy consumed by all our data centers and about 90% of that was taken up by CPUs. In 2030 they're estimating 600 terowatt hours.

Ryan Eppley: Wow.

Waleed Atallah: And 90% of that is going to be for GPUs. M so we're seeing a complete shift from the CPU as the center of compute infrastructure and not this isn't by the way just for AI this is much broader than that although AI like naturally takes up all the oxygen in the room and all we can think about is like LLM's training inference ChatGPT all

Ryan Eppley: but going back you studied electrical engineering you know about 15 years ago you saw a lot changing at that time went to Intel

Waleed Atallah: Oh, 15 years. Jesus Christ. I just did the math that long [laughter] ago.

Ryan Eppley: Sounds sounds about right. I looked at LinkedIn, you know.

Waleed Atallah: That's right. I just [laughter] Sorry, I didn't mean to interrupt, but Jesus Christ.

Ryan Eppley: No. No. And you know, I bring that up just because I think a lot of people get excited about, oh, GPUs like, you know, 15 years ago, they were only really used for gaming. I think we were talking about this right before the podcast of just the change in the focus of GPUs has become so massive now, but it started pretty niche 1015 years ago. But you seem to be one of the first movers seeing this happen in your different kind of trajectory of career path. Like when was the moment where you said, "Wow, this is going to be a huge market opportunity. I have to go all in. I'm willing to take the risk as a founder." When was the real first moment that you saw I need to be ahead of the game in this transition in period?

Waleed Atallah: Yeah. So, so there's differing opinions, right? So I came out of school in 2017 and this was I guess you could say AI really kicked off in 2012 with the release of a model called AlexNet which was really good at detecting cats and that was the famous application of it actually the most important application of course naturally and so AlexNet came out in 2012 and it was a vision model that could detect cats superbly well. And every year after that, the vision models were getting really good by getting bigger. And so around 2017 when I was coming out of college, I had thought, all right, I'm going to do one of two things that seem like it's going to be really important in the future. Either one, I'm going to do AI because this AI thing is pretty cool, doing more than just cats, but cats especially, [laughter] or or two, I was going to do cyber security because it just seemed like a big topic.

I chose AI. I stuck at Intel where I thought that, you know, I actually fell for the propaganda originally. I was thinking, "Oh, GPUs, ah, that's made for graphics. You can make a custom chip for AI." And I still think you can in most cases, but like the GPUs have evolved so much. I mean, it became clear that they were the runaway winner. Especially because they were really the only game in town for a really long time. Like Nvidia libraries were so good that you could actually make use of a GPU without having to write a ton of really niche CUDA code.

Ryan Eppley: Yeah. And obviously us prepping for this, we understand Nvidia is the what feels like winner take all right now, but there are some really other massive companies with their own huge technology kind of stack. Does your software work across different vendors like AMD? Is it only for Nvidia today? Like how does that work and kind of your approach?

Waleed Atallah: Yeah, so our approach to developing this software is basically to build a pipeline such that we can teach a language model, a new language for a new hardware really fast. That's what we optimize for. How do we teach a language model to learn how to code for AMD to learn how to code for Cerebras or Groq or this hardware or that hardware? So we do support multiple platforms right now. We support Nvidia, AMD, and Tenstorrent today. And we have a lot of other ones in the pipe, too.

Anna Kirk: And so I've often heard people describe Nvidia as it's a software company that also sells hardware. Can you maybe double click on [clears throat] how Mako might might interact with something like CUDA and whether or not Nvidia is trying to build this or whether or not you kind of see those software layers being complimentary.

Waleed Atallah: Yeah. So, so you know people talk about the CUDA mode and people and people are right like Nvidia is a software I think they have more software engineers than they do hardware engineers. Oh, and Jensen talks about it even a week or two ago at the GTC in DC. He has this slide that shows all the different CUDA libraries that Nvidia engineers had developed and he says this is the treasure of our company. And it's true the CUDA li he says it t it's taken them 30 years to build up that that level of support. And the challenge is that for most software developers, most people have no idea how to write CUDA and they don't want to know. If they have to look at CUDA kernels, something has gone terribly wrong.

Ryan Eppley: Yes. Yeah. We were joking about that right before the podcast started. We're like, you know, that's not where you want to find your your time spent.

Waleed Atallah: Yeah. And so Nvidia is so good. People use the NVIDIA GPUs because someone at NVIDIA had spent the time to build this like little Lego block that you can essentially use to build out some sort of application layer. And so most people are fine using the Lego blocks that Nvidia provides. But there's a lot of opportunity outside of that. And so Mako generates CUDA code. It's like if you want to create a new Lego block, a different color, a different shape, whatever it is. So if you need like if you're building like a completely novel algorithm then you need to write your own CUDA eventually. That's where Maker would come in.

Ryan Eppley: Well maybe walk us through and specifically for our our our audience why is it so hard to write for GPUs? Like why is it so hard to program these things? Shouldn't just be building like any other kind of like code base out like why is it so painful to do this manually today?

Waleed Atallah: Right. So what could take you maybe like one or two hours to write in Python could take literally months for an expert to rewrite that for GPUs. And the reason why it's so difficult is because you're not just defining an algorithm. You're actually mapping the algorithm down to specific hardware features. So on any chip like ultimately you have three components. You have memory, you have compute that does the math, and then you have pipes that connect the memory to the compute networking. And so a GPU today is so complicated that the kernel actually has to map specifically how you move data from the memory into the compute, how you crunch those numbers and where you put the those numbers afterwards.

This is challenging because for a algorithm like ChatGPT a sing like when you press enter if you say hi and press enter I think it's something like like two trillion math operations happen for it to return back like hi how are you I'm ChatGPT and so those two trillion math operations need to be mapped in what order are you going to execute those two trillion operations as you can imagine there's a fast way to do it and there's a really really slow way to do it so writing CUDA is about how do I find the fastest most efficient way to slice up these two trillion math operations such that I can have a good stream of, you know, data moving around and numbers being crunched.

Anna Kirk: How do you future some future proof something like this? You know, I imagine this is such a moving target in terms of like you're always keeping up with the latest generation

Waleed Atallah: Yeah. Of GPUs.

Anna Kirk: Especially as you know, folks like OpenAI start to diversify their partner base. H how do you keep up with that? And does that make your life at Mako harder or easier in some respects?

Waleed Atallah: Well, so you know, supporting new hardware platforms is always a challenge. So to f future proof it, we use AI.

Ryan Eppley: Oh, yeah.

Waleed Atallah: Believe it or not, to build AI. No, seriously though, like one of the big challenges for having LLMs that can generate this kind of like complex GPU code is that like now we're at 12 months cadences for a new GPU. The B200 came out 12 months ago and then 12 months later the next generation Nvidia is going to come out and the same is for AMD. And every time a new generation comes out, you actually have to rewrite all that low-level code. So it is and you compound that with, you know, Google TPUs, which now those are shipping to other partners. I think they said Meta is going to be installing and running their own TPU cloud. Other NeoClouds are looking at doing it too. Amazon Trainium, Anthropic uses Trainium.

There's the Qualcomm, they announced their data center chips. So there's a ton more like this is proliferating. We have the Gro chips, the Cerebras chips. So it's definitely going to be like a challenge. And especially like the reason LLMs get good at coding is because we have so much training data to train an LLM to write code well. So there is a, you know, orders of magnitude less data available on GPU code, especially for new hardware like the new B200 that comes out or the new Rubin chip that comes out next year from Nvidia. So this data is just not going to be in the pre-training corpus of these LLMs.

So what we've done to overcome this is we've developed you know it's something of a continuous learning system. It's this is how we train LLMs to learn how to write code for new hardware. We do a lot of reinforcement learning. And then we have a long-term memory subsystem built into our coding agent that lets us basically teach it how to use new hardware features. So it's definitely something that you know it's always a moving target. It's definitely a challenge to keep up with, but ultimately becomes the edge for Mako because we've we've developed a way to kind of stay on the cutting edge of new hardware.

Ryan Eppley: And I guess, you know, you brought up coding agent. Is this the experience that you're delivering for your customers today? Similar to a cursor or a clawed code or how do how do you take your approach because I've seen a lot of flavors in code assistants the co-pilots all the way through like you know just kind of like set and forget and come back 10 minutes later and something's done for you like where do you fall in this kind of like map of coding agents

Waleed Atallah: that's a good one because I'm actually not totally sure we've landed on like like anyone like cursor cognition Codex whatever it is the wild wild west right now I think that we haven't really landed on what like the ideal interface case is especially as we get more into like multi- aent stuff like the set and forget like we think of it in terms of two different ways. There's like the synchronous mode that's like cursor tab complete where you're working with it and you expect a result back ASAP. And then there's the asynchronous mode which is you hit go you know take a shower, take a nap, whatever and then you come back and check on your coding agent to see the progress later.

Ryan Eppley: Hey buddy, you did a pretty good job.

Waleed Atallah: Yeah. So we've been working on both. I think the async mode is a lot more interesting for this. Like we had an example with a specific chip where we let the agent run for 12 hours in a row. So this was a like hit go on Friday and check on it over the weekend type thing. And this was actually after about 12 hours was when we saw a result that was literally five times faster than the human handwritten kernel. So, so incredible results. Something that would have taken a human literal months to do happened overnight in 12 hours. So, so we're really interested in the async mode. We're currently developing products for the synchronous mode, too, just because we have all these cool models. It's like might as well. But honestly, there's a lot of testing right now. I don't I don't know if we found the right one like

Ryan Eppley: Yeah. Yeah. I have so many opinions on this, too, you know? I use Yeah. I have a feeling that we will move more and more async, but there is something nice about feeling in control. I call it some people call like a code editor or an IDE. I like to call it like a command center now. Yeah. Where like you can go out and send things to do, but I still feel in control. Where do you think the hockey puck is going? Like where do you think this will be in not just like next year, next quarter, but things are going so fast, but yeah, it's 2030. We've all seen the cursors, the makos of the world, all these different technologies be so helpful to writing important software for all different types of use cases. How does that experience go? Will it be one one person will find it and we'll all adopt that interface or that kind of experience or will there be totally different flavors for different use cases like just like riff with us. What do you think will happen in 2030 for this type of code assistant?

Waleed Atallah: That's a tricky one. Have you like when I first started using the code assistants like they constantly shocked me even as someone who's like been in AI like working on models in 2017 to then ChatGPT was a surprise. The Claude Sonnet 3.5 model was another surprise and seeing the results like with Claude code and codecs and other cursor stuff. It's still a surprise. I feel like I keep trying to change my expectations and I keep getting surprised. So, it's really tough to tell. Like it can go in one of two ways. One, you know, we have solved coding. That's what a lot of the like job's done.

Ryan Eppley: Everyone packed up. Yeah. Super.

Waleed Atallah: Yeah. Yeah. No, I'm not kidding. There's a lot of like the super intelligence people who say, "Oh, by by 2027 it's over. All the all the all the work is going to be done by AI. It's the last software we need to invent and we're done." I don't subscribe to that belief. Some people do and I see their reasoning. They see the straight line. They're like, "Well, if the straight line keeps going then is what's the new Moore's law for this for coding agents? We got to come up with something."

Ryan Eppley: Probably should be.

Waleed Atallah: But yeah, they they're incredible. Have you tried like doing like vibe coding for like a random app or website? Either of you?

Ryan Eppley: Absolutely. Absolutely.

Anna Kirk: I was going to say like I know you were talking about sync versus async. I'm an instant gratification girly. So like something like replet is just like it's so addictive to me because it's like it it populates it with dummy data. Like you can see it right in front of you. Like it's not as powerful as some of these other like tools.

Ryan Eppley: Great output from Replit.

Anna Kirk: It's not it's not great.

Ryan Eppley: Was my first entrance into this where I was like sick of trying to find a template for a website and I just said, you know what, someone on on Twitter or X put it on like, oh, vzero.dev and I was like, wow. The first time I used it, I was like, I don't think I'm ever going back to a Figma or manually designing things. Like I'm just going to ask this thing that apparently understands my intentions pretty well and I get to see it within 30 seconds. Oh my gosh, like that that was a game changer for me.

Anna Kirk: I am curious like you mentioned that your async offering was potentially much more performant. I'm curious, you know, there there's got to be some trade-off still between like abstraction and instant results and like what whatever the accuracy threshold is. Like I'm curious kind of how you're seeing that diverge and whether or not you're going to see you know different user bases behaving differently in that regard and then obviously getting different types of output.

Waleed Atallah: Yeah, I think so. The synchronous mode like that's for kernel engineers. That's for somebody who knows what a kernel should look like and they'll use like a tool to, you know, create their kernel faster than they would have otherwise, right? I think why we're so excited about async is because we've seen it in our company already. We're taking normal software engineers who have zero experience with CUDA and with the async agent, they're able to generate high quality CUDA kernels. Yeah. So, so, so that's why we like that one more. I think I don't I'm not totally sold though. I don't think this is like, like I said, I don't think it's the final right finished thing. I don't think we've landed on the right like product shape.

Ryan Eppley: Well, you're only two years in, something like that.

Waleed Atallah: A year and a half, two.

Ryan Eppley: Yeah. Yeah. So, you know, early in the in the game, but you've made incredible strides. I think something that we always like, you know, audience members to be able to hear from from, you know, builders like you is just walk us through like where you are for your company today, like how much you raise, the team composition, where the office is, just some of those details for for the listeners just to be able to follow you after this.

Waleed Atallah: Yeah. So, we have 13 full-time employees now. Six of us are here in New York City. That's the main office. Six of us are also in Poland. So, our CTO is Polish. He runs the office in Gdansk, right? And then we have our sales guy in San Francisco, which we should be. I feel like I'm there every two weeks it feels like. But you know, like I said, we've been off to the races for a little bit. We're doing a lot of really exciting work. We have a really close collaboration with OpenAI. So we actually have access to training GPT5 with reinforcement learning. So this is something that we got early access to. It was a really close partnership. We've had really strong results. We've trained the best LLM in the world at writing CUDA.

Ryan Eppley: Wow. Bar none.

Waleed Atallah: It beats GPT5. It beats the rest of the propri proprietary and open source models. And this was used with our, you know, secret continuous learning training recipe.

Ryan Eppley: Yeah. Yeah. The secret sauce.

Waleed Atallah: Yeah. So, so we're really excited about that. We hire kernel engineers, we hire compiler engineers, we hire LLM researchers. Our LLM research lead, his PhD thesis was training LLMs to write GPU code.

Ryan Eppley: You're kidding. Super, you know, on the nose. How many of those are there in the world today?

Waleed Atallah: He might be the first. I think he might have been the first to actually get that.

Anna Kirk: Yeah, it's a great great pickup. It is, it would be remiss of me if I didn't also ask you this question. Why did you decide to set up shop in New York?

Waleed Atallah: Oh, so it's funny. We our chief science officer, Dr. Muhammad Abdel Fata. He's a professor at Cornell Tech. And he told me that he would work more if the office was closer to him. The closer the office, the more he would work. And he said this because he and he told me later, he said, "I thought you were bluffing. I thought for sure you were going to open the office in San Francisco." So I put the office right across the street from his office, which is in Cornell Tech campus on Roosevelt Island. So I can literally like from our office I can through the window see where peek out of the blinds be like all right like wave him over right and it worked perfectly. He spends a ton of time with us a huge huge ad.

Ryan Eppley: Smart man. Smart man.

Waleed Atallah: So we're on Roosevelt Island in New York. If you ever find yourself there for some reason or tram or boat.

Anna Kirk: I love Roosevelt Island. Like it's weirdly it's one of the weirdest hills that I will die on is that it's one of the best places in New York. It's hard to get to, but like once you're there, it's so safe. The history is so cool. Like I don't know if you've ever done weird any of the like walking tours or a bunch of like old assign asylums that have since been converted into like luxury apartment buildings. It's crazy.

Ryan Eppley: Still crazy people in them.

Anna Kirk: Yeah, it was a [laughter] different different kind of crazy people. Like a little cursed.

Waleed Atallah: I heard like a bunch of crazy I don't I don't know these for sure, but they said that like during like the Spanish influenza or something, Roosevelt Island was the quarantine island for New York, the asylums and then god knows probably some other crazy things.

Ryan Eppley: Well, you heard it first. It's a cursed island, but good things happening on it. Good things happening on it. Well, we love to always have our guests weigh in on some current events. So, we teed up a couple of articles that came out literally just this week. Anna, why don't you share our first one?

Anna Kirk: Sure. So, for those of you who don't have the patience to read the 30-page report, I'll announce it for you. McKinsey came out with their 2025 state of AI report, they surveyed over 3,600 employees and over 200 seuite level executives to get their take on AI and get a pulse on what the progress has been so far with enterprise adoption. Nothing that I found too too surprising. TLDR, lots of pilots in experimentation phase that have yet to make it to production and even fewer companies actually seeing result in terms of that flowing to operating income on the B bottom line. I'd be curious to hear both of your thoughts on this too. I mean, we saw the MIT report that came out a couple months ago that was like catastrophizing, like oh no, like 95% of pilots are failing. My take is that this isn't necessarily a bad thing and having a high failure rate at least means that companies are experimenting early, which I think can actually do good things for enterprise adoption long term. Like what do you what are your guys' thoughts?

Ryan Eppley: You know, I feel like there's never been more pull from enterprises than right now. And I'm sure this might have happened in a technology generation before us or something of that nature, but the magnitude of when I personally, in my experience, talk to a VP of technology at a Fortune 500 company, they're looking for me as much as I'm looking for them. You know, I think their hunger to beat their competitors and actually tell their seuite like, "Hey, we're adopting AI." It almost doesn't matter what that AI does yet. It just matters that they have an initiative that they can tell their boss's boss's boss or so forth.

And so to your point, I thought you nailed it of it's great to see that there's a lot of desire in the market to adopt. And I don't know if the I think what was it 67% is a necessarily like oh like AI is failing. I think that's just there's a lot of founders building a lot of important tools but it takes time to build a great enterprisegrade product. Let's just be honest. And so I think there's great signal that there's demand in the marketplace, but it takes time to build these tools. Well, that's my take on and that's my personal experience when I when I sell to enterprise. You know, how has Mako gone about your enterprise like playbook?

Waleed Atallah: Yeah, I think it's a lot easier to make a demo that's like super flashy and it's a lot harder to make something like robust enough for production quality usage. I think another like weird one too is like when Microsoft Word first became a thing, people are saying, "Oh, like we're going to be so much more efficient. We're going to spend so much less time typing things up. We have this word processor." but you almost couldn't measure the efficiency gain in there. Like we ended up with a lot more pages of written documents, but there wasn't exactly that measure of efficiency gained. So I think that there's there might be a little bit of that happening sometime too, right? Like do we actually get more productive? I think in some cases the answer is certainly a yes but not in every case.

Ryan Eppley: I also feel like having read that article what went through my mind is I think there is a huge amount of that 67% that failed that actually just replaced that pilot with someone else's pilot that you didn't even know existed. You had a competitor pop up in the last couple months that had a flashier demo that they put together and there was a shiny new object and they said, "Hey, really love what you're building, but we actually found another, you know, competitor of yours to go with." I think that's happening as much, too. So, maybe that's a good thing. Competition is good for the marketplace. I think there's a lot of new builders building important tools that enterprises want. It's just a matter of who's actually going to win their category now.

Anna Kirk: Yeah, it's also like it's a matter of who has a buying power to, right? Like you know, especially at a lot of these enterprises like it's so multithreaded to get any kind of technology innovation passed and so I'm kind of curious like you this report in particular I think as well as the MIT one like I think primarily looked at kind of number of pilots. I imagine there is an allocated budget for AI that yes has in aggregate increased over time but this maybe that's spread thin across a higher number of pilots and so they're actually like they they're not able to kind of push forward as many initiatives as they'd like to. One point from the report I also thought was really interesting is that they McKinsey looked at what they call like AI top performers. Which is any enterprise that is reported to have seen 5% EBIT improvement, operating income improvement or more attributable to AI. And that those top performers have an aggressive agenda when it comes to kind of mandating AI initiatives top down pushing that through across all of their key executives and their key stakeholders.

Ryan Eppley: You know I am kind of curious for a lot of these companies that are hiring like a head of AI title.

Waleed Atallah: You know it's funny to I heard I heard a really funny one with that.

Ryan Eppley: Woodier.

Waleed Atallah: I heard that Crocs hired a head of AI. The shoe company.

Anna Kirk: Well, someone has to optimize like how silly they're going to look on your feet. I'm [laughter] putting GPOS in the heel.

Waleed Atallah: I want my I want my Crocs to talk to me. Yeah, that's what I want.

Ryan Eppley: Croc Croc Talk [laughter] mobile that right now here first. That's good.

Anna Kirk: But no it's interesting like a lot of these enterprises are hiring ahead of AI ahead of innovation whose like sole task is to experiment you know and play around and maybe maybe their mandate is less so what is the efficacy of this across different business units and more so like hey let's just kind of play around and see what happens and they don't know exactly what the game is but they want to be in it

Ryan Eppley: so you can't risk not doing it right every it's a ton of pressure for boards to have some type of plan and I just feel like it's happened so fast and everyone's talked about it's happened so fast but truly I don't know if there's been a faster takeover. I understand the internet I was a huge I've watched almost all the documentaries on like how the internet kind of came about and all the hype and things like that. I think this is like 10x that though because we now have social media and other means that people are like sharing the hype in a different way and it's more than like hyped up. It's like people are getting real efficiencies but it's not equal. The efficiencies aren't equal.

So when you see someone else in a different department be like oh we just onboarded to this you know bookkeeping AI software and now we just take a photo and all of a sudden like it's in our Quickbooks or anything. You look at your department you're like oh we're in procurement. Like why don't I have that? Like where do I do that in my company? And so I feel I feel like there's enormous pressure right now. I don't know what the outcome's going to be, but I just feel the pressure in every room I walk in. Either they have a plan, they're working on a plan or they are executing really well and you're trying to figure out how you can be a part of their plan. But there's something that every every boardroom is talking about that I feel is a good time if you're building for AI. You can walk in any room and know that someone will take you seriously if you have that flashy demo and you can prove an ROI at least in theory there's an opportunity for you to get a pilot. So those 67% pilot failures I think that's a good sign that people are willing to take a risk on a new technology.

Anna Kirk: Yeah, it is it is interesting though like that that point about if you can prove an ROI like to your point earlier about like word docs like how do you how do you encapsulate what the ROI is on Microsoft Word? You know maybe it's I don't think so but maybe it's made us less efficient in some capacities. So I' I'd be curious to hear your take on you know h how do you quantify that especially like the McKinsey report mentioned you know the top performers in AI are looking at upside accelerators as well as like cost cutting measures for AI. So like how do you even begin to quantify what that looks like?

Waleed Atallah: So AI is really weird in that sense for me. So previous software pick one I don't know Stripe, Visa, Excel, if any of these were were 10 times as efficient if they got 10 times better over the next 10 years or whatever it is. It wouldn't change much. Like Visa is already really fast to charge you something. Stripe already makes it easy to collect payments and pay people, right? Excel does its thing. Well, I don't even know what 10x would be, you know, for some of these things. So, but for AI software, it's completely different. It's not a tool. It's actually doing work or at least it's promising to actually do work. So, when you scale these up, it's not that you have 10x XL, whatever that [laughter] looks like. It's that you actually have the potential of doing 10 times more work, ideally revenue generating work or revenue recovering work or something around the capture of value that your company is supposed to do as the reason it exists. So I think that's like one of the weird differences that's kind of hard to quantify. So like we recognize that oh like in the case of Mako for example it's not that you get this 10% faster software which you do and it's really cool it's nice it's more that we're doing 12 months of work in two days

Ryan Eppley: you get that 10% much faster

Waleed Atallah: yeah exactly and so it's I think it's the same for a lot of the like measuring ROI for a lot of these other AI softwares that there's this idea that like we can actually you do more work with the same amount of people and we can just turn this knob. If I dump more GPUs on it, more work gets done and my ROI or my my bottom line gets better. So I think that's like one of the weird things about like AI software that it's supposed to actually do the work rather than be a tool that somebody uses to do the work.

Ryan Eppley: It's a good segue because we have another couple articles teed up in one bucket of, you know, there's some clear winners right now, right? The more recent one that I was really excited about is Gamma. I don't know if you've tried Gamma.

Waleed Atallah: I have. You know, I tried it way back when.

Ryan Eppley: Yeah. No, it's gotten really good.

Anna Kirk: It was like all right.

Ryan Eppley: Yeah, it's gotten really good. I really like what they're up to. And so they just raised at a $2.1 billion valuation. Skyrocketed what they're up to. And another one is obviously Lovable's been making a ton of headlines. I think they posted what 8 million users now. I don't know if that's active monthly, but eight million users. And there's a ton of others. We brought up Replit earlier. So there there seems to be some some winners in big ways. Yeah. In efficiencies of labor for their kind of target market and demo. Do you see this sustaining this mass adoption of AI or are we like we've we've made it and now it's going to like start peaking like how many more users can you get? Yeah. But the labor market itself is trillions. So maybe there's a ton of efficiencies. Like what's your take on that?

Waleed Atallah: So I think especially like lovable is a really good example. Lovable replet those they've expanded beyond the software engineer. They've made coding accessible to a huge portion of people who never thought they would ever learn to write code and probably still haven't learned to write code. But now they can create code that does useful things. Canva has made everybody into a designer. Clay Clay's a great example. Clay has made everybody into a GTM wizard.

Ryan Eppley: They made the GTM wizard.

Waleed Atallah: Exactly. So I think like what's what's most interesting about those companies that you mentioned is that they expand beyond their original demographic. Beyond the software engineer, beyond the, you know, maybe some people like they would always just bother somebody else to create the PowerPoint slides or better yet they would make the super simple just bulleted wall of text. M but now that everybody feels empowered to do you know higher quality work by using these kind of tools and I think that's kind of how they've won.

Ryan Eppley: I think also we live in a bubble where I know gamma you know gamma I know level you know lovable and there's probably like 90% of the world I think never thought about not even once about about AI notetakers about AI presentation makers about any of that and one day as the market continues to grow they'll be like huh oh neat I can I can do this myself yeah like one of my good friends since childhood he makes a ton of presentations for his job he's in you know sales sales and marketing role for for enterprise and when he's meeting with clients like he usually makes his own slides and he didn't know about GAM I was like this is a product made for you man like this is for you how do you not know about this like I've known about this for a long time and he's like I don't know let me try it out he got back to me literally yesterday he's like wow this is awesome

Anna Kirk: I'm just so curious and maybe this is less so relevant for something like Gamma where like you can just like make elements that you can stick into a slide. So, it's actually really great because like you I feel like companies aren't that good at like generating their own slides yet that actually look like customized and fit for purpose. But like Gamma's great and like pretty easily usable in like an enterprise context today. For some of these like vibe coding tools like what is this doing generationally when you think about like behavioral like behavioral tendencies given that a lot of these consumers are also enterprise employees. You know like it's very transferable and it's great for increasing access and increasing like some level of fluency but you know in terms of like coding in enterprise stack like you still need people who know to some extent how to like speak or at least audit assembly like you still need to know people who can like code in C. Like how do you think this is going to impact like employee attitudes towards that type of coding? You know, are people going to get more laxed? Are you just going to hire different people?

Waleed Atallah: Well, I'll tell you when we because we vibe code extensively. But it never goes to production. Yeah. Like we'll we'll test things out. We'll test new features with customers. We'll try different UIs, but whenever we actually want to build something scalable, like we'll pretty much throw it all away and like rebuild it the way we think it should be built. And that might be a testament to one of two things. Number one, we're not doing it right. Like maybe if we did give like really explicit testing conditions and super elaborate instruction docs, the app would be able to whip it up perfectly. Yeah. Or, you know, software is not there yet. Tech's not there yet. So I think that like I'm not totally sure.

Ryan Eppley: So it's like the calculator coming up in like the 50s or something, right? [clears throat and laughter] We know it's going to change the way.

Waleed Atallah: Yeah, we know it's going to change the way things are done. That's without a doubt. I'm This is not a nothing burger. This is legit. This is going to change everything forever. This is going to be the way it's done. Not totally sure the role it's going to play in all the really advanced edge cases.

Ryan Eppley: Yeah. Yeah. Yeah. I don't know. I feel compelled to always be leaning forward like if this enables us to build more impressive technology. It's always a good thing. I have gotten the push back from, you know, my my friend groups at times when I'm so tech optimist. They're like, "Hey, but what about the people's jobs and people graduating that are getting, you know, it's harder and harder to get junior dev jobs." you know, I don't know if there's a silver lining here, but I think it's always going to be good long term. I think it's a bumpy ride to get there, though.

Anna Kirk: Yeah. I've had this conversation with quite a lot of people or two around like what it will do culturally to like the apprenticeship model. You know like if if a vibe coding agent is essentially equivalent to a fresh out of college junior developer in terms of its accuracy and how you can use it. If if not more efficient because you don't have to teach it then then what happens to the working world next? If you don't have to teach it like will will we lose that culture? Will that always be important?

Waleed Atallah: Yeah. Yeah. It is I've heard and I know firsthand it's really hard for junior engineers to get jobs out of college. We're seeing a lot of people just go into like a master's degree because they might as well because they couldn't find a job. And the amount of people who have like volunteered to do unpaid internships is way too high. That's like my barometer, I think. [laughter] Like the amount of people say, "Look, I'll just do an unpaid internship. I just need the experience." so I think it's it seems like it's becoming an issue.

Ryan Eppley: Yeah. AI macroeconomic impacts. I'm sure there's like a couple factors going into it. It's a weird one. What else we got on on the docket? What's our next article on the docket?

Anna Kirk: So here's an interesting one for the week. Michael Burry, the investor, the hedge fund investor made famous by Christian Bale's character on the big short, he actually posted on X this week, accusing hyperscalers like Oracle and Meta of inflating their projected earnings by basically estimating that their chips will have a longer, usable lifetime over time than they than he thinks that they will. So he estimates that that kind of accounting methodology is understating levels of depreciation across the industry by about just under 180 billion between 2026 and 2028

Ryan Eppley: small figures teeny tiny I mean when you're talking billions in the in the world of hyperscalers like quote the other I think it was a month or two ago he's like we have to burn like a hundred billion dollars we'll do it or something like that he said it so casual I forget the exact number but it was a pretty big one. But you're a great person to kind of maybe weigh in on this one. Like are data centers and people building these these infrastructure are they inflating their kind of projections? How how do you feel about that?

Waleed Atallah: So there is without a doubt there has been this insatiable demand for compute driven mostly by you know five companies. I think like Mark Zuckerberg or someone said that the risk of not building this out far outweighs the risk of doing it and wasting a lot of money. And that's because I think they believe in this, you know, tremendous like improvement in quality of an intelligence of models. I saw the Michael Burry thing. I didn't dig too deep into it. The whole depreciation thing is interesting too because people always for forever have been messing around with depreciation on the balance sheet in terms of whether it's real estate or GPUs or anything. So I'm not I'm not totally convinced. I do think it's something that's like pretty interesting. Yeah. I did not know about like the small changes where they say oh we're actually depreciating over eight years instead of five. I didn't know that was happening. So that is you know suspect. I do think there is just such a like people have talked about like you know the circle jerk the open anti goes to

Ryan Eppley: partnership partnership yeah go to video

Waleed Atallah: it's like if we just keep passing $20 in around in a circle do we get to claim that revenue every time the 20 bucks passes I made 20 I made 40 I made

Ryan Eppley: well actually you can just lever up and so you can claim even more [laughter]

Waleed Atallah: right my advice and so by the way we have now made $20 in No I'm kidding but so you know some there is some funny business going on when a company that doesn't have the money makes a signs a contract to buy a certain amount of hardware. So that does obviously raise some red flags. But just like you know I think it's like easy to or easier to justify these things when you see like the rocket ship of revenue like OpenAI and Anthropic like their revenue has grown so insanely quickly. Yeah. And so you look at future earnings, you want, it's easy to say, oh, this is obviously going to continue. The models will keep getting better. We are going to expand more and more into the economy and own more of the actual productive capacity of the world. And it's going to be through these GPUs.

Ryan Eppley: Yeah, I'm glad someone's taking an antagonist approach, you know, and obviously he's a very famous investor and he gets a lot of headlines, but you know, I think he shorted Palantir and he's lost on all of those shorts so far. So you never know who's the real far so far. If you watch the movie, you know, he was negative for a [laughter] while, right? It comes, you know.

Anna Kirk: I mean, I do think like some of this news came from like, oh, he put had he had a put order and like we don't basically it's a security that will like increase in value assuming that you know the stock price decreases. So, it's usually a bet. It's considered a bet against a stock price moving in a certain direction. But we don't know like he could have been hedging. He could still have a position. Like I think being a little bit too like reading into this too much is maybe not looking at the right signals. But I think they're probably I mean wed can you think of any reason why it would make sense for the lifetime of a given like generation of GPU chips to increase with time.

Waleed Atallah: So the for example the Nvidia A100 GPU which came out a couple years ago now we even still have reason to use that GPU so it's a lot more affordable than the newer ones the H100 or the B200 it's still like a pretty decent piece of hardware and I think it's past the three years since that chip first came out at this point so three years is the usual cycle of a chip in a data center, whether it's a CPU or GPU or otherwise. So I still see a world where we would pay for access to A100s. It better be cheap, but we'd still pay for it. So you can still kind of like ring those for for some more revenue. So it's not totally out of this world.

Anna Kirk: Eight years, which I was it eight years? I feel like I saw that. Eight years.

Waleed Atallah: What was a chip from eight years ago? I don't think we'd use that.

Anna Kirk: I like I've heard I've heard four. I've heard I feel like I've heard a lot of different I'm not totally sure. I didn't read the whole article. I just read the headline.

Waleed Atallah: No, no. I think you're right that like I can't remember which hyperscala. Yeah. No, I could be wrong.

Ryan Eppley: It's also not just the GPU running. It's also how well are they cooled and the infrastructure around them. Like it's how well you treat your car. Yes, your car can depreciate, but like if you treat your car better, maybe it might not depreciate as the average car.

Anna Kirk: Yeah. But depreciation also is not tied to like the useful life of the GPU at all. Like that's totally they're totally divorced. One's like basically just accounting and one is like I said, how long can you use the A100 for?

Waleed Atallah: GPUs do have like a way higher failure rate than other types of chips. They get hot, you use them for a long time continuously, things start to break if you don't cool them properly. So there's definitely, you know, I guess like you could argue the useful life is, you know, for in question there. But they the two are are very divorced. Like in real life, like people depreciate on completely different schedules than the useful life of a chip.

Ryan Eppley: Anyways, we need to get an accountant on the show ASAP. If anyone has an accountant in their life,

Waleed Atallah: you ask ChatGPT.

Anna Kirk: Yeah, literally we could do that right now. The interesting thing about that also is that Burry on the same post on X did say stay tuned. More to come November 25th. So, we'll check back in.

Ryan Eppley: Couple couple podcast. Here's here's the real question. Do you guys own Nvidia stock and would you sell?

Anna Kirk: I don't own direct Nvidia stocks, just ETFs. But I own AMD. I'm big on AMD. Been on AMD for a while. Still still bullish generally on the

Waleed Atallah: Yeah, I'm telling you, Intel's coming back. Intel's coming back.

Ryan Eppley: Yeah. Coming back. Coming back. I think there'll be an overreaction. And like if if you're looking at the next couple weeks, I think there's going to be an overreaction, especially when whatever happens on November 25th happens. Plus, with like SoftBank selling some of their stake, I would hold.

Anna Kirk: Yeah. Yeah, I would hold if I were you. What about you, Waleed?

Waleed Atallah: Would I hold? Yeah, I think I [laughter]

Ryan Eppley: He's been his whole company all that.

Waleed Atallah: I Yeah, it better keep going up. [laughter] right. I actually I do think there's going to be a correction. Yeah, across probably the entire economy, but especially the that kind of a market. Basically what I'm saying is if you have a startup, raise now. Raise now. Raise right now.

Ryan Eppley: All right, we got one more article teed up for us. This one is more like sentimental and like kind of consumer consumption of AI. There's a lot of backlash of some of the upcoming holiday clips that have started being aired and you can clearly see that they are generated with AI and some of the consumers are saying hey like for Christmas or maybe Thanksgiving like we don't feel like it's appropriate to generate these very very important sentimental moments. How do how do we feel about AI generated content for the holiday season coming up?

Waleed Atallah: That's a good one. First of all, slightly related, I think next year or within the next two years, we're gonna see the first political candidate who's one of their primary things is their anti- AI. I think like like there's going to be this like consumer backlash outside the pitchforks outside the data center. I think I think it's coming. There's going to be an anti- AI political candidate. Anyways, about this I you know, I kind of get it, right? Like it's like the handmade pottery versus the factory produced porcelain cup, right? Like one of them is you just like it more. Why? Cuz somebody put effort into it. Mhm. And I think it's probably similar, right, for that people want the And sure maybe the fact we can make something really crazy and cool and detailed that a human couldn't like possible probable actually. But I think for a lot of these things, people are going to continue to value the actual human created stuff a lot more. I think there's going to be less writers in the future, but we'll pay them more because people will like that human touch writing and a lot more. They'll be there'll be relative.

Ryan Eppley: They'll be drowned out by the AI slop. That's [laughter] push.

Anna Kirk: Right. Right. Authenticity is most important now with slop being so, you know, prominent.

Waleed Atallah: Yeah. So, I think there's going to be, you know, always an appetite for slop as there always has been. I think there's there's going to be an appetite for soft, but there's also going to be an appetite for weird. Like I'm really excited for the like counter reaction to this to be like what is the weirdest and like we've already seen some of this today like what is the weirdest content that is so off-the-wall that it could never have possibly come from an LLM.

Ryan Eppley: Interesting challenge. So we're all okay with AI generated content for the holidays?

Anna Kirk: For the holidays in particular? I don't really have a strong stance one way or another. Like I watched the ad. I was like, "Oh, that's somewhat wholesome and just as commercial as I would expect from a Coca-Cola ad around Christmas time." Like but some of the criticism that it was getting is like, "Oh, like the lighting is off in this one scene." I'm like, I didn't notice any of that.

Ryan Eppley: Everyone's a critic. All right. Do Do you get AI slop marketing emails? Can you tell when you get like a obviously AI generated email like, "Hey, Willie, GPU optimization space is super hot right now." Anyways, unrelated page.

Waleed Atallah: Yeah. No, I usually can tell, but they're getting pretty good. They're getting pretty good at figuring out how to make me not feel like they're generated. Yeah, I guess like selection bias. If I haven't noticed, then maybe it's just

Anna Kirk: Yeah, it gets into the spam box either way. [laughter]

Ryan Eppley: It goes to spam either way.

Waleed Atallah: Yeah, it's now the slot box.

Ryan Eppley: Slot box. You got to trademark that.

Anna Kirk: Yeah, right. [laughter] Can you tell? Do you think you can tell when you see like AI generated Oh, I mean like videos, images or or writing?

Waleed Atallah: Sometimes sometimes no. Like there are some like AI generated videos where like you can kind of see around the edges like there will be some weird movement of the hair or things like that, but like I wouldn't say that I'm any better at picking up on it than your average consumer.

Anna Kirk: Yeah. Getting harder.

Ryan Eppley: Yeah. Well, when we end the show, we always like to ask a final question. We're here in New York. It's always New York based. The question of the week for us is what is your favorite New York City building, monument, structure, statue, anything that you are like, "Wow, like this this is super cool. I like that this is in my city and I can look at it." You know, what's your favorite building in the city?

Waleed Atallah: First of all, so I've lived in like a bunch of different cities and New York is definitely like the best city for that kind of stuff. There's so many like like I take the ferry to Roosevelt Island and I leave, you know, it's night times the skyline, you see it all. It's almost like it's like wow, holy we're here. [laughter] So there's so many different like choices, but the one that I've kind of been obsessed with lately is the Domino Sugar Factory.

Ryan Eppley: Ah, in Brooklyn. That's cool.

Waleed Atallah: It's like a office space now. And I was thinking of moving our because I want our office to be in like an epic building. Mhm. And this place used to actually be one of like the biggest sugar refineries in the United States for a while. And that feels like, you know, epic, right? [laughter] Like this used to produce the sugar for the whole country or for some portion of the country or whatever.

Ryan Eppley: Yeah. They did a great job with it.

Waleed Atallah: Yeah. And it it looks it's like it's got the old on the outside and when you pass it on the boat, this is why I think about it because I sit on the boat and I pass it every single night. It's like lit up real nice. I know somebody who works there apparently it's really nice on the inside. Yeah. So that that's been my I stare at it every day practically when I come home. That's the top of my mind.

Ryan Eppley: Love it. What about you, Anna?

Anna Kirk: Oh man, a couple I can probably think of. Like I had I had the opposite problem with with this were like, yeah, there are just so many to choose from. It was hard to pick one. So I narrowed it down to two. One of them is the Jenga building just because I think it's such a fun kind of playful part of the New York skyline. The other one that I will add as a shout out to Roosevelt Island is the Smallpox Hospital there. It is so spooky and creepy, but when you look at it, there's like beautiful iV growing around the side of it and you know, they're I don't think they're allowed to renovate it. U, so it's just kind of slowly like [laughter] crumbling. But it's just a really interesting part of New York that you don't really know about unless you know to look.

Ryan Eppley: The smallox building.

Anna Kirk: Yeah. Well, it's just spooky and cool. Like, it was on my mind with Roosevelt Island, so why not?

Ryan Eppley: I mean, the ghost of Rob would appreciate that.

Anna Kirk: Probably haunting it right now. Yeah.

Ryan Eppley: You know, when I was writing this question, I was like, I don't even know which one I'd pick. You know, I have a feeling that it would probably be the public library building. It's beautiful right at Bryant Park. I also I'm just a sucker for for history and like kind of this iconic class nature of New York of the Empire State Building. It holds so much more sentimental value to me. But honestly it's I wouldn't even call it a building, but if you walk down I think it's like 22nd or 23rd, you can see Teddy Roosevelt's little plaque where he was like born. It's literally like chest high when you walk by. Like you can walk by and not even notice it. But like I've seen it a couple times and I go over I'm like put my hand on I'm like, "Oh, Teddy Roosevelt like was born here. I didn't realize he was born in New York City." so it's like my favorite thing to like just pass by and just, you know, feel that kind of history of this iconic city. But yeah, what a great city to live in. Super happy to be here. So, thank you again so much for being on the show. This was really fun. And we can't wait to see what Mo does next.

Waleed Atallah: Yeah, thanks for having me. This is a good time.

Ryan Eppley: Thanks, Will. Send us your swag. We'll wear it on the show.

Waleed Atallah: Done. I've got some sink jackets [laughter] for