Rebuilding the Internet for the AI Era

Transcript generated automatically from the episode audio and lightly cleaned up (filler words removed, misheard names corrected where clear). Speaker labels were added afterward and may contain errors.

Rob May: Hi everyone. I'm Rob May. I'm the co-founder and CEO at Neurometric.

Ryan Eppley: And I'm Ryan Eppley, co-founder CEO of RootAccess.

Rob May: And this is the AI in NYC show. Welcome everybody to the AI in NYC show. We are shooting from studio near Times Square in the greatest city in the world. And our sponsor today is BePresent. So if you want to stay off your phone, they have a judo move where they use the same tricks that people use to keep you on your phone, they use it to keep you off your phone. So you should go check it out. We'll put something up on the screen. Our guest today is Zach Smith from Do you guys pronounce your company Datum or Datum? Do you have a preference?

Zachary Smith: It's such an interesting question. Yeah, I always respond with that with do you say data or data?

Rob May: I say data.

Zachary Smith: So then you know how to say okay datam.

Rob May: All right. Well so welcome to the welcome to the podcast today. Give us the quick high level overview of Datum before we get started and how you guys sort of started it.

Zachary Smith: Yeah. Well thanks for having me on. I would say I'm a lifelong New Yorker but it seems none of us are. But awesome city and been kind of rolling around the tech scene here since I failed out of Juilliard in 2001 and decided that playing classical double bass was not my way to a stable condominium apartment in Manhattan. So it's really cool to be part of the tech scene here in New York and what you guys are doing. So appreciate that.

So Datum is all about reinventing the primitives of the internet. So, I've grown up around an internet that was all about connecting everything. And we had this long period of time where we're just trying to get all the networks online and people were getting broadband and we were adding content and websites and trying to figure out how everybody could participate in this big weird worldwide web experime experiment.

And so I had some early learnings in setting up peering exchanges and how networks actually form which is very communitydriven. The internet itself is pretty communitydriven. But we're we're kind of a lot of things are changing that and so number one we have some architectural problems AI in particular and the way people are developing applications or the way things are being developed is just really breaking the kind of centralized internet that we've had for a long time. And then number two is that there's this really big movement towards I call it the splinter net like we have a lot more rules. We have a geopolitical environment that's not all hugs and kisses to say the least. And so AI is becoming a big part and I think in general the internet is such a huge part of society that we're seeing a lot more complexity. So Datum's about building an open network cloud, a way that we can bring those superpowers of how the internet works, upgrade parts of it, what we think need to happen, and give those capabilities to the next few thousand clouds that are being born right now. So that they don't have to go and figure out every little piece of fiber through routers themselves.

Ryan Eppley: Birthing clouds. What a what a concept.

Zachary Smith: That's right.

Ryan Eppley: I think you coined the term alt clouds. Yeah. Talk us through what what is it alt cloud?

Zachary Smith: Well, see, I have an identical twin brother, so it's really confusing about if he coined it or I coined it, but I'm here and he's not. So, I'll go with me, right?

Ryan Eppley: We're going to claim it first.

Zachary Smith: Yes. So, you know, in the I'm going to take it back really quick, but in the early 2000s when I got into the web hosting world, there were thousands of web hosting companies. There was ISPs. I mean, you probably you grew up in, you know, a neighborhood, they had a local ISP, probably. Maybe it was Dialup. Maybe it was DSL. And then there was this kind of big movement towards oh we got to get stuff online. So you had tons of little web hosting companies. And so for me it was this kind of golden age post 1996 the telecommunications act that kind of deregulated the monopoly where there was a lot of new service providers. And I kind of felt that we're going through another period of that today because so many software companies are being born as clouds. There was a whole new wave of infrastructure clouds happening who were servicing this new need around AI in a very different model than the hypers scale clouds. And then it seems to be that like every you know there this line that blurred between what is a SaaS service you know that I log into and what is a data service because it holds and manages all my data and what is a cloud service became pretty murky like this this typical IAS SaaS delineation to me didn't make sense anymore as people were assembling applications out of all kinds of different services.

So we decided that we needed to figure out how we could kind of umbrella this group. And we decided on altcloud because Neo cloud to people meant GPU data centers, right? And we're like no no that's not what we mean at all. We mean like what about Databricks? They're cloud too, right? And isn't like Vercel cloud? They're like oh yeah. So like how do we you know make this hey we're operators? You know there's a group of companies who who if their service goes down kittens cry on the other side. People rely upon this for their businesses or applications or whatnot. They have some sort of ondemand or consumptivebased experience that feels very kind of usage or I can get started versus you know kind of I have to build it. And they're global right you know that works wherever we want it to work. So those were kind of the key adjectives we ran around. And so we started compiling this awesome altcloud list on GitHub which now has like I don't know about thousand clouds and more being added every day. As you know there's this whole new renaissance of new service providers.

Ryan Eppley: Yeah. Super cool. And before we go from double base Juilliard right into all clouds like walk us through how how do you even get this company started and off the ground? What what prompted you over the last couple years?

Zachary Smith: Well I was fortunate I've I've been you know really fortunate to have built a couple companies and you know exited them a couple times and my my first experience was more of an emotional one. I bootstrapped a company with this guy Raj for 11 years and you know when when we sold the business it was like losing your firstborn child you know it was anything you knew your friends your your work what you spend all your time identity everything tied up I tell people this story when I sold my first company and I the day I got the paperwork and all I needed was my signature I walked around Cambridge Massachusetts like I carried it around for like three hours like am I going to do this like it's hard it's very emotional it's hard because everything you know especially and for me I've done this for my whole 20s.

So my difference between being, you know, a young kid, whatever, adolescent, teenager, and an adult was really just defined by this this period. But I took two years off. The first, you know, six months was like some sort of drug addiction removal process, you know, like don't look at your email, don't call your like you could just could not stop think I couldn't stop thinking. That was different. The second time I like went through more therapy, you know, had to talk it out, you know, and so I knew that after my brother and I built a company and sold it in 2020, you know, we knew like, hey, this like going to end and that's okay. So, we went through a transition period. You know, kind of helped integrate that business and then knew we wanted to build again, right? I love to build primarily I love to build communities and teams and I was sensing that there was just this really awesome opportunity around a new wave that I hadn't participated in since the early 2000s of this the way that people are building it's just so different and so we knew we wanted to start something and so we actually incubated the idea with a good friend Alex Bennick. Yeah.

Rob May: A mutual a mutual acquaintance up in Boston.

Zachary Smith: And he was just leaving battery kind of starting his own firm. So, he was all in idea mode and we were he knows networks really well and what we decided to build at Datum is a little little crazy, right? It's like building a new telco for the age of the internet doesn't is not very normal, right? You know, people are building, you know, chat bots and, you know, AI orchestration something or other. And we're like, you know, what we should do is we should build a whole new internet. And most investors, most VCs wouldn't even understand what you're doing.

Rob May: Yes, this is true. They're they're not as technical as people think.

Zachary Smith: And so, you know, having us just swim in that for a while. So, we spent about a year just kind of letting it set, clearing our minds. I had a really really amazing mentor, this guy, Bill Luby, was an investor in both my businesses. And I was complaining to him in 2000. It wasn't even having a talk. I was actually just complaining. We would go on walks around lower Manhattan and he's like six foot seven, so he walks very fast. I'm like always out of breath.

But Bill, you know, I was telling him, hey, this is this is my, you know, company called Packet. And we're like, I don't have any money and we don't have any customers and, you know, nobody knows who we are. And he's like, Zach, quit your bitching. You know, like, this is the best time. I'm like, this doesn't feel like the best time. It feels like kind of really hard and I'm not really sure what's going to happen. And he's like, "No, this is the best time because you have no past. You have no books that you have to close because you don't have any customers. You have no, you know, we've always been doing it this way. Your process like nothing. You So, what you spend 90% of your day thinking about is the future." He said, "At some point, you're going to be in a big business and all you're going to think about is the now and what just happened." Yeah. And so we knew we needed to let that after we left the Equinix business in 2023, my brother and I, we just got to let it leave. So we stopped thinking about the now. Yeah. And start spending all your time thinking about the future.

And I think that's just so important in infrastructure companies because you can't build an infrastructure company in a year, even with all the cool, you know, Opus 4.6. Yeah. Like you, it takes a decade or two to really physically change how networks work and whatnot. And so we thought okay we need to spend more time thinking about the future. So that was this just germinating period of about a year and then we kind of bootstrapped the business with some funding that we had just to see if we were completely crazy because honestly I didn't know at 45 with some gray hair and you know you get habits right and I'm like shoot am I really able to build in this new age? Things are different, you know, and so we did a little bit of just experimenting before we decided to raise some capital and we raised a preede round about a year ago and started getting serious. So that's how it happened.

Rob May: Yeah. Very cool. Do you have a perspective on you mentioned earlier that like like in the 90s I think a lot of people don't know this like networks were very different like I remember I used to TNET into other people's computers, right? And like you had to know the address and so the tools we're going to we're going to talk about DNS again like it's always that so well but think about this right so like the tools have made all that stuff easier and abstracted way so you don't need to know it u but in some ways it's made it less power there was a group of people that would never have have learned that right they can now do it but there was a there was a group of people that are semi-technical that would have learned it if they had to and used it if they had to and then didn't and so do you so in some ways these tools that made it easier maybe made it less decentralized ized or something. I do you have so looking back at what happened last time, do you have a point of view on where you want it to go this time and what's what what what's important as a society?

Zachary Smith: Yeah, it's almost we actually did not plan this but I love this question. So I was really fortunate that in 2001 when I got into Linux web hosting in order to put a website on a virtual host on Apache on a Linux server you had to do a lot of things. Yeah. You had to go to like net access out in New Jersey in Alex's data center and ask to borrow a rack and then because you needed a switch because you had to get on the internet, you needed some IP space and so you could do BGP and then after you figured out how to do that, you could figure out how you were going to like run an internal network and then you got to figure out how you're going to get BIOS to work with Linux because man, this don't don't work, you know, etc., etc., etc., right?

And then once you were on and you had some customers, you had to figure out how like you were going to keep track of it and bill it and meter it and you know whatever. And so you had to write some software to do that stuff. And the nice part is you really went full stack and it was accessible. It wasn't like we had millions of dollars. You could bootstrap this. You could borrow this from that guy. You could buy a $3,000 switch and be on the internet because there the whole routing table was not that big. Yeah. You know, like everything was kind of more accessible. And you know, Alex would talk to you and rent a half cabinet to you for 500 bucks a month.

And now you got to kind of get in with like a gigawatt or something like that and a billion dollar capital deployment if you want to play in the big boy internet. And so I think it is a lot less accessible right now even though we have never had more access to technology through hypers scale clouds and through other resources. There's almost been this kind of ability to be more intimate and aware in the past few years of you know dinner table conversations about power and data centers and chips and things that I didn't think my mom would be asking me about you know but she's like what's with all this chip shortage what's with the DRAM but it's also kind of this inability to go touch it so where do you get started and so that's one of the main missions that we have at Datum is I think Jacob and I wrote it down like our like what would be success and it's like help a thousand new clouds, you know, survive and thrive.

Because I think a world where you have three or four, you know, monopolistic clouds feels a lot like the '9s with AT&T kind of running the show and we all just kind of bought whatever they gave us. And boy am I super happy that that you know had some competitive nature because otherwise I don't think the internet has exist would happen right so I'm hopeful that there's enough oomph I think there's some real risks right now because there are some serious advantages to people who have unlimited balance sheets and some structural issues in the way I think the chip companies are and data center businesses are kind of going about their thing. But I'm hopeful that we can build a thousand clouds and that'll be more diverse. And I actually, you know, like I don't know, we can get into politics. Are we allowed to do politics?

Ryan Eppley: Yeah. Yeah. Totally.

Zachary Smith: Cool. Right. I was super pleased for one that we had both OpenAI and Anthropic with foundational models so that way one of them could say no to the Department of Defense. Yeah, I'm super pleased. Like good thing there were two, right? Or at least two, right? Because if we didn't have that, it'd be a really difficult choice for everybody. So, I'm really glad that, you know, we had that choice and I want to see more of that over time.

Rob May: Oh, well, I don't know if you looked at what we do, but I like I started a company to make intelligence free under the view that models are going to fragment and you're going to start building on systems of smaller models and you only hit these frontier models when you need to. Yeah. For certain things and so yeah, I think the world's going that way. I mean, it similar to what you think the network's going Yeah.

Zachary Smith: Exactly right. Like use what you need. Because that's where you're going to get more stuff, right? And it's Yeah.

Rob May: And it's interesting because it it ties back to like the same idea. It's part of what I love about New York and it's part of what I love about the culture of New York is cities become very homogenized, I feel like. Right. You go to Seattle, it feels away. You go to Dallas, it feels away. You go to you know, Boston feels way. Boston's very academic intellectual, that kind of stuff. Orange County, California. Yeah. Yeah. And you know, yeah, you know, they all have their different like standards of beauty and way people dress and hobbies people have. And you come to New York and it's just like so much of everything. And there's still all these like hole-in-the-wall pubs and record stores and like that you don't see anywhere else. And I know there's less than there used to be, but the whole city just hasn't been gentrified the way that other places have. And I think that's awesome.

And I like as a child of the 90s as well who was into like you know programming in the early internet in 1992 like I it just like I kind of want that world back because it was cool. Yeah. Yeah. Let let the weirdos be out and it's actually bad because to your political point when you when you have these igopies it actually like they aggregate all the eccentric weirdos under like one thing sometimes, right? And this is how you get like MAGA or something like that, right? And so it's like I think keeping, you know, letting everybody have their space in the in the right way is really really important.

Zachary Smith: Amen to that.

Ryan Eppley: Do you have a thesis on what type of cloud is like best for for your service right now? Are you thinking about like what sector that you're probably optimizing for?

Zachary Smith: I mean, first off, you have to be somewhat infrastructure adjacent in order to appreciate what we're trying to do. Which is really a connectivity play. We think that you know number one applications have been moved like this. You know 10 years ago we're still doing it. We've got a server in US East. Maybe we did it via a console or an API but there was a we knew it was a computer with a VM and a disc and we chose what OS we're going to put on it. We put a firewall on it. You know we like did it basically. Maybe we did it with DevOps and APIs and a whole bunch of automation. And then we made a million of them. But it was still the same kind of like client server computer model that we've been working on since mainframes went out of style.

And I think like that's changed. Number one, we have things like distributed systems that have gotten a lot better. The local first movement probably what you're talking about with on, you know, local models. I mean, I had an investment banker I was chatting with yesterday. It was a good friend who's like he's like, "Dude, I was drinking four espressos and stayed up till 5 a.m. Working on OpenClaw. My laptop and then I had to set up a Linux server and do these like weird commands that I copied out. I'm like, dude, you're like a CIS admin hacker. You just don't have awesome. Yeah. Yeah. Like way to go, man. Yeah. Throwing around that pseudo a little bit. Yeah. 100%.

So, you know, I think that applications are moving everywhere, you know, and also the way that people are building applications. They're assembling, you know, JavaScript front end with a back end of this and a pub sub here and whatever. So, so I think we're just moving application intelligence and data. That's the mo that's going to be my key thing in so many places.

And then with this kind of fragmentation of service providers, which I think is really powerful, you know, enterprises now no longer just think multicloud as Amazon and Azure, right? I mean they're all using Databricks and Anthropic and you know you name it the sovereign cloud ABC like they got all stuff right so the dam has broken so now you've got all these critical services and that point I made earlier about more rules which are going to be around this is illegal in this state you have to do French data for French citizens and French data centers only and never let it leave the country like how you going to do that you're going to have to build networks around that stuff right? You're going to have to choose who you curate and connect with. So, I'd say to answer your question more succinctly, clouds that are basically interested or dealing with data, and have some sort of target customer that is a enterprise, a real business, not just developer users, which is I love the developer users, but like they just want to move fast. Yeah. Yeah.

And it's really like, oh, but if you're a developer user who works at Bank of America, you're regulated. These these rules that crop up because some lawmaker export controls, Belgium made something. You don't get to just like ignore it, right? And so that's the target that we're looking for is a wave of builders who are really curious and interested to adopt new ways of curating and connecting. They're probably doing it at a velocity that's super different than the world of, you know, telecom, right? Which was send a ticket to the network team and two days later get a service now request and they say their firewall port is open, right? Like that's not going to work anymore and have some sort of target towards enterprise users who have the pain.

Rob May: Yeah. So, does some of this tie into this new version of the networking world that you look at in the future? How does this tie into the concepts of net neutrality that were sort of debated all through the Obama administration, the first Trump administration, and then we're like, I don't I don't know where the rules stand today. I know they've sort of gone back and forth under the administration, and I text largely stopped talking about it. But are these same concerns whether they were right or wrong, I don't know, but like are the are these concerns people have about the future of networking?

Zachary Smith: Well, I think so let me let me let me go to how I think about how the internet, right? I'm going to give a very you know my first interaction with very weird stuff on the internet. So in early 2000s I joined with this guy Raj and we're building this hosting company Voxil on open source and open source wasn't just like oh it's cheap and works. It was kind of like because it matters man. Yeah, you know, this was like we're going to like take away this proprietary stranglehold from Solaris, IBM, and Microsoft, and we're going to make software free. Let's let's everybody should be able to work on a database, not just the ones who can go to it and get a big license for something or other, right? So that's like how do you get Linux, how do you get Apache, how do you get like MySQL, how do you get software? And we, you know, there's a movement called the free software movement, you know, which is probably the more radicalized version of open source. But we really believed in that. Okay. And then it's starting a company that was going to provide a place where you could use open source including operating systems. You know, you had to like become a service provider. Okay. Now you have content.

So is content all good? No. Turns out there's a lot of bad content. Yes. And I remember because like 6 12 months in we start getting all these letters and you get calls from some random, you know, police officer, FBI agent being like, you got to take this down. This is, you know, child porn or this is like invi inciting violence or this is, you know, whatever that's illegal in this. Oh my god, where are you? Like who are you? Right? Yeah. And so the common answer was listen, send us a warrant, you know, to take down this content and we will, right? And almost all the time it was never, you know, they would never go through that process or didn't have it. But there was some truly objective content and we had to answer this question like are those aligned with our morals?

Oftentimes not, but are we going to be judge and jury for what is allowed to be, you know, online? And the answer was no, we're not. And so we you would have policies which say hey we don't want to have customers we don't want to take money from people who are inciting violence or you know basically the main thing was violence or or or some sort of you know I'm going to call it illegal or suspect act right in the jurisdiction that we are in but you did have to kind of take a line there on what you felt the internet was about right and I think fast forward today. Although in ways we have this like supery big internet with all kinds of stuff on it and people have come somewhat desensitized to you know objectionable content or whatnot. You know we've turned it from I am a neutral hosting company or network and in order to be judge juror have to inspect every packet to advertising businesses whose entire thing is inspecting and promoting content. Right. Yeah. And I just don't believe that they can take the but we're neutral and we you know we're going to work under the Telecom Act and we're just a common carrier. Sorry. Right?

You're completely not like if you wanted to be a common carrier then we would figure out like others would figure out what promoted and whatnot. People would see what they saw. They would make phone calls to their you wouldn't be like making money on the number of times the weird stuff got you know pushed. So I think the common carrier side of net neutrality is pretty messed up right now because of the concentration of the ad driven model. Yeah. And my hope is that we're going to see, you know, some learnings from the past 25 years of consolidation there. And I'm not in a position to do any like regulatory monopolistic breakups. I don't think that's currently on the foresight of our our administration either. But I certainly hope that competitive natures will allow more companies to come in and eat the lunch of the big guys and make them, you know, a little weaker.

Rob May: So, it's I mean, it's interesting. I think it ties I didn't put this in the news for the week but related to the thing that you just said there was an article that came out Walmart came out and said they've been doing this experiment with ChatGPT commerce through e-commerce through ChatGPT and their conversion rates rates were like onethird of what they were through their own stuff interesting and it it's as you're saying that it's like one of the things I'm thinking about AI is as you like I love it as a kid who grew up a command line you know programming with DOSs and then eventually Unix or whatever I like going back to text and chat, but as it takes away some of the visual abilities to display lots of ads and all that, like it could start to take your argument in a different direction, right? Which is like the future of the internet has fewer clickable ads displayed, you know, because you can't like like if I ask ChatGPT a question, it can't be like here's here's the answer and here's nine advertised. Like maybe it can serve one or something. So, like I'm hoping that the ad concentration breaks up a little bit as a result of AI. I mean, do do you guys like knowing that about Walmart and OpenAI, does that

Ryan Eppley: oh, there's too much money on the line. They will adopt and then change and adapt and I mean, I think the what you're talking about is like a markdown file, right? Like, you know, AI is just going to give you a very very textbased markdown file of what you're looking for. They're not going to give you the fluff unless you're on a service. And I know that was the big thing we talked about a month or two ago is OpenAI thinking about advertising. How would they introduce that? That was like a big thing during the Super Bowl ads because all the, you know, Anthropic Yeah. You know, ads came out trolling them basically. Right. Right. So ads will will always make a rise because there's just so much money on the line for them.

Zachary Smith: Well, I mean I'm going to make a counterpoint. I agree like there is it's there's so much concentration of how do I say gravity in the ad driven model for the web. And I think we saw some in the crypto movement which was like no own your own data. I mean you know kind of is right but you know what I mean again like the decentralized web which still became centralized. It's like but cryptos now own the top 100 wallets own 60% of Bitcoin or something right? But the decentralized concepts I think were pretty popular, right, with people, which is like, wait, I can own my own data. That's pretty interesting. And I think that over the past few years, we've seen AI make technology like everybody knew that they were getting advertised to with and it was like, oh, it's fine. I just talked about this and now it's advertising it to me. It's super weird.

Wonder how that happened. Oh Siri, you know but you know ads was creepy but like AI is right in your face you know everybody can understand wow this thing remembers it knows a lot about me it's getting really personal in terms of my data I'm telling it all this stuff like I'm telling it way more and I think that there is a subset of our you know general populace who is you know more interested in owning that data and less interested in giving it away at least they're aware of this transaction to a much bigger degree versus like I to watch free videos all the time. This is amazing. I mean I kind of know I'm being outro but like you know whatever. Now this is like super clear and people are paying interestingly enough you know for these you know AI services to do a lot of very important things for them.

And I think that what I'd hope is that over the coming, similar to the Anthropic OpenAI DoD situation, we'll have companies who aren't going to be in the ad driven model business has built we want local models and your data to work for you and we don't want to serve you ads, but we are going to charge you 10 bucks a month. Yeah. You know, to do all this stuff for you. And some people are going to be like that's great cuz I wasn't interested in uploading all my bank statements to the public internet you know or whatever. So I think like that is the tension is there. I'm not sure that we have reached any kind of tipping point and there's a lot that but you know I think there we're going to see more models which is good.

Ryan Eppley: Yeah. Well one of the leaders in kind of the decentralized at least training side of things is Prime Intellect. I don't know how much you've spent time with them. I think they're they're interesting because they're trying to say, "Hey, we see that basically the models for OpenAI and Anthropic, things like that. They own a lot of the infrastructure. They own the entire process and we really don't know how the sausage is made outside of them releasing white papers when they want to, right? But Prime Intellect is coming up and saying, "Hey, we can decentralize this model training process around the world." I don't know if you've you've talked to them, but I'd be interested on your take with them when you when you find out more about them.

Zachary Smith: Yeah, I mean I'm I'm pretty pretty big on an open future. And I think that cat's out of the bag if my you know 45-year-old investment banker friend is hacking on his laptop on the terminal. Yeah. Like this this is going to get way easier to be like I'm going to have tooling that'll take what has been pretty much a gooey to me and turn it into an actual workstation that runs stuff. Yeah. Like that's probably one of the biggest untapped computing resources we have on the planet is people's MacBooks sitting around, right? And I know that Apple's taken a very very different approach. I saw a capex graph in SemiAnalysis. Yeah. You know, 14 billion while everybody else is doing 10x or more that, right? They look, you know, they're taking a very different approach, privacy oriented as Apple has been for the past few years.

And so let's wait and see. But I think like decentralized and distributed access, you're going to get forced there because there going to be a lot of country rules. It used to be one big internet and maybe then it was China was a little bit separate and then Russia and now Indonesia and then now it's like states in America have different rules and you know we're way we're ahead of the European Union on regulating you know kind of GDPR style controls. So I think that this is all going to get fragmented anyways. Maybe the US is still going to be or the big model, you know, big big markets will still be pretty concentrated. But I think technology and companies will pro, you know, will crop up that will be able to service a more privacyoriented internet and I'm excited for that to be discovered.

Rob May: I mean, related to that, we've we've got an article from the Deep View that we're going to put up here about the four AI trends that shaped Nvidia GTC. So their big you know conference was last week and I think the most interesting thing that came out of that was their push for open models right their their new Nemotron model it's actually really really good in terms of like sort of what do they call it like intelligence density or whatever the stuff that you can use it for for the size it is in parameters and then and they're partnering basically with all the second tier labs you know thinking machines mistrol all those guys to really push the open source thing so I think that bodess well for the future that you see

Zachary Smith: 100% And at least in Nvidia, this is like, you know, kind of at their own. They're like, well, we want, you know, everybody to have the

Rob May: Right. Right. Spend more of that money on our hardware.

Ryan Eppley: Yeah. Very, very, very smart. And it's interesting, right, because they are a centralized player obviously at one part of the stack in how dominant they are, but it seems that they want all the other layers to fragment for various reasons, which I think is probably a good thing. Yeah. I mean, I'd love to unpack that a little bit.

Rob May: Yeah. Well, they you know, if you think about if you think about their risk, right? Their risk is that when when a lot of industries start, they go off and they build sort of like full stack pieces, right? Your fir your first SaaS applications had to go build their hosting and sort of everything else. And then as people do that, then you get to the horizontal layers where you can launch S3 and you can launch Heroku and all that kind of stuff. And I think they're I think they're very worried about like one of the things that didn't happen that I thought would happen is somebody would have gone to buy like Groq or SambaNova, one of those chip companies and been like we're going to make a whole competing stack because there hasn't been like I think we came up in the LAMP stack days right where that was like a standard even though it was open. There hasn't really been a standard like that on the AI side. Yeah.

And I think Nvidia, like CUDA and PyTorch maybe were like the closest things to standards. And now I think I think Nvidia seeing people encroach on their CUDA mode and everything else is like we want to fragment this thing out again and make you know partially to create new opportunities and new ideas partially because they have so much money they probably don't know what to spend it on and but I think it's a little bit more existential. So for Nvidia Yeah. Yeah.

Zachary Smith: So for like I grew up in the early the first wave of Intel and AMD competing when AMD created Opteron 64-bit chip. Suddenly you could address you know real memory. And for a while there they were just like you know taking Intel the cleaners. But Intel had this really nice verticalized model which they had the fab and the whole kind of Windows you know ecosystem tick tock and they really were able to out innovate and AMD just fumbled even though they invented and they have this perpetual license x86 so here we go this is amazing they got 64bit you know they're crushing it then they basically died okay then the second wave came around and here comes what's different right AMD in 2015 Intel's collecting, you know, 98% of the data center market share, like literally all of it, you know, and AMD is just coming out from the ashes, but they go fab right to TSMC and they basically focus in on the hypers scale customers and they crushed it and Intel just couldn't catch up with their verticalized model. But I think it also represents this underlying we'll take this to Nvidia in a second.

This underlying competitive issue which is that for the past 40 years there's been this high R&D but high margin chip business that kind of like called all the shots right basically like we make the thing we design it and we give all the reference boards we make sure the software world works with it whether that's we sit on Red Hat's board or we hubby hubby with Windows or like all the ecosystem is working and then we give it to OEMs to make and do all that selling instead of making 80% gross margin they get to make 10% gross margin that's the Dells and HPs and you know super micros and related they get out to market for us well we go back and like you know make next chip right and the hypers scale guys come along and they're like wait I'm going to actually own the customer and I want the 80% margin I don't want to give that to my supplier I see Intel as my supplier it's not like Amazon No, it's like Amazon powered by us, right? And so here's this like movement towards this distribution channel that, you know, Google and Amazon in particular have verticalized their own silicon and they they they'll buy your AMD stuff, but it's a supply chain thing for them, not like, oh, it's your customer, you're driving me to demand.

Now, that has changed with Nvidia, you know, because suddenly out of nowhere, not out of nowhere, but like everybody wants Nvidia needs it. So that's like back to the days they give me x86, right? And it's pretty powerful.

Rob May: I love the anamanopia.

Zachary Smith: You like that? Yeah. I mean like for for those of us who grew up in the 80s strong branding for Olympics, right? You know, and so like this this you know this brand of like I want Nvidia, I don't want AI, I need Nvidia AI, right? Is real and it's strong. But the hypers scale companies like Google does not need Nvidia, right? And they have a much better business model called like I don't got to sell new chips. I just got to sell Gemini, right? Or give you value like all this like I get to lock and own the customer in my world and this thing on the bottom called the chips is my extra money down here, right? And that's Nvidia's whole business.

And so I think like they have to fragment and get that distribution model going because in the end the long-term just even capital on they like Google can cause new TPO to show up in market. Nvidia actually can't. They have to like have people buy it, put it in data centers, build data centers that actually work around it. 800 volt data centers. How are we going to convince everybody to do that when they just finished doing 400 volt? Business is like tens of billions of dollars of capital you got to go get and so like theirs is much more complicated and so I believe that it's not only in their best interest they have to create more points of distribution to compete with the you know hypers scale verticalized model otherwise they'll toast like in the end you know the economic model is just too powerful for as a service

Rob May: I mean they've got the best view of the AI ecosystem of anybody right they see all the players they're invested in a lot of and they've made some very, very, very savvy acquisitions. I think I think Groq was savvy. I think Mellanox was savvy. Like, yeah.

Zachary Smith: So, Groq was was an essential one for them. I think more back to the existential crisis. I think they were like, "Oh, yeah. We need this before someone else eats that up."

Ryan Eppley: Yeah. I mean, what's our take on on Terra Fab, right? That just got announced like I think it was this week, right? I don't even remember how long ago it was now. But I mean it's a massive deal and it's exciting. But I don't know how you think about that when you're going through how you've seen the last 10 years progress or even last 20 if you really go back to the first AMD kind of pitfalls now their rise again. Nvidia's kind of like the king. But like will Elon come in and start thinking about this lowest of levels?

Zachary Smith: I mean let's go back to the better one of Intel attempting to reestablish what is it 14A. You know, I mean, it's fabrication leadership. I mean, they they burned a lot of money to basically not actually be able to get anywhere close to TSMC, right? Yeah. And I think that this is not I mean, even just pure engineering humans, you know, it's so hard to build a fab, right? I mean, I just don't I think we should continue to invest in it from a national standpoint for sure. I'm just really unclear how somebody goes and builds a new fab in 2026. Maybe you go and drive a lot of partnership or demand with Intel, you know, while you play TSMC, but like GlobalFoundries, you know, fine for the old stuff, but I just don't see how you compete in two nanometer. I mean, like it makes everything else we're talking about, power and space and data centers, laughably simple. So when you're worried about the curvature and vibrations of the planet, you know, while you're doing lithography, like there's only there's only a few people who know how to do that right now. Maybe they go work for Elon.

Rob May: I'm going to I'm going to say I'm going to give a take that's going to make me very unpopular with the tech community. I think Elon's lost his touch. I think his I think his focus on politics and his sort of drug use and all that kind of stuff and the crazy that he tweets now compared to how he was, you know, five, six, seven years ago, he's in a bubble, man. Yeah. And I think he you know and I think you see this I think you see this in the failures of Twitter. I think you see this in him merging his companies together because he sort of has to keep them alive and stuff like that. And so I'm yeah so I'm I'm skeptical which is and it's super disappointing man because he was like the hero like he was like the ultimate like don't care do it for the tech kind of guy.

Ryan Eppley: Dream big, figure it out later.

Rob May: Yeah. And like I think we all we all admired him so much and I think now we're like hope I don't end up like that guy. It's the hero that lived long enough to become the villain kind of story.

Ryan Eppley: Yeah. Yeah. I take the perspective, you know, it's kind of like Warren Buffett with America. Never never bet against the US, right? I think you should never bet against Elon. I just know that there's some rocky moments, but like there's something really incredible going on here. I just don't know where it's going to land, but I think it's going to be net positive. I just don't know if it's two to fiveyear net positive or 5 to 10 year net positive. I think it's an incredible moment here. I don't know if it was the right business decision to merge all the companies together, but I mean the amount of, you know, just technology he's created is, you know, you can't deny it.

Rob May: Yeah. You know, catching catching rocket 2030 version of AIC show. We'll we'll see where we are with back here then.

Zachary Smith: Yeah. Well, I think the maybe more like potential disruptive way of the chip business beyond like create new fap, okay, maybe, but like probably not. Is actually just fix the business model. The business model today like this is actually where I think we're sitting in New York City, which is just the ultimate place for financial and capital innovation, you know, on the planet. We've figured out how to securitize mortgages. We got like ABS's. Like we know how to like, oh, it's a big industry that needs a lot of capital. Yeah. Let's figure out how to do that more efficiently. Yeah. Yeah. And this is the one area that we have just really been complete doofuses in my opinion.

Because in the chip industry, like that's the technology. We're all just, as I like to say, you know, all of us AI businesses, hosters, networks, we're all just working for Intel. Yeah. You know, and I mean, now we're working for Nvidia, right? But we're all just distribution and layering on top of the real technology, which is like none of this would be possible without Nvidia making their chips. Okay. Everything else, oh, which flavor of Linux, what model you're using, what, you know, distributed runtime, you know, it's all just an innovation on top of like Broadcom, Nvidia, and a whole bunch of like IP from Qualcomm and Intel probably, right? And so they got all this stuff and it's incredibly capital intensive in their business model.

And yet they push it out to cloud service providers like a neocloud who's supposed to go figure out how to buy it all, build a whole building, like finance all that stuff, build the customer thing, get it to market. It's like what? You know, like they're supposed to do all of that, which is why you've started to see some innovation in like financing a GPU against the customer. Okay, cool. So we're going to create that like an asset class. Yeah. Okay, cool. But like, you know, I see that, you know, just I don't have my phone on me, but like this company has done so well with like a dominant premium position with the Apple ecosystem of taking what they used to distribute through Verizon and say, "Go get your, you know, 2009, go get your iPhone from Verizon. They'll finance it because, you know, nobody wants to pay $1,000 for a phone, but they'll pay 50 bucks a month and you put it on your bill and have your relationship. We call Verizon because, you know, we're going to help you with your whatever."

And then over the past, you know, 15, 20 years, they just like made it a service. They're like, "Do you want a new iPhone?" Like just come in the store. I always say like they'll come on roller skates and before you know it, you got a new iPhone. Barely barely cost you anything. And suddenly you got the new one. A couple hundred bucks, but like they take care of secondary market. And they're not just going and like reselling the phone to like wherever. They're taking it apart. They're getting all the stuff out of it. They're getting rid of that. And we don't do any of that in the chip industry. Yeah. We have zero secondary market. We have zero circularity. I'm like, car companies figure out how to sell you a brand new car every year. And yet, you know, there are like nine ways to do it. You know, rent it, lease it, take back, pre-own. There's there's a company in Brooklyn that's starting to refurb GPUs that are coming back on from like sort of like 2019, 2020 era GPUs. Maybe we'll have them. This is the equivalent of like Joe's Auto Shop selling you a used car versus like certified pre-owned from from Mercedes.

And so I think like if somebody came to market with a new chip and instead of saying, you know, we got to do is we got to go get those into Dell servers, we got to figure out how we're going to sell them to that guy and then we can go bank another chip, why don't they just get people on a chips as a service business? Like let's just fix the business model. Now, it would require financing that chip company differently, but boy would it like change the game because now you get super low cost capital instead of 80% gross margins to Amazon who's financing somebody else's chips most of the time. Well, maybe that's 10% financing. You know what I mean? Like, so there's real ways that this can change, but I think you got to think about the business model, not just the product.

Ryan Eppley: So, we're almost at time. We always end up with a question related to New York specifically. So curious if you have an opinion on this. It's always a best or worse question. So this week is best French fries in New York City.

Zachary Smith: I do love my French. Well, I have a rule which is if you go to a fancy restaurant, I'm like kind of like a midbrow eater. Like I'm not like, you know, fast food, but I'm not, you know, whatever. But I kind of have a rule. If you go to a nice restaurant and they have a burger on the menu, you need to order it. So the Wall Street Hotel. Yeah. Yeah, downtown. Went there recently. They have like a prefix, you know, like restaurant week situation. If you're not from New York, restaurant week is where it's at, man. $26, you get like a real sit down lunch. It's like 10 restaurant weeks per year. But yeah, I they never ends really restaurant week, but it's amazing. And the Wall Street Hotel has a restaurant, I don't know, it's La Mashant or something. Can't It sounds very fancy. They have a $26 prefix menu. They have a gourmet smash burger with a thing of fries. 100% worth it, right? And not even more expensive than going to Shake Shack if you really add it up for

Rob May: I do the same thing. I get the burger at a lot of nice places for that reason, but I've never had fries as good as Five Guys. All right. Anywhere that they are.

Zachary Smith: Five Guys, you're going with Five Guys. Five Guys, a New York City original, right? I don't know if it is, but they're here, but yeah, I didn't think it was.

Ryan Eppley: Honestly, I'm I'm sad that I don't know the diner's name, but I just had some incredible fries. I think it's like on 43rd and ETH or something like that. Great, great diner fries. I don't know.

Zachary Smith: Diner fries are the best, especially at 2 am.

Ryan Eppley: Yeah, it just hits when you need.

Rob May: Awesome. Zach, thanks for coming on the podcast today. If you want to check out Datum datum.net datim.net. All right. And, thank you guys for watching. Please subscribe. Rate us if you like it. Don't rate us if you don't. And, send us send us comments, guests we should have on, and stay tuned for, stay tuned for our conference this summer and some other stuff that we're doing. Thanks for watching. Check out BePresent and you know you'll be a better person for it.