Ex-Meta Engineer Turned Founder: Inside ScaleBuild AI

Transcript generated automatically from the episode audio and lightly cleaned up (filler words removed, misheard names corrected where clear). Speaker labels were added afterward and may contain errors.

Anna Kirk: The next. Welcome to the AI in NYC show. I'm Anna Kirk, sales lead at Thread AI.

Ryan Eppley: And I'm Ryan Eppley, co-founder CEO of Root Access.

Anna Kirk: And today we have our guest Ujwal Roy who's founder of ScaleBuild AI. Ujwal do you want to just give a quick introduction to yourself and kind of your background also what motivated ScaleBuild AI?

Ujwal Roy: Yeah sure. So first of all thanks for having me here and great to be in the AI in NYC pod. So I'm why I started ScaleBuild AI is I need to talk a little bit about my background. What was I doing before that? So I worked for companies like Meta, Micron, Intel mostly focused on the hardware side of things. So I was in the enterprise cloud division where we were serving like the top hyperscalers in the world. So when I worked for Micron and Intel, Meta, Google, Microsoft, Amazon, most of these top internet companies that were building data centers at very very high speed, we used to put them in the bucket of hyperscalers.

So Micron and Intel was supplying the compute and memory chips or memory and compute chips, right? So Micron was was the biggest supplier of DAM and SSD or is the biggest supplier of DAM and SSD in the US. The only bigger companies bigger than Micron is SK Hynix and Samsung but that's located in Korea. So in the US we had the biggest responsibility of delivering these storage and memory chips to the hyperscalers and when it came to compute then at that time when I was working in the corporate world Intel was still the category leader. So, Intel was delivering chips like Intel Xeon scalable processors to the hyperscalers over here and working in the enterprise cloud division in both of these companies and then later on working in meta for OCP open compute platform which is again their data center hyperscaler division and it's an architecture that Meta has now open sourced on which they're building all of their own data centers, AI clusters, things like that.

One thing that I quickly realized is that this AI thing is taking off and I had an early view in the data center business and also in the workload. What exactly are these data centers processing and I could see that AI is going to be like electricity pretty soon where every company in the world they cannot say that yeah we don't need electricity to operate like electricity touches them one way or the other and then later on when the internet came 30 years ago now internet is touching every company in one way or the other right like whether it's banking and finance or healthcare education like you name it whatever whatever vertical you talk about, electricity has touched them, internet has touched them, and I could see that AI is actually going to be touching every company. And just like in the com boom or the dot era, you could see that the first thing that every company needed was a website, right?

Like a dot website. And that's how they entered the internet. And then later on they created portals and things like that where their customers or their employees could interact and work on the company on the digital layer on the internet. So I could actually tell that every company in the world now will need AI and there has to be AI system integrators that actually delivers AI to them because AI was so brand new and it was created by few select groups of people in the world that we didn't even have enough people to actually just deliver AI or basically a company XYZ could not just go out in the market and open the position of AI engineer or AI system integration engineer and then hire out because that role didn't exist, the training didn't exist.

So that's why I started ScaleBuild AI to fulfill that gap that created that was created immediately in the market after these model companies started coming out with APIs and we were one of the first system integration company that were integrating OpenAI APIs to the companies that already has large amount of historical data or they're generating a lot of data on a daily basis and the first work that we're doing in ScaleBuild AI was building custom rags and at that time like our customers looking at rag working so flawlessly they were actually thinking that oh wow it's magic and they were willing to pay top dollar for it now anybody can can build rag but basically that is how like ScaleBuild AI had started like we wanted to be the AI system integration company in the Bay Area I had never really thought about building it into a really really large company I thought it's always going to be an AI agency kind of like mom and pop even though the name has scale in it. I was not thinking about scaling it to like millions of users or anything like that but basically that was the idea behind ScaleBuild AI.

Ryan Eppley: When when you say you saw that this was a pretty endemic across almost every industry you still had to start somewhere. Was there a specific sector that you started with or a specific kind of niche that you said hey we're going to take our first 20 30 customers from?

Ujwal Roy: Right. So that's a great question. So I could actually tell that the first so AI is going to disrupt almost every industry vertical but the first industry vertical that it would disrupt greatly was the SaaS industry software as a service. So we could actually tell that most of these software they will need AI capabilities later on even aentic AI capabilities. So SaaS industry was the first thing that that we targeted.

Ryan Eppley: Okay. And did people like get concerned when you first came to them be like, "Hey, we're going to give you this AI thing because you've been doing this for quite some time now, right?" you were one of the first ones to really come out with this kind of service. Were people looking at you like, "Why would I pay you so such top dollar to get access to this?" Or were they pretty floored away in their early discovery process?

Ujwal Roy: Yeah. So when we first started in 2022, people were not even willing to talk to us. It was really weird. But one year later once ChatGPT and these kinds of technology started going viral. CEOs started getting pressure from the board that you have to think about your AI strategy and things like that. So we actually shifted gears in our company rather than saying hey we are AI system integration company. We started saying that we are AI strategy consultants and if you are a CEO and you are under pressure from your board to come up with a strategy we have an answer for you now right we can actually build you a strategy but we not just build a strategy we also implement it so we have an AI engineering team a team of like AI engineers scientists data architects which we could actually use to also implement the whole architecture on your business

Ryan Eppley: and now things have progressed so rapidly in the last 3 four years that at the time there was only maybe one or two big labs that you could really API and get like some success out of but now there's a whole host of open source models tons of different frontier ones not only just in America now but in other countries are you being asked to deploy multiple different models for your one customer or are they pretty tied to just one

Ujwal Roy: right so whenever it comes to enterprise level deployment most of the or most of the customers that we are dealing with they do not really trust the frontier model companies like Anthropic or OpenAI. The first question they ask is will our data be shared with their server or even if they are like end to end encrypting all the information that is going in training their model.

However, it's not encrypted to the model. It's encrypted to the engineers or employees that are working in these top labs. But the models can still see the data and whatever data it ingests maybe sometime it can also release that data. Yeah. To someone else you know. So that's why that's the top concern. So most of the time whenever there's a case of enterprise deployment they actually do prefer trusted open-source models. Mhm. So and also there's a large push in enterprise in keeping the data on prem in most cases right like especially in the in the banking they don't even want the data to go out and hit like OpenAI or Anthropic cloud you know so and also the other thing is even for the companies that might not care about the data leaking to these frontier AI model companies maybe their data is not that sensitive.

For larger companies, the cost of token is so high and recently we're hearing the news that Microsoft had to roll back on its clawed subscriptions for employees because the bill was like $500 million a year or something like that, right? They were paying anthropics. So but that's just mostly in like coding, right? Like people are using Claude Code and that's where most of their tokens are being consumed but even if you are running some AI model to do process automation or to provide like higher value solutions to your end customers the cost of processing on these top labs APIs is pretty high.

Yeah. So if you want to be cost efficient in today's world then it's best to have your own either your own on-prem GPU cluster or some rented GPU cluster and you are running some open source model that is like completely safe to use. It's not connected to any server and whenever some new model drops you actually manually upgrade it and you are saving the token cost and also you are saving the risk of like getting your data exposed to the public.

Anna Kirk: I'm interested to hear a little bit more about you know a lot of scale builds origins were kind of in SaaS and there's been a lot of talk recently around like the future of SaaS and whether or not those types of companies are going away whether or not they're getting displaced or we'll just be forced to transform very quickly.

Ujwal Roy: Yeah. Yeah. Exactly.

Anna Kirk: What's your perspective on that? I mean, I feel like we've been kind of going back and forth within like the kind of public debate, but do you see SaaS companies as kind of will they be displaced? Will they be able to take advantage of that kind of distribution and data mode in time? Like kind of where do you see that landscape shaking out?

Ujwal Roy: Yeah. So 100% of the SaaS industry will actually transform or it has already started transforming and I say that there is a big gold rush going on right now where 18year-old or 15 year old even college kids who understand AI pretty well and they understand the tech stack. What they're doing is they're looking at some really successful SaaS products and then turning that into an aentic AI product just copying the features and everything and then launching at a as a new product in the market. So either these SaaS companies have to transform themselves.

It's called AI transformation. They need to go through the AI transformation or they will be disrupted or they will be run over by these new AI product companies that are being launched like every day. I think we have like if not thousands hundreds of like serious AI companies that are being launched, right? So SaaSpocalypse has already started and the SaaS industry that we know of or even the business model in SaaS those things are completely changing like SaaS industry survived on subscription model for the longest time like doesn't matter whether you use their software or not this month you're still going to pay like pay per seat per year.

Ryan Eppley: Yeah pay per seat per year

Ujwal Roy: like let's say $35 per seat per per month you are paying. So the now the even the pricing model or the business model it's changing in a way that you only pay for the amount of tokens you used. Yeah. If you did not use that particular SaaS this month then maybe you paid zero. Yeah. Or you pay like very very minimal amount to keep your subscription and keep your data hosted. So yeah there is a disruption in the SaaS industry but also in the in the business model.

Ryan Eppley: Mhm. And on like I guess like we talk a lot about the SaaS apocalypse in our own kind of way and we've you know covered that in a lot of episodes. What about in the physical world? Are you are you seeing any of your own customers? Are you dealing with any of the physical world people building airplanes or cars or things like that using AI effectively?

Ujwal Roy: So we're not really into robotics even though that's an area of interest to me. I do keep tabs on the industry news and things like that. So we are ScaleBuild AI is not working with any physical AI yet. However, physical AI is the next big disruption in the world and this is like the brand new industry revolution where the physical machines or physical systems that we're seeing that is going to get intelligence or is already starting to get in intelligence. So we all have seen like humanoid robots. I think ever since we were kid we could see like Japan they were working like Sony or Toshiba they were coming up with humanoid robots and one day they were supposed to serve us in our homes as our servants you know like do chores like laundry like cooking cleaning but we could never really achieve that level of sophistication right like none of the companies in the world could build a hand that could actually mimic a human hand right I mean it could move do things like that.

But if if we actually do the mathematical modeling of human hand on in different things that we're doing like sometimes we're driving, sometimes we're cleaning, sometimes we are maybe like wip wiping the counter, sometimes we're carrying the grocery bags, right? Or like holding a child, holding an egg versus holding a dumbbell, right? So if we do the mathematical modeling of our hand, it's actually a very very complicated mechanism and building such complicated mechanisms were not even possible before we had the thinking models that we have today.

So the best thing about these AI models are that they could actually be plugged in into the physical AI and all of a sudden like physical AI can actually do much more things. The hardware to support that we have always had it because we have always had like stepper motor servo. That is kind of the like programmable mo motor that can release enough amount of RPM or torque to do different kinds of things. So hardware platforms were already there. What was lacking was intelligence and now that we have this intelligence we can actually plug in intelligence in smart way and make everything around us like very smart like even the physical products like light bulb right so like light bulbs or these lamps or any of the lights that you that you see in this room it has electricity touching it but it's not connected to the internet so even the verticals that internet could not penetrate AI is going to so penetrate

Anna Kirk: quick follow-up question because I'm curious about that especially in the context of your like the origins of your career being in more of the data center industry like compute side of that equation. And I know we have limited time today, so I want to make sure we get to at least one news article, but how do you think about like what does an inference need now thinking through like, okay, we're going to need to have a lot of these like relatively you know, like low latency type use cases if we are going to employ different kind of like robots in our homes or things like that where the expectation about that real-time interaction is much more kind of hightouch in the physical world.

Ujwal Roy: Yeah. So there is actually there's a new breed of companies that are starting. These are like new kind of semiconductor companies that are very different from how AMD or Intel or these companies might have started. So we are actually building or we're seeing a lot of startups that are building like very fast inference chips. And these chips are actually built for AI. These are built for inference so that you train on like heavy data centers or like huge GPU cluster that you can rent but when when you are running inference it could actually run like multivariate inference at a very high speed. So that problem will be solved by these AI chips that are coming out and Google is coming out with its own Amazon and then there is like 100 plus AI chip startups that actually work.

Anna Kirk: There was one that just, you know, came out of Stealth today with about a billion in funding called Etched. They've been around for the last couple years, but they weren't very public about, you know, their release and what they're up to. They built an amazing team. I don't know if you saw the release of Etched or it's very exciting competing against Nvidia, though. Like I'm I'm curious like it's a very big market.

Ujwal Roy: Yeah. Yeah.

Anna Kirk: And to your point, like we're going to need more specialization probably on the

Ujwal Roy: and side they're actually solving the gap that Nvidia is leaving. Yeah. Yeah, because Nvidia is very very focused on like training. I mean both training and inferencing. So Nvidia is like they're actually trying to become the general purpose compute business. However, there are like mission critical computes or like specific purpose compute that these startups are.

Ryan Eppley: Yeah. And outside of the startup world, I think everyone, you know, everything we talked on so far was public sectors, enterprises, whether startups, things like that. Really curious about your take. It was one of the articles in the news for for the week is the Pentagon and other agencies within America and other countries to be honest are trying to figure out how to get talent to work for the government. There's so many needs the government has, right? Like if you've ever even tried to pay a parking ticket, it's still a terrible experience. And yet we have these frontier models being built right in front of us with, you know, people that walk around our cities, but none of them typically work for the government. And it's usually because there's a government kind of cap on your salaries. The article is questioning like, should we increase salaries for AI roles in the government? Should we try to play a patriotic card as they the article kind of touts like what is the right play for governments to start getting their hands on amazing top tier AI talent.

Ujwal Roy: So I think if we're looking at any institutional use and if we consider US government and institution then US government is the institution that is the largest consumer of AI models and if you just look at like NSA or like these different departments within the government, they are like a hyperscaler of their own. They own like large data centers like Google or Meta and they have to because they're actually processing the data of all the citizens in and I don't mean to say in a like Edward Snowden way that they are actually smiling at us or anything but they just need to process like they need to process like social security benefits Medicare Medicaid things like that they need to process like anytime you go to a government website and you are taking a service it has to be hosted process somewhere. So government and already we can see that government is the largest consumer of like anthropics government cloud and the kind of AI that we have access to on Anthropic or open AAI even the latest models government's AI the version of the same AI model that government is accessing is at least 15x faster you know wow because their hardware stack that's running the same AI model is like 15x larger

Anna Kirk: I mean when you think about that on the human capital side like what is required to like continue becoming competitive within this space more like broadly the Trump administration just laid off like almost 300,000 workers 20,000 of those were IT workers like how do you think about kind of the disparity between the people who have to maintain those underlying systems and the people developing the AI and is that out of is that out of whack?

Ujwal Roy: I mean so we need to be clear about what do we mean when we say people developing AI is government trying to develop a frontier model or is the government trying to be a consumer of the frontier model and it looks like US government has no no interest or they have never disclosed of having an interest in developing their own model looks like they're just fine using the existing open-sourced or close closed source model on the government cloud

So for them it's open source anyways right and when it comes to human capital like the kind of talent that you need to build these frontier models it's not the same kind of talent that you need to deploy these frontier models. So even if the government has not the top scientists in the world who are working on the system integration and deployment then that is fine and the traditional IT workforce is not the same workforce that would be doing this AI system integration. They will actually be needing the new kind of workforce and I think US government is best positioned in the world to take advantage. I mean if if we see maybe the second best positioned government in the world is the Chinese government but the US government is best positioned when it comes to like access to the talent because US has I mean whenever we say the top labs in the world we are always talking about these for-profit companies like Anthropic and OpenAI but we do ha we do have literal top labs in the world at Stanford, MIT, Harvard, right? So, so the government has access to those talents and if the US government really needs the top talent in AI, then they literally have to compete with the private sector because ultimately people might not feel that patriotic where US government is paying them $60,000 a year and OpenAI is paying them half a million dollars a year, right?

Ryan Eppley: Better pay up. Yeah. Well, what's next for you at at ScaleBuild? What's what's coming down the pipeline the rest of this year.

Ujwal Roy: So at ScaleBuild AI we're actually changing our direction a little bit. So we have done a lot of integrations learned a lot by doing the work and we're on the edge of where the AI changing every day. We're actually literally on the heartbeat of where the change is occurring and we are betting big on aentic AI. So we're moving into the products where we have three different swim lenss and we're building a product in each of these swim lenss and we're not trying to be a frontier AI model company but we're trying to become a consumer AI company where we use the frontier AI models in building the useful products that people can use and they can get much more in less one thing that I've always been passionate about is powering solopreneurs, you know, like the single person entrepreneur like how do I give them a team of AI agents that just listens to the boss, the soloreneur and then helps them run their business, not just run but scale the business and make the dream of this billion dollar solopreneur businesses come true. So that's what our focus is on scale for for rest of the year.

Ryan Eppley: Very exciting. Well, yeah. As we as we wrap up, we have a question of the week for us here. What's what's our New York question?

Anna Kirk: Yeah. So, Ujwal and I, so we were talking kind of before the recording started about the fact that you are relatively new to New York, probably spent like six months here over the past kind of year or so. So we figured for today's question, we would do something you're familiar with. I'll let you kind of give your own background and share that fun fact if you want to, but favorite music venue in New York.

Ujwal Roy: So for me it's Mr. Purple. I like the indoor, outdoor, the views of the entire skyline and also their their music resonates with me because like I told you I'm myself an EDM DJ. I don't have a Soundcloud or anything. I'm not that cool, but I do like to make music in private and listen to it myself. Sometimes if my friends that have heard my music and they like it, they're like, "Hey, can you come play at my house party?" Or I don't charge. I do it for free just for fun. So yeah, I like Mr. Purples like the music vibe, views, everything.

Anna Kirk: Amazing. Maybe that's like a kind of like subsidiary of ScaleBuild AI is like your own DJ practice like the solo solo builders space.

Ujwal Roy: I do want to build an AIDS though.

Ryan Eppley: Yeah. Yeah, it it'll happen, you know. I thought we were talking about just like any venues because in my mind I went to room 52 a handful of times. It's a very small like comedy club. It's fantastic. It's just like really good vibes. 40 people max maybe. Private bar. It's really great. It's like right here in Manhattan. So, that's where where I'm at.

Anna Kirk: Love it. Music venue. I think they've hosted some music, but it's much more comedy club.

Ryan Eppley: All right. Well, we'll still count it. We'll still count it. It's effectively comedy as much of an art. What do you got?

Anna Kirk: I got to go with Smalls Jazz Club. It's a classic. It's Have you been there before?

Ujwal Roy: Not yet.

Anna Kirk: Oh my god. You have to go. It's like So, it's in the West Village and you enter and it's like you're entering directly into someone's basement. So you climb down this like super narrow staircase, but then it opens up and it's just like the warmest kind of coziest vibe. You're really up and close with musicians and it just feels a lot more like intimate. It feels like it feels like

Ujwal Roy: I'm going to write that down.

Anna Kirk: A hug. Yeah. Yeah. It's one of my favorite places in New York.

Ryan Eppley: Well, just for our guests out there, camera's right there. Where can they keep up to date with ScaleBuild and yourself personally?

Ujwal Roy: Yeah, sure. So, I'm available on LinkedIn. I do write there even though Elon Musk said a couple days ago that I think he said something like it's gay to write on LinkedIn or something. So I do write there. I have quite a following there. What we're doing at ScaleBuild AI scale.ai. Check it out and you can see the latest and greatest things that we're doing over over there and connect me with LinkedIn. I'm not I'm I'm on X but I think I have like 43 followers or something like that. So very powerful.

Anna Kirk: Could be 50 after this. Great. And for any of our audience, we're AI and NYC. Feel free to kind of like, subscribe, follow, all that jazz. Write in if you have something that you want to talk about or you'd like to be on the show.

Ryan Eppley: Awesome. Cool. All right. Thanks so much. The next