Waymo Trucking to Autonomous Construction with Boris Sofman, CEO and Co-founder of Bedrock Robotics

Right now, in the construction space, where you have these really, really critical industries, gigantic volume, fairly low margin, but [they're] very rational and thoughtful about the pain points that allow them to be more efficient. And so the idea of how do you build an automated system and try to make it scalable across environments and platforms, that was a very, very direct learning directly from our time at Waymo where we were doing exactly this and trying to unify an operating system that could scale across all these use cases. You are fundamentally a manipulation machine. So it's a manipulation problem where an excavator's a giant hand that's six degrees of freedom and manipulating the environment in all these diverse ways. All the other machines are too, whether you're a bulldozer or a compactor, you're manipulating the environment in some way. So now you have a physical manipulation problem that is not just you being a participant in the road. The physical world manipulation makes this a more volatile problem. Simulation becomes more interesting and complex, but you also are in a world where you're in a closed structured environment, so you have a little bit more control on what's in scope and what's out of scope and how you slice up the product.

Welcome to the Main Branch podcast. This is your host, Cagla Kaymaz. Main Branch is a show where I talk to founders and early adopters about building, deploying, and scaling world-class AI startups. Hi, Boris [Sofman]. Thanks for joining me today.

Yeah, thanks Cagla. Real pleasure to be on.

You're the cofounder and CEO of Bedrock Robotics, an autonomous construction startup, and you raised already over $350 million in funding. Tell me in your own words, what are you building?

We're enabling autonomy for heavy machinery. So these are the sort of machines you see in construction like excavators and wheel loaders, bulldozers, dump trucks, but they're also in other sectors like quarries and mines and agriculture and lumber and garbage movement. And so we're enabling these machines to be fully autonomous and allowing the contractors that use and operate those machines to work faster, safer, and more predictably.

I have a lot of questions for you about Bedrock, but before we get there, I was actually really excited to have you on the pod because you've been one of the OGs in robotics. So you have your PhD from CMU back in, I think you graduated in 2010, and then you started a company back then — Anki — where you were doing consumer robotics and already shipped millions of robots in homes, from my understanding. And then you went to Waymo, ran autonomous trucking there. So you've really seen it all since 2005 or something like that.

Yeah, it's been a fun journey. (laugh)

Yeah, it's amazing. Can you walk me through the technology and the capabilities? What has changed? What has been exciting?

It's funny, I was laughing because back when I got into robotics, there was this really deep pocket of people pushing on autonomous driving and other related areas at Carnegie Mellon in particular, where I think it was a big multiple of the size of investment in these areas than anywhere else. And now it's this really hot area, everybody's excited, physical AI this and world model that. And back then [it] definitely wasn't the case. We all did it because we were a bunch of nerds that liked to see physical things go and try to do intelligent things. And you definitely got into it for the love of the technical challenge and some of these holy grails, like autonomous driving, that really have attracted a lot of us. But the world's changed a lot and the technology's changed a lot. I remember back then the entirety of robotics was more classic approaches of search and heuristics and really understanding and representing the world in some way.

We started to use machine learning and interesting ways to start to do online learning and real-time adaptation and perception systems that could be trained to understand the environment. But fundamentally, a big portion of the solutions were trying to capture intuition of costs and structure in the world. And that was actually the foundation that continued for a pretty long time where deep learning, I think, transformed perception in a robust way where now you could do supervised learning and start to really detect things you care about: people, vehicles, lanes, all of these aspects. But it was a very gradual evolution. I had a detour in consumer robotics with Anki where we were able to leverage these amazing step functions and capability from smartphones like iPhone and Android phones where suddenly things like cameras, microcontrollers, wireless chips, batteries, motors, they started to cost 10% of what they used to cost because of the high volumes that they were being produced at.

And so all of a sudden we realized you could try to reinvent the intelligence interactivity in the physical world with these types of sensors. And I think that kicked off a pretty interesting trend where the cost of sensors to compute was always a big driver of feasibility in robotics. And suddenly cameras started to fall, LiDAR came out at really high prices, which was fine for research applications and autonomous driving. Now you have really capable LiDAR for a few thousand or even a few hundred dollars in the case of some automotive LiDAR. And so there's this really fascinating progression where things that used to be very expensive or very R&D driven or have to be reinvented inside of a company like Waymo suddenly started to become much more attainable. And then I was really fortunate to be at Waymo during a time when there was probably the biggest transformation in robotics of the last 20–30 years where we really shifted from these structured approaches to just directly learning from large-scale data to pick up the most subtle patterns that you would never be able to solve these problems with in any other way.

And I'm sure we'll dive into that, but it's been probably the most exciting transformation in my entire career is seeing what's possible now with learning from massive-scale data and being really clever about how you pick up those nuances and create a scalable system.

So it sounds like initially it was more like access to hardware and the systems and R&D getting cheaper. And then [in the] last couple of years it has been more on the data and the training and the model side. I guess switching a little bit to Bedrock and your origin story, what happened around the time you were starting the company that made you say now is the great time to leave and start this?

Yeah. Waymo was just a giant transformation on, to some degree, hardware, but massively the algorithmic side. And there was constant progression in algorithm. I think deporting was probably not quite this significant, but it certainly was a giant step function where people were blown away at what you could do in structured learning problems. What happened at Waymo was there was ... I came in in summer of 2019 and I came in to build out and lead the autonomous trucking application, but then also very quickly was asked to take on the perception team, which was supporting both the cars and the trucks. So we helped launch the Jaguars into San Francisco and then start to scale it. And then a lot of our team that's actually now at Bedrock, we led the first freeway driverless launch, which was a really fun problem. It was a really amazing full-circle moment where it was the area that got me into robotics to begin with.

But what really blew me away is that, as a company, we took a pretty gigantic bet that shifted the whole technology stack to really embrace these large-scale learning-based approaches, particularly on the behavioral side where up until then it was very much in the same spirit of these sort of structured solutions. They're more sophisticated, but you're still trying to engineer your way to really complicated problems. And it wouldn't have worked across a lot of cities, particularly in really hard environments like San Francisco where you just have way too much nuances. And so you had this brittleness that would constantly get harder and harder to scale. And then we took a kind of a bet. The company bet on learning from giant scale, like hundreds of thousands or even millions of miles of human driving, to teach a system to navigate a very complicated environment like San Francisco where you have pedestrians, cyclists, intersections, double-parked cars, cut-ins, all this weird behavior. And not only did it start to pick up on and capture those nuances and it worked incredibly well after some time, but what we were very pleasantly surprised by and we were hoping for but you never knew until you really pushed it was how well that started to generalize across other environments and platforms, even. And so we would go to Los Angeles, Phoenix, and Austin, it would get easier and easier.

By today, you have 270 million driverless miles at 20 times safer than a human. And opening up a new city like in Asheville or Seattle is largely a qualification operations problem. Even though there's always long-tail engineering work, it really generalized, you start to get more and more power from these models. And we saw the same thing going from car to truck, from surface street to freeway where we needed 10% more data to get these models to driverless quality. And that was pretty eye-opening where you're operating across very different platforms. A truck is crippled vehicle dynamics and it's 80,000 pounds in an articulated trailer and we're training off of Jaguar data in San Francisco. That was really shocking. And then you start to go to driverless on freeways and new platforms and you start to realize that this approach has massive generalization power. You start to mirror the sort of intuition that you build with LLMs where you're learning the genuine foundation of what it means to drive and see the things that you see on the road.

And as you change environments or you change platforms, more and more of that heavy lifting gets done by that model and not the tuning and the extra work that you do. And that was the catalyst for us where we realized that this is a superpower in a physical world. We saw this happening in LLMs and in the digital side. At Waymo, we obviously saw it. And so we started to think about what other applications are there where you can leverage this to solve previously unreachable heights of capability in applications. We have very diverse platforms and diverse nuances of capabilities. And that took us to heavy machinery where you have exactly that.

In what ways? I mean, size of the trucks you were working with are probably closer to what you're doing now than Anki days, but in what ways —

We keep getting bigger. (laugh)

Yeah. In what ways was your Waymo trucking experience on the perception team super similar to what you're doing now versus what did you have to learn the last two-three years?

A fully loaded truck could be like 80,000 pounds. I think we're working with 110, 150,000 pound excavators. And so it's like we're definitely getting bigger and bigger. I don't know what's next though.

What's next? (laugh)

It's a giant transformer or something. Yeah. It has this really fun sense of some that's familiar, very familiar, and then some that's very different. So what's familiar? The essence of how do you think about architecting the system and thinking about safety and autonomy, thinking about hill climbing on a machine-learning model and getting to really high quality. How do you build the muscle to understand data and curate it and be able to splice it in? The idea of system architecture to cover your sensor needs and blind spots in order to be safe. How do you think about simulation, how do you think about evaluation and qualification, the infrastructure, even the partnerships where logistics ... Trucking companies have very similar feel to me that we are experiencing right now in a construction space where you have these really, really critical industries, gigantic volume, fairly low margin, but very rational and thoughtful about the pain points that allow them to be more efficient. There's a lot of big carryovers and positives there that actually can be huge tailwinds. And so the idea of how do you build an automated system and try to make it scalable across environments and platforms, that was a very, very direct learning directly from our time at Waymo where we're doing exactly this and trying to unify an operating system that could scale across all these use cases.

Now at the same time, you have a few things that are pretty different — some positive, some negative. You are fundamentally a manipulation machine. So it's a manipulation problem where an excavator's a giant hand that's six degrees of freedom and manipulating the environment in all these diverse ways. All the other machines are too, whether you're a bulldozer or a compactor, you're manipulating the environment in some way. So now you have a physical manipulation problem that is not just you being a participant in the road. At the same time, you bypass the hardest part of autonomous driving, which is the interactions with others around you, like the agent interactions where you're never fully in control of your own safety, you're only safe relative to your behavioral interactions with others. These are interesting trade-offs. The physical world manipulation makes this a more volatile problem. Simulation becomes more interesting and complex, but you also are in a world where you're in a closed structured environment, so you have a little bit more control on what's in scope and what's out of scope and how you slice up the product.

You can become very productive and valuable as a machine that can do subsets of work that are very high volume and not necessarily solve everything. And then the interface becomes trickier where there's a lot more nuance and diversity to how you define the goals. And so it was a bunch of gives and takes where I think we were very smart about being able to bypass some of the most capital-intensive and time-intensive elements of Waymo, but we're also appreciating that there's other really challenging problems. At Bedrock, at the same time, we're able to get to market massively, massively less expensively than what it took Waymo. We went to fully autonomous deployments in just over two years.

That's amazing.

And so I think we were able to break apart the problem in a way that allowed us to start to commercialize while we keep expanding the scope.

You went from basically getting started to having your first excavator operator-free, autonomously deployed in at least in two sites or maybe two states in multiple sites. That's quite impressive. How did that happen? Maybe talk about this demo-to-production gap like we always talk about in robotics.

Right.

How did you manage to do that? Obviously things are cheaper, you have all these learnings, but maybe a two-part question. One is what enabled you to do that? And then two, what was the hardest part of this last mile of getting to full autonomy?

Yeah, we're on with four customers now and four sites and we have I think many, many more customers we're starting to ramp with. It's interesting because I think coming out of Waymo, we appreciated the complexity of the zero to one, but also how that's only the beginning of the challenge and the complexity to go then to tens, to hundreds, to thousands and beyond completely changes the shape of the problem. And you're constantly reinventing yourself as you hit these new things that weren't a bottleneck at 100 machines suddenly become a bottleneck at 500 or 1,000.

What's an example of that, Boris? What's not a problem at 100 but it could at larger scale?

[I can] give you a few examples. So one is operations. You can be pretty heavy-handed on your operations and your support structures and ratios that allow you to bypass maybe some of the constraints you have. But the moment you start really scaling to large numbers, it becomes infeasible to continue to scale that, whether it's financial or just operational. At low zero to one or tens of machines, you can be very meticulous on the scope of what's in scope and what's out of scope. So if you're an autonomous car company, you can avoid nighttime, you can avoid certain weather conditions, you can do what you need to do. If you're operating 1,000 machines in a city and it starts to rain, you can't just stop in the road at 1,000 places and you can't bring down your whole service. And so you start to have to really be scalable.

The other really interesting thing is that from a safety standpoint, a lot of companies abuse the zero to one where you can go driverless through an exposure argument where you go and you just test like crazy, test the hell out of it. You basically 1,000 to one, 10,000 to one is sort of miles in volume and you can overfit to almost anything. And then you go driverless and you pat yourself on the back, but you didn't actually create a scalable safety framework because the real test is when your fully autonomous miles, fully driverless miles, dwarf your testing miles so that when you get surprised by something, there's a really high chance you get surprised in a fully driverless deployment. That's when it's a real test. And so that's why I've always thought that until you get to probably on the order of 50–100 driverless machines on the road in a very meaningful scope that isn't hyperconstrained, that's not really the test until you hit that point.

Even at Waymo scale, when we had the power outage, I don't know, six months ago in S.F., a bunch of Waymos were stuck because I guess there wasn't enough training data and they hadn't quite figured out how to handle power outages and the stop sign.

Well, yeah, the funny thing there is that it's another one of those operational learnings, actually. They were fully capable of handling the stoplights that were out. There's just a precaution that they have in the system where, when you have a stoplight that's completely out, you have — just for extra safety redundancy — they have a remote supervisor that just kind of confirms that you're good to go and there's nothing really weird. And so it was like a denial of service attack where suddenly many hundreds of cars simultaneously start pinging and it was just a purely operational decision that wasn't even ... From a safety standpoint, the cars know exactly how to handle these circumstances. And so that's just a reminder that when you have these rare correlated failure opportunities, it gets pretty scary and you need to have an ability to not be vulnerable to it or to default back to another behavior that's actually totally fine and still fits within your safety framework.

And so when we approached our zero to one, we tried to be as aggressive as we could on containing scope, but making sure that that scope wasn't some toy example but was actually a really meaningful scope that could allow us to start to scale. And so in our case, that was mass earthwork. So think of the sort of work you do when you're doing large-scale earthwork for a heavy industrial project like a factory or a data center or a warehouse or a water treatment facility. A lot of these projects, you're moving hundreds of thousands or millions or even 10 million cubic yards of earth. And this could be four months, 12 months. We're going to be on a project that's 24 months of work.

And so you have high density, high usability, but it's still really complex, but you can actually focus on getting to market and make that be a big beachhead. And then we tried really hard to be honest with ourselves that we weren't overfitting or hyperoptimizing for a commercial deployment [where] part of it was going to be a dead end, but that this was a genuinely really good milestone that allows us to then run the flywheel and start to scale quicker and quicker. The zero to one was hard, though, because you're bringing up autonomy, safety, operations, customer relationships, all your tooling, simulation, qualification, metrics. We started to appreciate all the things that got developed over a dozen years at Waymo that we had to bring up for the first time all at once.

Maybe you can talk about different parts of your system because you guys totally crushed it, at least for the zero to one. And it's good to hear you're thinking about more as a generalizable problem versus we're going to overfit to this one task and we're going to show really good results and then it's going to take us another two years to do the second task. From a hardware perspective, you're retrofitting existing equipment, right? What does it look like?

This is a huge unlock. So talking about the differences with Waymo, you physically had to go and change every platform — car or truck — to be able to be safe for driverless on public roads as you're redesigning your brakes, steering, actuation, electrical to be redundant and to integrate the right way. Most cars are not drive by wire and so it's a huge amount of work and you're doing surgery where it might look like a Jaguar, but it's nothing like a Jaguar on the inside — a traditional Jaguar, in any case. In our case, we're able to retrofit existing heavy machinery and put our sensors and compute and integrate it into the machine to where these are $600,000 machines that already exist. We're able to turn them into autonomously capable. And we could do this in a handful of hours. It's 100% reversible. And so that becomes a really powerful enabler.

We can go to market and not have to — both in cost and time and qualification complexity — we didn't have to reinvent the machine. And so then we could really focus more of our energy on the software side where ... You were asking about the architecture. We pushed even more aggressively on the learnings that we saw at Waymo and we have an end-to-end model that is the core of our autonomy model. We complement it with some additive safety mechanisms that are intentionally independent, but the autonomy model is end to end. We take as an input sensor data, machine signals of all kinds, POs that we compute for the machine, a goal. So we literally take in the goal representation of what you're trying to achieve in the world and any other context that's relevant, and then the output is effectively guiding the behavior of this high-degree-of-freedom machine.

And you can have multiple heads on it for other things like funny things like honking when your truck is full. That's how, on a construction site, an excavator tells a dump truck that it's full and ready to go. And so even something like honking is learned in our system.

So Boris, you were mentioning your autonomy model. What's your POV on world action models versus VLAs? Is it closer to one? Are you predicting the future states or are you predicting what the action is going to be? Maybe you can first talk about the differences between approaches and then what's your current approach?

Yeah, they're both kind of interesting because VLAs have a lot of appeal in the sense that you're intentionally deciding what the next kind of action might be as a function of all this context. World models are relevant for us as well, particularly on the simulation side because how the world evolves when you manipulate it is actually an incredibly hard problem that you can tackle through physics, you can tackle through learning. Eventually the right answer for almost everything is learning where how does earth manipulate and transform based on properties, moisture, the bucket size, the things you're doing, things in it, rocks. These are very well suited for learned approaches, especially when you get into large-scale data. For us, we've done a huge amount of approaches and we're continuing to optimize this in real time. We've used diffusion policies, we've used all sorts of things, but fundamentally what we're doing is we're predicting the next action for the machine. But the inputs are kind of a giant bore of sensor data, cameras, LiDAR, machine state, goal state, everything.

And this is very much in a Waymo fashion where it's almost like a kitchen sink of all the context that might be relevant. In a similar fashion to transform our architectures and LLMs where you're learning the relationships between this complicated set of features of words, you can do the exact same thing on relationships of sensor data, objects, world state, goals, everything. And you're learning this gigantic relationship that allows you to then output the logical behavior for the machine given the goals and the context and everything that's there. It's actually shockingly beautiful and nuanced where you come out with really delicate behaviors like digging, raking, organizing, final finishing. We've had machines do pivot turns by pushing on their bucket and turning. We've had all these incredibly subtle behaviors. And what's beautiful about that is that you're learning these fundamental relationships between the world and your motions and your behaviors and how you work your way towards a goal.

And we're seeing an acceleration, where as you add more scope, you're actually learning a whole lot more than what we've cleared. And so every time we open up a new context for what we want to do, the learning rate is actually far quicker and we need less and less data. And so we're learning from these complicated relationships to really predict what the right behaviors might be. And that's an architecture that actually generalizes quite well based on what we've seen in other contexts.

But then it always goes back to data, right? So you said you need less and less data over time, but I'm assuming you don't have the ... In the LLM world, you had the internet-scale data and you still maybe can use a little bit of YouTube data, but I doubt that there's anything super useful for construction you can reuse there. How are you approaching data both from a sim perspective but also collecting from the real world?

Data's always core to these, and I think there's no substitute, especially when you have these very complicated high-dimensional physical problems. You'll never be able to simulate the exact physics of the earth, the way a hydraulic system works in an excavator, the impact of giant rock and soil. So these are all things that ... [It's] how demolition works. You have to have a root in real data. We partner incredibly closely with our customers and our design partners where we both collect data with them and we go and actually deploy our autonomy with them.

And do you have the right to train using their data?

We do. It's part of an agreement that we have where it's frankly for everyone's mutual benefit where these systems are effectively we get to test with them and they're learning on exactly what use cases they care about. And then the biggest win that they have is that they get to be first in line to be able to take advantage of being able to leverage this in the field. And so at the end of the day, we are learning more and more about what excavators do in the world, whether it's earthwork or demolition or material handling or moving a concrete barrier in the middle of the night on a highway or way weirder things that we've seen excavators do because they're so versatile. And so you start to learn the foundations of what this machine and machines in general can do to the physical world. And so if you intuitively think about this, and let's say you got incredibly capable as an excavator doing certain types of work, you change your attachment to do demolition, the motions are similar, the understanding of the environment's similar, these things carry over.

You want to go to a bulldozer or a wheel loader where you're pushing earth or moving, manipulating the environment in a different way. Based on what we saw with cars to trucks, I would expect a really large subsidy where you get a lot of shortcut in that foundation, especially since you're operating in very similar dynamics. And so we end up using this data very heavily now. We're at the same time very smart about how we curate it, how we slice it, search it. We use simulation, and so we invested a lot into being able to do closed-loop simulation, which basically means that you can simulate a behavior that actually manipulates and modifies the world that loops back in your simulator, which is a lot harder than just replaying something. We use simulation pretty heavily and we can create synthetic examples.

And then the last really useful tool that ... There's a number of others, but one other really useful tool is structure testing where you can go and you can create and upsample a really large amount of exposure to things you may not see in a normal distribution but become really useful. And so in the case of ... At Waymo, this will be things like a toddler jumping out from behind a car. You're not going to drive 200 million miles waiting for that to happen a couple times. You're going to go and create this at large scale through synthetic data and structure tests on a closed site and then make sure that you're very comfortable with this even though you're exacerbating probably a million x this sort of exposure that you would otherwise normally have.

And it doesn't sound like you're working with any external data vendors. It's all in-house?

We are not. And the thing is nobody actually has the data we need. This isn't the sort of problem you can learn from a webcam footage or there's no internet-scale data of this. We actually need and benefit from the full details of what the machines do, the physics of the interaction of what does the push of a joystick do in the interactions with the earth? How do you understand the physics of the hydraulics and the pressures that go through it? We collect it ourselves. Now, we're very open in cases where it might be relevant, but it's very hard to match exactly the data that's necessary for a physical-world process. In the digital world, it's I think a lot simpler, especially when language ends up being the common core route for a lot of it.

I asked you last time I saw you about what's your thinking about as these more generalist models from Pi and the Generalist, as they get better, would you consider using them? And I guess your answer was they don't have the type of data and it wouldn't work for you. Maybe you can talk a little bit about why [or] why not?

I think it's very impressive the work that they do. I think both those teams are actually quite incredible. We've gotten a lot of inspiration from them because fundamentally we're a manipulation problem, just a very different kind. We have grown to appreciate how much you have to deeply understand every element of the vertical stack of your system from the exact sensors you choose to the way you structure the model, the data, the valuation, everything, all the way through the controls, the POs, the safety systems, and how do you capture all of it. The challenge with any of these more horizontal generalizable model approaches is that there's zero data of the scale that existed on the internet where, if you're OpenAI, you're Anthropic, you have an almost infinite dataset of exactly the feature space that you care about — language — available for you. That just isn't the case here.

And not only is there no dataset, but then every application's dataset matters a ton. And to have a robust safety case and to optimize the uniquenesses of your product and your unique hardware and your unique platform and sensors, you kind of have to go full vertical. And so it would be almost like somebody saying, "Hey, I trained my robot hand on manipulating mugs and dishes. Let's go put it in a driverless car." It's just a completely different context that has no similarity to it. And so I think that there's genuine generalizability to classes of work, for example, local manipulation with a hand. I get that. And loosely you could argue maybe gives you a tiny shortcut, but the specifics of operating 110,000-pound excavator with all the complexity of the system and all the nuances and the workflows that are so unique to each customer, these are the things that will actually have a pretty big moat.

And that's why Waymo's now penetrated so deeply. And even when companies were focused on nothing but driving, it was so hard to do. And now you could argue Waymo's starting to get towards a foundation model of a generalizable model for driving after a pretty giant amount of progress they've made. That's just my opinion. And I think there'll be classes of problems that are simpler that maybe you could use it that way, but when you're really rethinking the full stack and you have a safety-critical system, I don't think it works that way.

A lot of these construction OEMs like John Deere and Caterpillar, I'm assuming they would be willing to pay a lot of money to have access to a model like yours, but they're also competitors and maybe they're partners on the hardware side. Would you consider making your models available to other construction autonomy companies? And then how do you view the competition from OEMs?

We think of Caterpillar, Deere, and others as more partners than anything, where they're a key player in the ecosystem, their machines are marvels of engineering that are astounding how reliable and capable they are, and they're already incredibly well-suited for us to be able to engage with and partner with. In a successful world where this skyrockets the utilization of these machines and productivity accelerates and more jobs get done and the labor constraints start to dissipate, more machines will get sold and used and recycled and worked through as you progress. Now, we're not in the business of selling our model to let others go and do autonomy.

However, there's a very, very natural complement where you will get the sort of partnerships that Waymo has and had with Chrysler, with Jaguar, with Daimler Freightliners, with Daimler Trucks, with Zeekrs, with Hyundais. You're bringing together the superpowers of both companies. And there's a world where we work together with OEMs to make manufacturing time integrations of our systems and lean into each other's strengths and manufacturing and supply chain and maintenance on the OEM side, and then ours on the autonomy side and the insights of how you actually coordinate these machines and you get a win-win. And so I think there's room for both. Even though some of these companies have tried to do autonomy, I think these are just unique challenges and we'll push as fast as we can. And I think there's also a reason why Waymo was in a better position to solve autonomy than a BMW might've been, even though they had the vehicle platform and a lot of the shortcuts on that front.

Yeah. It's interesting that Waymo and Zoox, like Amazon and Google tend to, right now at least they seem like number one and two that we are seeing. It's not necessarily the Audis and BMWs of the world.

Yeah. So Waymo's scaling in a way that nobody's even remotely touching. And I think until somebody does it, all the talk about Autonomy 2.0, it's all garbage. At the end of the day it's just like, well, Waymo didn't hold still. Waymo's reinvented itself six times and pushed on all these things, and so people have to break through until they can go and start throwing stones. But Tesla, I would put as the next most capable competitor in the sense that ... My view is that they're further off from level four — full autonomy — than they believe they are. FSD is an incredible level-two system, absolutely astounding, but there's a deceptive gap between the best level-two system and what it takes to be level four where there you're optimizing for the things that you and I would never see in our lifetime, but when you put 10,000 or 100,000 cars on the road, you'll see them at pretty high frequency.

And then they also have the capital, the talent, the compute, and the existential need to go and actually crack this. And so I do think that if I were to guess, okay, who's at the table in 2030 and is really formidable players in level four, I would put Tesla as the next likely after Waymo.

So we were actually talking about this on our team this morning. When do you think we'll have more than 50% of the cars on the road in the United States being autonomous?

Good question. There's one annoying headache is that a car on the road is going to live there for 12 years. And so if you were 100% ready and it was zero extra cost or whatever, it would still take you a decade to phase in.

I guess put retrofit down if there's a way to make the cars that are existing autonomous.

So the thing is, this is not a space where you can do that because ... And the main reason is that when you're at high speeds, every single thing that can fail will fail at some frequency. And so a big portion of what we were doing was getting redundancy in brake, steering, and actuation, where if your brakes fail, a human can muscle an emergency brake, do something, bring the car to a stop. An autonomy system won't be able to. Same thing with steering. So you need a redundancy on everything so that you're able to pick up and actually fail safely. And so I think from a safety standpoint, you won't be able to do it. But I think that personal car with full level-four autonomy will start to become something that's sold probably in the beginning of the '30s, let's say in four years or something like that, four or five years. Waymo's already really generalizable, the cost is coming down, and that'll continue to happen. They'll remove the geofences. And then at that point it becomes how do you make this a really intuitive experience for personal car ownership, which is still ... When you look at the stats, ride hailing is like 1.5% of all miles. Trucking logistics is maybe 13, 14%, and then most of the rest is personal car ownership. And so I would guess it starts happening in really meaningful volume on ride hailing over the next few years where you're going to see ... Waymo's already close to 50% in San Francisco of ride-hailing traffic.

Oh really?

Yeah. It's kind of wild. And so I think it starts to spread to 20 cities this year and many more beyond. I think they accelerate really aggressively and start getting into the billions of miles. Ride hailing probably increases in market share a bit just because it becomes such a wonderful experience and more predictable. And then personal car ownership starts ramping within five years. I would say ... I'm a little bit pessimistic because these things just take time to adapt. I think the early '40s you start to have 30% penetration, something like that, where it's like you're getting close to a third to half the cars on the road being fully autonomous, and part of that is the adoption cycle. A little bit of that is the continued technical evolution. And then a big part of it is just the fact that physical cars will still exist until they're just physically not being sold anymore. We saw the same thing with electric car adoption and so forth. It just takes time for some of these things.

I mean, EVs have taken much longer than I thought. And then the manual versus automatic cars, now it's more like nostalgia, at least in the US, to drive a manual car, but I guess we'll eventually get there. I can't wait for a car to just drive me to Tahoe and back.

Oh, it's just incredible. When you sit in a Waymo, it's just clearly the way it should be. We feel the same. We go and we talk to our partners and we're building a data center outside of Austin and you go and stand there with a partner and it's just you see there's 50 to 70 machines on this site. Some of them have hundreds. A lot of these sites are in the middle of nowhere and you have these tents that have to get set up to house people to do the work. And so you look at it and you see the first fully autonomous machines moving earth with nobody in it and just beautifully boring hour after hour after hour. It has the same sort of feeling as when you sit in a Waymo for the first time and, two minutes in, you're checking your phone and it just feels normal.

It's just a step function in the way it should be. And then when you can digitize that whole experience and plan for it in a way that you only could with a digital system like this, then suddenly you can transform transportation, construction, mining, all these areas. I think it's kind of exciting. The next two decades, I think this is the big theme is that most of the world's GDP is still physical. It's building, it's transporting things, manufacturing things, growing, harvesting. These are going to be the big transformative areas I think in the next few decades.

I guess going back to your stack, I was curious, how do you think about inference? So it might've been different at Waymo, maybe you had more connectivity. I'm assuming right now for your deployments, these are probably middle-of-nowhere type of sites where you don't necessarily have full connectivity. What runs on edge and in the cloud in terms of training and then once you have deployments, how are you splitting up the workloads?

Yeah, this philosophy is actually fairly similar to Waymo. So we do have connectivity both through cellular and satellite, and so that helps us. It's actually very important for interface. So it's like you want to be able to see and monitor and give intents. Waymo likewise. The philosophy that is safest and most practical, which we share with Waymo, is that anything that is safety critical has to be on the edge where you don't have to have dependencies on latency, connectivity, and things that are totally out of your control. And so if you are running cycle counts, some main process of five times a second, you can use remote assistance interface and so forth, but it should not be something that you're 100% relying on for safety criticality. It might be, in the case of Waymo, a confirmation on interpreting a weird sign or, in our case, dealing with something that was found in the ground and we're checking some monitor that triggered.

We have a computer on the edge. We use Nvidia chips. They have really great chips that are targeted towards level two, level three automotive. And so that becomes a brain behind the experience. Now, it doesn't mean you don't do things in the cloud. We do inference for the actual autonomy per machine locally. In the cloud, you have the monitoring, the overview, and the analytics and everything and the interface of the machine. But when you start to get many machines that work together, whether it's a bunch of excavators coupled with dump trucks and bulldozers and wheel loaders and you have a team of 10 machines that's operating on a site, the orchestration layer is in the cloud because that's where you keep a shared state of everything that's happening and you're effectively doing a large-scale reasoning model on how do you break up the work and actually go and achieve it.

That behavior is not time critical or safety critical. It's breaking up intents as a supervisor basically to all of these digital operators on the site. You end up having this nice parallel, similar to Waymo, where the actual behavioral autonomy and safety is all local, but then you coordinate the fleet remotely. Now, off board, we go and we do all of our training and simulation and everything in the cloud, in AWS or wherever. And there you're taking advantage of the fact that you can scale to huge numbers, you have no time sensitivity and so forth. But it's actually quite an advantage to be able to just run locally on the edge where you have a lot more consistency and predictability of your system.

So you said you're using Nvidia chips. There's some companies that are innovating on inference chips on the edge. One, what do you think of that? You don't have as much of a space constraint potentially as some of the other robotics companies because you're working with larger heavy machinery versus a humanoid, but also largely — I'm asking this kind of selfishly as an investor — but where do you think are the opportunities today? If you were starting a company tomorrow and you had the hardware and software, the skill set wasn't an issue, you could do anything, where do you think are the opportunities that I should be looking into?

It's a good insight that inference and training are very, very different, and I think inference is starting to dominate use cases faster than most people predicted. I think that was a sign that, okay, this isn't dotcom where you have dark fiber that's just not being used. It's like, oh, it's all being used. And a lot of the hyperscalers have flipped over into over 50% inference at this point, which is a very telling statistic, and that's hyperconstrained. Training, there's not much reason to not use cloud capabilities of this type. Inference, in our case, for the reason we discussed, we do it on the edge. We use Nvidia chips because they're hardened and ruggedized for large-scale automotive use, industrial use, defense use. That's very helpful.

And there's a large base of development on them. The tools are pretty fantastic. And so we've used Orin chips and Thor chips, which are their newer chip that is targeted towards level two, level three automotive, maybe level four, which is helpful. That's useful because we know that it's high capability, reliability, and great tooling. Now, in the long term, there's a lot of value to being able to do inference on various kind of scales very efficiently. So there's everything from are you doing inference on an LLM that's huge amounts of billions of parameters? Or is this a real-time model that has to run very quickly? And then you might be having hundreds of millions, half a billion, 1 billion, a couple billion-parameter model.

The ability to do that in a very effective way, knowing that that's going to be a use case, I think there is an opportunity there where you see this at large scale with Cerebrus and you might see it at a smaller scale. It is difficult because it's not just the chip, it's the confidence on the supply chain, it's the tools around it, it's the reliability. So there's other dimensions as well. But generally we have a buy-versus-build model on this on the hardware side where we're leaning into automotive-grade cameras, we're leaning into LiDAR companies. We try to buy off the shelf where we can. We do a unique integration on the compute side, but the chips, we try to basically de-risk our supply chain so that there's no uncertainty that this component will be around in two years or three years.

So on the hardware side, you said you're more like buy versus build. On the software side, when do you decide to buy versus build? And I'm assuming because you're in this more vertical ... We talked about this, data and the models are more specialized. Maybe some of it is there's just not enough capable vendors out there for your industry, but if they were available, when does it make sense for you to build on the software side versus partner?

In general, I think we're optimizing for some function of time and money. So it's like if it truly helps us, we'll go for it. So a lot of cloud infrastructure I think is very much in the buy category because it's just such standardized and long-term developed use cases. What we're finding is where we're replacing most companies that would've been tools that we use but we start building in-house is on things like dashboards and analytics and health monitoring and fleet monitoring where it's so easy now to build your own tools of this type using the AI capabilities these days. And you have an extra advantage that if you build it yourself, you have full API access where you can have a giant data lake and all of your systems are interconnected and you can query them with a model. That becomes harder if you're buying a bunch of closed systems that are holding their own data.

And so that's been one of the motivators where, in the time it takes us to learn and deploy a new piece of software, we oftentimes realize we can just go and build it ourselves. And so that's become really interesting for data understanding and analytics, interfaces, and all of these aspects. Now, at the same time, we're constantly experimenting with a lot of external tools like optimizing, labeling, data mining and curation, analytics. We've experimented with simulation tools externally where you want to be able to control it, but we don't want to have to build a physics engine if we can avoid it, things like that. But this is the part that's hard. I would say that's shaken up a lot in the last even six months, six to nine months. It's a little bit harder to predict on what is really safe as an externally purchasable tool, especially when it's not a standardized use case where you're comfortable locking into an API and not worrying about the fact that you're depending on another tool.

Sounds like you would rather work with more established, stable vendors because your use case is so safety critical.

For the most part, and it depends on where. So if it's a tool that facilitates our development speed, we'll go and experiment and try it. If it's something that's going to be part of our supply chain and hardware deployment, you want the credibility of a pretty large-scale company and user base that just de-risks and helps you. It's kind of interesting. And then I'm shocked that a vast, vast, vast majority of our software now is written by AI, which happened so quickly. Literally that wasn't the case a year ago. And all of a sudden we're literally getting Claude to debug hardware in weird race conditions and start to operate within our whole company to expand hope scope and run experiments that get deployed. Literally we have an agent that's requesting physical tasks of our test operations team. It's kind of crazy.

I used to be an engineer before investing, and I remember getting so frustrated with certain bugs where you can't replicate them and you're spending hours on end trying to figure out what's happening. And now it literally takes five seconds and there's already a —

Totally. We literally had Claude find a race condition in the kernel where some drive wasn't getting mounted and it was happening super rarely. And we set up an experiment where it had access to a bunch of machines — hardware to loop machines — on a bench and it could power cycle them and run them and then play with anything it needed to in a sandbox. Over the course of a weekend day, it found an issue where some kernel update had accelerated the timing and it created a race condition that if this finished first, it didn't mount correctly, and then it suggested a patch and it found some bug that nobody had reported. It's just crazy.

(laugh) It's funny, there was a chat room with the Claude instance and then a chat room with everybody else gossiping about what was happening, which is really funny. And so now we're trying to experiment with this much more aggressively, but I think it's an advantage to be a new company in this age. I think if you're a 10-year-old company, it's emotional and physically hard to go and take a team that's like 50 people but could have been four and then go through unemotionally how do you evolve that or shift people or do whatever versus we're a team that's one and probably needs to be four. And so you want to use AI as aggressively as possible so that you can replace a team of 50. And so there's zero emotions about it. It just becomes a superpower that you can run with.

So you guys have your first deployments now. What's the vision? Let's think about the next five to 10 years. What's Bedrock doing in 10 years?

Yeah, so I think in five to 10 years, let's say construction, if you are a contractor, you are no longer micromanaging your wheel loader and your bulldozer. You go and define a project, you bid for it, you use automated systems to bid for it. It automatically says, "Hey, okay, you want to do this factory? It's like 6.2 million cubic yards of earth. This has to get moved there, you got to pour concrete here, do all these things. All right, it's going to take you 11 months, two days, and six hours using the machines you specified." And the firm could be like, "No, no, no, we got to finish this in nine months." And it'll say, "Okay, you got to add three excavators, nine dump trucks, two bulldozers, and a compactor, and then you'll be done in nine months with a 97% confidence." And then you go and you launch the thing and you deploy the machines.

The machines are all all-autonomous, being orchestrated in the cloud. You have real-time visibility, one second latency on exactly what the earth looks like, what the productivity is, how fast it's going, the variances of your estimates revolving in real time. You are running 24 hours a day where you fuel them up at night and the machines just go off and run. It unblocks the contractor's fleets where they can stack more projects together. They do more work, they're 5%, that margin becomes 10% or 15% because now you're utilizing the same CapEx much more heavily. It starts to do operations on their broader fleet where a company operating in the Dallas-Fort Worth area, you have 15 projects. The system could tell you, "All rigt, transport on Thursday afternoon, these two machines from site seven to site two, that's the most optimal thing that you can do." And you start to effectively digitize this whole process where you are enabling a totally unprecedented level of productivity and totally unconstraining a world that today is supply constrained and jobs are getting turned down and costs are skyrocketing.

The impact is that everybody becomes more productive. It's expansionary like the Industrial Revolution, and it can actually be deflationary in cost where the houses get cheaper, the roads get cheaper, and you don't have to price compete with the data centers to build a high school. And then the same concept starts to expand into quarries, asphalt plants, mines, agriculture, and other areas. But you create this digitized process where the sum is way bigger than the parts, and it's not just a bunch of operators digitally working, it's actually the orchestrated system that becomes a manufacturing process that you can fully track and optimize digitally. And I think it's pretty incredible. Safety issues plummet where it's right now the deadliest job in the country — 20% of all injuries and deaths are in construction — productivity skyrockets, developers start doing more projects because they're taking out 9% interest loans, and if you can compress a schedule, your IRR skyrockets, everything transforms. It's going to be pretty amazing.

Yeah. One of my favorite words is "efficiency." I'm hearing a lot of efficiency from what you described.

Yeah, absolutely. And this is the most beautiful part is that there's so much inadvertent waste in these hypermanually optimized industries. We have a partner, for example, that does a lot of heavy civil work, and they gave us a really fascinating stat. They track very closely match factors between dump trucks and excavators, and how long does it take to do a cycle of a dump truck getting loaded, dumping, and coming back again. And if they were able to reduce that cycle count by one second, it would be $600,000 more profit per year, just one second on average. These are really, really long and oftentimes inefficient cycle counts. And so you just have so much latent efficiency opportunity in there, [it] blows my mind. I think it's a beautiful industry, and what I love is that we're not trying to replace contractors or OEMs or anybody else. We're actually just making the whole system more productive.

I like to ask my guests something about a more distant future. Obviously, no one knows. This is just us stretching our imagination, but what do you think the world looks like from a physical AI perspective in 2050?

A lot of the physical work is automated: driving, farming, kind of everything. I do think on that window, humanoids become really prevalent. I think we're kind of at a potential false start right now where the technology's incredible, but focused applications make sense. The general solutions I think might be off by a while, and I think humanoids will be there. I think that the amount of support to humans that's driven by machines is going to skyrocket, where today that's digital. I think we'll see the parallel on the physical side where the same way you have so much software that makes your life a little bit easier, I think there'll be the equivalent for cooking and for cleaning and driving and everything else.

And laundry.

And laundry. Yeah, the ever-present laundry robots. (laugh) The very expensive laundromats of today that are humanoid robots will get a lot cheaper. I'm an optimist. I actually think that it's way easier to see the disruptive potential of AI than to see all the positives. We saw the same thing in the Industrial Revolution where, by the time the dust settled, yeah, jobs got displaced, but actually what happened is the productivity skyrocketed, the number of jobs actually increased, and the average salary doubled because you're overseeing much more productivity. Now it's happening quickly and quicker than other revolutions, but I think on the tail end of it, it's going to be pretty exciting because the bottom 1% of education is going to be better than the top 5% of education today. A lot of classes of diseases will probably start to get resolved. I'm actually quite excited about the personalized aspect of AI, both in physical world medicine and everything related.

I agree with that. Boris, thank you so much. This was pretty awesome.

Thanks, Cagla. This was a lot of fun. Pleasure to have the conversation. Really enjoyed it.