When an area becomes consensus but is still somewhat novel or hard to parse for investors, they often try to distill companies into very clear-cut and easily “findable” metrics in order to make decisions.
In many cases, this wave of chaos, rapid change, and complexity pushes investors to retreat into discussing a constant in startups; Founders. And as one searches for Founder-Company fit, they then anchor on the next clearest signal; Founder Pedigree.
In technologies that are Pre-Breakthrough or Cambrian Explosion moment this framework feels valid.
Investors are looking for those responsible for changing the world and doing something that has never been done before, and as those people seek to commercialize their breakthrough, the idea is that nobody understands it better than the original “inventors” and the inventors will be necessary to push it from a research breakthrough to production-ready as a technology.
As an industry shifts from pre-breakthrough to post-breakthrough, high-pedigree founders are able to raise staggering amounts of money, capitalizing on investor narrative distillation and hype cycles. We have been seeing this within AI over the past 24 months, as we live in a post-Transformers and post-GPT world, with most of the well-funded AI Labs and companies being founded at first by the original authors of Attention is All You Need, and now with a second wave coming from those early on at these larger labs (or those who were senior hires for short periods of time).
This secondary dynamic is not a rare one in the 1-2 years after a breakthrough as the incumbent monolithic feeling entities rise massively in value, creating golden handcuffs for some and parachutes for others. The early employees then often look at the nuanced disagreements they have with their incumbent approaches and candidly have the confidence to go out on their own, and leap into the founder pool (resulting in the second wave dynamic above).1Or these things happen at big companies and the leaders fracture in ambition or sequencing and both leave. This is arguably how one could describe the origin stories of Nuro and Aurora in the autonomous vehicle space.
However, as a given space’s breakthrough(s) becomes well-understood, investors tend to overrate the rate at which the people responsible for the breakthrough will be able to take a given technology towards a scalable product or down a cost-curve while navigating the idea maze of commercialization and GTM. Those responsible for a given breakthrough may not be best suited to figure out how to innovate & capture value as founders, though it is likely they will be incredible team members.
Thus as a founder and investor, the key thing to understand perhaps is the rate proliferation of “key” knowledge in a breakthrough area and how much that knowledge compounds internally versus at an industry level.
The canonical example in AI is the thought that a year ago people would say there are ~50-100 people in the world able to train models at the parameter size and scale of GPT-4. That number has greatly expanded on the infrastructure side and will continue to expand as people adapt Mixture-of-Experts approaches and less reinventing of the wheel happens on the back-end.
Within core innovation and architecture in AI , the rate of learning has historically spread like a wildfire due to open publishing and a sense of collaboration amongst the largest labs from 2014-2020. This rate of proliferation will certainly change within AI due to competition, a lack of publishing, and perhaps more consolidation of talent and less talent bouncing from lab to lab (I talked about this more in the appendix of On AI Transparency and Alignment)
Now that we’re approaching model quality that will be high enough value for a variety of applications, it’s likely that we are going to start to see similar proliferation of knowledge happen with local (and perhaps <200B parameter) models on the AI side as most of the collective research is still open and a large portion of the open-source community is collaborating online to push the limits of base models like LLaMa-2 or others.
It’s likely that we will continue to see research pedigree over-indexed on within AI, despite my core belief that an understanding of the bleeding edge + pace + creativity could be just as successful at pushing forward innovation in this next stage of AI progress.
In our 2022 Annual Letter we wrote:
We also are looking closely at novel techniques that could emerge. In creative AI we went from GANs to diffusion models for image generation. Could there be something better like OpenAI’s recently published Consistency models? Will Transformer models be the final model to rule them all?
While it is currently unclear to us what will change, we have a hard time believing that nothing will. We’ll have to keep looking closely.
These statements are of course of incredible importance to understand.
Will novel paradigms or breakthroughs come from the existing players, effectively betting that the best researchers are able to continue to innovate on novel concepts better than others while in the Founder seat?
While linear progress likely will continue to compound with the talent and capital (and thus compute) moats of the largest research labs, it’s possible that entirely new architectures , as alluded to above, or new building blocks that are their own breakthroughs, could create non-linear breakthroughs. Whether those breakthroughs outpace performance of Scaling Laws is an open question.
It’s also possible that this innovation push could come not from even scaling models but instead from scaling the research done for LLM agents or other adjacencies to model performance (as I wrote in depth about in The Dark Forest of R&D and Capital Deployment in AI). This concept was also well-covered from an alignment perspective in this post by Paul Christiano at Alignment Research Center (formerly of OpenAI) and we’re starting to see people try to setup open-source mixture-of-experts communities as well.
Again, understanding the bleeding edge.
Perhaps the biggest limiting factor here will not be the talent but instead will be the politics, technical debt, and social capital that sits within these large groups that have accumulated and spent tens of billions (hundreds?) collectively on a transformer-centric approach.
This idealogy of being pot-committed is a lot of how we had conviction to partner with Wayve who had a very non-consensus approach to solving and scaling the self-driving problem. Put simply, the larger players just weren’t going to toss out their traditional approach now that they were seeing linear progress and had billions of dollars of budgets allocated to them because of their confidence and bet on this approach.2Many have since pivoted/progressed towards Wayve’s approach It took a team of relative outsiders to take a shot on an end-to-end approach in 2017 versus the slew of ex-Tesla/Waymo/Cruise people who spun out and raised capital post-Cruise acquisition to build self-driving startups.
While AI does seem somewhat unique in this regard, my gut tells me as other other emerging technical categories start to show obvious breakthroughs, we will see this dynamic play out again across Bio, Robotics, and likely other deep tech categories (quantum, ARVR, etc.)
Bio, has historically been funded almost exclusively on pedigree in the past decade+, however now we are seeing interdisciplinary teams as equally if not more competent than traditional biotech OGs as bio shifts to a more engineering-centric and platform-driven paradigm.
We’ll likely see similar traditional pedigree-driven dynamics emerge within Robotics in the coming years. As the progress made in large models and breakthroughs like RT-1/2, Code as Policies, and more continue to ripple through the community, investors will likely index heavily towards those from these labs or from the large labs that have been dismantled or refocused (hello OpenAI Robotics team).
While many of these people will be paramount to creating the breakthrough, building a venture-scale robotics company will not only be about those who are able to implement advanced hardware + embodied intelligence in the real world, but also those who are able to push forward the creative thinking of the modality or use-case of a given robot in order to build the next great Robotics company.
My advice for those at the top tier labs would be to make sure you are partnering with incredible product people on the founding team.
Taking this back to where we started, our view continues to be that we are in The Most Important Century3The origin for this framing which means that we will likely have a large number of “breakthrough” moments in the coming years in deep tech that are near impossible to pattern match to.
In a world where asymmetry and exponentials are becoming more and more understood, it’s likely that there will both be far more volatility as well as opportunity for Founders regardless of classical pedigree and it likely means investors should be careful over-indexing or pattern matching into what the obvious view of excellence looks like.
All of this leads me to keep coming back to the venn diagram above which originally described our Computational Creativity thesis, our seed investment in Runway, and perhaps describes building at the age of post-breakthrough, pre-scale innovation.
A mixture of the best scientists in the world who have done things that have never been done before and perhaps a different cohort of artists tasked with pushing these breakthroughs at scale to create wonder.
Founding teams building at the intersection of Science and Art.
Recent Comments