OpenAI vs. Open Source
Who wins in a head-to-head competition of the Top 10 competitive moats?
Last week, everyone was shook by the leaked internal memo from Google that declared “we aren’t positioned to win this arms race” and that both they and OpenAI were being rapidly overtaken by the silent killer that is open source.
Here’s a spicy excerpt from the memo:
“We’ve done a lot of looking over our shoulders at OpenAI. Who will cross the next milestone? What will the next move be? But the uncomfortable truth is, we aren’t positioned to win this arms race and neither is OpenAI. While we’ve been squabbling, a third faction has been quietly eating our lunch.
I’m talking, of course, about open source. Plainly put, they are lapping us. Things we consider “major open problems” are solved and in people’s hands today.'
While our models still hold a slight edge in terms of quality, the gap is closing astonishingly quickly. Open-source models are faster, more customizable, more private, and pound-for-pound more capable. They are doing things with $100 and 13B params that we struggle with at $10M and 540B. And they are doing so in weeks, not months.
—Leaked Google memo, “We Have No Moat”
I wanted to answer the question for myself, who do I think is going to win the race? OpenAI or Open Source? I ran an informal poll on Twitter and it seems most (but not by much) are betting on Open Source.
So I put together a list of the Top 10 competitive moats, wrote out my thoughts on who would win in each category and assigned points to try to determine a winner.
Certainly not scientific but interesting to see the results nonetheless.
Let’s get into it!
Welcome to Making Product Sense
Join thousands of product leaders and founders from the top companies in the world, who are learning how to hone their product sense.
1. Brand recognition
If you asked a random, non-technical person off the streets if they've messed around with any of the A.I. tools, in my experience, out of those that say yes most will have used ChatGPT. They've made big enough waves early on and have such a low barrier to entry, that it's a natural starting point for the A.I.-curious.
🤖 +1 OpenAI
2. Intellectual property
Intellectual property such as proprietary algorithms or datasets is all fine and well if you're competing against other closed-sourced competitors. But if you're competing against the masses of open source contributors, it's much slower to build in secret.
We’ve seen glimpses of this by how quickly Alpaca and Vicuna followed the release of LLaMA (2-weeks and 3-weeks respectively), reaching 92% of the quality of ChatGPT, up from 68%.
🌎 +1 OpenSource
3. Network effect
Right now, network effects are slim because each model exists in isolation. You can't really collaborate with anyone yet. However, OpenAI released plugins in Beta which has the potential to have some network effects.
OpenAI can negotiate with the biggest brands to build plugins for ChatGPT because they have the brand recognition and the capital to potentially help subsidize development costs (no idea if they are or not). Assuming open source models can't catch up anytime soon, this could be a point in their favor. Plugins by household brands like Zillow and Expedia means more users. More users means more plugins from top-tier brands.
🤖 +1 OpenAI
4. Economies of scale
Google straight up admitted that open source was eating their lunch when it came to the cost of development. If you're working on Reinforcement Learning from Human Feedback (RLHF) internally, it's going to take a ton of resources.
“They are doing things with $100 and 13B params that we struggle with at $10M and 540B. And they are doing so in weeks, not months.”
—Leaked Google memo, “We Have No Moat”
And if we’ve learned anything from Waze’s community-sourced approach, when you're rallying people from around the world to contribute for free... yeah, it's gonna be cheaper.
🌎 +1 Open Source
5. Cost advantage
This one is tricky. ChatGPT is already free and even ChatGPT Pro is reasonably priced at $20/mo. Open source tools on the other hand are abundant but often unpolished or require a bit of technical know-how to run so the friction is high. I would argue, much higher than shelling out $20/mo.
🤖 +1 OpenAI
6. Superior product or service
Currently, the point would go to OpenAI. However, I'm going to give the point to Open Source because we've seen how fast they're moving. As Sam Altman said, the era of large language models is already over. The new era of creative solutions to difficult problems will naturally fall to open source.
🌎 +1 Open Source
7. Switching costs
There's no loyalty when it comes to A.I. tools. We've seen the tides shift from one tool to another within a day. Tools that effectively do the same thing, but one just happens to get a bit more hype for some reason.
Especially with the introduction of tools like LangChain, different models can be integrated into the same product so there isn’t even a big switching cost for models. So the only switching costs I can think of are ChatGPTs plugin ecosystem. While slim, I think the point goes to OpenAI.
🤖 +1 OpenAI
8. Regulatory barriers
The A.I. landscape is changing too quickly for regulation to keep up. So the odds are that any regulation that comes down the pipe over the next year won't do an adequate job.
More than likely, it's going to be geared toward regulating the larger companies like OpenAI, Google, Meta, Microsoft and others since those are the leaders who are helping inform A.I. policy. That means that the open source community which can move infinitely faster and outside of the geographic and legal bounds of America where the big tech companies are based, will have the upper hand.
🌎 +1 Open Source
9. Distribution network
Open source has an opportunity to win distribution but the chances aren't great. The problem with open source is that it's often fueled by a group of highly passionate users which is nice, but it doesn't get eyeballs. It needs a champion. React is championed by Meta. Android is championed by Google. But open source A.I. models just don't have that going for them (yet). They won't win the wider audience, especially if Microsoft is championing OpenAI's models.
🤖 +1 OpenAI
10. Talent
OpenAI seems to have the leg up on talent. Their team is stacked. But do they have the talent of the masses? You know that jelly bean counting experiment where you could ask a bunch of people how many jelly beans are in a jar and average out the responses and get really close to the actual answer? Well open source is kind of like that. Average out all of the contributions across thousands of talented minds and you're likely to get a better answer than you would from a small group of 1%-ers.
🌎 +1 Open Source
👑 The Winner
So. If you’ve been counting, you know that's 5 points to OpenAI and 5 points to Open Source. Split down the middle. To be honest, I was hoping for a clear winner one way or the other but, as with most things in life, the future isn't clear.
Here's what this tells me…
We're going to see some insane progress over the next 12 months. Competing models, new tools, creative use cases, and fresh regulation (maybe). Much of that innovation will come from the open source community and a lot will continue to be driven by private companies like OpenAI.
At the end of the day, I imagine that open source will soon overtake private companies in leading the charge on A.I. development. But I don't foresee a bunch of non-technical folks adopting open source models to run locally or fine tune themselves. Instead, I have a hunch that OpenAI will shift toward developing ChatGPT into a powerhouse consumer-facing app that will capture the minds and wallets of the general public.
Open Source → will lead Technological Development
OpenAI → will lead Consumer Adoption
I could be wrong about all of this. I’d love to hear your thoughts in the comments.
Do you have a counter argument for any of the competitive moats?
Do you think the tables will turn and one will overtake the other?
That’s all for this one - I’ll catch ya next week.
—Jacob ✌️
Do you believe that as models or agents become more personable, users may find it increasingly difficult to switch to other options due to the emotional connections they form, resulting in higher switching costs specifically for personalized models?
Given that OpenAI has chosen to avoid incorporating personality into their products thus far and appears to maintain that approach, it is possible that switching costs could be reduced in favor of open-source alternatives.
That is some useful knowledge - producing edition