GitLab, AI Eating Software, and Syndrome from The Incredibles

Welcome to Tidal Wave, a research newsletter about software, internet, and media businesses. And as always, if you’ve been forwarded this email (you know who you are), you can become a subscriber by clicking the link below.

---

Last week GitLab held a Fireside discussion where they launched and showcased a number of their AI and ML features. Much to the chagrin of its investors, GitLab has seemingly lagged both its direct (i.e., GitHub) and peripheral competitors in launching their own generative AI products. 

But, admittedly, this was a bit unfair for GitLab as the company had been investing in AI and ML functionality. It just has not been in the primary form factor that most investors and customers have become familiar with (i.e., GitHub CoPilot). For example, in 2021 GitLab acquired UnReview, which uses AI to suggest potential merge request reviewers. At that time, the company explicitly highlighted its ongoing development of AI-assisted features and emphasized its significant investments in MLOps tooling, which represented another area of investment for the company.

“Integrating UnReview’s technology into the GitLab platform marks our first step in building GitLab’s AI Assisted features for DevOps,” said Eric Johnson, CTO of GitLab. “By continuing to incorporate machine learning into GitLab’s DevOps Platform, we are improving the user experience by automating workflows and compressing cycle times across all stages of the DevSecOps lifecycle. We’re also building new MLOps features to empower data scientists.” (Source)

But UnReview and GitLab’s other AI-related investments fall into the category of “under the hood” projects i.e., they improve the product but people cannot see them. While these investments make GitLab’s product incrementally better they do not have the visceral effect that products like CoPilot do on both customers and investors. So the underlying reason for GitLab’s AI Strategy call was in reaction to GitHub CoPilot (and its many copycats). The company wanted to demonstrate to investors that they did indeed have the ability, wherewithal, and intention to invest and compete with CoPilot et al¹. 

GitLab is not alone in this regard. In the face of the AI tidal wave and democratization of LLMs, most software companies will have to kick off or accelerate an investment cycle with a focus on embedding AI functionality into their product(s). These investments and the associated marketing are targeted at both prospective customers and investors. 

GitLab’s AI & ML Bets and Product Announcements:

The company’s near-term focus is to use AI to drive automation within the DevSecOps workflows and the features/products GitLab launched were primarily focused on that. Here are a few of the features the company announced:

Code Suggestions: AI-assisted code recommendations and suggestions, which help developers write code more efficiently with fewer errors. The functionality is analogous to GitHub CoPilot and is based on Google’s foundation models. 

Suggested Reviewers: Once a developer finished writing the code, GitLab can recommend potential reviewers of the code or merge requests. This feature is an extension of the aforementioned UnReview acquisition. 

Merge Request and Review Summaries: When submitting code for review, developers can use LLM to generate a summary of the change. And on the flip side, reviewers can summarize their review when sending the code back to the developers. 

Explain Vulnerability: The LLMs can analyze new code and existing code for security vulnerabilities and explain the vulnerability to developers/SecOps. 

The company announced a few other features but the overarching theme is that the company is leveraging ML and AI to automate different aspects of DevOps workflow.

GitLab is investing quite aggressively in the opportunity, in part due to competition. Fortunately, GitLab naturally has an edge given its positioning as the core repo and the CI/CD platform for its existing customer base. The impact on the company’s P&L is unclear, primarily because the features are new and the company has not disclosed how they are going to monetize. 

Some of the features will just be included as part of the product and may be available to everyone. Some of it will be tiered and some of it will be charged separately, some on a user basis like per user per month, and some on a consumption basis, where you pay for the computer specifically for that feature in the form of tokens. (Source)

I would expect that the most likely outcome is that GitLab goes with a hybrid monetization model. The company will give the features that are not developer-centric (e.g., issue summaries) away for free and charge for the developer-specific features (e.g., code suggestions) on a per-seat basis. Companies that are charging for the CoPilot-esque features have priced their products at ~$10-20 per month. Some companies like Sourcegraph, which offers universal code search, are monetizing the features by bundling them into their core offering because it’s adjacent to their existing value proposition. 

Adapted from Pragmatic Engineer

Assuming a $20 per seat price and 10-20% penetration across the current user base in the first ~24 months, these features would theoretically add 2.5-5% incremental revenue and 350-650 bps to the company’s growth rate in FY ‘25. The impact would not be immaterial but is also unlikely to move the needle for investors, some of which are more concerned about headwinds on the company’s seat expansion².

More importantly, that revenue uplift is hard to square with the (stated) aggressiveness of GitLab’s investment in generative AI:

Yes, I think a lot of what we do, we generative AI is pretty recent. We really changed how we allocate our resources. And I think now as we look at our development, about 1/4 of our efforts are in generative AI, but I think it's a pretty recent development. So we still have to see a big impact from that.

Now “¼ of our efforts” does not necessarily equate to GitLab spending ¼ of its budget on generative AI. But for argument’s sake, let’s assume that ¼ of the executive team’s time and the company’s net new products are going to be focused on generative AI³. Effectively, GitLab is meaningfully pulling forward its R&D spend. The company did have an AI and ML vision, it just happened to be years in the future and management was taking an incremental approach.

GitLab’s AI and ML Bets

There are a few reasons why GitLab is focusing to such an extent on AI, ML, and generative AI:

Expansion Beyond DevOps. The company sees a long-term opportunity to expand beyond DevSecOps to serve the needs of the engineers and data scientists working on AI and ML projects, which would allow GitLab to serve more personas i.e., seat and TAM expansion.

It’s the current thing. Frankly, AI is the current thing. Customers are interested in learning more about how they can use AI to improve productivity at their company. And investors are looking to discern companies which companies are aligned with or against the current thing.

Competition: GitLab’s competitors in each of their core DevSecOps workflows have announced competitive AI products. For the sake of its marketing message and its IR message GitLab kind of has to have a response.

All the points are interrelated. As previously mentioned, GitLab has been investing in AI and ML for a few years⁴.

GitLab’s investments can be categorized into three buckets: 

• Applied AI: features and products that use AI and ML to improve the DevOps workflows e.g., AI code generation, suggested reviewers, etc. Generally, most generative AI products fall into this category. 

• MLOps: MLOps is the process by which developers, data scientists, and engineers work together to build, train and maintain ML models.

• DataOps: help companies process data workloads i.e., ETL, which is a prerequisite for building models. 

Source: It’s GitLab, the logo is right there. Just kidding, here’s the link.

The near-term focus is using applied AI to improve the existing DevOps⁵ experience for developers. Because for one, ModelOps and DataOps landscape was crowded before AI took over the zeitgeist and has become even more crowded since. In addition, GitLab’s DevSecOps roadmap is quite expansive as it is and arguably the company has undelivered on their announced features⁶.

With Applied AI features such as code generation and suggested reviewers, GitLab can serve their existing persona, capture more of their workflows, and make their product stickier. The idea is that over time, the company captures more of company’s DevOps budget and raises prices to realize an ROI on those investments. That’s kind of always been the GitLab thesis and applied AI can simplistically viewed as an accelerant to that thesis.

It’s not meaningfully different than what Microsoft is doing with their CoPilot products. Or what Salesforce is doing with their suite, or what Atlassian is doing with their suite, etcetera. 

The DataOps and MLOps bets were designed to help GitLab expand beyond DevSecOps. Both DataOps and ModelOps were (and are) designed to be long-pole investments to position the company to become an “AllOps” platform: 

We see it with data and operations teams creating DataOps; we see it with machine learning and ops teams creating MLOps. As more companies – and more teams within a company – rely on our platform, we are positioned to become the AllOps platform – a single application for all R&D. (Source)

Given that these bets are likely going to be immaterial for GitLab in the near term, the “why” for GitLab’s investments in those relatively crowded categories is more interesting:

We see ModelOps as a big opportunity for GitLab. ModelOps is a combination of data ops and MLOps. It's everything you need to do to add AI to your application. In GitLab, we're adding AI to our application to GitLab, but with GitLab, customers are creating applications and they now need to AI to that. We want to help them on that road. We already started building this functionality in 2021 in collaboration with the wider community. That allowed GitLab's CICD runner support for NVDI GPUs. More recently, we added the capability to link MLFlow experiment with GitLab experiments.

Later this year, we plan to introduce a model registry allowing our customers to store version, deploy and track the health of their AI and ML models, natively within GitLab. We want to help our customers be more productive every significant application is going to have both code and AI, and we want both managed with GitLab. (Source)

Gitlab (like others) believes that AI is going “eat” software. The idea is that every major software application will have some Applied AI features. Or said another way (more cynical way), Applied AI functionality is going to be table stakes. In this world, there are not going to be bright lines between MLOps and DevOps. Developers will need to work with data scientists, ML engineers, etc. to build their own models and/or fine-tune and implement 3rd-party models. GitLab’s vision to become the one platform for all software developers means they serve the needs of data scientists and data engineers. 

Vision: Identify opportunities in our portfolio to explore ways where GitLab can provide a better user experience for Data Science and Machine Learning across the entire Machine Learning life cycle (model creation, testing, deployment, monitoring, and iteration). (Source)

AI is Eating Software

You’re already seeing “AI-eating software” play out in GitLab’s core DevOps market. One of the main reasons that GitLab even needed to have an “AI Strategy” call is because the company’s direct and peripheral competitors have launched applied AI products. The first to do so was GitHub CoPilot, and recently Atlassian announced Atlassian Intelligence which provides LLM technology across Jira, Confluence, and Bitbucket. More recently, HuggingFace (a future MLOps competitor for GitLab) announced a partnership with ServiceNow to offer code assistance. 

Importantly, it’s the current thing with clear value. Both investors and customers are trying to understand who is well-positioned to offer those features. Investors are also interested in understanding who is not well positioned. In light of Microsoft’s aggressive move with CoPilot and their traction, there’s definitely been a perception that GitLab has fallen behind the competition.

GitLab (like Google) was caught off guard by Microsoft. Unfortunately, the threat to GitLab is more pronounced because GitHub is a far more formidable competitor to GitLab than Bing can ever be to Google.

So in reaction, GitLab has pulled forward its AI and ML investments. 

And when everyone's super…

The imperative to pull forward “AI investments” is not limited to GitLab. For example, most of Google’s recent keynote was focused on highlighting all the ways the company already has AI embedded in its products and the investments they are making. Some of this is just calling products “AI” that were previously not labeled as such.

And over the last few months, almost every B2B software company with more than a handful of engineers has announced features that will incorporate generative AI into their product. A few examples are below:

• Glean

• Outreach

• Zoom

• Five9

• Thoughtspot

For these companies, and other software companies, OpenAI and Microsoft’s moves have presented a Catch-22.

One of the reasons that GitLab could only make incremental AI and ML investments is that the cost to train LLMs was previously prohibitive for any one company to incur. Fortunately and unfortunately, Sam Altman and OpenAI have effectively played the role of Syndrome from The Incredibles. 

Source: Disney

OpenAI brought commercial-grade generative AI to the masses. And by doing so, they also forced other vendors (e.g., Google) to open up / commercialize their models, which made it feasible for all companies without deep pockets to incorporate generative AI. 

But to quote Syndrome:

I'll give them heroics. I'll give them the most spectacular heroics anyone's ever seen! And when I'm old and I've had my fun, I'll sell my inventions so that everyone can be superheroes. Everyone can be super. And when everyone's super no one will be. 

By “opening” up OpenAI, OpenAI gave access to the technology to everyone⁷. The challenge for incumbent software companies is going to be that if everyone has AI features, no one really has AI features. Not literally, of course. It just means the bar for AI features is rapidly increasing and if you are offering similar features as your competitors based on the same out-of-the-box model, there is going to be a cap on how much you are able to charge for those features⁸. 

The other implication has been that companies that have previously built their businesses around NLP and generative AI are under siege. Or said another way, are no longer super.

But the rapidly advancing technology threatens other companies that have spent years crafting software to automate tasks or to build machine-learning models, including richly valued startups such as Databricks⁹, Snyk, Zapier, and Talkdesk….

”If companies like Grammarly don’t figure out what is their unique competitive mode pretty soon, they’re going to be quickly replaced by every other text-based interface that integrates LLMs” (Source)

An area where this thesis has been effectively A/B tested is AI copywriting marketing. Some companies had early access to GPT and were able to quickly scale up. But once GPT-3.5/4 was released GA, their early technical advantages faded and competition increased. The scaled players – the ones that got early access – might still be the long-term winners but it’s also non-controversial to say that competition has meaningfully increased for them. 

The pattern extends to more mature categories as well. Generative AI is creating an arms race among incumbents and effectively increasing competition and the R&D burden.

Take the example of Box and Dropbox. Box recently announced AI functionality that can be used across their content storage products, called Box AI:

Box AI will make it easier than ever to uncover and share insights, find timely answers to critical questions, and effortlessly create content based on an organization’s data in Box. The company also announced that it will integrate OpenAI’s most advanced AI models with the Box Content Cloud, further enabling new ways to understand and create content on Box… 

Box AI will bring foundational AI models to where their content is already securely stored, making the files inside of an organization more useful and valuable than ever before. (Source)

And a few days prior to Box’s announcement, Dropbox announced they were laying off 16% of the company’s workforce because of an AI imperative:

The opportunity in front of us is greater than ever, but so is our need to act with urgency to seize it. Over the last few months, AI has captured the world’s collective imagination, expanding the potential market for our next generation of AI-powered products more rapidly than any of us could have anticipated. However, this momentum has also alerted our competitors to many of the same opportunities. (Source)

The company would articulate the AI imperative more explicitly on the next earnings call

The first objective is around building AI-powered product experiences centered on organizing cloud content. We believe for many years and the potential for AI to completely transform and knowledge work and we've been investing in automation and machine intelligence features to help our users organize their cloud content and search and discover more easily.

Did Dropbox lay off 16% of its workforce because Box was going to announce an AI feature set? No.

But both companies understand there’s a strategic imperative to incorporate AI into their platform soon rather than later especially given its relevance for enterprise search and the Microsoft/Google threat. After all, no one wants to be a “Buddy” in a post-GPT world. 

Applied AI features without question make the products better but the actual economic payoff is less certain. In some instances, companies have been able to charge for the tool (e.g., GitHub CoPilot) but in other instances, the functionality is free for customers (e.g., Atlassian Intelligence). 

In the previous investment cycle (cloud), investing in the cloud transition made economic sense. The trade-off in gross margins was worth it and over time, that became the standard. But with AI, it may just lead to incremental R&D spend for some categories unless they find a specific way to realize the ROI.

Atlassian is a good example because I think that’s the model that incumbent application software companies will follow. Atlassian is offering Atlassian Intelligence as a cloud-only product and using it as an incentive for the company to speed up its cloud migration. My model is that mature categories (e.g., CRM) are segments where AI will primarily serve as a “carrot” for something (typically bundling or capturing more usage/workflows). This will inevitably require accelerated investment across the P&L forcing an investment cycle for incumbents (see: Box vs. Dropbox), which was not previously planned nor expected by either investors or management¹⁰. 

On the other hand, the parts of the market/companies that I think are going to buck this trend are (1) less mature categories or (2) companies that take some tech risk (e.g., build their own domain-specific model). The second one is tempting because it has the allure of “being proprietary” and “having data moats” but it will be a more capital-intensive model (and more likely to fail). But leaning into this type of uncertainty is kind of required if you’re trying to pull away from the crowd or compete with incumbents. From Jerry Neuman

To be successful a founder must seek out uncertainty and then must manage the company through it and emerge from the other end with a moat. Uncertainty creates a difficult trade-off for entrepreneurs. Without uncertainty they will immediately face competition from many others, including some who are better resourced.

Theoretically, GitLab could have come out and said “we’re going to build our proprietary code-gen model that will be better than GitHub CoPilot”. But investors would’ve probably had a conniption. Because for one it’s clearly going to require a lot of money and two the payoff is entirely uncertain. And investors like the certainty of outcomes, unless the management has a great deal of credibility for taking those types of risks.

For those reasons, GitLab and most other companies have decided to invest in “certainty”¹¹ (i.e., take existing foundation models and fit them to their use case). By definition, this means the returns on increased development spend are going to move towards an average with little room for outlier returns¹². On the flipside, the threat for incumbents will likely come from companies that do take the innovation or technical risk such as betting on the fact that models are getting smaller and easier to train.

Fin. But as with every AI and ML post, I’d like to end with: well anyway, it’s early and we’ll see.

Next week I’ll be going deeper into Google’s I/O Conference and its developer Keynote announcements.

---

If you’re finding this newsletter interesting, share it with a friend, and consider subscribing if you haven’t already.

Always feel free to drop me a line at ardacapital01@gmail.com if there’s anything you’d like to share or have questions about. This is not investment advice, so do your own due diligence.

1. Investors have other concerns about GitLab but those fall outside of the purview of the AI discussion.

2. Topic for a future post.

3. For the purposes of our discussion, it’s important to keep in mind that companies are increasingly using “generative AI” as a catch-all for all things directly and tangentially related to AI and ML. And for simplicity, this author will commit that sin from time to time. 

4. In fact, many companies (e.g., Workday) have been making AI and ML investments, but those investments generally were “under the hood” and were designed to make the products and workflows incrementally better. But no one really launched a product that had the visceral effect that ChatGPT and generative AI have had. 

5. I’m really tired of writing “DevSecOps”

6. This is one of the reasons that “Depth over Breadth” is an FY ‘24 goal for the company.

7. In another world, Microsoft would have acquired OpenAI, and kept the technology for itself and GitHub. In this alternative universe, GitLab would have been knee-capped. They do not have the resources to train a model that can compete with OpenAI but because OpenAI gave access to everyone, other owners of models had to too. And GitLab can leverage Google’s foundation models to compete with GitHub. 

8. Especially in this budget environment

9. Databricks should not be on this list.

10. One of the reasons that I like Salesforce and Workday is that they have already made many of these investments historically and are poised to incorporate/take advantage of this wave.

11. To be clear, I don’t blame these companies. I along with most other investors would not giving many management teams the benefit of the doubt of executing on the “proprietary” model approach.

12. Someone will point to the AWS analog as a counterpoint. But I’ve written before about this and how most vanilla SaaS has seen narrow moats and declining returns over time.

So things have happened, the cost to scale has gone up, barriers to entry have gone down, and finally, these companies are increasingly priced to perfection. Investors have broadly fallen in love with the SaaS business model because these companies check a lot of the boxes that investors care about (recurring revenue, high-gross margins, sticky customer base, and high incremental margins.

Naturally, this dynamic has led to excess capital, more competition, poor management discipline, and increasing capital intensity in the industry.