10 product growth lessons from 4M users

Building 4 growth teams, scaling to 4 million users, and running 100s of experiments.

TryHackMe (cyber edtech) has grown to over 4 million users, with 1.5 million joining in the last year alone. All with no paid acquisition or established marketing team. We’ve run >100s of experiments, and our growth team function has matured tenfold over the last few years, and I thought I’d share my top 10 growth lessons.

We have 9 product squads (making up 50% of the entire company), that each have their own core focus areas. 4 of these are growth squads that work on different parts of the funnel: activation (our first and longest standing squad), engagement, monetisation and acquisition.

Having run hundreds of experiments, built growth teams from the ground up, completed many Reforge courses, worked closely with product and growth advisors, churned through books and articles, listened to growth podcasts, and learnt through my own failings, I thought I’d share some lessons.

#1 Quality over quantity

It’s obvious, but overlooked. We care about acquiring, activating and retaining users that are the best fit for the product. Sign ups are a vanity metric. Who cares if you have 1 million users signing up if only 10k end up being active.

When analysing experiment results, we filter for “gold users”. We still look at all users, but want to make sure we’re analysing data, making decisions and spending time building a better product for users that genuinely will get value from the existing product. As Paul Graham said, you want 100 people love your product, not 1000 people sort of like it. You’ll find those 100 people will bring in 100x more.

Also, you don’t know how many sign ups are coming from bots. My opinion is you should delay battling bot accounts (especially in the early days). Focus on growing and learning, when its becomes a problem, or your product has a high cost associated, then work on it.

We don’t have any control over user registrations as TryHackMe’s growth is 100% organic / word of mouth, but if you’re spending money on paid ads, it’ll be even more important to bring in the best users for your CAC to ROI ratio.

#2 Not every software engineer is right for a growth squad

Growth engineering is focused on creating fast experiments to learn and find quick wins that drive product growth. Development is extremely quick, MVF-like, and is usually ripped out of the code - you find the easiest and lowest effort way to engineer with quality and unit tests being an after thought.

This is some engineers worst nightmare, and will slow the squad down should they be tasked with growth work. Finding the right type of engineer to go into growth has been transformative. The work is extremely high paced, hacky, but incredibly rewarding as you make such a direct impact on company growth - the work of our growth teams have contributed to millions of dollars of revenue to the company.

There is a great article here on “What is growth engineering”. I’ve undersold how exciting it can be to be a growth engineer, but the point of this lesson is, the wrong engineer in the squad can be detrimental to the success of the team. When hiring engineers, we try to understand if they’re better suited to growth or core engineering.

#3 You can run experiments without engineers

You don’t always need engineers to validate ideas - there are faster, cheaper ways to learn. Use tools like Typeform, Figma, HotJar, run smoke tests, even email to test demand, messaging, or workflows before writing a single line of code.

#4 Make data self-serve

Relying on the data team for all analysis will slow you down. Surface-level analysis should be done by the squad. PMs should set up their own dashboards, track metrics, and explore trends themselves; a skill we look for when hiring.

The data team is a huge unlock and can supercharge a growth team, but you don’t always need them. They should be focused on the highest-leverage questions - so they’re not spending hours on basic requests. All our PMs know how to use Amplitude, and we have a data dictionary that breaks down all events to make self-serve analysis easier.

#5 Share lessons (failed and successful)

Growth teams are very expensive, but the insights they get are incredibly valuable. If the rest of the company doesn’t hear about those learnings, it’s a big missed opportunity. Even failed experiments show how users behave. Sharing these learnings with teams like product and marketing helps everyone make better decisions.

We do quarterly growth debriefs that anyone in product or marketing can join, post updates in Slack about ongoing experiments and what we’re learning, and hold regular squad calls to keep everyone aligned.

You also want the rest of your teams to be really reactive to the lessons. When they hear about a lesson, they should be proactive in trying to apply it to their work.

#6 The whole company should contribute growth ideas

This isn’t anything new, Duolingo wrote about it a while ago, but having the entire company able to submit an experiment idea is a big unlock. You can source ideas from staff that have different experiences with your product; customer support, lesson creators, sales, customer success - all see different parts of the business.

We try to encourage growth squads (not just the PM, but entire squad: designers and engineers) to run workshops with the rest of the company to crowdsource ideas.

On another note, what I’m really proud of at TryHackMe is people in different departments will share user stories to different teams; for example, our Head of Sales shares insights that feed directly into activation and engagement.

#7 Time box experiments to force solutions

Parkinson’s Law: tasks take as long as the time you give them. If you give an engineer 10 days to develop an experiment, it will take that much time, but you’d be surprised how quickly you can move by asking “what does a 2 day solution look like”.

The problem when you grow as a company is things slow down, and experiments that took 1 day now take 1 week. You introduce loads of process (e.g. 2 code reviews, unit/integration tests, “only pushing on certain days” etc..). At TryHackMe any squad can push to production at any time, and we have measures to help mitigate potential issues, but we don’t overcorrect if something small goes wrong.

Experiment results compound and help with product growth. If you learn more, you’ll build a better product with better activation, engagement and conversion rates. Which is why it’s so important that “every second matters” - move as fast as you possibly can.

#8 There is no failing in growth

A failed experiment isn’t a failure, because you’ve learnt something. I actually hate when an experiment performs the same - I like negative (or obviously positive) changes, because you know it’s having some sort of impact.

#9 Balance “big bets” and “quick wins”

It’s easy to fall into the trap of chasing quick wins. They’re fast, measurable, and feel productive. But to unlock big step-change growth, you need to take bigger bets that feel riskier but that can move the needle meaningfully. We try to have at least 1 big bet in each growth squad per quarter.

#10 The whole squad needs to be bought in

You want your entire squad focused on the problem, not just the PM. Everyone should be contributing ideas, clear on the metric you’re trying to move, and genuinely excited about making an impact. Ideally even engineers sit in on user interviews (a page taken out of Amplitude's playbook), and read user feedback. We share user feedback and recordings to the squad to give everyone as much context as possible.

Closing comments on growth

I like to write for early stage startups. So one last thing, don’t hire growth teams until you’re ready and have some momentum (or “product market fit”) - it’s the entire companies job to grow, and if you hire a Head of Growth, or Growth PMs too early, you’ll end up wasting both time and money. You can’t run experiments without existing traction, traffic or users.

Founder led growth is crucial - and as the company grows its expected that you’re more “hands off” - but this should not be the case. Founders have so much context about the entire business, and that involvement should be welcomed by the teams. It’s why when I read “Founder Mode” based off a talk by Brian Chesky (CEO and Airbnb Founder), I was so happy to learn the way I wanted to operate wasn’t wrong and often misunderstood.

I’m sharing 2 more product growth lessons on my Slack - join to find out (its free).

Next post will be: Talking to users for fun and profit.