Experimentation sounds intimidating, like something only scientists in lab coats do. And with all the buzz around it these days, it’s easy to get caught up in the hype. But really, it all comes down to being curious, iterating on your ideas and using data to guide you. Think of it as iterative exploration — set a goal, try something, learn from it, apply your learnings, and try again.

Why experiment?

Experimentation helps you avoid making costly mistakes. By now, we should know that following our heart (or the loudest voice in the room) to make our product decisions leads usually to make big mistakes. On the other hand, it’s tempting to just ask customers what they want, but research and even surveys, especially about hypothetical behaviours, can be misleading. People are notoriously bad at predicting their own actions.

So, instead of asking “Would you buy this?” or “How much would you pay for this?”, find ways to observe actual behaviour in real-life situations. That is the reason at the core of experimentation.

When to experiment?

The short answer? Always! Experimentation isn’t just a one-off activity or something you do on the side of your product process. It should be an ongoing process, woven into the fabric of your product development lifecycle, from the earliest stages to ongoing optimization.

Here’s how experimentation can play a role at every step:

• Find and kill opportunities: In the early stages, use experimentation to rapidly test different ideas and identify promising directions. This helps you quickly validate (or invalidate) assumptions and focus your efforts on the most impactful opportunities.
• Validate research: Don’t just rely on market research or customer interviews. Use experiments to test your hypotheses in real-world scenarios and see how users actually behave.
• Build to learn: Adopt an iterative approach where every step of the development process. This allows you to continuously adapt and improve your product based on user feedback.
• Define your go-to-market strategy: Experiment with different launch strategies, marketing channels, and messaging to find the most effective approach for reaching your target audience.
• De-risk adoption: Use feature flagging to launch each iteration to a % of your users, so you can learn before going to your whole audience.
• Adjust your strategy: Use experimentation to refine your target audience.
• Define your pricing strategy: Experiment with different pricing models and tiers.
• Prioritize features: Use data from experiments to prioritize feature development and ensure you’re building the things that matter most to your users.
• Optimize (yes, of course also for this! 😉)
• Messaging for your marketing communications: Test different messaging to find what resonates best.

How to experiment?

Experimentation is indeed a word in the mouths of everyone nowadays. It seems such a great idea, to apply a scientific mindset in the way we build products. Makes sense: feels like the right thing to do to get the right answers. But hey, before launching yourself into it, consider a few things about the right mindset.

• Kill your darlings: We all have ideas we love, but a true experimenter is willing to put those ideas to the test, even if it means proving them wrong. Design experiments with the goal of disproving your hypothesis, not just confirming it.
• Embrace failure: no one likes to fail, but by creating the right culture of learning in your team you’ll get people to share learnings, Make sure psychological safety is there for people to feel confident sharing these things.
• Why before the what: Don’t just experiment for the sake of experimenting. Every experiment should be tied to a clear strategic goal.
• Don’t fall in love: Experiments should be designed to be quick, iterative, and inexpensive. It is easy to get attached to an experiment, be aware and avoid this at all costs!

Even with the right mindset, experiments can be misleading, too. How? Well, if you experiment with the wrong cohort, narrow your experiments too much to technology or just create stories about the data you get. How to avoid this?

• Get out there: Don’t limit your experiments to your existing customer base. The more diverse your testing ground, the more you’ll learn.
• Be creative: Experimentation doesn’t always require tech. There are no tech ways to deliver your service to your customers and learn, before writing a line of code, get creative!
• Include the human factor: Numbers tell a story, but they don’t tell the whole story. Combine the quantitative data from your experiments with qualitative insights from user interviews, feedback sessions, and observations. This will give you a deeper understanding of customer behaviour and help you make more informed decisions.

Experimentation is a team sport.

You can’t truly get the benefits of experimentation if that is only a thing that the product or design people do in your team. To get the real value of experimentation, it needs to be a team sport, and everyone needs to get involved cross-functionally.

Pay attention to your team’s maturity To truly embrace experimentation, you need to equip your teams with the right tools, processes, and mindset. If your cross-functional team is not collaborating properly, it will be impossible to embrace Experiments properly. Pay attention to where your team is, and make sure you have the basics covered.

Before you start, make sure everyone is aligned in the Why and know what are the goals we want to achieve. Also, a psychologically safe environment is non-negotiable: people feel comfortable taking risks and admitting mistakes.

Build to learn. Embed a culture of continuous learning into your development process. Every feature, every iteration, should be treated as an opportunity to learn something new. This means adopting an iterative approach, collecting data, and using that data to inform future decisions. Is not about building a car, is about finding the best way to move from A to B, together.

Embrace continuous discovery. Don’t lock yourself into a rigid roadmap. Instead, embrace a continuous discovery process where you’re constantly gathering feedback, testing assumptions, and adapting your plans based on what you learn. This allows for flexibility and ensures that you’re always building the right thing.

Beyond product teams

Experimentation isn’t just for product teams. Involve stakeholders from across the organization from marketing to sales to customer support. This will help you get a holistic view of the customer and the business, ensuring that everyone is aligned on the learning process.

and spread the experimentation mindset.

Ethical considerations

Last, but now least: do it ethically. Always remember that there are humans behind every experiment. Be transparent with them and respectful of their data.

Good luck! (and enjoy the journey)

‍