THE IDEA
Don’t put all your experiments in one basket
In a complex system, you can’t predict what will work. Expert opinion is unreliable. Past experience is an imperfect guide. The system is too interconnected and adaptive for any analysis to determine the right intervention in advance. This is frustrating if you expect certainty. It’s liberating if you’re willing to learn.
A portfolio of experiments is the response. Instead of choosing one intervention and betting everything on it, you run several small experiments simultaneously. Each tests a different approach, a different hypothesis, a different angle of attack. Some will work. Some won’t. Some will produce surprises. The portfolio doesn’t need every experiment to succeed. It needs enough to succeed - or enough to fail informatively - that the next round of experiments is smarter than the first.
The logic is borrowed from venture capital: most bets fail, a few succeed, and the successes more than pay for the failures. Applied to organisational change, social intervention, or personal development, the principle is the same. Don’t look for the one right answer. Run enough small experiments that you discover answers you couldn’t have predicted, from directions you didn’t expect.
IN PRACTICE
Many small bets, one big learning
A public health authority wants to increase vaccination rates in underserved communities. Instead of designing one comprehensive campaign, they fund five different approaches simultaneously: a mobile clinic in a shopping centre, a trusted community health worker programme, a social media campaign in the local language, a partnership with faith leaders, and a school-based family outreach. Three produce meaningful results. Two don’t. The authority scales the three that worked - and they now know something about the community that no amount of prior analysis would have revealed.
A product team doesn’t know which feature will improve retention. They build and test four minimal versions simultaneously: a personalised onboarding flow, a social sharing feature, a weekly digest email, and a gamified progress tracker. The onboarding flow shows clear signal. The others don’t. In four weeks, the team has an answer that a planning process would have debated for months - and the answer was different from what anyone in the room would have predicted.
A person isn’t sure how to meet people in a new city. Instead of committing to one strategy (join a gym, take a class, use an app), they try all three simultaneously for a month. The class produces the strongest connections. The gym is fine but solitary. The app is frustrating. One month of parallel experimentation produced more useful information than a year of sequential trying would have.
WORKING WITH THIS
Building your portfolio
Design experiments that test genuinely different hypotheses, not variations of the same one. Five versions of the same email campaign is one experiment, not a portfolio. A portfolio tests different approaches to the same problem - different channels, different mechanisms, different target groups, different theories of change.
Keep each experiment small enough to be safe and fast enough to produce feedback within weeks, not months. The value of the portfolio is speed of learning. If the experiments take a year to show results, you’ve lost the advantage.
Define success and failure criteria in advance. For each experiment: what would signal that this is working? What would signal that it isn’t? How quickly will we know? Having these criteria prevents the drift where inconclusive experiments continue indefinitely, consuming resources without producing learning.
Amplify what works. Dampen what doesn’t. This is the discipline of the portfolio - not just running experiments, but acting on their results. Scale the successes. Stop the failures. Feed the learning from both into the next round of experiments. The portfolio approach isn’t a one-time activity. It’s an ongoing cycle of experimentation, learning, and adaptation.
THE INSIGHT
The line to remember
In a complex system, the smartest strategy isn’t finding the right answer first. It’s running enough experiments that the right answer finds you.
RECOGNITION
When this is in play
You need a portfolio of experiments when nobody agrees on the right approach and the usual response is to debate endlessly. When the cost of running small experiments is far less than the cost of being wrong with a big one. When past experience doesn’t reliably predict what will work in the current context. When someone says “let’s just pick one and go” about a situation that’s too uncertain for a single bet. When the system is complex enough that the right answer probably isn’t in the room yet - it needs to be discovered.