The Limits of the Lean Startup Method

Advocates of the lean startup method for creating a business advise entrepreneurs, as well as corporate intrapreneurs, to document, test, and refine their assumptions about a new venture’s business model via customer conversations and experiments. My recent research on 250 teams that participated in an American cleantech accelerator program during the last 10 years found that while the lean approach can be effective, having a strong strategy is more important than conducting a tremendous number of market tests.

First, the good news: In general, the lean startup method works. We measured success by looking at how teams performed in a pitch competition in front of a panel of industry experts at the end of the accelerator program (a proxy, albeit an imperfect one, for long-term financial performance). Teams that elucidated and then tested hypotheses about their venture performed almost three times better in the pitch competition than teams that did not test any hypotheses.

Now, the bad news: There was no linear relationship between the number of validated hypotheses and a team’s subsequent success. In short, more validation is not better. I also found that teams that conducted both open-ended conversations and more formalized experiments with customers actually performed worse in the competition than teams that conducted either one or the other during the early stages of venture design.

One possible explanation for the diminishing and even negative return on customer interaction is an erosion of confidence: too much feedback from customers might cause the entrepreneurs to change the idea so frequently that they become disheartened. Another possibility is that the lean startup method, while efficient compared to the conventional approach of “build it and they will come,” still requires time, attention, and resources that are diverted from other projects.;