How to Test
One of the scarier and more difficult, though unfortunately necessary, parts of operating a business is testing the validity of new ideas. That’s because doing so risks time and capital and can be disruptive to your core business, but also if you don’t do it, your core business risks stultifying and being competed away.
Remember, either you or the world is always moving on.
To that end, we keep a prioritized list of new ideas for each of our businesses and try to be tackling at least one (and ideally the one with the most potential bang for the buck) at all times. The question is how best to do that?
Because I’m the one who keeps a very close eye on our capital stack, I’m generally a fan of incremental testing. This approach is born out of the lean startup/minimum viable product schools of thought, which is to say that if you have a hypothesis about a new initiative, the right way to go after it is to start small, iterate as you go, and feed it with more capital only as it proves itself out.
To that end, our COO Mark is constantly advising our businesses to test new things so long as those tests are (1) measurable and (2) above the water line. What above the water line means is that if the test fails, it’s not catastrophic to the business i.e., punching a hole in the side of a boat is not as bad as punching a hole in the bottom. And if you do find a win, then work on scaling it.
Of course, a catch-22 here is that if you have a big strategic idea and you’re only willing to start small and test incrementally, you may never get around to actually testing that entire big strategic idea. To wit, we have a retail business that is interested in assessing the opportunity of having a brick & mortar location. (And before you say that’s stupid the economics of selling online are way better, a true fact is that building a durable consumer brand requires selling through multiple channels.)
Now, they did the work and identified an ideal spot where we could open a store that would benefit from the right kind of foot traffic, and that could be staffed with people who already knew the brand well and stocked with inventory that could quickly be repurposed if we got indicators that the test wasn’t going well. It was a more significant initial capital outlay than I’d typically greenlight, but when we modeled it out, the returns from getting good data on how to open physical locations was very much worth it.
Unfortunately, the landlord came back and said we couldn’t have that space, but that we could have one that was not entirely dissimilar and also quite a bit cheaper. It wouldn’t benefit from the right kind of foot traffic, but would allow for the risk mitigation pieces on the staffing and inventory side.
As we thought about that, though it was less expensive, we decided that wasn’t a test we wanted to run. The reason was it wasn’t a good location, and if we ran the test and it failed, even if it didn’t cost us much in dollars, the cost of getting a false negative on a big strategic idea might in the long run turn out to be really expensive.
In other words, there is a fine line between the lean startup/minimum viable product/incremental approach and half-assing something. Further, it’s important to know the difference between the two when thinking about how best to test a hypothesis about your business. So in designing any kind of test, resource it sufficiently such that the conclusions you ultimately draw from it are meaningful. That may cost more upfront, but the cost of believing something to be true that isn’t is likely to be significant.
— Tim
