Startup Data Reliability Grid

Startup Data Reliability Grid: Good strategic decisions take account of market intelligence and the conclusions we draw from the data. This is especially true in new venture creation, where experience is limited. What we need to take into account is the reliability of the data we collect. It is some times tough to do, because we may have conflicting evidence. So, we need some methodical way of evaluating that market intelligence.

When I was in training in British military counter intelligence towards the end of the Cold War, we were drilled in comparing and contrasting. We had to evaluate the reliability of information sources—the context, as well as correlating that information with other items of information. We examined assumptions, and were critical about our beliefs. What I learned back then, can be applied to strategic decisions and market intelligence in new venture creation today.

Below is the Startup Data Reliability Grid, a simplified version of the kind of grid that we used in the military. When you set out the information graphically in this way, it is easier to see the ‘wood for the trees’. By comparing the individual items of data and plotting them against their sources, you gain confidence about the reality of the marketing intelligence that you have collected and can make better strategic decisions.

Startup Data Reliability Grid

So, for example, if you have market data (quantitative and qualitative) about your intended customers, put the numbers or opinions in the first column of the Startup Data Reliability Grid. The source of the data goes in the second column—and next the tricky part. Put your evaluation of the reliability of the source in the last column. Now you’ll be able to see in graphic and comparative terms each bit of data, to help your level of confidence in decision making.

It is vital to be concerned with probability of your data being right, since you can never be certain about the likelihood of your strategy working out precisely. Evaluate the degree of trust that you have in both the information’s subject and its source. You can then significantly reduce your level of uncertainty, or improve your confidence in the likely outcome of decisions based on the data. Another very useful technique you can use to check on reliability, getting to the bottom of things, is the Five Whys Technique.

Help from Military Intelligence

You may get more help from the US Army guidance on intelligence evaluation in your thinking about using the Startup Data Reliability Grid:

from Appendix B of the US Army’s manual FM 2-22-3 Human Intelligence Collector Operations, which can be found freely on the internet

New venture creation is hazardous at the best of times. Impulsive decision making is often the norm. Being a first-class noticer of all the information that surrounds you, or that you can collect, really helps ensure you are working on the best data.

Stopp to draw breath before ‘pulling the trigger’, by using the Startup Data Reliability Grid. This may help you to avoid costly mistakes. I know from my own new venture creation experience that I made early assumptions that were based on my gut feel, rather than good data examined reflectively. This approach to market intelligence seem a bit cumbersome, but it will probably save you time and money correcting mistakes later.