The art of dealing with incomplete and imperfect data: I want to put this into the water supply of some companies. I need it in soluble form.
The old thinking says we need to have all pieces, all the dots in place, a complete and accurate picture, before we move. The new thinking says, we move forward and, whilst moving, we’ll get the better data and the better picture. And if we seem to be going in the wrong direction, we recalibrate and regroup. But if we are going in the right one, we will arrive faster than anybody else.
The trick is awareness of the imperfection, therefore of the risk. No risk, no great gain, really. If full solid data is what you require to move, and if you minimise risk as much as you can, your success may be so, but probably not spectacular.
For ‘data’, the word I have been using, read all the insights, the pieces of information and wisdom, the historical market research, the latest trends, the latest industry reports. The stuff you are supposed to have before making serious decisions.
(But the mind is equipped with ways of dealing with impartial data, incomplete information and blurred strategies. It’s the heuristic brain we have. Why worry about it?)
Management training has told us that you’d better do the homework, that good management needs good data, so we have created generations of slow managers waiting for a perfect plan.
Parkinson (of the ‘Laws of’) said that ‘Perfection of planning is a symptom of decay. During a period of exciting discovery or progress, there is no time to plan the perfect headquarters. The time for that comes later, when all the important work has been done’
Trying to be perfect is the most imperfect way to address organisational problems.
We have been told, as well, repeatedly, that a system (of decision making for example) is only as good as the data that has been inputted into it. So, again, the search for the perfect, accurate data must surely prevail. But, modelling and, in particular, Multi Attribute Decision Analysis (MADA) tells us that most often what matters is the relative weight between variables, not their absolute.In the past I have run entire ‘big deal’ MADA exercises when the ‘big numbers’ inputted were completely relative: if this produces x benefit, it will produce twice as much. And if you think it is three times, no worries, we’ll model that as well. There is no absolute number here but a lot of accuracy.
Back to the organization. Imperfect, unfinished and ambiguous is the state of most data points, insight sets and market forecasts. Moving forward with those is an art, a game of trial and error. After a while, one gets very proficient at this, and errors decrease (but never disappear). You win by making less errors than others.