Thursday, October 1, 2009

Fixing "Problems" We Don't "Understand"

We were discussing statistics in our class and how to understand numbers you are presented with respect to their distribution. Particularly, we were focusing on prediction metics and describing how a residual value (calculated as actual minus prediction) is defined as biased if the mean value is not equal to zero (or FAR from zero, our heuristic is if the mean is approximately 20% of the standard deviation).

I proposed a solution that we could "correct" the bias (even if we didn't understand it) by making a permanent subtraction from our prediction model by the mean of the residual.

My teammate, Matt Clare, and the professor further refined this model to say it would work IF the residual pattern exhibits constant variability (the variation and volitility isn't changing).

While I am loath to use solutions which insufficently describe the underlying causes, this seems to be one of the FEW examples where math can actually be used to solve a problem without a keen knowledge of what is happening (assuming an intimate knowledge of the mechanics of why patterns occur, rather just a knowledge that they exist and are stable).

For residuals that increase with the size of the actual, I think that considering errors as a fraction of residual over actual (in the same way we normalize variation for residual based on actual size) can understand variation in the same way.

No comments: