12.1 A theoretical model for the data analysis process based on  (Page 3/3)

As Velleman (1997) points out, data analysis is a revival of Francis Bacon’s scientific method and could be considered the modernincarnation of that method. The history of this method resembles a movement from an internal sensemaking process, which can often be subjective, to an externalsensemaking process that tries to be objective. If so, we should expect data analysis to display a foundation based on sensemaking with added safeguardsagainst the biases that sensemaking is vulnerable to.

PRELIMINARY RESULTS

I followed Cox and Mallows suggestions and compared data analysis case studies and suggestions available in the statisticalliterature to the sensemaking model. In all cases most of the data analysis prescriptions fell into one of the four sensemaking steps. The remainingprescriptions were all “meta-steps” which dealt with the data analysis process itself (e.g, plan, understand the problem). These meta-techniques may beevidence that data analysis has incorporated safeguards against the vulnerabilities of the internal sensemaking process. A visual description of thecompliance of 11 papers:

LOOKING FORWARD

This preliminary analysis supports the hypothesis that sensemaking may provide a theoretical model for data analysis. Furtherstudy must address the question, “How can we provide a rigorous demonstration that data analysis follows a sensemaking model?” As Cox points out, only a smallnumber of data analysis case studies are available in the statistical literature. Future research may employ more direct methods such as observingactual data analyses or scouring computer code used to perform data analyses.

if a cognitive basis is demonstrated, cognitive science may provide opportunities to improve the activity of data analysis. Docurrent data analysis methods provide adequate safeguards to the well documented list of sensemaking biases?

Finally, a firmly established model for data analysis can be used to expand the academic understanding of the sub-field. Theauthor originally embarked on this study to address the lack of well defined objectives for data visualization techniques. A better definition of the purposeof data analysis methods may provide new opportunities to optimize data analysis techniques.

ACKNOWLEDGEMENTS

• The National Science Foundation
• Dr. Hadley Wickham

REFERENCES

Alexander, et al. (1977). A pattern language: towns, buildings, construction. Oxford University Press, USA.

Bailyn (1977). ‘Research as a cognitive process: Implications for data analysis’. Quality and Quantity 11 (2):97–117.

Becker, et al. (1987). ‘Dynamic Graphics for Data Analysis’. Statistical Science 2 (4):355–383.

Box (1976). ‘Science and Statistics’. Journal of the American Statistical Association 71 (356):791–799.

Box, et al. (1978). Statistics for Experimenters: An Introduction to Design, Data Analysis, and Model Building. John Wiley&Sons.

Cabrera&McDougall (2002). Statistical consulting . Springer Verlag.

Chatfield (1995). Problem solving: a statistician’s guide. Chapman&Hall/CRC.

Coplien (1996). Software patterns . Citeseer.

Cox (2007). ‘Applied statistics: A review’. Annals of Applied Statistics 1 (1):1–16.

Friedman (1997). ‘Data mining and statistics: what’s the connection? ’Computing Science and Statistics: Proceedings of the29th Symposium on the interface.

Hey&Trefethen (2003). ‘The Data Deluge: An e-Science Perspective’ pp. 809–824.

Klein, et al. (2003). ‘A Data/Frame Theory of Sense Making"’. In Expertise out of context: proceedings of the sixthInternational Conference on Naturalistic Decision Making, pp. 113–155.

Mallows (2006). ‘Tukey’s Paper after 40 years (with discussion)’. Technometrics 48 (3):319–325.

Pirolli&Card (2005). ‘The Sensemaking Process and Leverage Points for Analyst Technology as Identified ThroughCognitive Task Analysis’. Proceedings of International Conference on Intelligence.

Ribarsky, et al. (2009). ‘Science of analytical reasoning’. Information Visualization 8 (4):254–262.

Tukey&Wilk (1966). ‘Data analysis and statistics: an expository overview’. In Proceedings of the November 7-10, 1966, fall joint computer conference , pp. 695– 709. ACM.

Tukey (1962). ‘The Future of Data Analysis’. The Annals of Mathematical Statistics 33 (1):1–67.

Wild&Pfannkuch (1999). ‘Statistical thinking in empirical enquiry’. International Statistical Review/Revue Internationale de Statistique 67 (3):223–248.

Velleman (1997). The Philosophical Past and the Digital Future of Data Analysis. Princeton University Press.