Exploratory Data Analysis (EDA)
Exploratory data analysis (EDA) was introduced by John Tukey as an approach to analyze data when there is only a low level of knowledge about its cause system as well as contextual information. EDA aims at letting the data itself influence the process of suggesting hypotheses instead of only using it to evaluate given (a priori) hypotheses.
Explorative - opposed to Confirmatory - Data Analysis is like detective work looking for patterns, analomies or in general new insights and is usually done via graphical representations of the underlying data-set.
Explorative - opposed to Confirmatory - Data Analysis is like detective work looking for patterns, analomies or in general new insights and is usually done via graphical representations of the underlying data-set.
Exploratory Data Analysis (EDA) is detective work – numerical detective work – or counting detective work – or graphical detective work ... unless exploratory data analysis uncovers indications, usually quantitative ones, there is likely to be nothing for confirmatory data analysis to consider ... [it] can never be the whole story, but nothing else can serve as the foundation stone - as the first step.
[Tukey, 1977, p. 1-3]
Exploratory Data Analysis (EDA) is an approach/philosophy for data analysis that employs a variety of techniques (mostly graphical) to maximize
1. insight into a data set;
2. uncover underlying structure;
3. extract important variables;
4. detect outliers and anomalies;
5. test underlying assumptions;
6. develop parsimonious models; and
7. determine optimal factor settings.
The EDA approach is precisely that--an approach--not a set of techniques, but an attitude/philosophy about how a data analysis should be carried out.
1. insight into a data set;
2. uncover underlying structure;
3. extract important variables;
4. detect outliers and anomalies;
5. test underlying assumptions;
6. develop parsimonious models; and
7. determine optimal factor settings.
The EDA approach is precisely that--an approach--not a set of techniques, but an attitude/philosophy about how a data analysis should be carried out.
[Filliben, 2004]
[...] is concerned primarily with explorations and description of data, not with inference. The techniques are designed to identify fundamental, conceptually meaningful patterns and relationships in data and to call attention to observations that deviate greatly from those fundamental patterns
[Smith and Prentice, 1993]
Exploratory data analysis as the process of searching and analyzing databases to find implicit but potentially useful information, is a difficult task. At the beginning, the analyst has no hypothesis about the data. According to John Tuckey, tools as well as understanding are needed [Tukey, 1977] for the interactive and usually undirected search for structures and trends.
[Keim et al., 2006]
Furthermore, EDA can be used to support the selection of appropriate statistical tools as well as to provide a basis for statistical inference and further data collection.
Essential to EDA are graphical tools like box plots, stem–and–leaf plots, scatter plots, or timelines.
References
- [Filliben, 2004]: James J. Filliben, NIST/SEMATECH e-Handbook of Statistical Methods, Retrieved at: November 15, 2004. http://www.itl.nist.gov/div898/handbook/eda/section1/eda11.htm
- [Keim et al., 2006] Keim, D.A.; Mansmann, F. and Schneidewind, J. and Ziegler, H., Challenges in Visual Data Analysis, Proceedings of Information Visualization (IV 2006), IEEE, p. 9-16, 2006.
- [Smith and Prentice, 1993]: A.F.Smith and D.A. Prentice, Exploratory data analysis, A handbook for data analysis in the behavioral sciences: Statistical issues, pages 349-390, 1993.
- [Tukey, 1977]: John W. Tukey, : Exploratory Data Analysis. Addison-Wesley, 1977.