Empirical Evaluation of Interactive Visualizations for Preferential Choice: Difference between revisions

From InfoVis:Wiki
Jump to navigation Jump to search
No edit summary
No edit summary
Line 21: Line 21:


== ValueCharts+ ==
== ValueCharts+ ==
ValueChart+is a set of interactive visualization techniques for preferential choice and is an improvement of ValueCharts. It supports the DM in the 3 phases described above. In an Additive Multiattribute Value Function the DM’s objectives are hierarchically organized. In VC+ this hierarchy is displayed as an exploded stacked-bar.
ValueChart+is a set of interactive visualization techniques for preferential choice and is an improvement of ValueCharts. It supports the DM in the 3 phases described above. In an Additive Multiattribute Value Function the DM’s objectives are hierarchically organized. In VC+ this hierarchy is displayed as an exploded stacked-bar. The ValueChart+ follows the information-seeking mantra by Ben Shneiderman: '''overview first, zoom and filter, then details on demand.'''
 


== Evaluation Part A: Controlled Study ==
== Evaluation Part A: Controlled Study ==
Line 51: Line 52:




The subjects had to test the ValueCharts+ in the sensitivity analysis and inspection phase — no questions directed at the experimenter were allowed. Test domain was a set of hotels in the Vancouver area. 20 subjects were tested, 10 tested the VC+H and the remaining 10 the VC+V. Each subject performed each task, writing down the answer to applicable tasks that asked a question about the data. This procedure war re-iterated 5 times. The authors looked closely at their results to find an indication of whether one version of VC+ was a better fit than the other during the decision making process.
The subjects had to test the ValueCharts+ in the sensitivity analysis and inspection phase — no questions directed at the experimenter were allowed. Test domain was a set of hotels in the Vancouver area. 20 subjects were tested, 10 tested the VC+H and the remaining 10 the VC+V. Each subject performed each task, writing down the answer to applicable tasks that asked a question about the data. This procedure was re-iterated 5 times. The authors looked closely at their results to find an indication of whether one version of VC+ was a better fit than the other during the decision making process.
 
Finally, in the second step of the analysis the authors determined, for each task, what interface the subjects performed better. VC+V performed better on all five inspection tasks and also performed better on three out of the four sensitivity analysis tasks.





Revision as of 16:27, 27 May 2009

An Empirical Evaluation of Interactive Visualizations for Preferential Choice by Jeanette Bautista and Giuseppe Carenini

UNDER CONSTRUCTION

Short Description

The authors of this paper tried to not only show the usefulness of Value Charts to support preferential choice, which is finding the best option out of a set of alternatives. Furthermore, they compared two types of Value Charts: a horizontal version against a vertical version. The outcome of this extensive user study was that Value Charts in general and in particular Vertical Value Charts (abbreviated VC+V) seemed to be very effective in supporting decision making.

Process of decision making

The process of effective preferential choice can be divided into 3 steps according to prescriptive decision theory.


Step 1 / Model construction phase: the decision maker (abbr. DM) finds objectives, which are important to him/her. The degree of importance is also chosen.

Step 2 / Inspection phase: DM analyzes his/her preference model as applied to a set of alternatives.

Step 3 / Sensitivity analysis: DM is able to answer "what if" questions - such as "if we make a slight change in one or more aspects of the model, does it effect the optimal decision?"

In the development of interactive tools for preferential choice, we argue that full support for - and fluid interaction between - all three phases are essential in making good decisions.
[Bautista and Carenini, 2008]



ValueCharts+

ValueChart+is a set of interactive visualization techniques for preferential choice and is an improvement of ValueCharts. It supports the DM in the 3 phases described above. In an Additive Multiattribute Value Function the DM’s objectives are hierarchically organized. In VC+ this hierarchy is displayed as an exploded stacked-bar. The ValueChart+ follows the information-seeking mantra by Ben Shneiderman: overview first, zoom and filter, then details on demand.


Evaluation Part A: Controlled Study

In Part A, the authors took a quantitative approach by performing a controlled usability study to see how users performed the primitive tasks of the PVIT (Preferential Choice Visualization Integrated Task Model). Part A starts in the sensitivity analysis phase. The task is the following: 5 questions (based on the PVIT model) should be answered:

What are the top 3 alternatives according to total value?

For a specified alternative, which ob jective contributes to its total value the most?

What is the domain value of objective x for alternative y?

What is the best alternative when considering only objective x?

What is the best outcome for a objective x?


Mapped to the house domain, for example, we get the following tasks:

List the 3 highest valued houses.

For HouseX, which is its strongest factor according to your preferences?

How many bathrooms are there in House1?

Which is the least expensive house?

What is the best bus-distance?


The subjects had to test the ValueCharts+ in the sensitivity analysis and inspection phase — no questions directed at the experimenter were allowed. Test domain was a set of hotels in the Vancouver area. 20 subjects were tested, 10 tested the VC+H and the remaining 10 the VC+V. Each subject performed each task, writing down the answer to applicable tasks that asked a question about the data. This procedure was re-iterated 5 times. The authors looked closely at their results to find an indication of whether one version of VC+ was a better fit than the other during the decision making process.

Finally, in the second step of the analysis the authors determined, for each task, what interface the subjects performed better. VC+V performed better on all five inspection tasks and also performed better on three out of the four sensitivity analysis tasks.






Evaluation Part B: User Study

In Part B, the authors followed a more qualitative approach by observing subjects using the tool in a real decision-making context. In this second part of the study, they attempted to measure the users’ insight in the decision problem. Once the subjects had completed Part B, they filled out a questionnaire regarding their experience with VC+ in the decision-making process.


Datasets

Figures

Shown above is the horizontal version of a ValueChart+ presented by Bautista et Carenini

Shown above is the vertical version of a ValueChart+ presented by Bautista et Carenini.

The vertical height of each row indicates the relative weight assigned to each objective (e.g., size is much less important than internet-access). Each column represents an alternative, thus each cell portrays an objective corresponding to an alternative (bottom-right quadrant). The amount of filled color relative to cell size depicts the alternative’s preference.


Important Citation(s)

Suitable for which data types

Evaluation(s)

References

An empirical evaluation of interactive visualizations for preferential choice