VisWeek V&V

At VisWeek 2011, Robert M. Kirby, Claudio T. Silva, Robert S. Laramee, and William Schroeder participated in a panel entitled "Verification in Visualization: Building a Common Culture"

The slides used during the panel:

• Mike Kirby: File:KirbyVisPanel2011.pdf
• Bob [1]
• William Schroeder
• Claudio T. Silva [2]

The panel was recorded and will be available at http://vgtc.org/ "soon"

Inspired by the discussions at this panel, the following people would like to organize an effort to facilitate V&V at future VisWeek submissions:

• Britta Weber
• Tim Lebo
• Carlos Scheidegger
• Paul Rosenthal
• (feel free to add your name)

A few of us met for lunch immediately after the panel, and discussed a few points:

• SIGMOD has had something like it for 2 years, so we can borrow their model.
• Reproducibility takes (uncredited) time and effort; producing unreproducible papers is valued by the community.
• A grad student once published make files behind every published image - but stopped when he got a job.
• Publishing your code (without an established system) provides value because people grab it, use it, remember your name, and cite you in their publications.

The following people may provide useful suggestions for how to increase reproducibility within VisWeek publications:

• Claudio T. Silva - knows about SIGMOD's effort.
• Gordon Kindlmann - was involved in Claudios efforts
• "make file" SCI student - because he cared about it enough to make make files.
• Anita de Waard
• http://www.stanford.edu/~vcs/Papers.html

Some points:

• we should be careful not to confuse the two issues of V&V and reproducibility. V&V is roughly about a set of guidelines for ensuring that your code is correct; reproducibility is about making sure I can replicate your results.
• We should plan to have a decent set of guidelines in place so that if the VisWeek 2013 committee wants to use them, they can.

Related work

• http://www.executablepapers.com/ by Elsevier
• http://lemire.me/blog/archives/2010/04/20/the-mythical-reproducibility-of-science/
• http://www.ncbi.nlm.nih.gov/pubmed/20876290
• http://www.openscience.org/blog/?p=306
• http://www.snee.com/bobdc.blog/2011/10/publishing-academic-research-d.html