This folder contains all data relevant to the user studies from the paper:

Marc Spicker, Franz Hahn, Thomas Lindemeier, Dietmar Saupe, and Oliver
Deussen. 2017. Quantifying Visual Abstraction Quality for Stipple Drawings.
In Proceedings of NPAR’17, Los Angeles, CA, USA, July 28-29, 2017, 10 pages.
DOI: http://dx.doi.org/10.1145/3092919.3092923

If you choose to use this data, please cite the paper above!


=== OVERVIEW ===

This folder contains 4 main parts:
- The introduction shown to the crowd workers prior to the questionaire (Study_Introduction.pdf)
- The pilot study data and results (PilotStudy_Acinipo and PilotStudy_WaterFlea)
- The main study data and results (MainStudy)
- The confirmation study data and results (ConfirmationStudy)
- The portrait study data and results (PortraitStudy)

Each study folder contains the images (source and stippling results for different tonal percentages) used in the study.
If you want to know more details on the layout or what tonal percentage means, please read the paper.


== USER STUDIES ===

The csv files were downloaded from CrowdFlower(R), which we used to crowdsource our user study. Each study consists of 3 files:

*_source.csv: 		The source file we uploaded (excluding image urls) with the user study questions
*_full.csv: 		Each crowd worker answer individually, plus some meta information provided by CrowdFlower(R)
*_aggregated.csv: 	Aggregated answer from the crowd workers (each question was answered 50 times)

As stated in the paper, our main study consisted of 1260 comparisons with 50 answers each, resulting in a total of 63000 answers.
However, there were always 1 comparison of each input shown after each other to the user, therefore we have 6 comparisons per row in the csv file (63000 / 6 = 10500).


What do the column headers mean? (the most important ones, some meta data added by CrowdFlower(R))

percentageXY:
For the X-th comparison (1-6, test images) the tonal percentage of the y-th side (1: left, 2: right)

visual_pcX: 
Result (from the crowd worker) of the X-th visual paired comparison (1-6 for test images in the main study) on a scale from 1 to 5 (1: left much better, 2:
left slightly better, 3: no difference, 4: right slightly better, 5: right definitively better)

_golden:
Wheather this question was a test question, used to filter out bad answers.

visual_pcX_gold:
The accepted answers for the X-th comparison.

visual_pcX_gold_reason:
A note shown to the crowd worker why their answers were wrong (feedback).


== NOTES ==

- CrowdFlower also gave us the IP addresses of all workers, we removed them for privacy reasons
- The "url" rows contained the location where we hosted the images, removed everything but the local paths