Deploying the framework on Mechanical Turk over a two-month period yielded 85K English descriptions for 2K videos, one of the largest paraphrase data resources publicly available.

We deployed the task on Amazon’s Mechanical Turk , with video segments selected from YouTube.

Figure l: A screenshot of our annotation task as it was deployed on Mechanical Turk .

We deployed our data collection framework on Mechanical Turk over a two-month period from July to September in 2010, collecting 2,089 video segments and 85,550 English descriptions.

We randomly selected a thousand sentences from our data and collected two paraphrases of each using Mechanical Turk .

We designed our data collection framework for use on crowdsourcing platforms such as Amazon’s Mechanical Turk .

We hope to engage these algorithms with more sophisticated users than those on Mechanical Turk to measure how these models can help them better explore and understand large, uncurated data sets.

We solicited approximately 200 judgments from Mechanical Turk , a popular crowdsourcing platform that has been used to gather linguistic annotations (Snow et a1., 2008), measure topic quality (Chang et al., 2009), and supplement traditional inference techniques for topic models (Chang, 2010).

Figure 5 shows the interface used in the Mechanical Turk tests.

Figure 5: Interface for Mechanical Turk experiments.