\section{Redirect paper to Crowdsourcing}

What do we have?
  We have a game based on having a structured dialoge between two players
disputing the correctness and optimality of claims in a particular family. 

What does the structure give us?
  It guarantees 
   a) focus : players remain focused on a particular claim at hand.
   b) Constructiveism : if the claim asserts the existence of an object x with a
                        particular property, then the proponent has to have an 
                        algorithm to construct x.
=========
What do we have?
  We have a game for searching

\begin{verbatim} 
Paper should be ready one week 
before due date: Jan. 25, 2013
and will be sent to Magy on 
that date or sooner.

Ahmed's task (Jan. 8):
Write section:
Mechanism Design for Crowdsourcing
(Formerly: The SCG Design Pattern 
with Applications)
Four subsections:
1. Optimization Labs
2. Agreement with two refutations
3. Perfect Labs
4. Less Competition (to support brainstorming)
(formerly less competitive payoff ...)

Ahmed proposes a good way to deal with implied games.
(games where some of the decisions
have been made)

Ahmed proposes a good way to deal 
with Lab decompositions.
What is the difference between lab decompositions
and problem decompositions.
In lab decompositions we don't DIVIDE
AND CONQUER? We TRANSFORM AND CONQUER?

Karl works out three examples of lab decompositions.
\end{verbatim} 

A very important property of our approach to crowdsourcing
is that we take good care of the crowdsourcing workers.
\begin{itemize}
\item
feedback: when points get deducted there is a demonstrated reason.
\item
examples: see knowledge base of claims and history of claims
\item
get rewarded for breakthroughs
\end{itemize}


Crowd Sourcing to Distinguish Good from Bad

The Scientific Community Game as
A Crowdsourcing Platform to Distinguish Good from Bad

Domain of requests (instances) and responses (solutions).
Responses are checked against valid(request,response).
Claims are about the relationship between requests and responses.
Claims are divided into good and bad claims.
Good claims are claims that are predominantly defended.
Bad claims are claims that are predominantly refuted.
Refutation is the complement of defense and is based on
the requests and responses exchanged.

Want to build artifact: good claims. And the corresponding techniques to defend them.

Definition from Communications of the ACM:
A CS system enlists a crowd of users to explicitly
collaborate to build a long lasting artifact
that is beneficial to the community.

Also CACM:
enlists a crowd of humans to help solve a problem
defined by the system owners.

The White Paper Version: Crowdsourcing is the act of taking a job traditionally performed by a designated agent (usually an employee) and outsourcing it to an undefined, generally large group of people in the form of an open call.


Four challenges:

(How to recruit and Retain Users?)

What contributions can users make?
provide requests and responses, propose and oppose claims about requests and responses

How to combine user contributions to solve the target problem.

Describe in detail how this is done in SCG:
bottom-up or top-down.
lab decompositions and meta claims (subject to refutations).
strengthening, correcting mistakes.

Break down a lab into simpler labs. Extend basic game:


How to evaluate users and their contributions.
breakthroughs
learning
activity
how many contradictions

Describe in detail how this is done in SCG

Related Work:

 Group organization. Warren Bennis in his book: Organizing Genius, The Secrets of Creative Collaboration, says: "you create an atmosphere of stress, creative stress, everyone competing to solve one problem."

Crowd IQ: measuring the intelligence of crowd sourcing platforms
Web Science 2012

we have our own way of measuring crowdIQ: count contradictions.

The CrowdLang paper by Abraham Bernstein at Web Science 2012

CrowdForge: Crowdsourcing complex work
(2011) ACM
from CMU
http://ra.adm.cs.cmu.edu/anon/usr/ftp/anon/hcii/CMU-HCII-11-100.pdf


We identify the coordination requirements necessary to crowdsource complex tasks, and describe a framework 
to support a variety of task types.  The framework systematically breaks complex problems down into simpler 
tasks by creating subtasks that define and create other subtasks and distributes these tasks to workers.  Output 
from subtasks can be evaluated and consolidated via additional outsourced tasks.

\section{Fuzziness}

http://jcr.sagepub.com/content/50/1/28.full.pdf+html

contains interesting references


