The Guardian set up a crowdsourcing mechanism to cull through thousands (millions) of expense line items from the MP expense scandal. A pretty amazing creative feat from concept to execution. Here are some short learnings:

1. Your workers are unpaid, so make it fun (the sorting of expenses went in 4 simple categories and there was a leader board to track progress)
2. Public attention is fickle, so launch immediately (no time for load testing, etc, build/launch in a week. Live QA. Fun times!)
3. Speed is mandatory, so use a framework (stand on the shoulders of giants;they used the Django Python framework)
4. Participation will come in one big burst, so have servers ready (they used Amazon EC2 cloud hosting to get around corporate procurement timelines)