Tweaking solutions in the open
One of the issues of services like Innocentive is that by design they provide suboptimal solutions to seekers. Why? There’s no tweaking allowed. It is almost imp0ssible to have the situation where the winning solution is also optimal. Optimization could be the element of public seeking of the solution, but instead it is left to the seeker.
Mathworks gets it right, in its Matlab programming contest:
8. Why are people allowed to tweak my code?
It can be frustrating to have someone modify your code just a bit and receive a better score.
One of the goals behind the MATLAB Contest is to learn more about MATLAB programming. Viewing and modifying other entries, as well as having your own entries be modified, is a great way to learn how to write better code.
From previous contests we’ve seen a recurring pattern – a particular algorithm is submitted and then “optimized”, and then suddenly a contestant submits a better algorithm to take the top spot. It’s then optimized, and the cycle continues. The top entries typically have been repeatedly modified by several contestants.
Though tweaking of entries that you see can be frustrating, it’s part of the evolution of the entries. The tweaking portion doesn’t represent the end of the contest but is just an intermediate phase; the brainstorming and tweaking never really ends!
At the end of the contest we’ll post an analysis of the entries showing how the entries developed over the course of the contest. This analysis includes highlights of the tweaking that went on in successful entries. At this point you can more clearly see if allowing code tweaking has hindered or helped the creativity involved in the contest. From what we’ve seen, the tweaking has made the contest more of a learning experience.
