In the spirit of the upcoming 2008 Olympic Games in Beijing an Editorial in the August issue of Nature Methods discusses the use of organized competition for evaluating algorithm performance. Such competitions, or “collaborative experiments”, have become very popular in some large communities.

We believe similar competitions would also be helpful for smaller communities. What do you think? Is it more trouble than it is worth or would it be a valuable way of pushing algorithm performance to new heights?