From the last Google Analytics (GA) study on the usefulness of "pivots_block" on 4 recommendation algorithms, we learned that the classical "relevancy" algorithm generated the better results. Therefore, we used the relevancy algorithm on D.O. from Dec/4/2008 to Jan/9/2009. And the average click-thru rate was 0.474%.

Then, we were interested in whether 0.474% was indeed a good indicator to claim "pivots_block" was useful to users. So we temporary switched "pivots_block" to the "New forum posts" on the module pages, and try to compare if the "New forum posts" could invite more clicks. It turned out that the average click-thru rate for "New forum posts" between Jan/9/2009 and Jan/18/2009 was only 0.144%. That means, more or less, on the module pages "pivots_block" is almost 3 times more useful than "New forum posts".

Now we are much more confident that "pivots_block" and its intended goal of providing module recommendations is indeed useful to the community. So we are going to switch back to "pivots_block", and continue to make improvements to it.

One thing we learned from this study was that users seem to like more "serendipity" in the "pivots_block". One problem for the current algorithm is that the recommended modules for a particular module remains quite stable over time. And we are going to try out an approach of adding a little bit variability to the recommendations.

The new algorithm is coming soon!