In a previous post we mentioned the research on Dynamic Ranked Retrieval conducted by Professor Thorsten Joachims, Christina Brandt, Yisong Yue and Jacob Bank at Cornell University and that their paper was accepted for publication and then selected as one of the six Best Paper Candidates by WSDM 2011. While this is a bit belated, we are happy to have discovered the video of Professor Joachims presenting that paper.

From the 1m40s mark to the 3m20s mark a screen shot of Surf Canyon is used to demonstrate dynamic ranking in action. Professor Joachims then offers DCG analysis of a sample search senario and concludes, at the 5m50s, that, “by being dynamic, and adaptive, you can gain a lot of retrieval performance.” Surf Canyon is then mentioned at the 12m00s mark as an Interactive Information Retrieval Model. The Adaptivity Gain, defined previously as the increase in retrieval performance offered by dynamic ranking over traditional static ranking, calculated from empirical studies done on two collections of TREC queries labeled for multiple intents, is then presented at the 14m10s mark and described as “quite substantial” with NDCG going from 55% to 70%.

“When you think about how much effort search engines are spending to get a 1% improvement in NDCG, this is a lot and could potentially change the upper bound of how good you can get with a single ranking.” – Professor Thorsten Joachims