Citation in Cornell University Ph.D. Dissertation

October 30th, 2010

New Learning Frameworks for Information Retrieval” is the title of Yisong Yue’s January 2011 Ph.D. dissertation at Cornell University. (Professor Thorsten Joachims, his Ph.D. adviser, flatteringly reviewed Surf Canyon in the past.) While the thesis generally “proposes principled approaches to formalize the learning problems for information retrieval, with an eye towards developing a united learning framework,” it specifically discusses dynamic search interfaces and the value of disambiguating intent and implicitly re-ranking results in real time.

In §4.6.1, entitled “Beyond Predicting Static Ranking,” Yisong writes (emphasis added):

“It is common for information retrieval research to focus either on relevance estimation or user interface design, but rarely both simultaneously. However, for many tasks, it can be useful to model both jointly… One major limitation of result diversification over static rankings is that it sacrifices recall in favor of some minimal amount of utility for all usage intents – such a limitation could be dealt with by moving towards more dynamic interfaces.

Consider the example interface shown in Figure 4.8, which is inspired by and adapted from the SurfCanyon.com search engine [48]… by clicking or mousing over a result that matches the user’s intent, additional indented results are inserted into the original ranking… This interaction is quite natural, since the process resembles navigating a dropdown menu and since users are already familiar with result indentation. And yet even this one additional degree of freedom in content display can offer tremendous benefits…”

After the University College Dublin paper “A Recommender System Approach to Enhance Web Search and Query Formulation” and the Universidade Estadual de Maringá paper “An Approach to the Customization of Web Search Results,” this is the third academic reference to Surf Canyon. We’re delighted by the attention we’re receiving from the academic community.

Lastly, the team at Cornell recently drafted a brilliant paper, entitled “Dynamic Ranked Retrieval,” which dives deep into the study of real-time implicit ranking and offers statistical support for the “tremendous benefits” described above. It has been accepted for publication at WSDM 2011. While not yet public (we’ll post here when it is), we’ve been given permission to offer a sneak preview from the introduction (emphasis added):

“… most queries are ambiguous at some level. For such queries, there is often no single ranking that satisfies all users and query intents. While result diversification aims to provide a “compromise ranking” that provides some utility for all intents, diversification necessarily sacrifices recall…

The key idea is to make the ranking “dynamic” – namely, allowing it to change in response to user interactions after the query was issued.

From the user’s perspective, this may look as illustrated in Figure 1. This interface is inspired by and adapted from the SurfCanyon.com search engine…”

[Update 11/28/2010] The final version of “Dynamic Ranked Retrieval” is now available.

[Update 2/9/2011] The WSDM 2011 conference selected “Dynamic Ranked Retrieval” as one of six Best Paper Candidates.

Tags: - Top Posts - Research Testimonials

UCD Paper on Recommender System Approach to Enhance Web Search

October 18th, 2010

At the 19th Irish Conference on Artificial Intelligence and Cognitive Science (AICS 2008), researchers from the School of Computer Science and Informatics at the University College Dublin, Ireland, published a fascinating research paper entitled “A recommender system approach to enhance web search and query formulation.”

They begin by observing that “the traditional search interface has remained relatively static” before going on to describe “a recommender system approach to Web search which allows users to dynamically interact with the result-space that is of interest to them,” which they see as “an overlay interface as a complement to an existing search engine.”  This will sound familiar to anyone who has used Surf Canyon, which the researchers acknowledge (emphasis added):

“… Surf Canyon is an excellent example of a next-generation approach, and allows users to interact with results returned by existing engines; in short, users can select a result and receive recommendations drawn  from related results that appear further down a result list. Surf Canyon launched in the latter stages of our own research and presents an interface similar to what is proposed here.”

While there are significant differences in terms or implementation, their results are similar to what Surf Canyon has demonstrated:

“A key question is whether the resulting recommendations were found to be relevant. In fact… at least one of the 3 recommended results was selected approximately 25% of the time, a very significant indicator of relevance…”

Tags: Recommendations Research Testimonials

“Fully Dynamic” Search on TechCrunch

October 12th, 2010

Alexia Tsotsis at TechCrunch has posted an article about the new “Fully Dynamic” search being offered by Surf Canyon. As users know, Surf Canyon has been making search dynamic for some time. Recently, however, with the launch of Instant, Google has also started to make search dynamic. As explained by Mark Cramer, CEO of Surf Canyon, the two are not only compatible, but complimentary:

“Dynamic” is the future of search and we feel we have a critical element to add to that. Instant makes the page faster (dynamically updating the SERP after every key press), however, our innovation makes it more relevant (dynamically updating the order of results after every mouse click). The two together (Google Instant + Surf Canyon) make the search experience “Fully Dynamic.”

There are presently two ways to experience Fully Dynamic search: at the newly launched instant.SurfCanyon.com and, as always, by navigating over to Google after downloading our browser extension.

Tags: - Top Posts - Announcements Media

Pick of the Week on France24

October 7th, 2010

Francis Pisani, the highly-esteemed independent journalist who authors a technology blog for Le Monde and who also recently reviewed Surf Canyon for El Universo, chose to select Surf Canyon as the “Pick of the Week” for the France24 program Technophile, which was recorded in both English and French.

The portion about Surf Canyon begins at the 6m55s mark. Mr. Pisani starts with “Pick of the Week, use it, it’s called Surf Canyon,” before continuing with a demonstration and then:

This is artificial intelligence and [it makes your searches] more efficient.

The French version of the video is also available with the segment about Surf Canyon starting at the 6m37s mark.

Tags: Media Testimonials

Search as a Conversation – Second Part of Search Engine Land Interview with Gordon Hotchkiss

October 1st, 2010

Search Engine Land posted the second part of Gordon Hotchkiss’ interview with Mark Cramer, CEO of Surf Canyon, entitled “Search As Conversation.” The first part of the interview, posted on August 6th, focused primarily on Surf Canyon’s product and technology. The second part also includes a broader discussion about the industry as a whole and the increasingly dynamic nature of search. According to Cramer:

For the past 40 years since the inception of search, the way it’s worked is this: user enters a query, that query is matched to an index of documents, tremendous activity is deployed to try to determine the relevancies of those different documents, and a search result set is produced. But that search result set is static. There’s an order to those results. They go from 1 to 10 and then 11 to 50 million, and that order does not change. It’s stateless. What we have been doing for the past few years is essentially applying state to the search page in order to make the results dynamic. And if you consider “dynamic” to be something resembling a conversation in the sense that the search result page is actually responding to every input from the user to alter the content on the fly, then I think that’s an interesting way of looking at searching.

Tags: Media

Improving Your Search According to the Universe

August 24th, 2010

Writing for El Universo, one of the largest daily newspapers in Ecuador, Francis Pisani, the highly-respected independent journalist who authors a technology blog for Le Monde, reviewed a number of search technologies in a column entitled “Improving Your Search” and had this to say (translated from Spanish):

The most useful is Surf Canyon, which offers a “real-time personalized search” that uses an artificial intelligence formula to adapt to the specific demands of the user. The idea is simple: only a small percentage of responses offered by any search engine correspond to what really interests us. Surf Canyon interprets each click as an indication of what may concern them and then finds similar information in the results offered by Google on the following pages (2, 5, 10, etc.) which are then deployed immediately below the newly activated link. The selection improves with every click, saving the user time.

Tags: Media Testimonials

Where Is Search Going? – Search Engine Land Interview with Gordon Hotchkiss

August 6th, 2010

One of the most respected voices in understanding search user behavior, Gordon Hotchkiss is the CEO of Enquiro and a popular keynote speaker at search industry events. His ongoing series on Search Engine Land regarding the future of search has so far included conversations with Bing’s Stefan Weitz, author and CEO of Federated Media John Battelle and Google’s Johanna Wright. The latest installment in the “Where Is Search Going?” series is an interview with Mark Cramer, CEO of Surf Canyon.

Gordon had some very flattering things to say. After referring to one of the experiments outlined in our research paper, which was recently published by SIGIR, he replied, “A 40% uptick in user satisfaction by any measure is pretty impressive.” Later, after describing a little about the algorithm behind the re-ranking he responded, “That was impressive. An algorithm that recognizes the subtlety of subcontext has some decent IP cred behind it.” He concluded with:

“… I installed Surf Canyon and was fairly impressed. It was more helpful than I expected (though there may be few more skeptical critics of search user experiences than me)… If you’re not a search marketer [who often needs to see the “vanilla” version of the SERP], I’d definitely recommend taking it for a spin.”

Gordon’s next post in the series will be Part II of the conversation with Mark Cramer in which they will “speculate on where search might go in the future.”

[Update 10/1/2010] – Search Engine Land has published the second part of the interview.

Tags: Media Testimonials

Reaching the World with AMO Localizations

July 30th, 2010

As anyone who’s developed an add-on for Firefox is probably aware, Mozilla’s AMO pages offer a multitude of localization options for your add-on’s webpage: 33 different languages, including English, to be exact. This is an unparalleled opportunity to reach and communicate with users virtually everywhere around the world. However, actually translating your AMO page, quickly and economically, can be challenging. Outside of paying large sums for professional translation services, or hitting up foreign language-speaking friends, we know of two good options.

BabelZilla: www.babelzilla.org

While the UI might be a little obtuse and the process a bit complicated, many add-on developers have told us that they’ve had great success with BabelZilla. If you create an account and upload your add-on, volunteers will translate the embedded text for you. If you would also like to have the text on your AMO page translated, you could drop the strings in .property files. We were never quite able to get it to responses, and the activity seems somewhat low, but it’s free so you might want to give it a try. Apparently there are ways to kindly solicit translators on the forums.

Amazon Mechanical Turk: www.mturk.com

This is not free, but for a very modest amount of money you can get your AMO page quickly translated by native speakers. Create an account and then follow the instructions to upload the text of your AMO page. We used the “Basic Open-ended Question” template. On the Properties page you can specify the location of the worker. For virtually all of our translations we chose people living in the country where the language is spoken. The Reward is up to you, but we paid $5-10 per translation for our AMO page. Publish and then wait for the replies to roll in.

Some translations took a while to be returned, but the majority came in under 1 day, and some came back in hours. The average time a worker spent on a translation was 20-30 minutes. In the end, we were able to get 30 translations in just a few days:

Arabic (ar), Catalan (ca), Czech (cs), Danish (da), German (de), Greek (el), Basque (eu), Spanish (es-ES), Finnish (fi), French (fr), Gaeilge (ga-IE), Hebrew (he), Hungarian (hu), Indonesian (id), Italian (it), Japanese (ja), Korean (ko), Dutch (nl), Polish (pl), Brazilian Portuguese (pt-BR), European Portuguese (pt-PT), Romanian (ro), Russian (ru), Slovakian (sk), Albanian (sq), Swedish (sv-SE), Ukrainian (uk), Vietnamese (vi), Chinese (zh-CN) and Taiwanese (zh-TW). We’re still working on Farsi (fa) and Mongolian (mn).

There are some things to look out for.  When you get the translations you might want to drop them into http://translate.google.com/ which, while far from perfect, can help to make sure they are not way off base. Also, if you get a response from someone that took less than a few minutes, he or she probably just got a machine translation, which is not particularly desirable and naturally you could do yourself. Most people will not do this, but sometimes it might be worth specifying that you want a human translation. (As the Mozilla Devloper Wiki indicates, “You are kindly advised NOT to use any automatic online translation, which can bring unpleasant low-quality output to users.”) Lastly, if there are translations that look suspect, you can even go back into Mechanical Turk and set up another job to have the translations proofread by a different native speaker.

In the end it took a bit of effort, but making your hard development work understandable by so many people around the world makes it worth the while. Feel free to contact us should you have any questions or need any help.

Tags: Collaboration Tutorials

v3.0.6 – Real-time Featured Results and Private Browsing Support

July 8th, 2010

Our last release, v3.0.5, was at the end of March, so it’s been a while since we put out a new version. We’re pleased to announce that v3.0.6 went out on June 30th and contains real-time featured results as well as support for Firefox’s private browsing mode.

Tags: Announcements

The Future of Search: The Emerging Power of Real-time Personalized Search – July 13th Webinar with comScore

June 30th, 2010

On July 13th, Mark Cramer, CEO of Surf Canyon, will join Eli Goodman, comScore Search Evangelist, for a webinar discussion on the future of search and real-time personalization. To get an idea of the format, comScore offers a library of previous webinars. Here is a copy of the webinar invitation that comScore sent out this morning. We hope that you can attend.

comScore Surf Canyon banner

Internet users have come to expect search experiences that deliver immediate and relevant results.  Are you ready to take the concept of personalized search to the next level?

Please join me for a webinar on Tuesday, July 13, 2010, 2 PM – 3PM EDT as comScore teams with Surf Canyon to discuss: The Future of Search: The Emerging Power of Real-time Personalized Search.

Register Now

We will explore how the fracturing of the search market and emerging technologies enable searches that are not only relevant but are dynamic, real-time, engagements.  After reviewing the current search landscape and the limitations of personalization today, our experts will demonstrate critical factors that will drive our future search experiences.

You will learn:

  • What personalized search is, who does it now, and where current limitations lie
  • Where personalized search is heading and how search engines can adapt
  • How personalization will effect marketers and agencies, and how to prepare for it

I hope you’ll join me on July 13!

Cheers,
Eli Goodman
comScore Search Evangelist

[Update 7/28/2010] – comScore has now posted the slides and audio for anyone who missed the webinar and would like to check it out.

Tags: Announcements Personalization Presentations