adding suggest passthrough to snippy
Posted by ark, ,
Bear with me, this is going to be a long one.

Ever since I changed my default search engine to my own snippy url shortner I've only missed one feature. The suggest as you type from the original omnibox. I was fairly sure I could make it work though and today I got it working. But not without a lot of problems on the way.

First of all, the suggest as you type feature is part of opensearch. There is a url you hit with your partial search and it returns a bunch of suggestions as a small json reply.

Unfortunately when you specify a Search Engine in google chrome there is no where to enter a suggest url (it is pretty complex and maybe only 10 people would ever want to do that). so I had to find another way to get it into Google Chrome. Fortunately there's TWO ways to add it.

First you can put a little element at the top of your XML and the browser should pick this up and add it to the list of options. Sadly I never got this working with Chrome.

The other way is with a window.external.AddSearchProvider javascript call, you need to give a full url to this method and after a lot of work I found two problems. My XML had an empty Image element which caused Chrome to choke and then I also found out that this javascript method doesn't work on Mac Chrome currently. What a pain, Mac is my primary development environment. Chrome is pretty awful about telling you when adding a search provider fails.

I played around with it a while and finally found a way to do it on Mac. In 
~/Library/Application Support/Google/Chrome/Default/Preferences I found:

"default_search_provider": {
      "id": "5",
      "name": "snippy",
      "prepopulate_id": "0",
      "search_url": "{searchTerms}",
      "suggest_url": ""

and I just want to fill in that suggest_url

I quit all Chromes, edited it and restarted Chrome and tahdah, the file got reverted to what it was set to before :(

so I had a poke around and found Web\ Data had what I wanted in it
# sqlite3 Web\ Data

UPDATE keywords SET suggest_url = '{searchTerms}' WHERE id = 5;

And it worked! now Chrome was hitting my suggest url when I was typing in the omnibox!

Remember with my url shortner I can type [g something] and that will search Google for [something]

My first plan was to pick up the short url I was typing, look up the suggest url and just redirect them, but alas the returned json has the search query in it [something[, and not the short url plus the search query [g something]. Seems Chrome is a little picky about what responses it actually accepts and the search query must match what's in the omnibox.

So I moved over to actually requesting the suggest url and fudging with the json reply to add the short url keyword and then returning that. Worked great.

For the main google search I managed to find the suggest url on this page and that worked great. Then I had to find the other suggesty urls. Google News was my first attempt, I loaded google news opened up the javascript console and turned on resource tracking, then started typing, sure enough at the bottom of the resources it showed the search urls being hit. I could hit headers and copy/paste the full url. and add it to snippy. A screenshot is left.

But of course nothing is easy, Google News sends results back in a different JSON format, Google Images sends back results as JSONP and also the results have bold elements in it which need to be stripped out for Chrome. But I managed to strip it out and it's working.

Eventually I expect to allow you to add OpenSearch Description Documents directly which'll be kinda neat.

I also plan to allow me to specify 'promoted or super-public' snippy urls which can be displayed on a page and easily exported for other users to use.

One problem is, since the appengine app actually requests the suggest url, there are no cookies sent along with it to identify you which means the results can't be personalized by the server, which is annoying, not that I know any that do it.