This piece started as a review of Local.com‘s new mobile site (mobile.local.com). It’s got a simple, efficient user interface, understood addresses in every format I tried and it was very good at determining what I was searching for and returning relevant results. Most importantly, Local.com seems to return more results for my queries than other local search engines. I wanted to quantify what I was seeing and came up with some simple metrics for comparing local search engines.
The things I look for in a local search are, first and foremost, the number of relevant local results returned. Second comes usability which I define as a simple, intuitive interface that requires the fewest possible keypresses. Usability is also enhanced by features that save time and effort, like remembering the locations and search terms I’ve used in the past, or make the results easier to understand and use, like maps.
Here are the tests I came up with:
-
Query Results – I ran two test queries in each application. I used locations I was familiar with so I could judge the relevance of the results.
– ‘sushi’ near ‘94132’ which is a largely residential San Francisco zipcode. One point for each sushi bar found within 2 miles
– ‘Western Wear’ near ‘Dublin CA’ a San Francisco suburb. I gave 1 point for each shop selling cowboy clothes within 10 miles of Dublin, Ca
– duplicate and irrelevant results waste time and bandwidth so subtracted 1 point if there were any.
- Keypresses – the single most important measure of usability. The application requiring the fewest keypresses got 5 points, second place 4 points, 3rd 3, fourth 2, and fifth 1 point.
-
Features -Each of the following desirable features adds points to an applications score if present.
a) Click to call – This should be obvious, we are using a phone, the technology to make a call with a single key press exists and is easy to implement – 2 points.
b) Driving or walking directions from my starting point to my chosen target. Pretty much essential when you are in unfamiliar territory – 2 points.
c) Maps – appropriately sized, readable maps showing each business found in the context of nearby streets are a help in finding the businesses.
d) Saved Locations – Applications that remembered recent locations and let the user chose and reuse them got a bonus point.
e) Saved queries – Nice to have if you search for Espresso or Thai Food frequently – 1 point.
The winner! Local.com 27 points
Sushi: 9 in 2 miles from 94132 = 9
Western Wear: 8 in 10 miles from Dublin CA = 8
maps = 1
directions = 2
click to call = 2
keypresses 47 (tie with ASK and Live for best) = 5
no saved locations = 0
—-
Second Place: Windows Live 26 points
mobile.live.com/search/local.aspx
Sushi 11 in 2 miles from 94132 = 11
Western Wear 5 in 10 miles from Dublin CA = 5
1 duplicate = -1
maps = 1
directions = 2
click to call = 2
keypresses 47 (tie for 1st.) = 5
saves locations = 1
—-
Third Palace: Google 17 points
www.google.com/xhtml?site=local
Sushi 1 in 2 miles from 94132 = 1
Western Wear 6 in 10 miles from dublin ca = 6
3 dups = -1
maps (113×113) = 1
directions = 2
click to call = 2
keypresses 47 (tied for 1st) = 5
saves locations = 1
—-
Fourth Place: Go2.com 14 points
Japanese Restaurant (‘sushi’ doesn’t work) 6 in 2 miles from 94132 = 6
Western Apparel 3 in 10 miles from Dublin CA = 3
no maps 0
directions 1
click to call = 1
saves locations = 1
saves queries (categories) = 1
55/98 (ave = 76) clicks (5th place) 1 point
Notes: Go2 uses a combination of category lists and search boxes. It has a quirky user interface with multiple paths to the same result. The first time I ran the searches it took me 98 clicks. I was able to perform the same search in only 55 clicks after trial and error to find the most efficient path. I had to search for “Japanese Restaurant” as searching for “sushi” returned nothing (I didn’t count the wasted keypresses required to discover that).
—
Fifth: Ask.com 9 points
Sushi 0 in 2 miles from 94132 = 0
Western Wear 1 in 10 miles from dublin ca = 1
maps = 1
directions = 1
click to call = 1
keypresses = 47 (tied for first) = 5
—
Sixth: Earthcomer 7 points
mobile.earthcomber.com/phone/intro.do (review)
Sushi 2 in 2 miles from 94132 = 2
Western Wear 0 (not on list) in 10 miles from Dublin CA = 0
maps = 1
directions = 1
no click to call = 1
saves locations = 1
saves queries (categories) = 1
119 clicks (mens clothing) (last place) = 0
Notes: EarthComber requires picking a search category from a series of lists. There is no category for Western Wear or anything similar so the click count is based on searching for “Mens Clothing”.
—
Conclusions: Local.com returns the most results and beats Windows Live overall by a single point, none of the others are even close.
One thing that surprised me is that the two search engines that let you pick locations and things to search for from lists required many more keypresses than the ones where you key addresses and search terms. It’s conventional wisdom in mobile interface design that users refuse to type on phones and that a wizard-like series of pick-lists is the optimal design. That may be true for some types of mobile sites where there are relatively few items to choose from. But for search, typing using T9 takes many fewer keypresses, is faster, and I think less frustrating and more powerful. For example, EarthComer had extremely long scrolling lists of search categories. It took a frustrating number of keypresses to find that there was no category for “Western Wear” or anything similar (I didn’t count those keypresses in EarthComber’s total).
All of these local search engines, at least at the URLs listed, are US only except Google which seems to cover just about every place in the world and Windows Live which searches Canadian locations in addition to the US. If there’s a good local search engine in your country please share the URL in a comment.
Also, I’d give it 50 extra points for NOT including Western Wear anywhere within its search sites–as you can’t blame Earthcomber for maintaining a sense of style–unlike any Doug who wears Western clothing.
Just my humble opinion.
Dennis:
I see your point, I guess, but I like Earthcomber because it tells me what’s new when I am traveling in areas Im unfamiliar with. Plus, I don’t have to keep entering in search requests as I walk up and down the street.
You seem to miss that point. What I like about it is it informs you what’s around you without having to constantly enter it in.
Frankly, If I would have to ask what and where every time I wanted to know what’s around me, I wouldn’t use local. I’d call 411–like the rest of the world.
I think you missed the boat on this one.
Mike Blumenthal wrote, “Google allows you to save locations”. You are right Mike, but only if you use the new Yahoo oneSearch like version of Google Mobile search at:
http://www.google.com/m?uipref=3
I prefer Google’s classic mobile search:
http://www.google.com/m?pref=s&uipref=2
because it still lets you search for MOBILE web sites separately. See Nadir Garouche’s, Google Has Killed the Mobile Web for a great post on what’s wrong with the new Google Mobile search.
I guess that most people are using the new version especially now that Google’s removed the link on the settings page that let you switch back so I’ve updated the post to indicate that Google lets you save locations.
Dennis
Jim wrote;
“How is it logical to count 119 clicks for your Western Wear, if Earthcomber gives you all men’s wear in 1 click? Where are you getting 119? All the results in Mens? Did you click 119 times? Or complaining that they found a lot of menswear?”
Jim, it’s not just for mens wear that’s for the whole test case which involves two separate searches, one for “sushi in 94132” and the other for “mens wear in dublin ca” I wanted to be sure I didn’t make a mistake so I just repeated the test and got 121 this time – I must have missed 2!
Here’s my log, the numbers on the left are a running total of keypresses:
2 Hit “Find Things Now”, hit “1” for set location by address
9 Press down arrow 7 times to reach “zip”
12 Press “Edit” softkey, Press “Mode” softkey, Press “2” for “numbers”
18 Key “94132”, press “Done” softkey.
20 Down 1 and select “Set”
22 Select “Find Stuff” then press “5” for “Let’s Eat”
56 Press down arrow 34 times to reach “Sushi” (accesskeys only work for 0-9)
57 Select “Sushi”
Start over for new search
59 Hit “Find Things Now”, hit “1” for set location by address
63 Press down arrow 4 times to reach “city”
71 Press “Edit” softkey, key dublin, press “Done” softkey
76 Down 1 to “State”, press “Edit” sofkey, Key CA (takes 4 keypresses using T9)
81 Down 3, hit “Set” Right 1 Then “Find stuff”
95 Down 13, hit “Shopping”
121 Down 26 Hit “Mens Clothing”
Finally “Mens Clothing” is not the same as “Western Wear”. In fact none of Earthcombers’s 20 results for Men’s Clothing are Western Wear shops. That’s a fundamental flaw in using pick lists for something like local search. There are tens of thousands of things a user could be searching for. Earthcomber is useful for common searches but fails on things like “Western Wear” that only the end of the long tail is interested in.
Earthcomber would be a lot more efficient if I was using a device with a touchscreen, query keyboard or dedicated page down keys. But this site is about browsing on mainstream phones with just number keys and a D-pad. For that class of devices, Local.com is a lot easier to use. What phone do you use, Jim?
I missed that Earthcomber lets you save lists of your favorite categories, I’ll change the post to indicate that which will raise it’s score by 1 point.
Jim you also said “Also, Earthcomber does indeed have click to call, if there’s a phone number listed.” You are correct, I must have been testing that feature with a desktop browser, It looks like Earthcomber only formats phone numbers as click to call (wtai) when it recognizes a phone browser. I’ll correct that on the review too.
Dennis
Also, Earthcomber does indeed have click to call, if there’s a phone number listed.
I tried them all, too, and your 6th is my 1st.
How is it logical to count 119 clicks for your Western Wear, if Earthcomber gives you all men’s wear in 1 click? Where are you getting 119? All the results in Mens? Did you click 119 times? Or complaining that they found a lot of menswear?
You didn’t mention – and maybe you didn’t know – that Earthcomber lets you CREATE YOUR OWN lists online, so you could put Western Wear on a list with anything else you want. Then it would look for Western Wear along with anything else you like to regularly do or find or eat, with ONE click.
One. That’s slightly better than your winner: 47 key presses! to find 5 results?!? You have way more time on your hands than I do.
Search boxes suck if you have to spell out what you want each and every time. Does’t seem like the best use of “engines” if we have to spell it out every time to have them spit out one thing at a time.
Google allows you to save locations
Mike