Yahoo! Search Powered by Bing... But Not Identical

Summary: Since October 2011, Yahoo!'s search results have been powered by Bing. But despite using the same search ranking algorithms, the search results on the #2 and #3 search engines are still quite different. Today we use a couple of side-by-side comparisons of search results as case studies to discover what the differences are and what is causing them.

Yahoo! has been on quite the journey over the past several years trying to keep its head above water as Google seemingly takes over the world.  This summer their new CEO announcement even made mainstream news headlines because they had poached longtime Google exec Marissa Mayer. Prior to that, there were other major announcements that few outside of the search industry probably even heard about. First, in 2010, Yahoo! handed the keys to their search advertising over to Microsoft to be managed through MS Adcenter. Then a year later (and just one year ago), Bing began powering Yahoo!'s organic results as well. So for the past year, Yahoo! hasn't had any control over their own search results - organic or paid.

Presumably if Bing powers Yahoo!'s search results, the search results for each should be identical (style aside), but that's never been my experience, so I thought it might be helpful to review a couple of random examples today. There are of course a variety of reasons (budgets, ad auction, etc.) that you won't see identical ads, so let's leave those aside. We should also ignore, for the purposes of this post, when image/video/shopping results and the like are displayed within organic results. These now appear to be pulling from the same places, but styling issues mean more/fewer sample photos/videos/products are shown on either engine. There is value in exploring these other items at another time, but today I'd like to focus specifically on standard organic results. Again, if the same ranking algorithm is being used to produce the results for both search engines, shouldn't the organic results themselves should be identical? I would think so, but that's not what I'm seeing. Let's first take a look at the search results on Bing and Yahoo! for "ginger ale" (What? I'm trying to make it random and it was the first thing I saw on my desk!)

Bing & Yahoo! Searches for Ginger Ale

(click to enlarge)

Key Differences and Similarities:

• The obvious - Yahoo!'s results are roughly twice as long
• Again, image results are the same, just different in how many are shown. Another item that reaches outside of standard organics is the "recipes" section Yahoo! includes, which accounts for much of the length on that side.
• The "Food.com" result on Yahoo! is the first item that really jumps out at me as being dramatically different. It also bumps ebay down to #3 vs. #2 on Bing, and all results that follow of course
• Next 4 organic results are then the same on each (Amazon, Canada Dry, Wise Geek, and UC.edu), but then Bing shows something from zevia.com while Yahoo! squeezes in another eBay result before zevia.
• Bing ends with gingerale.org, but Yahoo! adds on a result from ginger-ale.com before 4 paid ads.
• The related searches pod at the bottom of each is also completely different.
• Bing is only showing 8 organics + image results. Yahoo! is showing 10 or 11 (depending on what that food.com item is).

Bing & Yahoo! Searches for Catahoula Leopard Dog

(click to enlarge)

Key Differences and Similarities:

• Surprisingly similar - First 7 organics are the same, and the image and video results actually appear in about the same positions on the page.
• Bing only has 8 organic results again, and for some reason they snuck the "related searches" in before #7, instead of placing them at the top or bottom of results.
• Bing's #8 result is for buckeyecatahoulas.com, but that's not listed at all on Yahoo. Instead Yahoo! has an Amazon listing sandwiched between 2 akc.org listings for #7-10. The odd thing about that Amazon listing is that it's virtually identical to the Amazon result listed earlier (#5). One takes you to prepopulated Amazon search results for "catahoula leopard dog" and the other takes you to the same thing, only plural - "catahoula leopard dogs".

Bing & Yahoo! Searches for Ostrich

(click to enlarge)

Key Differences and Similarities:

• At first these seem like the 'catahoula' results, where Bing and Yahoo! were actually quite similar (even showing images and videos in similar areas), but the first thing you'll notice is an extra nationalgeographic.com result squeezed in between results #1 and #2. #2 is a NatGeo result too, and you'll notice if you follow both of them, they take you to the same page. Seems like a bug to me, sort of like the Amazon issue I highlighted in the 'catahoula' example.
• Setting aside that oddball NatGeo result, regular organic results #1-4 are the same on both engines, but then things get a little crazy.
• Bing's #5 is Yahoo!'s #6, and vice versa. Same situation with #7 and #8.
• #9 and #10 are the same on both engines.
• For some reason Bing showed 2 additional results for a total of 12 organics, however when I tried to duplicate this after closing and reopening my browser, those last last 2 didn't show and the search results were reordered... until I refreshed. Very odd.

Bing & Yahoo! Searches for Halloween

(click to enlarge)

Key Differences and Similarities:

• Right off the bat, #1 is the same on both Yahoo! and Bing - halloweenmovies.com - but Bing shows it with a little preview image. Like author images in Google results, this would obviously increase CTRs, so why are they bogarting this capability and not letting Yahoo! get in on some of that?
• Yahoo! may not get the image on the #1 result, but they do get a big image on their #2 result which comes from the History Channel (history.com) and is a result not included at all on Bing. Sort of reminds me of the extra NatGeo result in the 'ostrich' example. Yahoo also includes a carousel of movies from the "Halloween" series, each of which (when clicked) takes you to a Yahoo! Movies details page. Bing doesn't include this.
• Other than the History item, #1-4 organics are the same on both engines.
• Bing includes a result (#5) for a page called "New England Halloween Guide" on bostonplus.com. This is an example of location based results masquerading as pure organics, or to use the term I recently coined, "local organics". We see things like this, even on Google, pretty regularly because our Comcast IP address shows us as being in Massachusetts. Yahoo! includes this same result, but at #7.
• Yahoo!'s #5 is a "local organic" as well - "Halloween in Western MA" on a site called Hilltown Families. Bing doesn't include this result at all (on page one anyway).
• Bing does include another "local organic" in it's #7 (of 8) position - an interior page on hauntedhouse.com - "Haunted Houses in Massachusetts.
• I verified that location settings were causing results #5 and #7 on both engines to show MA related results by turning changing my location manually on Bing and watching them disappear. Yahoo! was a bit trickier, but you can see where they've made "Massachusetts" bold in the search results even though my search was simply for "Halloween". In fact, this is also actually the case for #3 - Haunted Happenings Salem Massachusetts.
• #6 and #8 are the same on both engines - familyfun.go.com and the IMDB page for the original Halloween movie from 1978.
• Yahoo!'s #9 and #10 are additional results from History and Wikipedia respectively. Bing only includes 8 results, because it must be counting the shopping and image results toward the organic total.

So what do these simple case studies tell us? That despite being told, even in the footer of Yahoo! search results pages, that Yahoo is now powered by Bing: ... it's a little more complicated than that.

Conclusions about Yahoo! Powered by Bing Results

• In general, Yahoo! seems to generally show more results than Bing. Bing seems to include things like images, videos, shopping, etc. in their count of 10, rather than including those items in addition to 10 standard organic results.
• Yahoo! seems to include extra featured items with large images next to them for things like television stations (National Geographic, History, etc.)
• Bing and Yahoo! both seem to be using "local organics", like Google does, but like with everything else - there's a lack of consistency. The local organics that each of these two engines uses aren't exactly the same and they certainly aren't always in the same positions.
• The first 4 or 5 purely organic results seem to be the same on both engines, for the most part.
• Fun fact unrelated to search rankings: Items in Yahoo!'s search results all open in new windows when clicked.

Given all the variation in results that one might expect to be identical, I think there are a couple of things in play. For starters, Yahoo! still has some of it's own services (recipes, shopping, movie info, etc.) that it maintains rights to incorporate into organic results. I assume Yahoo! insisting on keeping these as part of their agreement with Microsoft. I would chalk other inconsistencies in regular organic results up to testing. It's no secret that all search engines test user interaction to determine what works and what doesn't and then over time they modify their results accordingly. Given that Bing does clearly "power" Yahoo!'s results, "testing" is my best guess as to why even the standard 1-10 organics, when you strip everything else away, don't match up precisely. On a related side note, it'll be interesting to see what CEO and new mother as of 11 days ago, Marissa Mayer had to say to her employees during yesterday's "Goals All Hands" meeting. Yahoo! has yet to publicly lay out what their future plans are, but the fact that they abandoned both paid and organic search over the past two years (while Bing seems intent on attempting to put up a fight against Google), would seem to indicate that they're headed off in a completely different direction. Then again, Mayer was with Google since the beginning so if anyone could get Yahoo! back in the search game, it might be her. Still, it seems like too much of an uphill battle and I would think a wise exec would recognize that. Time will tell. By the way, if you noticed anything, er, interesting about the shopping results on Bing for the "halloween" search, you're not alone - but I'll be posting about that as a separate item shortly. Stay tuned!