Comparing Search Functionality of Amazon, Best Buy and Walmart

  • Amazon claims just an extra 100 milliseconds (1/10th of a second) on its response time will cost them 1% in sales. That's a loss of $2.8 billion per annum.
  • Google found 500ms (½ a second) increase in latency caused traffic to drop by a fifth. That's a loss of more than $17.8 billion per annum.
  • Amazon accounts for more than 50% of product searches.
  • In a survey of about 160 e-commerce companies, 80% plan to keep their personalization strategy and increase inventions in that domain.
  • We compared the search features of Amazon, Best Buy, Walmart, Etsy, Shopify, eBay, Rakuten, Alibaba and others. However, this blog post is limited to a comparison of user experience specific to search functionality of Amazon, Best Buy and Walmart.
  • The comparison focuses on search features that can be analyzed with publicly available data. This includes relevancy, user experience design, update frequency, facets, filtering, contextual search, lack of critical functionality and sorting.
  • The comparison does not include precision metrics, query caching, linguistic analyzers, re-indexing (frequency, speed and quality), quality of scale, memory usage, cognitive abilities, and performance metrics comprising hardware and software utilizations, server durability and stress testing.
  • Success story: A small team at a large corporation spent more than $10 million in three years and ended up discarding the search due to below average experience. They accounted for less than 10% of searches company wide. We helped a large team in the same company with a strategy that would cost them less than $400k per annum for more than 90% of the company's searches. If you would like to create a world-class search engine better than the companies above please contact us at info@cazton.com.

Search engines have become an essential part of our life. Simple searches are used to find contact information, stock prices and music on our phone. Contextual searches understand user intention and serve an intuitive way to retrieve relevant information. Intelligent searches utilize query statistics and machine learning to guide the rank of a product. The more relevant the results, the more likely a product purchase is. It acts as one of the all-knowing representatives for your company, whom a person can go to anytime and ask questions to get the information about a product or service. People may want to search by category or topic, some would be interested in best reviews, some may be looking to sort by lowest price first or the other way round for best quality. A good user experience, through desktop or mobile, will probe your consumer to dig deeper in your search result to find the best product for him/her.

As search is the most important feature in any application, it is very important to build a robust, flexible and user friendly information retrieval strategy. There are three types of strategies:

  • Structured searches are those that are performed on clearly defined data types. A well defined relational schema is a good example of a structure search.
  • Unstructured searches are those that are performed on data that lacks structure. These include BLOBs (Binary Large Objects) and formats like audio, video, and social media postings.
  • Semi-structured search is a combination of the above two search strategies and reap benefits from both. It is apt for searching on JSON, XML or CSV formatted documents due to their partial document structure.

According to a research, a one second delay in displaying search results can cause a 7% decrease in sales.(1) For example, a site making $50,000 sales per day may incur a loss of $1.3M per annum. Many users' concern about their privacy has led to a significant growth in search market share for DuckDuckGo recently(2), but Google still dominates this space with more than 90% global search engine market. However, you may be surprised to know that more than 50% of product searches happen on Amazon(3), not on Google. About 95% of Americans purchase something from Amazon at least once in a year.

Machine learning (ML) and artificial intelligence (AI) are the driving technologies of the world. Companies utilize them to provide relevant results to the user by understanding the contexts, semantically and syntactically. A lot of organizations are shifting to personalization of shopping experiences by utilizing the demographic, age and psychological traits of the target consumer. In a survey of about 160 e-commerce companies, 80% plan to keep their personalization strategy and increase investments in that segment.(4) Voice, image and video search is also trending these days, but in this article, we will be focusing solely on text search.

We have segmented our comparison across various categories like simple queries, product queries having attributes, synonyms, misspellings or text description and complex queries. Our rating methodology is based on a weighted index that comprises relevancy, boosting, facets, contextual search, user experience and other factors. Let's briefly understand the search terminology before diving into the comparison:

  • Relevancy: It is the quality of retrieving closely connected results with respect to the search query. For example, a query for searching printers should provide printers and not laptops or mobile phones.
  • Precision: It means to provide the exact and accurate results to the user's query. Precision is used in search engines to measure the fraction of results that are relevant from the retrieved results. For example, for a user querying for gaming laptops, 7 results actually turn out to be a gaming laptop out of 10 search results, so the precision would be 7/10.
  • Hit Highlighting: It means to highlight the words or phrases in the search results matching with the search query. For example, a query for searching 16gb RAM laptops should highlight these keywords for better user experience.
  • Facets: They systematically classify information from search results to categories and subcategories to narrow down a search. It helps visitors to quickly refine their results without having to scroll through products when they are looking for something very specific. For example, a query for searching laptops would give a pricing facet with multiple price ranges to narrow down in a budget option.
  • Boosting: It is the process of assigning a value of how important a search result is from the other results. It helps to give extra weight to a result to show higher in results. For example, a query searching for a tablet on Amazon website can specifically boost Amazon Kindle Fire to the user.
  • Ranking algorithm: It is a mathematical system of calculating and assigning a score to a query result item. It provides a way to rank the results and majorly encompasses relevancy and boosting. For example, a query for searching a laptop would combine the results based on relevancy, boosting, and any other filters to arrive at a final score of ranking the laptops.

Here are a few assumptions we have made during our comparison process:

The relevancy is evaluated for the top 10 products of the search result (ignoring the sponsored products) as an average of all the queries in each category (simple, text descriptive, query with synonyms, misspelling, etc.) without applying any filter, or sorting.

The score to a category's criteria (user experience, facets, contextual search, etc) considers 5 as an average score.

The weights to a category's criteria are assigned based on the holistic impact to the user.

Now let's begin comparing the different e-commerce websites against our query categories.

Full Report

  • Simple queries.
  • Product query with attributes.
  • Table descriptive queries.
  • Queries having synonyms.
  • Incorrect spelling queries.
  • Complex queries.
  • User Interface Comparison.
  • Walmart - Amazon - Best Buy vs Ideal Search Engine.
  • What could Walmart, Amazon and Best Buy do better?