Cosmos DB vs MongoDB

• Azure Cosmos DB is comparatively faster than MongoDB in most common use cases, although MongoDB performs better over 1MB payload.
• MongoDB offers more flexibility in terms of setup. It runs on-prem, any cloud provider (Google Cloud Platform, AWS, Azure and more).
• We found Azure Cosmos DB to be 93% cheaper for reads and around 20% cheaper for writes than MongoDB Atlas depending on workload characteristics (see TCO table below).
• At Cazton, we work with different kinds of database technologies and have more than a decade of experience in creating highly scalable and efficient systems that use multiple database technologies that are complementary to each other.
• We have helped many Fortune 500 clients, mid-size companies and startups create a scalable data strategy with high performance.
• Want a detailed report for Cosmos DB, MongoDB, Cassandra, Redis, Postgres, SQL Server, MySQL, ElasticSearch and Solr? Contact us at info@cazton.com.

Just a few years ago, it was acceptable to have offline data processing that enabled employees in analyzing big data. That analysis brought continuous improvement to an existing business and the customer experience; however, there has been a paradigm shift. Customers expect real time predictions and they expect to act on them instantly. So, we need to use incoming information and map it with our existing data models to make real-time predictions.

Now, more than ever, big data analysis is a necessity for a company to excel and grow. The amount of data shared and transferred between humans is unimaginable and to manage, analyze, make real-time predictions and decisions using that data is a daunting task. Many organizations prefer polyglot persistence over a single data storage technology as they have to deal with different types of data.

Over the last decade the popularity of NoSQL databases have skyrocketed. This is mostly because of the increased number of devices including mobile as well as IOT. NoSQL databases have been a strong contender for data storage and processing requirements of modern applications. These databases can be deployed as distributed systems, which makes them more reliable and faster while querying for data. They are inherently more scalable than SQL databases as they allow you to scale vertically and horizontally whereas most SQL databases are vertically scalable by design.

In the technology industry, there can never be a “one size fits all” solution and hence one has to be ready to adapt to different technologies to solve different business and technical problems. At Cazton, we work with different kinds of database technologies and have more than a decade of experience in creating highly scalable and efficient systems that use multiple database technologies that are complementary to each other. We have helped many Fortune 500 clients, mid-size companies and startups create a fool-proof data strategy.

Over the last couple of years, we have seen tremendous growth and demand for both Azure Cosmos DB and MongoDB. Both these technologies are promising and offer features that often complement each other. However the question still remains, which among the two performs better? To analyze and understand which one is better, we ran benchmarking code which explicitly performed CRUD (Create, Read, Update, Delete) operations against both the databases.

What is Azure Cosmos DB?

Azure Cosmos DB is a globally distributed, multi-model NoSQL database service that provides strong consistency, extremely low latency, high availability, throughput and up to five consistency levels backed by solid service level agreements (SLAs). The core idea behind building such a database technology was to make it horizontally scalable and globally distributed. This means that the data gets much closer to the customers as it is replicated and stored across multiple Azure data centers in various regions. Azure Cosmos DB guarantees less than 10ms for reads and less than 10 ms latency for writes [MB1] less than 1Kb of data. Note that all reads and writes are served from a local region, and then replicated according to the selected consistency model. Learn more.

What is MongoDB?

MongoDB is a document-based distributed NoSQL database that can be used for modern, distributed and cloud based applications. It stores data in BSON format - binary encoding of JSON-like documents, and supports arrays and nested JSON objects. It offers SQL-like features like ACID transactions, ad hoc queries, joins, indexing, and much more.

MongoDB offers an option to run it as a service. This means you don't need to worry about setting up physical hardware, installing software or configuring for performance. MongoDB Inc., the company which makes the software for MongoDB, has a service called Atlas. You can leverage MongoDB Database as a Service on any cloud platform like Google Cloud Platform, Azure or AWS. However if you wish to keep it local, you can also install it as a service on Windows and Linux platforms. Learn more.

Benchmarking Azure Cosmos DB (SQL API) and MongoDB service on Atlas

The benchmarking effort aims to examine both Azure Cosmos DB (SQL API) and MongoDB service on Atlas and compare their performance using intuitive and easily repeatable scenarios in a cloud-based infrastructure. Using Azure and explicit CRUD operations we were able to create a benchmark that allowed each technology to shine and put its best foot forward. TCO estimates were created using public performance information and a straightforward, data-driven approach to calculations.

• Machine Specifications: We ran the benchmarking code on an Azure VM in the same region as the databases. Both Azure Cosmos DB and MongoDB service on Atlas were running in the West coast (US).
  
  

  Cloud Provider	Azure
  Machine Size	Intel Xeon Platinum 8272CL CPU 2.60GHz
  Cores	1 CPU, 8 logical and 4 physical cores
  Image	Ubuntu 18.04
  Software Development Kit	.NET SDK 5.0.301
  Host	.NET 5.0.7 (5.0.721.25508), X64 RyuJIT

• Strategy: The benchmarking approach included warmup steps to ensure the database instances operated in a representative manner and repeated operations to ensure a statistically significant set of results, including percentile grouping analysis. For the benchmarking effort, we tested CRUD (Create, Read, Update, Delete) operations on Azure with the database technologies and benchmarking code running in the same region. We tested common and uncommon scenarios to assess performance in Microsoft's Azure environment.
  
  We utilized BenchmarkDotNet and NBomber to simulate various load scenarios on the databases. The CRUD operations were performed with the following payloads: 1KB, 10KB, 100KB, 1MB, 2MB. Azure Cosmos DB has a payload limit of 2MB. We also made sure that both databases would simultaneously get hits with different user counts ranging from 1 to 1000 users.

Benchmarking Results

• For 1 KB Payload: Azure Cosmos DB performed better for create, update and delete operations whereas MongoDB did well for read operation.
• For 10 KB Payload: Again, Azure Cosmos DB performed better for create, update and delete operations whereas MongoDB did well for read operation.
• For 100 KB Payload: Here Azure Cosmos DB performed better for create and read operations whereas MongoDB did well for update and delete operations.
• For 1 MB Payload: As the payload size increases, Azure Cosmos DB continues to perform better for read operation whereas MongoDB exceeded in create, update and delete operations.
• For 2 MB Payload: MongoDB completely outperforms Azure Cosmos DB in this scenario.

Findings

As you can see for the most common use-cases with a payload size weighing 1KB, 10KB and 100KB, Azure Cosmos DB performed better than MongoDB. But as the payload size increased, MongoDB showed promising performance.

Azure Cosmos DB Benchmarking Result

MongoDB Benchmarking Result

Total Cost of Ownership (TCO)

Beyond throughput, there is the practical question of Total Cost of Ownership. We reviewed an analysis that maps Azure Cosmos RU/s to native MongoDB cores for various workloads with the primary goal of estimating Total Cost of Ownership (TCO). Results suggest Azure Cosmos DB API for MongoDB is best utilized for high-storage, read-heavy, variable-traffic workloads.

• Approach: To determine the RU/s for equivalent performance across Azure Cosmos DB for MongoDB API and given MongoDB Atlas SKUs, utilize Yahoo! Cloud Service Benchmark (YCSB), publicly available pricing data, and instances running on Azure Cloud. For MongoDB Atlas, investigate the impact of consistency configuration options on performance and TCO. The performance analysis targeted Azure Cosmos DB for SQL API, compared to this analysis which focuses on the MongoDB API and thereby accounting for a software solution that may migrate from one to the other based on the results.
• Workloads: Considering the different distributions of workloads, throughput is estimated based on following breakdowns of traffic:
  
  

  Writes	Reads
  90%	10%
  50%	50%
  10%	90%

• Throughput: The estimate is based on workload on an Atlas M40 instance.
• Storage:
  • Low storage: 100GB - throughput dominates TCO
  • Balanced storage: 4TB - max size of unsharded MongoDB Atlas
  • High storage: 25TB - sharded; storage dominates TCO

  For sharding, consider large M50 shards and small M30 shards (Azure Cosmos DB, however, is always sharded with 50GB shards and has no extra cost for sharding).

• Elasticity: For "high-throughput/low-storage" and "balanced scenarios" where throughput impacts TCO substantially, utilize traffic distribution:
  • 66% of the time, throughput is 33% of peak
  • 33% of the time, throughput is 100% of peak

Total Cost of Ownership (TCO) Results

Low Storage:

With low storage (~100GB), utilizing an unsharded Mongo M40 instance. MongoDB Atlas Azure M40 256GB estimated price was $1,095/mo.

Balanced throughput/storage:

Using the same throughput estimations as above in terms of ops/sec and RU/s, but with a larger customer data size of 4TB, which requires an increased cluster tier to M50. MongoDB Atlas Azure M50 4TB estimated price: $3,869/mo.

High storage:

Using the same throughput estimations as above, but scaling with an even larger customer data size of 25TB, which necessitates 7 shards of an M50 tier. MongoDB Atlas 7x sharded M50 w/ 8 vcpu, 4TB data disk per shard price: $27,536/mo.

Findings:

What these results show is a marked difference in price per month for an equivalent Azure Cosmos DB solution vs MongoDB Atlas for a variety of storage and workload scenarios. Low storage and high writes favor MongoDB while higher storage and higher reads favor Azure Cosmos DB in terms of how the prices scale, but the overall finding is that in this estimation Azure Cosmos DB is significantly cheaper for Total Cost of Ownership.

Azure Cosmos DB has five options for consistency. Eventual was used for the testing above. MongoDB separates the concerns into write concern and read preference, which adjust performance and throughput, and therefore also result in price changes in our scenarios due to changes in operations per second and estimated RU/s.

Read preference:

For example, switching read preference to SecondaryPreferred [MB4] on a 3x replicated MongoDB Atlas setup allows clients to read from either secondary replica, potentially halving read cost and therefore doubling read throughput for the same cluster tier.

With that setup in our low storage scenario: MongoDB Atlas Azure M40 256GB estimated price: $1,095/mo.

The increased operations/second (and RU/s), leads to an increased Azure Cosmos DB price but the pricing in comparison still favors Azure Cosmos DB. This is true for the balanced and high storage scenarios as well.

Write preference:

We can go one step further and also adjust write concern. Write concern governs the proportion of nodes in a cluster that must acknowledge a write to that cluster. This can directly lead to a lowering of insert cost in a similar 3x replicated cluster as above.

With write concern 1 and read preference secondaryPreferred in our low storage scenario: MongoDB Atlas Azure M40 256GB estimated price: $1,095/mo.

Note the further increase in operations/second (and RU/s) and resulting change in the Azure Cosmos DB price. Again note the comparison continues to favor Cosmos DB, which is also true for the balanced and high storage scenarios.

Overall, Azure Cosmos DB had admirable performance and succeeded at maintaining high throughput across the various scenarios. TCO remained modest as scale increased, compared to fixed costing of other solutions. These findings indicate that Cosmos is the more performant and cost effective solution, except for scenarios over 1 MB (which is not a typical use case). Obviously different needs are filled by different technologies, but Azure Cosmos DB has shown it is an effective solution for a wide range of needs and will satisfy high throughput workloads.

Disclaimer: These results are based on the benchmarking done on July 15, 2021 using intuitive and easily repeatable scenarios in a cloud-based infrastructure. Results and pricing can vary over time.

---

Our team has gained expertise all over the world and in all fields of the tech industry. They can quickly identify, predict, and satisfy our clients' current and future needs and that has enabled us to build our own framework, which is robust, flexible, scalable, increases speed of development and implements best practices. At Cazton, we believe that it is easy to do the right thing in business because there’s not much competition in doing so. We help all types of companies ranging from SMBs to fortune 500s in their digital transformation. And our clients trust us to provide them with the knowledge and skill to tackle every challenge and succeed at every opportunity.

If your company is building applications using Big Data and Artificial Intelligence or creating web or mobile applications on cloud, microservices or on-premises, please check our services to find out why Fortune 500 companies choose Cazton for building multi-billion-dollar revenue generating applications.

Cazton is composed of technical professionals with expertise gained all over the world and in all fields of the tech industry and we put this expertise to work for you. We serve all industries, including banking, finance, legal services, life sciences & healthcare, technology, media, and the public sector. Check out some of our services:

• Artificial Intelligence
• Big Data
• Web Development
• Mobile Development
• Desktop Development
• API Development
• Database Development
• Cloud
• DevOps
• Enterprise Search
• Blockchain
• Enterprise Architecture

Cazton has expanded into a global company, servicing clients not only across the United States, but in Oslo, Norway; Stockholm, Sweden; London, England; Berlin, Germany; Frankfurt, Germany; Paris, France; Amsterdam, Netherlands; Brussels, Belgium; Rome, Italy; Sydney, Melbourne, Australia; Quebec City, Toronto Vancouver, Montreal, Ottawa, Calgary, Edmonton, Victoria, and Winnipeg as well. In the United States, we provide our consulting and training services across various cities like Austin, Dallas, Houston, New York, New Jersey, Irvine, Los Angeles, Denver, Boulder, Charlotte, Atlanta, Orlando, Miami, San Antonio, San Diego, San Francisco, San Jose, Stamford and others. Contact us today to learn more about what our experts can do for you.