Database

Integrating microservices with Mongo.

Overview

Mongo is a document database. It's been chosen for it's ease of use and integration into applications, it's flexibility for unstructured data, as well as it's variety of features with the Atlas platform.

Purpose

Some of the key features that are used include: materialized views, Atlas search, and Atlas streams. The combination of features provided by Mongo greatly enhances the user experience.

Materialized views

Different from views, materialized views allows us to pick and choose data from various collections and store the result to disk. The advantage here is that the view can now be utilized in Atlas search for search indexing.

Atlas search

Instead of using an other third-party integration such as Elastic search, Atlas search sits on top of the existing data in Mongo which provides similar results with less headache.

Atlas streams

There are actually two parts to this feature, Kafka stream processing and change streams.

Integration with Kafka and Mongo allows us to consume state topics directly into Mongo and store the data with no write transactions needed from the microservices, meaning the Mongo service users for the microservices can be read-only.

Change streams allows for the creation for dynamic

Collections

Collections are similar to tables in SQL ,they are the storage container for data. Below are a list of collections stored in Mongo:

• Organization
• User
• Profile
• UserGroup
• UserPreference
• Project
• ProjectConfig
• Dataset
• Sample
• Annotation
• Audit
• Message
• Template