Database

Info

  • MongoDB (Non Relational )
  • Elastic search for document search //built on lucene
  • If relational db's needed (Postgres, MySQL/MariaDB), connect them using ORM or new graphQL/Prisma style
  • ETL(Extract, Transform, Load) tools,

Currently used version: 4.X (4.0.16)

WHY MongoDB ? (derived from huMONGOus)

  • JSON file database (less joins as related data stored together as JSON file) //like object oriented + transactions
  • flexible schema (Schema less)
  • Replicas : (Multiple copies) Master/ Slave concept -> disaster recovery, High availability and scalability (as read operations can be load balanced )
  • Sharding (distributed nature) - helps huge data spread across multiple machines, so memory & hardware power of machines are leveraged for processing data.
  • Rich query methods
  • Mongo shell is an javaScript interpreter
  • Special features like TimeToLive document index, capped collections , gridFS*..etc

MongoDB Variants (Community, Enterprise, Atlas)

MongoDB Community

Open Source

MongoDB Enterprise

Enterprise Edition Paper

Run at our infrastructure, extra advanced softwares & support.

1) Support

24*7 , emergency patches for MongoDB (SLA 1 hour)

2) Ops Manager (good)

(GUI + API) automate the deployment, monitoring and alerting, backup, and scaling of MongoDB.

Operator for Kubernetes

3) Security (good)

  • Kerberos and LDAP integration for access controls
  • auditing
  • PKI certificates, TLS, and Client-Side Field Level Encryption (currently in beta release).
  • Encrypted Storage Engine provides native, at-rest encryption in storage and backups, eliminating the complexity and overhead of integrating third party disk and filesystem encryption options
  • In-Memory Storage Engine delivers high throughput and predictable low latency for workloads with the most stringent response-time SLAs

4) MongoDB Charts

Fastest and easiest way to create visualizations of MongoDB data. You can create graphs and build dashboards, sharing them with other users for collaboration, and embed them directly into your web apps to create engaging user experiences.

5) MongoDB Connector for BI

use MongoDB as a data source for your existing SQL-based BI and analytics platforms such as Tableau, Microstrategy, Looker, and more

6) MongoDB Compass

(More features than community) GUI for MongoDB, lets you explore and manipulate your data

7) On-Demand Training

Online training & Private training delivered on-site

8) Customer Success Program

offers an initial onboarding process, as well as multiple check-ins throughout the year to ensure your apps are running properly

same above details : mongodb-enterprise-advanced , MongoDB Enterprise

MongoDB Atlas

Security Paper

Fully managed database as a service in custom cloud provider like google cloud , Amazon (AWS) , Azure ..etc (Available in Bombay, India)

  • All MongoDB enterprise features
  • No servers, patches, scaling, High Availability, Disaster Recovery & security issues.
  • Scale & configure machine powers runtime as needed. (granular pricing)
  • Continuous backup with on-demand Point in Time restore
  • network isolation by VPN, ip whitelisting, penetration testing & security audits..etc
  • query improvement suggestion

1) MongoDB stitch - (ISO & HIPAA compliance)

(Serverless architecture)

  • built in authentication (Email, API Key, Google/FaceBook, custom )

2) Document search

  • It is working on Full text search (on Lucene) //beta

3) MongoDB Atlas data lake

(Operational analytics, Machine learning & AI) //beta
Eg: spoke – a machine learning to answer questions & assign requests to right team

4) Key Server

  • Integrate with cloud provider key server for storing keys & multi-authentication

Atlas compare with DocumentDB/Cosmos

Sharding management

Sharding

MongoDB shards data at the collection level

  • If one shard machine goes down, other shard can still execute queries successfully. (good)
  • If queries do not include the shard key or the prefix of a compound shard key, mongos performs a broadcast operation, querying all shards in the sharded cluster. These scatter/gather queries can be long running operations. (careful)
  • You cannot change the shard key after sharding, nor can you unshard a sharded collection directly ! (careful)
  • query restrictions (careful)

Ideas to try

  • Can GIT be used with mongoDB storage to track / restore history points !!
  • Can GIT be used with file storage to track document modifications !! (Both cases schedular runs in a interval say 15 - 20min & commit changes)

(move this file content to mongoDB learning, point to that file)

Tools

Dev points

ETL

can synchronize data at real time between databases (so use for Postgres to MongoDB)