Upgrade mongodb from 3.2 to 3.4

Here's a quick guide on how to upgrade mongodb version on a sharded cluster hosted on AWS, AmazonLinux OS.

This is done, manually, on a cluster with ~30 TB data with minimal downtime (less than a minute).

Mongodb components:

• mongos: Server responsible for routing requests, balancing chunks in shards and entry point for all management.
• config servers: A replica set of primary-secondary-secondary nodes that store data of chunk-shard mapping in addition to other metadata required for cluster to function.
• Shard servers: Replica sets (each set with primary-secondary-arbiter/secondary). They hold the relevant data and respond to queries from mongos.

What do we want:

All mongodb versions have server and clients with v3.2.11. Requirement is to upgrade them to v3.4 (latest stable version).

How do we do it:

• Check out release notes and see if there's any breaking feature between versions. This is manual and has to be thorough.
• stop balancer on mongos. The chunks shouldn't run around when you're doing the upgrade.
• Take config servers' secondary nodes..Upgrade mongodb on each of them.
• Go to primary config server, do rs.stepDown() and upgrade mongodb on primary too.
• For each replica set, update arbiters first. Update secondaries and do rs.stepDown() and update mongodb on primaries.
• For mongos, stop the process. Update mongodb version. From mongos.conf, remove autosplit and chunkSize lines. Start the process.

Upgrade process on each instance:

• Create a file /etc/yum.repos.d/mongodb-3.4.repo with contents:

[mongodb-org-3.4]
name=MongoDB Repository
baseurl=https://repo.mongodb.org/yum/amazon/2013.03/mongodb-org/3.4/x86_64/
gpgcheck=1
enabled=1
gpgkey=https://www.mongodb.org/static/pgp/server-3.4.asc

• Run sudo yum install -y mongodb-org to install this version. It's recommended to stop mongod process while executing this command so that yum can cleanly replace binaries.

• In mongod.conf file, set the parameter:

sharding:
   clusterRole: <value>

where value could be either of configsvr or shardsvr depending on what's the role of server in cluster.

• Restart mongod process with sudo service mongod restart and check if desired version is running by logging into shell, mongo.

To make this process faster, you may choose to wrap up upgrade process in a single shell script and execute it in all servers one by one.