Note: Intended for self reference. We've to delete clients/nodes from chef when an ec2 instance terminates. We've followed the pattern of using instance id as chef node id. And the following script helps to delete nodes that are terminated. The script runs every 15 minutes. #!/bin/bash # Scan for

Scenario: Mono-repo with multiple projects hosted on github.. We need a CI/CD setup for this, whenever a pull request is raised. Inputs: * Code hosted on Github Continuous Integration/CI server: * CI server shall merge incoming branch into master in its environment, when a pull request is made. * Run unit

An ongoing document on terraform clean practices I follow. Type 1: Ingress defined inside the same resource as security group. resource "aws_security_group" "sample-sg" { name = "sample-sg" vpc_id = "${aws_vpc.default.id}" ingress { protocol = "tcp" from_port = 443 to_port = 443 cidr_blocks = ["0.0.0.0/0"] } } type

Due to various reasons, mongodb secondary goes out of sync and might be dead/stuck in recovered state or replication lag grows beyond acceptable limit. Instructions 1. Make sure primary is up and running. Login to primary mongo, let’s say calendar-mongo-rs-1-1. 2. Check acceptable mongodb replication lag. You can

In a typical setup, we need a resilient way to track down the errors. To think of it, we're writing code and we're generating the error. Thus, we should know it even before a user reports. While tracking logical errors might be a technical challenge, it's not difficult to track

I want to launch mongodb sharded cluster with chef and terraform on AWS.. show me steps! Let's name our mongos cluster as "Roads". Assumptions: There's a private VPC in AWS with subnets in multiple AZs. There's a bastion host launched in same VPC for ssh connectivity, as mentioned here: GHOST_

Here's a quick guide on how to upgrade mongodb version on a sharded cluster hosted on AWS, AmazonLinux OS. This is done, manually, on a cluster with ~30 TB data with minimal downtime (less than a minute). Mongodb components: * mongos: Server responsible for routing requests, balancing chunks in shards and

Meant for self reference. This is a gist of launching an RDS mysql instance with terraform. It uses module from here: https://github.com/terraform-aws-modules/terraform-aws-rds resource "aws_security_group" "sample-mysql" { name = "sample-mysql" description = "Allows services to talk to sample mysql" vpc_id = "vpc-xxxx" ingress { from_port = 3306 to_port

Down to dirty blog on setting up VPC with terraform. This is to setup a new VPC in ap-south-1 with three public and three private subnets. It'll also take care of NAT Gateways, Internet Gateways, etc. What the script does/creates: * There is 1 vpc in ap-south-1 * There are 3

Assumption: You've a kubernetes cluster running on EKS, and all the kubeconfig, kubectl, aws authentication are handled. What's EFK? Elasticsearch: A search database with support for REST api queries. Fluentd: Data collector typically used for collecting logs in a unified manner. Kibana: Visualisation tool for Elasticsearch data. Data flow: pods

This is for self reference. You may find it useful too. Objective is to deploy a kafka + zookeeper cluster in a dirty way. This is going to be on AWS. Assumes you can launch an ec2 instance with latest amazonlinux AMI. Quick intro of kafka and zookeeper: * Kafka is an

Context switch of an engineer usually comes on the top when we're talking of productivity. The problem, unlike CPU, humans can't do context switches by simply putting things in RAM, etc. That's a limitation. Solutions usually work towards reducing context switches. Building enough layers of communication so that engineer gets