Devops 101

I've been mentoring a few people in devops lately. Here's a glimpse of what I say on call when I introduce the whole devops paradigm.

This is non-exhaustive and meant for beginners. I'll add more to this with time.

There's a signup form for an intro+QnA session of the same at the bottom if you're interested.

Devops:

Traditionally developers and operations team were sitting in different places and there's a communication gap btw both parties over time as the person developed it and operating it are totally different. Devops aims to build a culture where developers take part in operations and operators take part in development, hence building the software holistically in a faster, reliable, secure way.

CI/CD:

Continuous Integration: Ensures every new code developer submits works seamlessly with existing code. This means, new code can be integrated continuously into existing code without breaking anything.

Continuous Delivery: Everytime there's a new code change ready for release, a build shall be made and automated tests are executed. This step ensures correctness, compilation as well as makes every release packaged and ready to be shipped.

Continuous Deployment: Taking further from above step, this stage ensures as soon as new package is ready, it gets shipped automatically (without any human intervention)..this can include steps like deploying first to staging/QA/performance environments and once validated, deploying to production.

Tools: Jenkins, Github Actions, Gitlab CI, GoCD, Spinnaker, etc.

Infrastructure provisioning:

When we say package is ready to be shipped, we need a infrastructure to execute the package, and use it. This step ensures we've desired infrastructure in place. This can happen before CI/CD in most scenarios.

Tools: Terraform.. infrastructure generally resides on cloud services like AWS, Google Cloud, etc.

Monitoring

Logging: Centralized logging ensures logs from various servers/places come to one place for quick querying and debugging.

Monitoring: Ensures we've relevant dashboards to understand how our system is behaving. This can help in debugging and improving performance.

Alerting: If something goes wrong, the software should alert people with an automated call or email. Alerts are generally set up with a thresholds (like requests are taking average 20ms instead of 10ms), anamoly disturbance, etc.

Tools:

• for centralised logging: Elasticsearch-fluentd-kibana, elasticsearch-logstash-kibana, Splunk, Humio, etc.
• for monitoring & alerting: Prometheus, Datadog, Newrelic, Nagios, Zabbix, Sensu, etc.

Docker and all that jazz:

If you're searching on google for devops, there's 11 in 10 chance that they'll talk about docker. Here's what all that's about.

Packaging:

While packaging software, there can be a configuration mismatch, or a drift in versions of underlying softwares. Something like developer has updated java version unknowingly and that's why it's working only on their machine. It works on my machine is a common joke that goes around in tech. To address this issue, people typically resort to container technologies like docker.

Every container is a process in linux machine, but it is isolated from other processes.
Think about it like an examination room with room full of students. While the whole hallroom+invigilators is a linux operating system, each student is a container and strictly don't share resources with others.

*P.S.: *The only difference is students in exam room can't talk to each other whereas containers can talk to each other for various reasons. Sorry for the bad example here. :P

Orchestration

Given that we can put any container in any machine without dependency on underlying operating system, it gives us flexibility to utilise our machines efficiently. We'll purchase only desired machine sizes to fit them all such that we don't have any excess nor are we starving any container. This mechanism of putting containers in appropriate machines is called scheduling.
Orchestration tools like kubernetes, docker swarm, ECS make this easier by handling scheduling and solving other challenges that come along with it.

Other advantages of orchestration are we can run a single logging agent, single monitoring agent, etc. for all the processes running in a particular machine.

Taking from above example, this is equivalent to allocating students in respective exam rooms in an effective way. And putting enough inviligators in each room as needed. ;)

Hope this helps. You may drop your questions to me at [email protected] or DM on twitter @acsrujan