Microservices in Practice

by Chenglim Ear, Senior Technical Lead


At Ask Media Group, we’ve transitioned from building monoliths to building microservices, and that transition has been an eye-opening experience. The ability to quickly build new services and understand existing ones is one of the key benefits in practice when working with microservices. It changes the way we work and the way we work together. For example, it has empowered our data scientists to quickly and easily build systems, leveraging models they’ve created for classifying queries. It has also enabled non-scientists to help our data scientists build the services. With a platform in place for building microservices, we’ve opened the door to exploring new ways to collaborate on a smart, fault-tolerant and highly scalable system.

When developers join the company, they are given a guide to our micro-component development platform, called Kram, which is used to develop microservices and more. In this guide, we give developers instructions for easily building a microservice and deploying it to production, all within GitLab. Developers can immediately begin to contribute a new service that can be deployed to production as soon as they start.

Micro-components are developed and deployed with pushbutton automation

We’ve moved away from lots of code in a few big things to lots of little things, and so the software development domain has changed. There’s a new set of best practices for developing software that we are discovering. In some ways, it’s like going from sculpting masterpieces out of large blocks of stone to building amazing sand castles out of grains of sand. One very important practice that we’ve employed is the practice of watching everything. We watch when services start and stop, and when they fail. We watch the failures and response time of every request, and we watch the amount of traffic each service is getting. This is done using technologies like OpenShift Origin, a flavor of Kubernetes, and linkerd, a proxy built on top of Twitter’s finagle. Every microservice consists of OpenShift pods and each pod is automatically paired with a linkerd container that is used to proxy incoming and outgoing requests. This allows us to watch the requests and send the metrics to our monitoring system. We have real-time graphs for everything and an alert system that monitor the metrics for failure conditions.

We use Grafana to visualize our production metrics per microservice.

While we have the usual process of incident escalation in place, a new practice for problem resolution has emerged from our experience building microservices–the practice of building self-healing services. Because our microservices are built on top of OpenShift Origin, services are deployed as Docker containers and mechanisms exist for watching the health of these containers. By leveraging the hooks provided by OpenShift, a service can let OpenShift know when it’s unhealthy and whether it requires a restart or just a temporary traffic shift away from the container. Since the functionality of restarting and shifting traffic is already handled by OpenShift, a service developer can focus on building the core functionality of the service.

In practice, we find that much of the operational functionality of a service can be automated, including the shifting of traffic away from bad pods to good pods. Our deployment process is literally a click of a button, and we achieve safety not by requiring tickets and approval for a deployment, but by automated checks and monitoring of all deployments. Because our services are built following some conventions, such as which port the servers listen on, we are able to build common tools for controlling and interacting with the services. Activities such as sending test traffic to a service and tuning its resource usage are enhanced with these tools. With all the supporting automation and tools in our micro-component development platform, a new service can literally be built and safely deployed to production by one person in a few minutes.

Building microservices is transformative and it aligns well with agile development practices. When we started, we only meant to build a microservices framework, but soon discovered that building microservices is not simply about following a new architecture. As we built our microservices, we practiced a new way of building systems and developed new practices from our discoveries. While we attempted to address issues and concerns that surfaced, it changed us and we began to see and think about things differently. To this day, it continues to transform all of us and help us be more agile.

After we successfully built our microservices on Kram, we started to promote Kram as a micro-component development platform instead of simply a microservices framework, to bring attention to its transformative potential. We have the opportunity to not simply follow a new architecture, but to embrace a new way of thinking. We can leverage the Kram platform to build any component the microservices way. For our developers who use Kram, there’s opportunity for exploration and innovation. New developers who join the company have an opportunity to be a part of the innovations and to work on the cutting edge, using Kubernetes, Docker, linkerd and all the new technology that we’ll introduce as we leverage the Kram platform. Learning the new technologies and practices of tomorrow is a cornerstone of our success at Ask Media Group.