To make your project as fault-tolerant and scalable as, for example, Spotify, you need to migrate to Kubernetes. There is a lot of controversy around it, but at the moment it is one of the best tools for containerising large projects. And, it must be said, the task of moving to it is not the easiest.
Our specialists have already accumulated enough experience in working with projects that at different stages needed Kubernetes implementation or support. But in order to competently implement such a large-scale process from any location, it is important not just to ask the client's team what is already ready or what is required from us. You need to have an understanding of all phases of the work so you don't miss anything important.
If you are planning to implement Kubernetes containerisation on your project, we offer our article to familiarise you with the main steps of this process. This way you will understand what to be prepared for and how long it will take. In this article, the Initlab team will share their experience of migrating various projects to Kubernetes and provide a step-by-step plan for a competent move.
11 stages of migration
Any description of such complex processes is just an "example in a vacuum". We have compiled our list based on our own experience and present it as a kind of ideal from our point of view. A successful migration should follow the following steps.
1. audit and document the current server infrastructure of the project. At this stage we determine what applications and on what servers are running, what technology stack is used. In case we are invited to a project with already launched process of migration to Kubernetes, we have to estimate the amount of work done and ready scripts that can be reused. The result of the audit is a document with a description of the current infrastructure and interconnections, as well as an assessment of the status of the migration process.
2. Formulating requirements for application scalability and fault tolerance. Here we find out whether additional application instances need to be launched when the system load increases. In addition, it is important for us to understand whether we need to run application backups to improve system reliability.
For example, we have an online shop that gets a lot of load on Black Friday. To prevent the site from slowing down and crashing, we need automatic deployment of additional web servers, which can be implemented with Kubernetes.
3. Preparing the application to run in a Kubernetes cluster. In this step, we check if the application meets the requirements of a 12-factor application. We described this concept with technical details using the example of running Drupal in Kubernetes in this article. If necessary, we refine and finalise the customisation of the application to meet the requirements. At this stage, our DevOps-engineers need to know the answers from the development team to the following key questions:
- does the application run without saving state? (stateless).
- does it store the state of the user interaction? (stateful)
For stateful:
- how is work with user sessions organised?
- what are the dependencies?
- how is the app set up and how is it run?
4. Containerisation of each application. Create Docker containers. If possible, use ready-made base images.
5. Design and implement a log collection and monitoring system for the application and cluster. At this point you need to decide whether it is necessary to organise log collection centrally.
6. CI/CD Automation. It is required so that developers can write application code that, as changes are made, will be promptly delivered to the cluster so that the relevant applications can be built and restarted correctly. In other words, so that any changes and developer commits are enforced through automation.
7. Prepare environments and migrations, create Kubernetes manifests. At this stage, the Kubernetes cluster is configured, the DBMS, queue software and other software required for the application are configured. A manifest for each service is prepared for its deployment into the Kubernetes cluster. Deployment of all services and application operation is launched and tested.
8. Testing under load, testing in case of failures. Does the cluster perform as designed? Does the current infrastructure support the required load/number of requests? This stage tests the failure of cluster nodes, how long it will take to move pods to a working node, and also tests the failure of one of the nodes in the clusters of DBMS, queues and similar software.
9. Application migration and launch. Services are updated by launching deploys for the latest versions of services, data in DBMS and files in external storages are updated. In this case it is required to suspend the application work for the time of actualisation so that the data on the old and new stand are not out of sync. It is also possible to switch the application work without pausing when the DBMS replicas are configured on the new stand. At the time of switching, they are put into the master role. Then the production domains are switched to the new cluster.
10. Final testing after launch. After launching the application on the main domain, testers perform full testing of its functionality in order to find bugs and further fix them. In case they are too serious and prevent further work of the application, their correction may take a long time. In this case, most likely, it will be necessary to switch back to the old stand and fix the errors found.
11. Monitoring and support of Kubernetes cluster. Monitoring is a regular check of performance of all Kubernetes cluster components: system components, as well as components running pods with services. Their operation is checked, as well as the number of restarts. If a service restarts frequently, it indicates problems that need to be eliminated. All additional software for the application operation is also monitored - DBMS, queues, storages, etc.
Cluster support - processing problems found through monitoring. In a developing project - adding new services: containerisation, manifest writing, CI/CD writing, deployment. Also integration of testing into CI/CD processes, integration of vulnerability checks in the code of services and docker images.
Canary releases
In order not to be afraid of large-scale changes when migrating applications and startups, Kubernetes allows us to deploy and rollback services of new versions. This means that we can perform so-called canary releases - partially launch a new version and migrate some users to it. This helps us to evaluate load changes, track errors, monitor the project status and, if everything is going well, smoothly continue the migration to the new version.
This process is automated. In case of critical problems, you can also automatically roll back to the previous version to correct errors.
Our cases
Of course, our experience is not limited to three case studies. We have selected them to demonstrate the entry to the project at different stages.
Project #1. Application for interaction with the car
Industry: car app: driving, refuelling, route tracking, fines and parking bills. Each functionality is a separate service that is launched in a cluster.
Language, application platform:
- Node.js - front and back end services;
- Python is the backend;
- PHP (Laravel) - backend.
What steps have been taken:
At the start of the project, all microservices were run separately on one single server. At some point, the number of services increased dramatically, to about 50, which made it difficult to manage them.
It took:
- design the architecture, migrate and launch the project from scratch;
- scale some microservices to speed up data processing;
- migrate from one cloud to another, which we also linked to the cluster configuration;
- to improve fault tolerance because all systems were in a single instance. One server for the application led to the fact that the failure of any server resulted in the unavailability of the entire application. Therefore, a cluster was deployed, where a pool of servers was allocated for services, and all systems (DBMS, queues) were deployed in cluster form.
We have also configured availability monitoring when starting services in the cluster. It checks the availability of services and notifies about problems in the cluster. Migration to the cluster allowed us to flexibly allocate server resources for services. A pool of servers with services running on them was allocated, and if one of them starts to consume a lot of resources, services are redistributed on nodes so that other services continue to work and the most resource-intensive service does not interfere.
CI/CD is configured in the cluster. For each service there is a check of its operability, and if incorrect functioning is detected, it is restarted.
Developers have an easier time: they can make their own changes in the application launch process.
Implementation period: 6 months.
Project #2. Tourist portal
Industry: travel planning service
Language, application platform:
- Node.js - front and back end services;
- Java backend.
What steps have been taken:
We had a task: to deploy a cluster from an existing dev cluster for a prod cluster, taking into account load and fault tolerance. The application was already developed.
It took:
- design the architecture of servers and services (DBMS, queues, cache) under prod;
- configure fault tolerance of DBMS, queueing and caching services;
- configure the servers for the k8s cluster;
- configure CI/CD for the cluster prod.
In this case study, a different service is responsible for the separate functionality of each microservice application. Front is a separate service, i.e. a display that accesses a large number of backend services. On the site there can be conditional hotels, cinemas, planes as separate services, which, when interacting through the front, start to address their correct backend service. Usually Kubernetes is used for applications that are divided into front and back, as well as for a large number of backend services.
A backend is a service that has no graphical interface. It responds to a request with text, from which the frontend builds the final display. For example, if we are interested in a theatre, the backend will respond with an array of performances with names, posters, and booking urls.
Implementation period: 6 months.
Project #3. State portal
Industry: public procurement
Language, application platform:
- Java
What steps have been taken:
The "big data" service in the portal ecosystem was intended to handle data that is so sensitive that we cannot refer to it. An archive with code and deployment scripts was handed over to us from previous developers.
It took:
- to deploy dev in the first place;
- then unfold the prod;
- fix deployment scripts, as all this was developed a long time ago and software and library versions are outdated in service dependencies;
- set up CI/CD for automatic deployment.
It took a long time to deploy the first dev stand, until we got in touch with the developers and they helped us to build the services. We had to put our own repository for mvn libraries and build libraries that are dependencies in services. After launch we were fixing service configurations and setting up integrations between them, and it was very difficult without knowledge of the project.
The prod was then rolled out in literally a couple of weeks as everything was updated and working.
Implementation period: 6 months.
A little more about the deployment timeline and the launch process
It may seem that we got some of the projects halfway through, but in fact all three cases were cluster deployments from scratch. And although in two cases we received scripts and code:
- there were infrastructural difficulties in Project 1, it was implemented together with project support, which delayed the relocation;
- in Project 2 we had to think through and organise the architecture + the process was quite long, because development was going on in parallel and for several months we were adding services and fixing the current ones. It took a long time to get to the ideal.
If all applications are containerised, i.e. 5-10 steps are performed, in big data experience, it can take about two weeks to migrate a finished working project.
Sometimes part of the third point is done by the customer. That is, we enter the project, audit the finished scripts, audit the quality of containerisation, but this doesn't happen very often - we mostly work from scratch. Without immersion in the project, you cannot be sure how well everything is already working and predict how the process will go in the future.
Starting at step nine, one of the most critical steps, is dependent on the architecture of the application. If it can be divided into parts so that one part runs in one place and the other in another, you can migrate partially and gradually fix migration errors.
But if all services represent one application and communicate with each other, as in Project #1, then the entire application is moved. In this case, it is very important to be able to test the new environment.
When migrating, we need to update data on the new cluster and then switch traffic to it. There are cases when we can do it without stopping - then we need to configure replication of all data to the new cluster, and there are cases when stopping is necessary.
It is very difficult and close to impossible to say in advance how long the migration will take. Projects that require Kubernetes implementation are mostly very large, where everything cannot be thought out and foreseen. Especially when devops does not know the nuances of the application and what it needs to work.
In the case where we are engaged for the entire phases from the first step to monitoring and support, we have to work closely with developers, conduct audits, document the application system and design the cluster, leverage Initlab infrastructure services for monitoring and CI/CD of Kubernetes cluster if necessary, or implement new ones according to the customer's requirements.
To migrate to Kubernetes or not?
There is no unambiguous answer to the question of transition. We can only speak with certainty about projects with large data sets and number of applications. As the number of the latter increases, server performance may become insufficient, and containerisation will be the only way out. Those who plan to expand and those who are already on the edge of their capabilities should definitely consider moving to a Kubernetes cluster that works on any platform.
If you are planning to move your project to a Kubernetes cluster for scaling and improving fault tolerance or your project needs k8s cluster support - contact us.
Add new comment