DevOps best practices and cases

12 Factor App: Less nerves for developers and managers

After frequently repeating the same actions, we come up with some templates and algorithms of actions that speed up the processes and make our lives easier. 12 Factor App has collected the experience of development and deployment of applications, which were done by its members while working on the Heroku platform.

These techniques will make your application ready for:

  • cloud hosting
  • horizontal scaling
  • continuous deployment (more in this article)
  • well-coordinated work of developers on its improvement and growth.

1. One app - one repository

Regardless of the size and importance of the changes, the code must be in git. And deploy to the prod, dev, test servers from the same repository.

The shared code used by several applications should be taken to a separate library and declared dependent. Only strongly linked applications should be located in the same repository. For example, the frontend and backend can be put into separate applications, which will follow the principles of the 12 Factor App.

2. Explicit dependencies

Applications should not have implicit dependencies, even if the library without its version does not cause problems for several releases.  Therefore, all system dependencies (OS build version), infrastructure parts (database, message queues, reverse proxy), programming languages, and library versions with packet manager (Python - pip, Node.js - npm) must be specified in the code.

In addition to the possibility of running the application on different platforms, this practice will help new developers to work on the project, rather than spend weeks searching for the appropriate versions of libraries.

Note: Docker isolates containers and explicitly declares dependencies, we recommend to specify the version explicitly and not to use latest tag.

3. Сonfiguration in separate files

The configuration is all parameters that change depending on the environment (where we launch):
  • logins, passwords and database addresses;
  • network address and port number
  • 3rd-party services and API keys.

It is good practice to store configuration files for each environment. Logins, passwords and other confidential data cannot be stored in a repository. They need to be added to the secrets management systems such as Hashicorp Vault or encrypted  with Ansible Vault. A good with the description of all parameters is much better than several knowledge transfer sessions.

A quite popular option is to store the config files in a separate directory in the same repository. And secret files should be stored in Hashicorp Vault or AWS Secret Manager.

4. Backing services and 3rd party

Backing services are any third-party service that is available over the network and is required by the application for normal operation. For example, databases (MongoDB, PostgreSQL), mail SMTP servers (Postfix), message queue (RabbitMQ), various APIs (Facebook, Twitter, Google Calendar), etc. 

Each 12 Factor application should be able to replace local service with 3rd party, for example, local PostgreSQL database to Amazon RDS. In this case, we must change only the variables in the configuration file, without touching the app code.

5. Deployment stages separation

Deployment of the application is performed in 3 steps:
  • Build - transform the code into an executable package, upload dependencies, compile files, run tests.
  • Release - combine the build with the configuration
  • Execution - launch the release and its processes

The build is started by a developer or DevOps engineer. It is desirable that the execution phase is automated and as simple as possible so that in case of emergency (server restart) our application will continue to work without any help from developers.

6. Processes

The application runs in a runtime environment as 1 or more processes. Its processes must not keep an internal state so as not to lose it when restarting (stateless). An application can use RAM or file system only as temporary storage, for example when reading files. Any user data should be stored in a permanent storage, e.g. a database.

Especially with session data, it is now very common to cache user data in process memory. 12 Factor App recommends not to use them because data can be lost due to a hardware failure or processor reboot. A great replacement for a sticky session is Redis or Memcached.

7. Port binding

An application can be run as a module inside a server, for example, a Java application inside Tomcat. However, the 12 Factor App must be completely self-sufficient, so we will declare the web-server library or framework as a dependency (Flask for Python, Spring in Java).

In this way we export our HTTP application to the world, binding it to a specific port. However, we can do the same with other types of applications, regardless of their protocol, such as ejabberd or Redis. And of course, with a link and port, we can refer to an application as a backed service.

8. Concurrency

12 Factor App recommends using different processes for different tasks. For example, HTTP requests are processed by a web process, while heavy and long tasks are processed by a worker, a background process.

This approach shows itself perfectly during the horizontal scaling (increases the number of machines), when each server performs one task or a group of related tasks. And application processes of 12 factors should not become beyond the control of the process manager (to demonize) but should react to a fall and handle reboot or shutdown events.

9. Fast startup and graceful shutdown

If the processes are stateless, they can quickly start and finish without errors, even in emergency situations. This helps stable and flexible scalability, high reliability, and fast new code and configuration deployment.

In case of a sudden termination, the workers executing background queries should return the current task to the message queue (RabbitMQ), and the web request processes should end them correctly. Also, the 12 Factor App should be able to repeat tasks and handle unexpected errors.

10. Similar dev and prod environment

Unless you use Docker and other containerization tools, your dev and prod environments may be very different. Especially if you use different tools at different stages, the 12 Factor App, on the other hand, will help to solve the main points:
Time: the code will get into the prod environment a few hours after it is written with the CI/CD system
People: the developer is involved in the deployment of his code or always in contact with DevOps engineer
Tools: the development and testing environment is the same as the production environment.

Now this problem is much rarer, thanks to Docker containers and declarative environment preparation tools like Ansible and Chef.

11. Logs

A log is a flow of events, so you should work with it in the same way instead of starting separate files. 12 Factor App recommends the application not to store and route logs but to write everything to stdout. A developer can view logs in the terminal, but we strongly recommend choosing a convenient tool for analysis and archiving, for example, Splunk, Fluentd, or Logstash.

This approach allows aggregating applications and 3rd-party logs (database, message queues). Thanks to the log analysis system, you can get valuable information about the state of the system and its parts, and see peaks of activity.

12. App administration tasks

Single and repetitive administration tasks (migration, database fixes) must follow the same rules as the other processes:
  • the task code is in the app repository (point 1)
  • all dependencies are explicitly declared (point 2)
  • the configuration is written into environment variables (point 3)

Before choosing a new tool or following the methodologies and manifesto, it is worth analyzing its necessity and impact on the project. Fortunately, the 12 Factor App points are weighted, confirmed by the creators' experience, real life cases, and our own personal ones. Its application will speed up your development, simplify the life of developers and help to scale the project.

If you still have questions or want to implement infrastructure the 12 Factor App, please contact us.