Cost optimization is one of the most important non-technical tasks when working with AWS. In previous articles, we described how you can save money on AWS. In this article, we will take a closer look at AWS Tags and cleaning with Lambda features.
Why use AWS tagsAWS Tags - key value assigned to a resource. Using tags in AWS Cost Explorer, we can see the costs for different periods on projects. Also, AWS Tags are used for several other tasks:
- Which employee is the contact person for this AWS resource?
- How many of our servers were updated with the latest version of the operating system?
- How many of our services have alerts been enabled?
- Which AWS resources are not needed during low load hours?
- Who should have access to this resource?
Before adding tags, it is worth making a plan to manage and add them.
AWS Tags StrategiesSelecting and planning the ideal strategy to use tags will take a huge amount of time - it will be an almost useless waste of time. So let's define the optimal one.
1.Understand tag typesAWS provides 4 types of tags: technical, business, security, and automation. Let's deal with each type in more detail:
- Technical - helps to identify the resource and simplify the work with it. For example, the name of an application or service, its environment (Ubuntu 20.04 LTS) and build version (1.3-dev).
- Business - to analyze the cost of each resource by teams and departments. For example, to find out what part of the money you spent on AWS for your new product.
- Security - to ensure compliance and compliance with security standards. For example, restrict access to meet the requirements of HIPAA or SOC agreements.
- Automation - to automate the removal or shutdown of unused resources in your account. For example, you can run a script that deletes unused servers with this tag.
2. Pick of the necessary tags and organizing processesDetermine the required AWS tags, considering your goals and the amount of resources in your account. You also need to determine the tag name format and assign responsible people for creation. In case of a small number of resources, 1 person will be sufficient, if you have several commands - delegate this task to them to determine the optimal number of people to complete the task.
3. Determine the name formatNaming convention negotiation of scalable AWS tags for your keys and values can be difficult, e.g. using snake_case or camelCase, which characters for spaces, etc. Be sure to read the AWS tag naming restrictions before choosing a single format.
A common template for the AWS tag format is to use lowercase letters with hyphens between words and colons for namespaces. For example, you may use something similar:
Tag key: cloudy:eng:os-version (company:team:tag)
Thus, first we specify the name of the company or project - cloudy, the command working on it - eng (engineering), and a specific property as the name of the tag: os-version.
4. Limit the number of AWS tagsThere are technical and practical limitations on the number of tags used. First, AWS has a limit of 50 tags for each resource. Moreover, it will be difficult for developers to keep many tags under control and remember them.
Fortunately, AWS creates some tags itself. For example, you don't have to store the EC2 instance creator, because Amazon adds the createdBy tag by default. Decide which tags you need and try to limit the creation of new tags
5. Automate AWS tag managementWith the growth of AWS resources in your account, it will become more and more difficult to maintain naming conventions and update tags. So you can use Terraform, CloudFormation, Cloud Custodian to manage AWS tags on your resources.
Amazon also offers tag policies, tagging by resource group, and resource tagging API to help you manage and assign tags in bulk. Automating as much of the tag management process as possible in the long run will result in tags of higher quality and easier to maintain.
6. Review and support AWS tagsTo keep your tags up to date, you need to review them. Depending on the amount of resources you deploy, it's worth setting a reminder to audit your tags quarterly, or selecting the person responsible for tags review and update each month.
Amazon Web Services provides a comprehensive document on recommended tag handling methods. Be sure to review it if you want to understand it better.
7. Cleaning up unused resources with AWS LambdaIn addition to automatically adding tags, you can use Lambda functions to remove long unused resources without tags, very often doing so for EC2 instances и EBS volumes.
Also if you use Lambda functions for production, it's worth working on their optimization.
Lambda functions deletionAmong the many technologies from Netflix you can distinguish Janitor Monkey for cleaning unused resources - AutoScaling Groups, EBS volumes and snapshot.
To get unused lambda functions use the tutorial from AWS on Python, and to remove functions use aws cli. If you prefer Node.JS, you can try this code.
Cost optimization and resource management automation for this are natural steps to reduce AWS accounts and better understand what the money is spent on. Tags and Lambda functions that clean up unused resources are the best way to do this.
Contact us if you still have questions or need assistance in reducing the cost of your infrastructure.