Creating backup for our data can be a demanding task. In Linux OS it can be done manually through SSH , or through scripts if we need some level of automation. If we have multiple servers, this task can become challenging and tedious. In AWS Cloud there are elegant and powerful solutions with proper scalability depending on the client’s request.
In this article we will comparetwo automated backup solutions. One is through Amazon EC2 service and the other one is using AWS Systems Manager. We will point out similarities and pros and cons to get a clearer picture about both of these processes.
This blog post will guide you through examples which will give you a basic understanding of these automated backup processes.
AWS Systems Manager has a capability called Maintenance Windows. Maintenance Windows can help you schedule several types of tasks:
Here is an example.
Figure 1 - Maintenance window
Here you can see type of tasks that you can register under existing window:
Figure 2 - Maintenance window actions
These tasks execute AWS documents like AWS-StartEC2Instanceor AWS-StopEC2Instance, which can be found navigating through:
Figure 3 - Systems Manager Documents
Figure 4 - Types of Systems Manager Documents
In AWS Systems Manager feature Documents users can store customized YAML format for a specific kind of execution. Navigate through:
We can create our own schedule. You will need to provide window details, schedule, and tags.
Figure 5 - Maintenance windows
To create window, set configuration:
You need to specify task details:
Figure 6 - Maintenance window creation
Figure 7 - Maintenance window creation settings
When you have created the window, you can open it and set Automation documents that you will use, targets, percentage, and IAM service role.
Maintenance window > Actions Register Automation task
Fill maintenance window details:
Figure 8 - Automation task details
Figure 9 - Automation documents
Figure 10 - Targets settings
Figure 11 - Input parameters
Figure 12 - IAM role setup
You can manage EBS volumes and AMI snapshots for your Amazon EC2 instances with the instructions above. AWS Systems Manager is a powerful AWS service that gives you a fully automated management of your Amazon EC2 instances.
Amazon Data Lifecycle Manager is an Amazon EC2 capability using which you can create various schedules for EBS volume or AMI snapshots simultaneously.
Through lifecycle policy you can choose EBS snapshot policy/EBS-backed AMI policy and backup a volume or an instance. There is a possibility of creating four schedules in one Lifecycle policy, like daily, weekly, monthly, and even yearly schedule for each instance. You can combine them as it suits your project needs.
You can create lifecycle policy under:
Figure 13 - Lifecycle policy creation
Under Create lifecycle policy you need to specify settings:
Figure 14 - Choosing Target
Figure 15 - Target description
Figure 16 - IAM role settings
Figure 17 - Tags
Figure 18 - Schedule configuration
Schedules can be set like CRON expression or schedule rate.
In case you need to modify or delete a policy, you can navigate through:
AWS Systems Manager has several “moving parts” that you need to combine for Amazon EC2 backup. It gives you capability to monitor, manage, patch, and backup Amazon EC2 instances manually or fully automated. In Systems Manager you can manage one Amazon EC2 instance with customized tasks or you can choose to manage several instances or more as a fleet. You can manage them through AWS Systems Manager Documents, AWS Lambda function or AWS Step Functions.
Maintenance Windows feature combined with AWS Systems Manager Documents can create a snapshot timeout if it runs more than 60 minutes. This needs to be handled in Documents by creating a custom Document which needs to be modified. You need to stop the instance, create a snapshot, and then start the instance. This, however, causes downtime and that can be tricky if it is happening in production.
Some PROD environments have dense ETL loads. If loads are running on Amazon EC2 instance that is maintained by AWS Systems Manager, the instance can be unavailable for some time because of maintenance which can make clients dissatisfied. On the other hand, Lifecyle Manager gives you an opportunity to create snapshots while instance is running. This can solve the issue with downtime while data is consistent after the snapshot. Best practice for AWS Systems Manager is to stop the instance, create the snapshot and start the instance to preserve consistent data and avoid corruption.
Lifecycle Manager has inaccurate policy start time (one hour from specified start time) which can cause confusion, and AWS Systems Manager is accurate regarding start time, so if you need accurate snapshot creation time this can be your choice. Maintenance window offers several steps for starting your snapshots. You can assign schedule for executing tasks on several Amazon EC2 instances in parallel. Here is an example:
With priority numbers (0,1,2) you can orchestrate when each instance will run AWS Document that will trigger stop, create snapshot, and start instance action. You can set priority number higher if you want some action to have lower priority than other. You can add up to 5 instances (or targets) in your orchestration.
AWS Systems Manager and Amazon Data Lifecycle Manager have great capabilities regarding backup creation. The purpose of this blog is not to direct you to use any of these two but to show you what can be set through them. Amazon Data Lifecyle manager:
**AWS Systems Manager: **
AWS Systems Manager has wider variety of settings and capabilities than Amazon Data Lifecyle Manager which is specialized for Amazon EC2 instances. In AWS Systems Manager you can schedule AWS Step Functions where each of them will schedule several AWS Lambda functions and create a vast orchestration of tasks and sub-tasks. If any direction should be given, use Amazon Data Lifecyle Manager for simpler backup tasks and AWS Systems Manager when tasks require higher level of complexity.