Automating AWS EC2 Instance Management with Ansible

Introduction

In this tutorial, we'll walk through how to use Ansible to automate the creation and management of EC2 instances on AWS. We'll cover creating instances with different Linux distributions, setting up passwordless SSH authentication, and automating the shutdown of specific instances based on their OS family. Additionally, we'll use Ansible Vault to securely manage our AWS credentials.

Prerequisites

• AWS Account
• Ansible installed on your control node
• Basic understanding of AWS and Ansible concepts
• Python and pip installed
• AWS CLI installed and configured (optional, for some verification steps)

1. Setting Up Ansible and AWS Environment

1.1. Install boto3 and AWS Collection

boto3 is the AWS SDK for Python, and the amazon.aws collection provides Ansible modules for interacting with AWS services.

pip install boto3
ansible-galaxy collection install amazon.aws --force

The --force flag is used to ensure the collection is reinstalled, resolving any potential version issues.

1.2. Create an IAM User and Generate Access Keys

Follow these steps to create an IAM user with programmatic access and generate access keys:

1. In the AWS Management Console, navigate to IAM (Identity and Access Management).
2. Create a new IAM user with programmatic access.
3. Attach the AmazonEC2FullAccess policy to the user (or a more restrictive policy tailored to your needs). Important: For security reasons, it is important to create restrictive policies instead of AmazonEC2FullAccess for this project.
4. Generate the access key ID and secret access key for the user.

1.3. Setting Up Ansible Vault

Ansible Vault is used to encrypt sensitive data, such as AWS credentials, within your Ansible playbooks.

Create a Vault Password File:

openssl rand -base64 2048 > vault.pass
chmod 600 vault.pass

This command generates a random password and stores it in a file with restricted permissions.

Create the Vault Encrypted File:

ansible-vault create group_vars/all/pass.yml --vault-password-file vault.pass

This command creates an encrypted YAML file where you'll store your AWS credentials.

# group_vars/all/pass.yml
ec2_access_key: YOUR_ACCESS_KEY
ec2_secret_key: YOUR_SECRET_KEY

2. Creating EC2 Instances with Ansible

2.1. Create the ec2_create.yaml Playbook:

---
- hosts: localhost
  connection: local
  tasks:
    - name: start an instance with a public IP address
      amazon.aws.ec2_instance:
        name: "{{ item.name }}"
        key_name: "rons-ec2-key"
        instance_type: t2.micro
        security_group: default
        region: us-east-1
        aws_access_key: "{{ lookup('ansible.builtin.file', 'vault.pass') }}"
        aws_secret_key: "{{ lookup('ansible.builtin.file', 'vault.pass') }}"
        network:
          assign_public_ip: true
        image_id: "{{ item.image }}"
        tags:
          environment: "{{ item.image }}"
      loop:
        - {image: "ami-084568db4383264d4", name: "managed-node-1"}
        - {image: "ami-00a929b66ed6e0de6", name: "managed-node-3"}
        - {image: "ami-084568db4383264d4", name: "managed-node-2"}

hosts: localhost and connection: local

This specifies that the playbook will run on the Ansible control node itself. This is required because you want to use the AWS API, not SSH, to create the instances.

tasks:

Define the actions to be performed.

amazon.aws.ec2_instance

This is the module from the amazon.aws collection that lets ansible communicate with AWS

loop:

This feature allows for many AMI to be created in the same code

2.2. Execute the Playbook:

ansible-playbook ec2_create.yaml --vault-password-file vault.pass

2.3. Verify the Instances:

In the AWS Console, confirm that the three EC2 instances have been created with the specified names and AMIs. You can also use the AWS CLI:

aws ec2 describe-instances --region us-east-1

3. Setting Up Passwordless SSH Authentication

To enable Ansible to manage the newly created EC2 instances, you need to set up passwordless SSH authentication between the Ansible control node and the instances.

Update inventory.ini:

Add the host information for each instance to your inventory file. Replace the IP addresses with the actual public IP addresses of your EC2 instances:

[all]
3.83.102.198
54.84.220.118
18.208.169.135

[redhat]
3.83.102.198

[ubuntu]
54.84.220.118
18.208.169.135

Copy SSH Key to Instances:

Use ssh-copy-id to copy your SSH public key to each instance. You will need the private key associated with the key name provided in your instances to the security to test the connection. Remember to replace the IP addresses, key file, and usernames with the correct values.

ssh-copy-id -i ~/.ssh/rons-ec2-key ubuntu@54.84.220.118

Repeat this for all instances, adjusting the username for CentOS (usually ec2-user).

This may work. Add new steps to create ssh keys from local machine and upload them to Ansible server so it can deploy to the created instances.

4. Automating the Shutdown of Ubuntu Instances

Create a playbook to shut down the Ubuntu instances based on the OS family.

4.1. Create shutdown_ubuntu.yaml:

---
- name: Shutdown ubuntu instances only
  hosts: all
  become: true

  tasks:
    - name: Shutdown ubuntu instances only
      ansible.builtin.command: /sbin/shutdown -h now
      when: ansible_facts['os_family'] == "Debian"

4.2. Run the Playbook:

ansible-playbook shutdown_ubuntu.yaml -k

The -k part may be required if you cannot use the ansible vault. It depends. You should specify the location of the keys with -extra var 'key_path=../../keys/aws_key.pem' to have this work

Congratulations!

You have successfully automated the creation and management of EC2 instances on AWS using Ansible! This detailed guide provides a solid foundation for automating your AWS infrastructure. Experiment with different AMIs, instance types, security groups, and tags to customize your setup. Also, be sure to refine your security practices to ensure your AWS environment is secure.