Use the AWS CLI to Customise and Automate Launching of your EC2 Instance
Introduction
In this article I will be walking you through setting up the AWS CLI (Command Line Interface) on your local machine and showing how we can use it to customise and automate the launch of our EC2 instances.
Firstly, we will configure an EC2 instance and set it up to install and run an Apache web server at launch.
Finally we shall make an Amazon Machine Image (AMI) using the instance we created earlier.
Pre-requisites
- A non-root IAM account with enough rights to perform the required actions. It is considered bad practise (and a security risk) to use your root account for daily administrative tasks.
- The access key ID and secret access key for your IAM account.
- A key pair for secure connection to your instance.
- A security group set up to allow SSH and HTTP traffic.
NOTE: I go over steps 3 and 4 in my previous AWS EC2 article here. Please reference that article if you need guidance.
Installing the AWS CLI (Windows)
First thing’s first. I need the AWS Command Line Interface installed on my local Windows 11 workstation.
I am just going to follow the latest official guide here: https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-getting-started.html
There are 3 sections in the guide that we shall refer to for a Windows setup. The guide is simple and straightforward, so I am only going to go through the items at a high level.
Pre-requisites — This section takes us through setting up your IAM account and getting the access key ID and secret access key.
Be sure to save your keys in a .csv file where prompted and store that securely.
Install/Update — This section details which installer to use for which Operating System.
For Windows, it’s a simple matter of downloading and running a straightforward msi installer. No post install reboot is required.
To confirm the install worked, type the following command in your terminal:
aws --version
I’m using PowerShell and the install was successful.
Quick setup
Finally, we use the aws configure command to input the required information which will allow us to access our AWS resources through the Command Line Interface.
I’m not going to paste my real keys here for security reasons, but this sample data shows what information needs input.
aws iam get-userThe aws iam get-user command is probably the simplest way to check that our keys are correct and we can connect to our AWS resources.
Of course, this command assumes our IAM account has the required IAM privileges! The output confirms that we do.
Now we are ready to create an EC2 instance.
Create an EC2 Instance that hosts an Apache web site
So we can again reference Amazon’s official documentation to get our basic command structure to launch an EC2 instance via the CLI: https://docs.aws.amazon.com/cli/latest/userguide/cli-services-ec2-instances.html
aws ec2 run-instances --image-id ami-xxxxxxxx --count 1
--instance-type t2.micro --key-name MyKeyPair --security-group-ids sg-903004f8
--subnet-id subnet-6e7f829eWell, it’s a pretty long command and the switches cover some things we need and not others.
However, we need to get the correct image-id, we need to specify our own key-name, security-group-ids and subnet-id.
We also need to find a way of installing Apache and configuring a simple web site.
First let’s get a valid image-id from the AWS Console.
Go to the EC2 Dashboard and click Launch instance.
Scroll down to Quick Start and we are just going to use the default Amazon Linux instance, so copy the highlighted AMI ID for use in our command.
If we go back to the EC2 Dashboard and click Instances, I can look at an existing instance I had created to get the key-name, security-group-ids and subnet-id values from the relevant tabs.
This instance was created in our default VPC and had a keypair created and a security group to allow SSH and HTTP traffic. I am just replicating that setup for this exercise too. Again, please reference my previous article here for more details.
The final step is to create a BASH script to install Apache Web Server and configure a basic web site. We need to call that file from our command line for it to be executed automatically.
If we were doing this manually, we’d just scroll down and open up the Advanced fields after pressing Launch instances, then paste our code into the User data field.
Searching on ‘user data in AWS CLI’ leads us to another official guide which has the information we need: https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/user-data.html
From the guide we can see we need to use the user-data file://<filename> switch to specify our BASH script.
Note that scripts entered as user data are run as the root user, so do not use the sudo command in the script.
# note - this script needs to be run with elevated privileges
# and is intended for an AWS Linux AMI.
#!/bin/bash
# update all packages on the server
yum update -y
# install apache http web server
yum install -y httpd
# install firewalld to allow use of firewall-cmd
yum install -y firewalld
# enable and start apache web server. also ensure it starts automatically after a reboot
systemctl enable --now httpd
# enable firewall
systemctl enable --now firewalld
# add http service to firewall and restart to apply change
firewall-cmd --zone=public --add-service=http --permanent
firewall-cmd --reload
# check for an index.html file and make a backup if it exists
if [ -f "/var/www/html/index.html" ]; then
mv /var/www/html/index.html /var/www/html/index.html.bak
fi
# download a royalty free landscape image and move it to /var/www/html
wget -O /var/www/html/bg.jpg https://mdbootstrap.com/img/Photos/Horizontal/Nature/full%20page/img%20%283%29.jpg
# create a new index.html file and add content
echo "<html>" > /var/www/html/index.html
echo "<head>" >> /var/www/html/index.html
echo "<title>Level Up In Tech - Project 5 - John MacLean</title>" >> /var/www/html/index.html
echo "<style>" >> /var/www/html/index.html
echo "body { background-image: url('bg.jpg'); background-size: cover; text-align: center; }" >> /var/www/html/index.html
echo "</style>" >> /var/www/html/index.html
echo "</head>" >> /var/www/html/index.html
echo "<body>" >> /var/www/html/index.html
echo "<h1>EC2 CLI Project - John MacLean - Gold Team</h1>" >> /var/www/html/index.html
echo "</body>" >> /var/www/html/index.html
echo "</html>" >> /var/www/html/index.html
This is the script I will use. It’s based on a previous script I created, but I’ve removed most of the status checking as we are not doing troubleshooting via the terminal.
For simplicities sake, I will just create the script in the default location on my workstation to minimise the chance of having any path related issues.
aws ec2 run-instances --image-id ami-09ee0944866c73f62 --count 1
--instance-type t2.micro --key-name LUITProject5KeyPair
--subnet-id subnet-0a2fd0a2e1babf30b
--security-group-ids sg-02328ff193be3ab8e
--user-data file://apache_auto.txt
--tag-specifications 'ResourceType=instance,Tags=[{Key=Name,Value=LUITProjectCLIEC2}]'So our custom command looks as shown. Note that I have formatted it so that it displays in the code box shown in a more readable way.
Our command ran fine and if we check the console, we can see a new instance running with the name we specified in the Tag.
If we take a copy of the Instance Id from our command output, we can then use the following command to get our instances public IP address.
aws ec2 describe-instances --instance-ids <instance_id> --query 'Reservations[*]. Instances[*]. PublicIpAddress' --output text
So we put our IP address into an external browser and…
…oh no! Looks like we need some troubleshooting. My first guess would be something in my script went awry.
So let’s SSH into our instance. Well, the default Apache Web Server location doesn’t exist, which means no index.html file and therefore no web page.
I do a bit more digging…
So it looks like the script to install Apache has actually not run at all. If I manually run the command to install Apache, it starts from scratch.
After some investigation, it seems that not having #!/bin/bash as the very first line of your user data code can potentially cause issues, so I remove the first two lines from my script, even though they are just comments.
I also rename my script from .txt to .sh — this shouldn’t matter, but I did it anyway.
aws ec2 run-instances --image-id ami-09ee0944866c73f62 --count 1
--instance-type t2.micro --key-name LUITProject5KeyPair
--subnet-id subnet-0a2fd0a2e1babf30b --security-group-ids sg-02328ff193be3ab8e
--user-data file://apache_auto.sh --tag-specifications 'ResourceType=instance,Tags=[{Key=Name,Value=LUITProjectCLIEC2Two}]'So the modified command is as shown, with a new instance name also.
After running the updated command, our new instance comes up…
…so let’s get the IP address and put it into a browser…
…success!
Creating and testing an Amazon Machine Image
Now we can use our working EC2 instance to create an AMI which we can effectively use as a template for generating future custom EC2 instances.
Let’s check the syntax of the command we will need here: https://awscli.amazonaws.com/v2/documentation/api/2.1.29/reference/ec2/create-image.html
The guides Synopsis shows the basic structure of the command — the switches in brackets being optional.
Note that the integrity of the file system in the resulting image cannot be guaranteed when using the no-reboot switch. However, we will use this option anyway as I need my existing instance to keep running for the moment and we are not in a production environment.
aws ec2 create-image --region eu-west-2 --instance-id i-05a831a958a6171aa
--name "LUIT_LINUX_APACHE" --no-rebootThis will be my command. Again, please note that I have formatted the command for readability so that it fits in the code box.
I’m specifying the AMI to be created in my nearest region, I am using the Instance ID of my working EC2 instance and setting a custom name for my AMI.
That looks good, let’s check the AWS Console.
Still on the EC2 Dashboard, we can click on AMIs in the left hand menu and see that our AMI has indeed been created.
Next, let’s create another new EC2 instance from the new AMI. Let’s first copy our AMI ID and go back to the CLI again.
aws ec2 run-instances --region eu-west-2 --image-id ami-0b88d6854bd566089
--count 1 --instance-type t2.micro --key-name LUITProject5KeyPair
--subnet-id subnet-0a2fd0a2e1babf30b --security-group-ids sg-02328ff193be3ab8e
--tag-specifications 'ResourceType=instance,Tags=[{Key=Name,Value=LUITProjectCLIEC2_AMICLONE}]'So our command to create a new instance is as before, aside from we are using the new AMI ID as our image-id and I’ve also specified a new name for our EC2 instance in the tag-specifications.
The command runs in our terminal with no errors. Checking in the console, we see the new EC2 instance has been created and is initialising.
Once up and running, let’s use the new Instance ID to get the IP address.
Let’s put the new IP address into a browser…
…yes! The EC2 instance cloned from our AMI works perfectly.
Conclusion
Well, I hope this article gives you a flavour of how useful the AWS CLI and AMIs can be.
Both can drive efficiencies in a production environment, allowing rapid deployment of machines tailored to your specific needs, bypassing tedious manual configuration.
Through automation, standards can be adhered to and individual errors eliminated.
This was my week 5 project with Level Up In Tech and my previous scripting experience definitely came in handy when getting to grips with the AWS CLI.
As ever, please feel free to contact me with any questions or to send me any feedback. I’m always happy to hear from anyone who reaches out.
