Creators of Amazon Elastic Compute Cloud (EC2) instances may stuff a script into the "user data" which will be executed on the instance's initial boot. This script may be useful to mount Elastic Block Store (EBS) volumes as file systems since those volumes cannot be attached to the instance until after the instance has been created and started.
This article presents a script which leverages the functions provided in the aws.rc
script described in a previous article.
The "user data" script described herein provides the following services to Redhat and CentOS instances:
-
Update OS software:
a.
yum update
b. Install/Update Python
c. Install AWS CLI
Create users, install their SSH authorized keys, and configure
sudo
Attach volumes, create file systems (if neccessary), mount, and update /etc/fstab
AWS Configuration
The scripts require specific configuration in AWS. These requirements are described in the next subsections.
Users
The script will configure all users specified in the instance's http://169.254.169.254/latest/meta-data/public-keys/. These key pairs must be imported either through AWS Management Console or the AWS CLI. The name of the key pair must be the same as the user name.
A primary use case is to create and configure ec2-user
on CentOS images to be consistent with Amazon's Linux images.
EBS File System Volumes
Any EBS volume that will be mounted as a file system must be configured with the following tags:
host
fstype
mntpt
Where host
is the http://169.254.169.254/latest/meta-data/local-ipv4 address of the newly created instance, fstype
is a file system type compatible with the mkfs(8) and mount(8) commands'-t
argument, and mntpt
is the local directory on which the file system will be mounted.
Once the EBS volume is formatted with a valid file system, the volume's uuid
tag should be updated with the file system's UUID. The user-data.bash
script will not format the volume if the uuid
tag is present. The user-data.bash
script will update the volume's uuid
tag if it successfully creates a file system on the volume.
Finally, the script will configure /etc/fstab to mount the EBS volume on the mntpt
directory.
Theory of Operation
The script:
- Update the operating system software
- Configures the users whose keys are specified in http://169.254.169.254/latest/meta-data/public-keys/
- Attaches, formats, and mounts any EBS volumes tagged with this instance's http://169.254.169.254/latest/meta-data/local-ipv4 address
The corresponding parts of the user-data.bash
script are described in detail in the following subsections.
Software Update
The software update consists of:
- Updating all packages managed by yum(8)
- Install and update python(1)
- Install the AWS Command Line Interface
export LANG=en_US.UTF-8
export LC_ALL=${LANG}
yum -y update
yum -y install python
easy_install --prefix /usr pip
pip install --prefix /usr --upgrade pip
pip install --prefix /usr --upgrade awscli
Users Configuration
For each public key specified in http://169.254.169.254/latest/meta-data/public-keys/:
- If they do not exist, create the user specified by the key name and create that user's home directory
- Add the openssh-key value to the user's
~/.ssh/authorized_keys
- Configure the user in sudoers(5)
for key in $(metadata public-keys/); do
username=${key#*=}
useradd -G wheel -m -s /bin/bash -U ${username}
userhome=$(eval echo ~${username})
mkdir -p ${userhome}/.ssh
echo "$(metadata public-keys/${key%%=*}/openssh-key)" \
>> ${userhome}/.ssh/authorized_keys
chown -R ${username}:${username} ${userhome}/.ssh
chmod -R go-rwx ${userhome}/.ssh
file=/etc/sudoers.d/user-data-${username}
echo "${username} ALL=(ALL) NOPASSWD:ALL" > /Users/ball/hcf-dev/blog/2018-08-22-aws-user-data-script/pom.xml
chmod a-wx,o-r ${file}
done
Attach and Mount EBS Volumes
For each EBS volume tagged with host
equalling the value at http://169.254.169.254/latest/meta-data/local-ipv4, a non-nil fstype
, and non-nil mntpt
:
- Attach the EBS volume to this instance
- Test if the volume contains data known to file(1) and, if it does not, create a file system of type specified by
fstype
- Determine the UUID of the file system and add an entry to /etc/fstab
export HOST=$(metadata local-ipv4)
export VOLUMES=$(ec2 describe-volumes \
--filters Name=tag:host,Values=${HOST} \
--output text --query 'Volumes[*].VolumeId')
if [ -n "${VOLUMES}" ]; then
for volume in ${VOLUMES}; do
fstype=$(ec2-get-tag-value ${volume} fstype)
if [ "${fstype}" != "" ]; then
device=$(next-unattached-block-device)
ec2-attach-volume ${volume} ${device}
if [ "$(file -b -s ${device})" == "data" ]; then
volume-mkfs ${volume} ${device} ${fstype}
fi
uuid=$(ec2-get-tag-value ${volume} uuid)
mntpt=$(ec2-get-tag-value ${volume} mntpt)
if [ "${uuid}" != "" -a "${mntpt}" != "" ]; then
mkdir -p ${mntpt}
echo "UUID=${uuid} ${mntpt} ${fstype} defaults 0 2" >> /etc/fstab
fi
fi
done
fi
mount -a
exit 0
The file systems are then mounted with mount -a
.1
systems were then connected on start-up.
user-data.bash
For reference, the complete user-data.bash
Ansible template is included below. The aws.rc
script is included through a relative path. If another tool than Ansible is used, the aws.rc
script must be included to provide the functions through that tool's appropriate mechanism.
#!/bin/bash
# ----------------------------------------------------------------------------
# user-data.bash
# ----------------------------------------------------------------------------
export LANG=en_US.UTF-8
export LC_ALL=${LANG}
if [ -e /usr/bin/apt ]; then
apt -y update
apt -y install awscli
else
yum -y update
if [ ! -e /usr/libexec/platform-python ]; then
yum -y install python
fi
if [ ! -e /usr/bin/pip -a -x /usr/bin/easy_install ]; then
/usr/bin/easy_install --prefix /usr pip
fi
/usr/bin/pip install --prefix /usr --upgrade pip
/usr/bin/pip install --prefix /usr --upgrade awscli
fi
# ----------------------------------------------------------------------------
# Functions
# ----------------------------------------------------------------------------
{{ lookup('template', '../../aws-rc/templates/etc/aws.rc') }}
# ----------------------------------------------------------------------------
# Create users and install respective .ssh/authorized_keys for public-keys'
# metadata
# ----------------------------------------------------------------------------
getent group sudo >> /dev/null 2>&1
if [ $? -eq 0 ]; then
wheel=sudo
else
wheel=wheel
fi
for key in $(metadata public-keys/); do
username=${key#*=}
useradd -G ${wheel} -m -s /bin/bash -U ${username}
userhome=$(eval echo ~${username})
mkdir -p ${userhome}/.ssh
echo "$(metadata public-keys/${key%%=*}/openssh-key)" >> ${userhome}/.ssh/authorized_keys
chown -R ${username}:${username} ${userhome}/.ssh
chmod -R go-rwx ${userhome}/.ssh
file=/etc/sudoers.d/user-data-${username}
echo "${username} ALL=(ALL) NOPASSWD:ALL" > ${file}
chmod a-wx,o-r ${file}
done
export HOST=$(metadata local-ipv4)
export VOLUMES=$(ec2 describe-volumes \
--filters Name=tag:host,Values=${HOST} \
--output text --query 'Volumes[*].VolumeId')
if [ -n "${VOLUMES}" ]; then
for volume in ${VOLUMES}; do
fstype=$(ec2-get-tag-value ${volume} fstype)
if [ "${fstype}" != "" ]; then
device=$(next-unattached-block-device)
ec2-attach-volume ${volume} ${device}
if [ "$(file -b -s ${device})" == "data" ]; then
volume-mkfs ${volume} ${device} ${fstype}
fi
uuid=$(ec2-get-tag-value ${volume} uuid)
mntpt=$(ec2-get-tag-value ${volume} mntpt)
if [ "${uuid}" != "" -a "${mntpt}" != "" ]; then
mkdir -p ${mntpt}
echo "UUID=${uuid} ${mntpt} ${fstype} defaults 0 2" >> /etc/fstab
fi
fi
done
fi
mount -a
exit 0
Within Ansible, the user-data.bash
can be expanded from the template
into a "fact:"
- name: user-data script
set_fact:
user_data: >
{{ lookup('template', 'user-data.bash') }}
An example fragment for creating the EC2 instance with the user-data.bash
script is given below.
- name: 172.31.0.4
community.aws.ec2_instance:
...
instance_profile_name: ec2-user
key_name: ec2-user
user_data: >
{{ user_data }}
...
References
- Source
- automount/autofs Executable Map for Amazon EBS Volumes
- Amazon Web Services (AWS)
- AWS Elastic Compute Cloud (EC2)
- AWS Elastic Block Store (EBS)
- Ansible
-
Previous versions of this script rebooted the instance and the file ↩
Top comments (0)