Static inventory will not serve the needs when tracking and deploying to multiple sources with hosts spinning up and shutting down in response to business demands. The solution to such a changing infrastructure would be ansible dynamic inventory. To accomplish this, you will need a cloud provider ini file (e.g aws.ini) and ec2.py (which ansible use to communicate to the AWS API to fetch the metadata for all the instances). This post focus on AWS(Amazon Web Service).
ec2.ini file provided by Ansible. All the cloud provider .ini can be found here https://github.com/ansible/ansible/tree/stable-2.9/contrib/inventory
ec2.py: EC2 external inventory script which dynamically fetches data from AWS API
To set up the ec2 external inventory script, copy the script to /etc/ansible/ec2.py and run chmod +x /etc/ansible/ec2.py. You will also need to copy the ec2.ini file to /etc/Ansible/ec2.ini. Then run ansible-playbook normally, the --limit args can be parsed.
To make a successful call to AWS API, configure boto. There are a variety of methods available, you can add setup your aws credential by setting up your ~/.aws/credentials or export the following environment variables:
export AWS_ACCESS_KEY_ID=’mYaWeSoMeAcCeSsKeY′
export AWS_SECRET_ACCESS_KEY=’IcAnToTaLlYpUtMySeCrEtHeRe′
Set up a few more environment variables to the inventory management script such as
$ export ANSIBLE_HOSTS=/PATH/ec2.py
$ export EC2_INI_PATH=/PATH/ec2.ini
This variable sets ansible-playbook to use the dynamic EC2 script instead of a static /etc/ansible/hosts file. Open up ec2.py in a text editor and make sure that the path to ec2.ini config file is defined correctly at the top of the script:
EC2_INI_PATH instructs ec2.py where the ec2.ini config file is located.
Make the script executable and test it by running
./ec2.py –list
$ ./ec2.py --list
{
"_meta": {
"hostvars": {
"1.9O.1OO.12O": {
"ansible_host": "1.9O.1OO.12O",
"ec2__in_monitoring_element": false,
"ec2_account_id": "19OP23I42OF3",
"ec2_ami_launch_index": "0",
....
"ec2_tag_App": "olonje",
"ec2_tag_Env": "Staging",
"ec2_tag_Name": "Bami Gbe",
"ec2_virtualization_type": "hvm",
"ec2_vpc_id": "vpc-IaMhIdDeN"
},
"4.OOO.122.1OO9": {
"ansible_host": "4.OOO.122.1OO9",
"ec2__in_monitoring_element": false,
...
"ec2_key_name": "emi-moni-profile",
"ec2_root_device_name": "/dev/xvda",
"ec2_tag_App": "ounje2",
"ec2_tag_Env": "prod",
"ec2_tag_Name": "Production Stack",
"ec2_virtualization_type": "hvm",
"ec2_vpc_id": "vpc-IaMhIdDeN"
}
}
},
"ami_1k02nd83js84k6apO": [
"1.9O.1OO.12O",
"4.OOO.122.1OO9"
],
"tag_App_olonje": [
"1.9O.1OO.12O"
],
"tag_App_order2": [
"4.OOO.122.1OO9"
],
"tag_Env_Staging": [
"1.9O.1OO.12O"
],
"tag_Env_prod": [
"4.OOO.122.1OO9"
],
"tag_Name_app_Worker": [
"1.9O.1OO.12O"
],
"tag_Name_Production_Stack": [
"4.OOO.122.1OO9"
],
"vpc_id_vpc_IaMhIdDeN": [
"1.9O.1OO.12O",
"4.OOO.122.1OO9"
]
}
To fine-tune the output and other features that aren’t applicable refer to the ec2.ini file. For multiple AWS accounts, you can pass --profile PROFILE name to the ec2.py script.
$ cat ~/.aws/credentials
[default]
aws_access_key_id = <default access key>
aws_secret_access_key = <default secret key>
[profile tito]
aws_access_key_id = <tito access key>
aws_secret_access_key = <tito secret key>
[profile ounje]
aws_access_key_id = <ounje access key>
aws_secret_access_key = <ounje secret key>
To get the inventory for the ounje account run AWS_PROFILE=ounje ansible-playbook -i /etc/ansible/ec2.py myplaybook.yml. Since the EC2 external inventory provides mappings to instances from several groups, we can filter the target instance based on resource tag. Speaking of Tags.
Tags
Each instance has a variety of key/value pairs associated with it called Tags. The most common tag key is ‘Name’, though anything is possible. Each key/value pair is its own group of instances, again with special characters converted to underscores, in the format tag_KEY_VALUE e.g. tag_Name_Web can be used as is tag_Name_redis-master-001 becomes tag_Name_redis_master_001 tag_aws_cloudformation_logical-id_WebServerGroup becomes tag_aws_cloudformation_logical_id_WebServerGroup
When Ansible interacts with a specific server, the EC2 inventory script is called again with the --host HOST option. This looks up the HOST in the index cache to get the instance ID, and then makes an API call to AWS to get information about that specific instance. It then makes information about that instance available as variables to your playbooks. Each variable is prefixed by ec2_. e.g
ec2_dns_name
ec2_id
ec2_image_id
ec2_instance_type
ec2_ip_address
ec2_tag_Name
ec2_vpc_id
...
To see the complete list of variables available for an instance, run the script by itself:
/etc/ansible/ec2.py --host ec2-12-1O1-14-12.compute-1.amazonaws.com
Note: API calls to EC2 can be slow. Therefore, API results are cached. The number of seconds a cache is considered valid can be updated in the ec2.ini file in /etc/ansible/ec2.ini. Default setting is
...
cache_max_age = 300
...
You can set it to cache_max_age=0.
TESTING OUR DYNAMIC INVENTORY ABOVE
Running ansible-playbook deployment. Using sample.yml
- name: Deploy Ounje Artifact
hosts: all
vars:
- app: "e-status "
- env: "staging"
tasks:
- name: Get Instance IDs
ec2_remote_facts:
aws_access_key: "moYaCoverSecretMi"
aws_secret_key: "amalaShitaLoSureJuMafo"
region: "us-east-1"
filters:
"tag:Env": "{{ env }}"
register: gbebodi
- name: Store ounje hosts
debug: var=gbebodi
- name: Store Ounje EC2 instance ID
set_fact:
instance_id: "{{ item.id }}"
with_items: "{{ gbebodi.instances }}"
- name: Store EC2 instance ID in var
debug: var=instance_id
Run command:
$ ansible-playbook -i /etc/ansible/ec2.py --limit "tag_App_order2" -u ec2-user sample.yml -e env=prod
In the above example:
-
ansible-playbookis the standard command to run an ansible playbook -
/etc/ansible/ec2.pypoints ansible inventory to use the ec2.py to callout to aws API to get the ec2 instance metadata -
--limit "tag_App_ounje2”This limit the host you want to target by instructing ansible-playbook to pull the metadata of all instances with a tag of keyAppand valueounje2. This enables the ansible-playbook to know where to run the tasks, know which instance you want to access and limit the activities to the only ounje2 -
sample.ymlis the name of our playbook file -
–ea way of parsing env variable to the playbook, or to override the default env variable -
-uindicates the user ansible needs to use to login to the instance.
output
$ ansible-playbook -i /etc/ansible/ec2.py --limit "tag_App_ounje2" -u ec2-user sample.yml -e env=prod
[DEPRECATION WARNING]: ec2_remote_facts is kept for backwards compatibility but usage is discouraged. The module documentation details page may explain more about this rationale.. This feature will be removed in a future release. Deprecation warnings can be disabled by setting deprecation_warnings=False in
ansible.cfg.
PLAY [Deploy Ounje Artifact] ***********************************************************************************************************************************
TASK [Gathering Facts] ******************************************************************************************************************************************
Monday 23 September 2O19 O1:17:25 -O4OO (O:OO:OO.141) O:OO:OO.141 ******
ok: [1.OO.1O9.111]
TASK [Get Instance IDs] *****************************************************************************************************************************************
Monday 23 September 2O19 O1:17:28 -O4OO (O:OO:O2.234) O:OO:O2.376 ******
ok: [1.OO.1O9.111]
TASK [Store ounje hosts] ***************************************************************************************************************************************
Monday 23 September 2O19 O1:17:29 -O4OO (O:OO:O1.481) O:OO:O3.857 ******
ok: [1.OO.1O9.111] => { "ounje": { "changed": false, "failed": false, "instances": [ { "ami_launch_index": "O", "architecture": "x86_94", "block_device_mapping": [ { "attach_time": "2O19-O6-23TO4:54:57.OOOZ", "delete_on_termination": true, "device_name": "/dev/xvda", "status": "attached", "volume_id": "vol-298hfn3f348ff4f348f" } ], "client_token": "", "ebs_optimized": false, "groups": [ { "id": "sg-Of11425adOe7f7d1a", "name": "launch-wizard-13" } ], "hypervisor": "xen", "id": "i-1O293847561OO293", "image_id": "ami-1k02nd83js84k6apO", "instance_profile": null, "interfaces": [ { "id": "eni-nf32f2fn48489r348", "mac_address": "41:9O:i7:dc:69:lO" } ], "kernel": null, "key_name": "emi-moni-profile", "launch_time": "2O19-O6-23TO4:54:56.OOOZ", "monitoring_state": "disabled", "persistent": false, "placement": { "tenancy": "default", "zone": "us-east-1c" }, "private_dns_name": "ip-1-OO-1O9-111.ec2.internal", "private_ip_address": "OO7-OO-69-42O", "public_dns_name": "ec2-1-OO-1O9-111.compute-1.amazonaws.com", "public_ip_address": "1.OO.1O9.111", "ramdisk": null, "region": "us-east-1", "requester_id": null, "root_device_type": "ebs", "source_destination_check": "true", "spot_instance_request_id": null, "state": "running", "tags": { "App": "ounje2", "Env": "prod", "Name": "Production Stack" }, "virtualization_type": "hvm", "vpc_id": "vpc-IaMhIdDeN" } ] }
}
TASK [Store EC2 instance ID] ************************************************************************************************************************************
Monday 23 September 2O19 O1:17:29 -O4OO (O:OO:OO.145) O:OO:O4.OO3 ******
ok: [1.OO.1O9.111] => (item={u'ramdisk': None, u'kernel': None, u'instance_profile': None, u'root_device_type': u'ebs', u'private_dns_name': u'ip-1-OO-1O9-111.ec2.internal', u'block_device_mapping': [{u'status': u'attached', u'device_name': u'/dev/xvda', u'delete_on_termination': True, u'attach_time': u'2O19-O6-23TO4:54:57.OOOZ', u'volume_id': u'vol-298hfn3f348ff4f348f'}], u'key_name': u'emi-moni-profile', u'interfaces': [{u'id': u'eni-nf32f2fn48489r348', u'mac_address': u'41:9O:i7:dc:69:lO'}], u'persistent': False, u'image_id': u'ami-1k02nd83js84k6apO', u'groups': [{u'id': u'sg-Of11425adOe7f7d1a', u'name': u'launch-wizard-13'}], u'spot_instance_request_id': None, u'requester_id': None, u'source_destination_check': u'true', u'id': u'i-1O293847561OO293', u'tags': {u'App': u'ounje2', u'Name': u'Production Stack', u'Env': u'prod'}, u'public_ip_address': u'1.OO.1O9.111', u'monitoring_state': u'disabled', u'placement': {u'tenancy': u'default', u'zone': u'us-east-1c'}, u'ami_launch_index': u'O', u'hypervisor': u'xen', u'region': u'us-east-1', u'ebs_optimized': False, u'launch_time': u'2O19-O6-23TO4:54:56.OOOZ', u'public_dns_name': u'ec2-1-OO-1O9-111.compute-1.amazonaws.com', u'state': u'running', u'architecture': u'x86_94', u'private_ip_address': u'OO7-OO-69-42O', u'vpc_id': u'vpc-IaMhIdDeN', u'client_token': u'', u'virtualization_type': u'hvm'})
TASK [Store EC2 instance ID in var] *****************************************************************************************************************************
Monday 23 June 2O19 O1:17:29 -O4OO (O:OO:OO.173) O:OO:O4.176 ******
ok: [1.OO.1O9.111] => { "instance_id": "i-1O293847561OO293"
}
PLAY RECAP ******************************************************************************************************************************************************
1.OO.1O9.111 : ok=5 changed=O unreachable=O failed=O
Monday 23 June 2O19 O1:17:29 -O4OO (O:OO:OO.11O) O:OO:O4.287 ******
===============================================================================
Gathering Facts ------------------------------------------------------------------------------------------------------------------------------------------ 2.23s
Get Instance IDs ----------------------------------------------------------------------------------------------------------------------------------------- 1.48s
Store EC2 instance ID ------------------------------------------------------------------------------------------------------------------------------------ O.17s
Store ounje hosts --------------------------------------------------------------------------------------------------------------------------------------- O.15s
Store EC2 instance ID in var ----------------------------------------------------------------------------------------------------------------------------- O.11s
Playbook run took O days, O hours, O minutes, 4 seconds
With no hostfile to maintain, you can deploy to freshly created instances spun up by terraform without having to worry about maintaining an hostfile.
Top comments (0)