Ansible Subelements Lookup Example

Suresh Vina -

When you're working with Ansible, you often come across situations where you need to deal with lists inside of lists. Imagine you have a bunch of servers, and each server has its own set of services to manage.

The subelements lookup plugin is designed to iterate over a list of dictionaries and a specified sub-list within each dictionary. Instead of writing complicated code to dig into each layer, subelements lets you glide through the outer list and then dive into the inner list easily.

What we will cover?

  • Subelements syntax
  • Subelements example
  • What are item.0 and item.1?
  • Subelements example with NetBox

Subelements Syntax

To use subelements in your playbook, you write a loop that tells Ansible what main list to look at and which sublist to go through. Here’s what a simple line of code looks like. 

loop: "{{ query('subelements', your_main_list, 'your_sublist_key') }}"

your_main_list is where you have all your main items (like servers), and your_sublist_key is the name of the sublist inside each main item (like tasks for each server). Ansible will then loop through each main item and its sub-items in turn.

Ansible Subelements Example

Suppose you have the following data structure defined in your playbook.

servers:
  - name: server1
    services:
      - name: httpd
        port: 80
      - name: sshd
        port: 22
  - name: server2
    services:
      - name: nginx
        port: 8080

This data structure represents a list of servers (servers), and each server has a list of services (services). Each service has a name and a port.

Now, let's say you want to loop through each server and its associated services to create a firewall rule for each service's port. Here's how you can do it using subelements.

---
- name: Configure firewall rules for services on servers
  hosts: localhost
  gather_facts: no
  tasks:
    - name: Allow service ports through the firewall
      ansible.builtin.debug:
        msg: "Allowing {{ item.1.name }} (port {{ item.1.port }}) on {{ item.0.name }}"
      loop: "{{ query('subelements', servers, 'services') }}"
  • We're using the ansible.builtin.debug module just for demonstration purposes to print a message. In a real-world scenario, you would replace this with a module that configures the firewall, like firewalld, ufw, or a custom module.
  • The loop is using the query function to call the subelements lookup plugin.
  • The first argument to subelements is the list we want to iterate over, which is servers.
  • The second argument is the key within each server's dictionary that contains the list to iterate over, which is services.

What are item.0 and item.1?

In the context of Ansible's subelements loop, item.0 and item.1 are the variables that reference the current items in the loop from the list of sub-elements that you are iterating over.

When you use subelements, you are typically dealing with a list of dictionaries, where each dictionary has a key that contains a list (the sub-elements). The subelements loop gives you each sub-element in turn, along with its parent element.

Here is what item.0 and item.1 represent.

  • item.0- This is the variable that references the current parent element. In a loop that iterates over a list of servers, where each server has a list of services, item.0 would be the server currently being processed in the loop.
  • item.1- This variable references the current sub-element from the list within the parent element. Continuing the above example, item.1 would be the current service for the server in the iteration.

Real Use Case with NetBox

I found a perfect moment to use subelements when I was setting up devices in NetBox with Ansible. If you are a network engineer, you might already be familiar with NetBox—it's a great tool for managing and documenting networks.

So, I had this task where each device in my network had a bunch of interfaces. In Ansible, I had a list that looked something like a record for each device. Inside each record, there was another list detailing all the interfaces on that device.

---
devices:
  - name: DC-Core-01
    site: DC_01
    type: Nexus7700 C7706
    role: Core
    rack: R_01
    face: front
    position: 10
    interfaces:
      - name: Eth1/1
        type: SFP+ (10GE)
      - name: Eth1/2
        type: SFP+ (10GE)
      - name: Eth1/10
        type: SFP+ (10GE)
      - name: Eth1/11
        type: SFP+ (10GE)
  - name: DC-Core-02
    site: DC_01
    type: Nexus7700 C7706
    role: Core
    rack: L_01
    face: front
    position: 10
    interfaces:
      - name: Eth1/1
        type: SFP+ (10GE)
      - name: Eth1/2
        type: SFP+ (10GE)
      - name: Eth1/10
        type: SFP+ (10GE)
      - name: Eth1/11
        type: SFP+ (10GE)
  - name: DC-FW-01
    site: DC_01
    type: PA-3410
    role: Firewall
    rack: R_01
    face: front
    position: 15
    interfaces:
    - name: Ethernet1/1
      type: SFP+ (10GE)
    - name: Ethernet1/2
      type: SFP+ (10GE)
  - name: DC-FW-02
    site: DC_01
    type: PA-3410
    role: Firewall
    rack: L_01
    face: front
    position: 15
    interfaces:
    - name: Ethernet1/1
      type: SFP+ (10GE)
    - name: Ethernet1/2
      type: SFP+ (10GE)

Using subelements allowed me to write a playbook that could go through each device and then, for each device, set up its interfaces in NetBox without writing complex loops.

---
- name: "Devices Tab"
  connection: local
  hosts: localhost
  gather_facts: False
  vars_files:
    - devices_vars.yml

  tasks:
    - name: Create Device
      netbox.netbox.netbox_device:
        netbox_url: http://10.10.10.22:8000
        netbox_token: MYTOKEN
        data:
          name: "{{ item.name }}"
          site: "{{ item.site }}"
          device_type: "{{ item.type }}"
          device_role: "{{ item.role }}"
          rack: "{{ item.rack }}"
          face: "{{ item.face }}"
          position: "{{ item.position }}"
        state: present
      loop: "{{ devices }}"
    
    - name: Create Interface
      netbox.netbox.netbox_device_interface:
        netbox_url: http://10.10.10.22:8000
        netbox_token: MYTOKEN
        data:
          device: "{{ item.0.name }}"
          name: "{{ item.1.name }}"
          type: "{{ item.1.type }}"
        state: present
      loop: "{{ devices | subelements('interfaces', 'skip_missing=True') }}"
  1. Creating the Device - I looped through the list of devices, where each device was defined with its site, type, role, and so on. This loop just went through each device and set it up in NetBox, simple enough.
  2. Creating the Interfaces - This is where subelements came into play. Instead of just looping through devices, I needed to loop through each interface of each device. With subelements, I could do this in a single loop by telling Ansible, "Hey, for each device, also go through its interfaces list." And if a device didn't have any interfaces, skip_missing=True meant Ansible would just skip it and move on.

By using subelements, I could efficiently create all my network devices and their interfaces in NetBox with a clear, easy-to-maintain playbook.

References

https://docs.ansible.com/ansible/latest/collections/ansible/builtin/subelements_lookup.html

https://www.buildahomelab.com/2018/11/03/subelements-ansible-loop-nested-lists/