Nornir Inventory Deep Dive and Filtering (III)

In the previous parts, we've covered how to create the Nornir configuration file and inventory files to get started. Before we even wrote our first script, we created those four essential files. But is that the only way to do it? You might have noticed that we saved our credentials in the groups.yaml file, which isn't ideal for production environments. So, let's explore some alternative methods to pass in those details to Nornir.

Nornir Network Automation Full Course

Nornir is a Python library designed for Network Automation tasks. It enables Network Engineers to use Python to manage and automate their network devices.

PacketswitchSuresh Vina

Nornir and Nemiko Plugin

In this part, we will explore the Netmiko plugin. The Netmiko plugin is used within the Nornir framework to simplify interactions with network devices.

PacketswitchSuresh Vina

Initialize Nornir Without a Configuration File

Just to jog our memory, in our previous examples, we used the config.yaml file, and our Python script referenced this file when initializing Nornir, as shown below.

#config.yaml
---
inventory:
  plugin: SimpleInventory
  options:
    host_file: 'hosts.yaml'
    group_file: 'groups.yaml'
    defaults_file: 'defaults.yaml'

runner:
  plugin: threaded
  options:
    num_workers: 2

from nornir import InitNornir
from nornir_netmiko.tasks import netmiko_send_command
from nornir_utils.plugins.functions import print_result

nr = InitNornir(config_file='config.yaml')

results = nr.run(task=netmiko_send_command, command_string='show ip arp')
print_result(results)

However, an alternative method is to pass the parameters directly into the InitNornir object. Let’s take a look at how this is done.

from nornir import InitNornir
from nornir_netmiko.tasks import netmiko_send_command
from nornir_utils.plugins.functions import print_result

nr = InitNornir(
    inventory={
        "plugin": "SimpleInventory",
        "options": {
            "host_file": "hosts.yaml",
            "group_file": "groups.yaml",
            "defaults_file": "defaults.yaml"
        }
    },
    runner={
        "plugin": "threaded",
        "options": {
            "num_workers": 2
        }
    }
)

results = nr.run(task=netmiko_send_command, command_string='show ip arp')
print_result(results)

In this script, we've directly defined the configuration settings within the script itself, using a Python dictionary. This approach eliminates the need for a separate config.yaml file. We initialize Nornir with all necessary inventory settings (like host, group, and defaults files) and runner options (specifying the number of workers) explicitly stated in the InitNornir call. This method allows us to set up everything needed for Nornir to run tasks within the script itself.

You can also use a combination of both methods as shown below.

from nornir import InitNornir
from nornir_netmiko.tasks import netmiko_send_command
from nornir_utils.plugins.functions import print_result

nr = InitNornir(
    config_file="config.yaml",
    runner={
        "options": {
            "num_workers": 1
        }
    }
)

results = nr.run(task=netmiko_send_command, command_string='show ip arp')
print_result(results)

In this combined setup, we initialize Nornir with both a config_file specified and inventory settings directly embedded within the InitNornir call.

My preference is to maintain a separate configuration file as much as I can so, I'm not cluttering the Python script.

Inventory Inheritance Model

In Nornir, inventory resolution helps determine the final set of attributes each host will have based on its association with groups and the defaults defined. This process combines data from three primary sources - hosts, groups, and defaults files.

To demonstrate inventory inheritance, I'm going to model a network with a mixture of devices from different vendors, each assigned specific roles.

Defaults File

This file serves as the baseline configuration for all hosts. Every host will inherit settings from here unless overridden by group or host-specific settings.

#defaults.yaml
---
username: superuser
password: notsecret

data:
  dns: 192.1615.15

Groups File

Hosts can be assigned to one or more groups. Attributes defined in a group will override the defaults unless specified directly in the host’s configuration.

#groups.yaml
---
cisco:
  platform: cisco_ios

arista:
  platform: arista_eos
  username: arista_user
  password: supersecret

manchester:
  username: manc
  password: password
  data:
    vlans:
      100: domains
      101: servers

london:
  data:
    dns: 1.1.1.1
    vlans:
      10: users
      20: printers

Hosts File

This is where individual devices are defined. Each host can inherit properties from the defaults and their associated groups but can also have unique settings which override any inherited properties.

#hosts.yaml

---
pack-swa-001:
  hostname: 10.10.11.10
  groups:
    - cisco
    - london
  data:
    type: access
    role: campus

pack-swa-002:
  hostname: 10.10.11.12
  groups:
    - cisco
    - london
  data:
    type: access
    role: campus

pack-swd-001:
  hostname: 10.10.11.20
  username: admin
  password: admin
  groups:
    - cisco
    - london
  data:
    type: distribution
    role: campus

pack-swc-001:
  hostname: 10.10.11.25
  groups:
    - arista
    - manchester
  data:
    type: core
    role: edge

pack-swc-002:
  hostname: 10.10.11.26
  groups:
    - arista
    - manchester
  data:
    type: core
    role: edge

To get a quick overview of the hosts and groups configured in Nornir, you can use nr.inventory.hosts and nr.inventory.groups. These commands will display the information in a dictionary-like format, which is easy to read and understand.

When you run nr.inventory.hosts, you’ll see all the individual hosts listed by their identifiers and nr.inventory.groups shows the information about the groups.

In [1]: from nornir import InitNornir
   ...: nr = InitNornir(config_file="config.yaml")

In [2]: nr.inventory.hosts
Out[2]: 
{'pack-swa-001': Host: pack-swa-001,
 'pack-swa-002': Host: pack-swa-002,
 'pack-swd-001': Host: pack-swd-001,
 'pack-swc-001': Host: pack-swc-001,
 'pack-swc-002': Host: pack-swc-002}

In [3]: nr.inventory.groups
Out[3]: 
{'cisco': Group: cisco,
 'arista': Group: arista,
 'manchester': Group: manchester,
 'london': Group: london}

Example 1 - pack-swa-001's Attributes

To kick things off, let's examine our first host - pack-swa-001. We'll look at its hostname, username, password, groups, and data. The data attributes are handled slightly differently, but we'll get into that too.

When we initialize Nornir and look up pack-swa-001, we find that:

In [19]: from nornir import InitNornir
    ...: nr = InitNornir(config_file="config.yaml")

In [20]: host = nr.inventory.hosts['pack-swa-001']

In [21]: host.username
Out[21]: 'superuser'

In [22]: host.groups
Out[22]: [Group: cisco, Group: london]

In [23]: host.platform
Out[23]: 'cisco_ios

• The username is 'superuser', which comes from the default file, since it is not specified in the host's own settings or its groups.
• The groups associated with the host are cisco and london. These are the groups from which the host inherits additional attributes.
• The platform for the host is 'cisco_ios'. This is inherited from the cisco group settings.

In [24]: host.data
Out[24]: {'type': 'access', 'role': 'campus'}

In [25]: host.type
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
Cell In[25], line 1
----> 1 host.type

File ~/Documents/blog/nornir_course/venv/lib/python3.10/site-packages/nornir/core/inventory.py:368, in Host.__getattribute__(self, name)
    366 def __getattribute__(self, name: str) -> Any:
    367     if name not in ("hostname", "port", "username", "password", "platform"):
--> 368         return object.__getattribute__(self, name)
    369     v = object.__getattribute__(self, name)
    370     if v is None:

AttributeError: 'Host' object has no attribute 'type'

• The data attribute shows {type: 'access', role: 'campus'}, which are directly configured under the host in the hosts.yaml file.
• When we try to access host.type using the dot notation, we receive an AttributeError. This is because 'type' is not a direct attribute of the Host object; rather, it is a key within the data dictionary. The same goes for the dns attribute, which is inherited and doesn't directly exist on the Host object.
• To access all the attributes, including those inherited from groups or the defaults file, we need to use the dictionary-like method (i.e., host['attribute_name']).

In [27]: host['type']
Out[27]: 'access'

In [28]: host['dns']
Out[28]: '1.1.1.1'

In [29]: host.keys()
Out[29]: dict_keys(['type', 'role', 'dns', 'vlans'])

• When we do this for 'type', we correctly get 'access', and for 'dns', we get '1.1.1.1', which is inherited from the london group.
• Using host.keys(), we see all the keys available, including those that are inherited (dns and vlans) and those that are directly set (type and role).

Example 2 - pack-swc-001's Attributes

For our second example, we'll take a look at the pack-swc-001 host, which belongs to the arista and manchester groups. Interestingly, both groups define a username and password. So, which credentials does the host actually use?

It will use the ones from the arista group. Why? Because arista is listed first in the groups section of our host's definition. This is an important thing to remember. The order of the groups in the inventory matters. Nornir will give priority to the settings from the first group in the list when resolving conflicts between groups.

So in our case, pack-swc-001 will inherit the username and password from arista, which are arista_user and supersecret respectively. The credentials from manchester will not be applied here, but other non-conflicting settings from manchester will still be inherited.

In [31]: host = nr.inventory.hosts['pack-swc-001']

In [32]: host.username
Out[32]: 'arista_user'

In [33]: host.password
Out[33]: 'supersecret'

Hiding Secrets in Nornir

So far, we've kept our usernames and passwords in YAML files (hosts, groups, or defaults) for the sake of simplicity. However, this isn't secure, is it? We shouldn't be saving sensitive information in plain text files. So, what can we do about this?

As I mentioned at the very beginning of this course, Nornir is pure Python, which means we can use any Python method to hide our secrets. Going back to our username and password example, you don't have to store them in the YAML files. Instead, you can pass them as attributes after initializing Nornir, like this.

from nornir import InitNornir
from nornir_netmiko.tasks import netmiko_send_command
from nornir_utils.plugins.functions import print_result

nr = InitNornir(
    config_file="config.yaml",
    runner={
        "options": {
            "num_workers": 1
        }
    }
)

nr.inventory.defaults.username = 'admin'
nr.inventory.defaults.password = 'admin'

results = nr.run(task=netmiko_send_command, command_string='show ip arp')
print_result(results)

If you want to have group-specific passwords, you can assign them directly to specific groups like this.

nr.inventory.groups['cisco'].username = 'cisco_admin'
nr.inventory.groups['cisco'].password = 'cisco_pass'

But of course, our ultimate goal was to hide them, right? You can use Python's getpass or environment variables to hide these credentials. Here’s how you can do it with getpass

from nornir import InitNornir
from nornir_netmiko.tasks import netmiko_send_command
from nornir_utils.plugins.functions import print_result
import getpass

username = getpass.getpass('Please enter the username: ')
password = getpass.getpass('Please enter the password: ')

nr = InitNornir(
    config_file="config.yaml",
    runner={
        "options": {
            "num_workers": 1
        }
    }
)

nr.inventory.groups['cisco'].username = username
nr.inventory.groups['cisco'].password = username

results = nr.run(task=netmiko_send_command, command_string='show ip arp')
print_result(results)

Inventory Filtering

Inventory filtering is another useful feature of Nornir. For example, say you only want to run a command or make changes to a specific group of devices. This is exactly what inventory filtering is for. You can choose exactly which devices you want to work with by filtering your inventory based on any of the attributes. We'll get into how this works with some hands-on examples next.

Let's start with our hosts.yaml file that we’ve been using. If we take a look at the hosts without any filtering, you'll see all the five devices listed.

In [1]: from nornir import InitNornir
   ...: nr = InitNornir(config_file="config.yaml")

In [2]: nr.inventory.hosts
Out[2]: 
{'pack-swa-001': Host: pack-swa-001,
 'pack-swa-002': Host: pack-swa-002,
 'pack-swd-001': Host: pack-swd-001,
 'pack-swc-001': Host: pack-swc-001,
 'pack-swc-002': Host: pack-swc-002}

Now, suppose we want to run tasks only on devices that have a specific role like 'campus' or 'edge'. We can use filtering to narrow down our target hosts using the syntax nr_object.filter()

In [4]: nr.filter(role="edge").inventory.hosts
Out[4]: {'pack-swc-001': Host: pack-swc-001, 'pack-swc-002': Host: pack-swc-002}

In [5]: nr.filter(role="campus").inventory.hosts
Out[5]: 
{'pack-swa-001': Host: pack-swa-001,
 'pack-swa-002': Host: pack-swa-002,
 'pack-swd-001': Host: pack-swd-001}

You can also filter the inventory with multiple attributes as shown below. Here, we can look for devices that have a role 'campus' and type 'distribution'.

In [2]: nr.filter(role="campus", type="distribution").inventory.hosts

Out[2]: {'pack-swd-001': Host: pack-swd-001}

Advanced Filtering - Filter Functions

Nornir's Filter Function allows you to use custom logic to filter hosts. By defining a function that takes a host object and returns True or False, you can control exactly which hosts pass through the filter.

In the following example, we've created a function named has_swa that checks if the string 'swa' is part of the host's name.

from nornir import InitNornir

nr = InitNornir(config_file="config.yaml")

def has_swa(host):
    return 'swa' in host.name

nr.filter(filter_func=has_swa).inventory.hosts

#output
{'pack-swa-001': Host: pack-swa-001, 'pack-swa-002': Host: pack-swa-002}

Here is another trick I found in this blog. Here I can target specific hosts that are part of a specific platform.

def filter_platforms(task):
    platforms = ["ios", "eos"]
    return task.host.platform in platforms

nr.filter(filter_func=filter_platforms)

Advanced Filtering - Filter Object

The Filter Object in Nornir lets you build complex filters easily. It's useful when you need to apply multiple criteria to find specific devices in your inventory.

In this first example, we're using the Filter Object (F) from Nornir to filter hosts based on their group membership. First, we import the necessary modules and initialize Nornir with our configuration file.

In [1]: from nornir import InitNornir
   ...: from nornir.core.filter import F
   ...: 
   ...: nr = InitNornir(config_file="config.yaml")
   ...: 
   ...: arista = nr.filter(F(groups__contains="arista"))
   ...: arista.inventory.hosts
Out[1]: {'pack-swc-001': Host: pack-swc-001, 'pack-swc-002': Host: pack-swc-002}

Next, we create a filter named arista using the Filter Object. We specify that we want to filter hosts whose groups attribute contains the string "arista". This filter is then applied to our Nornir inventory.

In this second example, let's look at how to use the logical OR operator (|) with the Filter Object in Nornir to combine filters. This operator allows you to specify multiple conditions, and it will match hosts that satisfy at least one of the conditions. Here, we use the Filter Object F to create a condition where the host's type can be either "access" or "distribution".

In [4]: from nornir import InitNornir
   ...: from nornir.core.filter import F
   ...: 
   ...: nr = InitNornir(config_file="config.yaml")
   ...: 
   ...: arista = nr.filter(F(type="access") | F(type="distribution"))
   ...: arista.inventory.hosts
Out[4]: 
{'pack-swa-001': Host: pack-swa-001,
 'pack-swa-002': Host: pack-swa-002,
 'pack-swd-001': Host: pack-swd-001}

Here is another exmaple using the AND (&) operator.

In [7]: from nornir import InitNornir
   ...: from nornir.core.filter import F
   ...: 
   ...: nr = InitNornir(config_file="config.yaml")
   ...: 
   ...: arista = nr.filter(F(type="access") & F(role="campus"))
   ...: arista.inventory.hosts
Out[7]: {'pack-swa-001': Host: pack-swa-001, 'pack-swa-002': Host: pack-swa-002}

If you want to learn more about Nornir Filtering, here is the official guide