Operations with Python
Table of Contents
preparing the work environment#
First, I have the habit of working in virtual environments. I create one and activate it:
Wassim@linux:~/private/labscripts$ python -m venv venv
Wassim@linux:~/private/labscripts$ ls
venv
Wassim@linux:~/private/labscripts$ source venv/bin/activate
((venv) ) Wassim@linux:~/private/labscripts$
I initially wanted to install a virtual environment for Linux and another one for Windows in the same directory. It looks like the installer failed me silently. No exe or bat files were installed. So, I deleted the venv directory that I created weeks ago on my Linux machine using rmdir /S, and recreated it on Windows with python -m venv .\venv.
I upgrade the pip package:
((venv) ) Wassim@linux:~/private/labscripts$ pip install --upgrade pip
Requirement already satisfied: pip in ./venv/lib64/python3.12/site-packages (23.3.2)
Collecting pip
Downloading pip-26.2.1-py3-none-any.whl.metadata (4.6 kB)
Downloading pip-26.2.1-py3-none-any.whl (1.8 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.8/1.8 MB 19.3 MB/s eta 0:00:00
Installing collected packages: pip
Attempting uninstall: pip
Found existing installation: pip 23.3.2
Uninstalling pip-23.3.2:
Successfully uninstalled pip-23.3.2
Successfully installed pip-26.2.1
((venv) ) Wassim@linux:~/private/labscripts$
I only have the pip package installed. I need pyEZ, Netmiko.
Installing pyEZ:
((venv) ) Wassim@linux:~/private/labscripts$ pip install junos-eznc
Collecting junos-eznc
Downloading junos_eznc-2.8.2-py2.py3-none-any.whl.metadata (1.9 kB)
Collecting lxml>=3.2.4 (from junos-eznc)
Downloading lxml-6.1.2-cp312-cp312-manylinux_2_26_x86_64.manylinux_2_28_x86_64.whl.metadata (3.3 kB)
Collecting ncclient==0.7.0 (from junos-eznc)
Using cached ncclient-0.7.0-py3-none-any.whl
Collecting scp>=0.7.0 (from junos-eznc)
Downloading scp-0.16.1-py2.py3-none-any.whl.metadata (4.6 kB)
Collecting jinja2>=3.1.6 (from junos-eznc)
Using cached jinja2-3.1.6-py3-none-any.whl.metadata (2.9 kB)
Collecting PyYAML>=5.1 (from junos-eznc)
Using cached pyyaml-6.0.3-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl.metadata (2.4 kB)
Collecting paramiko>=3.5.0 (from junos-eznc)
Downloading paramiko-5.0.0-py3-none-any.whl.metadata (3.7 kB)
Collecting six (from junos-eznc)
Using cached six-1.17.0-py2.py3-none-any.whl.metadata (1.7 kB)
Collecting pyserial (from junos-eznc)
Using cached pyserial-3.5-py2.py3-none-any.whl.metadata (1.6 kB)
Collecting yamlloader (from junos-eznc)
Using cached yamlloader-1.6.0-py3-none-any.whl.metadata (8.5 kB)
Collecting pyparsing (from junos-eznc)
Downloading pyparsing-3.3.2-py3-none-any.whl.metadata (5.8 kB)
Collecting transitions (from junos-eznc)
Using cached transitions-0.9.3-py2.py3-none-any.whl.metadata (97 kB)
Collecting MarkupSafe>=2.0 (from jinja2>=3.1.6->junos-eznc)
Using cached markupsafe-3.0.3-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl.metadata (2.7 kB)
Collecting bcrypt>=3.2 (from paramiko>=3.5.0->junos-eznc)
Using cached bcrypt-5.0.0-cp39-abi3-manylinux_2_34_x86_64.whl.metadata (10 kB)
Collecting cryptography>=3.3 (from paramiko>=3.5.0->junos-eznc)
Downloading cryptography-50.0.0-cp311-abi3-manylinux_2_34_x86_64.whl.metadata (4.3 kB)
Collecting invoke>=2.0 (from paramiko>=3.5.0->junos-eznc)
Downloading invoke-3.0.3-py3-none-any.whl.metadata (3.2 kB)
Collecting pynacl>=1.5 (from paramiko>=3.5.0->junos-eznc)
Using cached pynacl-1.6.2-cp38-abi3-manylinux_2_34_x86_64.whl.metadata (10.0 kB)
Collecting cffi>=2.0.0 (from cryptography>=3.3->paramiko>=3.5.0->junos-eznc)
Downloading cffi-2.1.1-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.metadata (2.5 kB)
Collecting pycparser (from cffi>=2.0.0->cryptography>=3.3->paramiko>=3.5.0->junos-eznc)
Downloading pycparser-3.0-py3-none-any.whl.metadata (8.2 kB)
Downloading junos_eznc-2.8.2-py2.py3-none-any.whl (217 kB)
Using cached jinja2-3.1.6-py3-none-any.whl (134 kB)
Downloading lxml-6.1.2-cp312-cp312-manylinux_2_26_x86_64.manylinux_2_28_x86_64.whl (5.2 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 5.2/5.2 MB 28.0 MB/s 0:00:00
Using cached markupsafe-3.0.3-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl (22 kB)
Downloading paramiko-5.0.0-py3-none-any.whl (208 kB)
Using cached bcrypt-5.0.0-cp39-abi3-manylinux_2_34_x86_64.whl (278 kB)
Downloading cryptography-50.0.0-cp311-abi3-manylinux_2_34_x86_64.whl (4.7 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 4.7/4.7 MB 27.4 MB/s 0:00:00
Downloading cffi-2.1.1-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (221 kB)
Downloading invoke-3.0.3-py3-none-any.whl (160 kB)
Using cached pynacl-1.6.2-cp38-abi3-manylinux_2_34_x86_64.whl (1.4 MB)
Using cached pyyaml-6.0.3-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl (807 kB)
Downloading scp-0.16.1-py2.py3-none-any.whl (8.9 kB)
Downloading pycparser-3.0-py3-none-any.whl (48 kB)
Downloading pyparsing-3.3.2-py3-none-any.whl (122 kB)
Using cached pyserial-3.5-py2.py3-none-any.whl (90 kB)
Using cached six-1.17.0-py2.py3-none-any.whl (11 kB)
Using cached transitions-0.9.3-py2.py3-none-any.whl (112 kB)
Using cached yamlloader-1.6.0-py3-none-any.whl (7.8 kB)
Installing collected packages: pyserial, six, PyYAML, pyparsing, pycparser, MarkupSafe, lxml, invoke, bcrypt, yamlloader, transitions, jinja2, cffi, pynacl, cryptography, paramiko, scp, ncclient, junos-eznc
Successfully installed MarkupSafe-3.0.3 PyYAML-6.0.3 bcrypt-5.0.0 cffi-2.1.1 cryptography-50.0.0 invoke-3.0.3 jinja2-3.1.6 junos-eznc-2.8.2 lxml-6.1.2 ncclient-0.7.0 paramiko-5.0.0 pycparser-3.0 pynacl-1.6.2 pyparsing-3.3.2 pyserial-3.5 scp-0.16.1 six-1.17.0 transitions-0.9.3 yamlloader-1.6.0
((venv) ) Wassim@linux:~/private/labscripts$
I install the netmiko package:
((venv) ) Wassim@linux:~/private/labscripts$ pip install netmiko
Collecting netmiko
Downloading netmiko-4.7.0-py3-none-any.whl.metadata (8.3 kB)
Collecting ntc-templates>=3.1.0 (from netmiko)
Downloading ntc_templates-9.2.0-py3-none-any.whl.metadata (4.2 kB)
Collecting paramiko<5.0,>=3.5.0 (from netmiko)
Using cached paramiko-4.0.0-py3-none-any.whl.metadata (3.9 kB)
Requirement already satisfied: pyserial>=3.3 in ./venv/lib64/python3.12/site-packages (from netmiko) (3.5)
Requirement already satisfied: pyyaml>=6.0.2 in ./venv/lib64/python3.12/site-packages (from netmiko) (6.0.3)
Collecting rich>=13.8 (from netmiko)
Downloading rich-15.0.0-py3-none-any.whl.metadata (18 kB)
Collecting ruamel.yaml>=0.17 (from netmiko)
Using cached ruamel_yaml-0.19.1-py3-none-any.whl.metadata (16 kB)
Requirement already satisfied: scp>=0.13.6 in ./venv/lib64/python3.12/site-packages (from netmiko) (0.16.1)
Collecting textfsm>=1.1.3 (from netmiko)
Using cached textfsm-2.1.0-py2.py3-none-any.whl.metadata (2.7 kB)
Requirement already satisfied: bcrypt>=3.2 in ./venv/lib64/python3.12/site-packages (from paramiko<5.0,>=3.5.0->netmiko) (5.0.0)
Requirement already satisfied: cryptography>=3.3 in ./venv/lib64/python3.12/site-packages (from paramiko<5.0,>=3.5.0->netmiko) (50.0.0)
Requirement already satisfied: invoke>=2.0 in ./venv/lib64/python3.12/site-packages (from paramiko<5.0,>=3.5.0->netmiko) (3.0.3)
Requirement already satisfied: pynacl>=1.5 in ./venv/lib64/python3.12/site-packages (from paramiko<5.0,>=3.5.0->netmiko) (1.6.2)
Requirement already satisfied: cffi>=2.0.0 in ./venv/lib64/python3.12/site-packages (from cryptography>=3.3->paramiko<5.0,>=3.5.0->netmiko) (2.1.1)
Requirement already satisfied: pycparser in ./venv/lib64/python3.12/site-packages (from cffi>=2.0.0->cryptography>=3.3->paramiko<5.0,>=3.5.0->netmiko) (3.0)
Collecting markdown-it-py>=2.2.0 (from rich>=13.8->netmiko)
Downloading markdown_it_py-4.2.0-py3-none-any.whl.metadata (7.4 kB)
Collecting pygments<3.0.0,>=2.13.0 (from rich>=13.8->netmiko)
Downloading pygments-2.21.0-py3-none-any.whl.metadata (2.5 kB)
Collecting mdurl~=0.1 (from markdown-it-py>=2.2.0->rich>=13.8->netmiko)
Using cached mdurl-0.1.2-py3-none-any.whl.metadata (1.6 kB)
Downloading netmiko-4.7.0-py3-none-any.whl (294 kB)
Using cached paramiko-4.0.0-py3-none-any.whl (223 kB)
Downloading ntc_templates-9.2.0-py3-none-any.whl (649 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 649.8/649.8 kB 15.5 MB/s 0:00:00
Downloading rich-15.0.0-py3-none-any.whl (310 kB)
Downloading pygments-2.21.0-py3-none-any.whl (1.3 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.3/1.3 MB 22.0 MB/s 0:00:00
Downloading markdown_it_py-4.2.0-py3-none-any.whl (91 kB)
Using cached mdurl-0.1.2-py3-none-any.whl (10.0 kB)
Using cached ruamel_yaml-0.19.1-py3-none-any.whl (118 kB)
Using cached textfsm-2.1.0-py2.py3-none-any.whl (44 kB)
Installing collected packages: textfsm, ruamel.yaml, pygments, ntc-templates, mdurl, markdown-it-py, rich, paramiko, netmiko
Attempting uninstall: paramiko
Found existing installation: paramiko 5.0.0
Uninstalling paramiko-5.0.0:
Successfully uninstalled paramiko-5.0.0
Successfully installed markdown-it-py-4.2.0 mdurl-0.1.2 netmiko-4.7.0 ntc-templates-9.2.0 paramiko-4.0.0 pygments-2.21.0 rich-15.0.0 ruamel.yaml-0.19.1 textfsm-2.1.0
((venv) ) Wassim@linux:~/private/labscripts$
The list of installed packages grew. This is because junos-eznc and netmiko require additional packages.
((venv) ) Wassim@linux:~/private/labscripts$ pip list
Package Version
-------------- -------
bcrypt 5.0.0
cffi 2.1.1
cryptography 50.0.0
invoke 3.0.3
Jinja2 3.1.6
junos-eznc 2.8.2
lxml 6.1.2
markdown-it-py 4.2.0
MarkupSafe 3.0.3
mdurl 0.1.2
ncclient 0.7.0
netmiko 4.7.0
ntc_templates 9.2.0
paramiko 4.0.0
pip 26.2.1
pycparser 3.0
Pygments 2.21.0
PyNaCl 1.6.2
pyparsing 3.3.2
pyserial 3.5
PyYAML 6.0.3
rich 15.0.0
ruamel.yaml 0.19.1
scp 0.16.1
six 1.17.0
textfsm 2.1.0
transitions 0.9.3
yamlloader 1.6.0
((venv) ) Wassim@linux:~/private/labscripts$
Configuring Junos using Python scripts#
prepare the NETCONF service#
I need to activate NETCONF first. As a reminder, this is a non-productive, isolated virtual lab environment. So I do not mind including the complete output of the key generation process:
root@leaf1:~ # cli
root@leaf1> show configuration system services
root@leaf1> edit
Entering configuration mode
[edit]
root@leaf1# set system services netconf ssh
[edit]
root@leaf1# commit and-quit
Generating RSA2 key /etc/ssh/ssh_host_rsa_key
Generating public/private rsa key pair.
Your identification has been saved in /etc/ssh/ssh_host_rsa_key
Your public key has been saved in /etc/ssh/ssh_host_rsa_key.pub
The key fingerprint is:
SHA256:56yVAnFp/VYywpu05Z8CMo/RPFRDqT69UZWJvf5kUj0 root@leaf1
The key's randomart image is:
+---[RSA 2048]----+
| .+.o o|
| + ..o +.|
| . + *.+ ..o|
| + =.O +.Eo|
| . S.X.+....|
| . Xo=o..oo|
| o *..oo+.|
| + .. .|
| . |
+----[SHA256]-----+
Generating ECDSA_P256 key /etc/ssh/ssh_host_ecdsa_key
Generating public/private ecdsa key pair.
Your identification has been saved in /etc/ssh/ssh_host_ecdsa_key
Your public key has been saved in /etc/ssh/ssh_host_ecdsa_key.pub
The key fingerprint is:
SHA256:oMEJE28oBNwJMBrQLes10m1RB+usTb/J2c1/nG/goMs root@leaf1
The key's randomart image is:
+---[ECDSA 256]---+
|@+*o. .o.. |
|o+oO... o |
|o .+*.... |
| .o.+oo+ |
| . o.o S |
| . + . . . |
| . . .. o o.|
| o.= o..+|
| E.. oo=|
+----[SHA256]-----+
Generating ECDSA_P384 key /etc/ssh/ssh_host_ec_p384_key
Generating public/private ecdsa key pair.
Your identification has been saved in /etc/ssh/ssh_host_ec_p384_key
Your public key has been saved in /etc/ssh/ssh_host_ec_p384_key.pub
The key fingerprint is:
SHA256:xj0kUalrEB3GOq7g283KG2lcu4S3BwMEZqwVzTZoTZ4 root@leaf1
The key's randomart image is:
+---[ECDSA 384]---+
| .=O. o+o.. |
| o=.B...o.. |
| + oE. o... |
|. . +..+ |
| o.oS.o |
| .. ++oo . |
| . .*.++ |
| .+.* o. |
| ..=o=. |
+----[SHA256]-----+
Generating ECDSA_P521 key /etc/ssh/ssh_host_ec_p521_key
Generating public/private ecdsa key pair.
Your identification has been saved in /etc/ssh/ssh_host_ec_p521_key
Your public key has been saved in /etc/ssh/ssh_host_ec_p521_key.pub
The key fingerprint is:
SHA256:HWLxt+z7oX5NgTVrM7IdiLhqkVV32os9XlKe9pPJG/s root@leaf1
The key's randomart image is:
+---[ECDSA 521]---+
| . |
| o . . + |
| o = + B o|
| . = = * O.|
| S o o BoB|
| o . . o.B+|
| o . ==*|
| o o.B=|
| . .+o.oE|
+----[SHA256]-----+
Generating ED25519 key /etc/ssh/ssh_host_ed25519_key
Generating public/private ed25519 key pair.
Your identification has been saved in /etc/ssh/ssh_host_ed25519_key
Your public key has been saved in /etc/ssh/ssh_host_ed25519_key.pub
The key fingerprint is:
SHA256:GTnu0CBp8RhQbEv3oT2h2lZgnfLI11/evFxjggC3t8Q root@leaf1
The key's randomart image is:
+--[ED25519 256]--+
| .++ . . |
| +*=.=o |
| o==oX*+o |
| ...=+B*.E . |
| o.oS.+.oo o |
| . oo o...o+|
| . . + +|
| o |
| |
+----[SHA256]-----+
commit complete
Exiting configuration mode
root@leaf1>
root@leaf1> show configuration system services
netconf {
ssh;
}
root@leaf1>
To be able to reach the OoB mgmt addresses of the Junos devices in the virtual lab, I had to set up static routing:
- added a default route on each Junos device toward the OoB gateway,
- added a static route on my Linux machine using
nmcli connection modify {Conn_Name} +ipv4.routes "{Prefix/MaskLength Next_Hop_IP_addr}"
I verify that I can establish a NETCONF session to a Junos device:
Wassim@linux:~$ nc -v 172.17.81.44 830
Ncat: Version 7.92 ( https://nmap.org/ncat )
Ncat: Connected to 172.17.81.44:830.
SSH-2.0-JSSH_4.1
^C
Wassim@linux:~$
prepare the SSH service#
I will also need SSH service since I might use Netmiko. I needed to add the ssh version and the port number. Otherwise, the connection was still refused by the Junos device:
Wassim@linux:~$ ssh [email protected]
ssh: connect to host 172.17.81.44 port 22: Connection refused
Wassim@linux:~$
[edit system services ssh]
root@leaf1# show
protocol-version v2;
port 22;
[edit system services ssh]
root@leaf1#
Wassim@linux:~$ ssh [email protected]
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@ WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED! @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY!
Someone could be eavesdropping on you right now (man-in-the-middle attack)!
It is also possible that a host key has just been changed.
The fingerprint for the ED25519 key sent by the remote host is
SHA256:GTnu0CBp8RhQbEv3oT2h2lZgnfLI11/evFxjggC3t8Q.
Please contact your system administrator.
Add correct host key in /home/Wassim/.ssh/known_hosts to get rid of this message.
Offending ED25519 key in /home/Wassim/.ssh/known_hosts:107
Host key for 172.17.81.44 has changed and you have requested strict checking.
Host key verification failed.
Wassim@linux:~$
My machine seem to have stored an older version of the Junos device fingerprint. I modified the fingerprint entry for the host with the new value:
Wassim@linux:~$ vim /home/Wassim/.ssh/known_hosts
Old value:
172.17.81.44 ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIPPZFFLO6dRGan/hhXDFOw1u8K9nl7pLWAkrXqzmGkiS
New value:
172.17.81.44 ssh-ed25519 SHA256:GTnu0CBp8RhQbEv3oT2h2lZgnfLI11/evFxjggC3t8Q
The error message disappeared:
Wassim@linux:~$ ssh [email protected]
The authenticity of host '172.17.81.44 (172.17.81.44)' can't be established.
ED25519 key fingerprint is SHA256:GTnu0CBp8RhQbEv3oT2h2lZgnfLI11/evFxjggC3t8Q.
This key is not known by any other names.
Are you sure you want to continue connecting (yes/no/[fingerprint])? ^C
Wassim@linux:~$
Wassim@linux:~$ nc -v 172.17.81.44 22
Ncat: Version 7.92 ( https://nmap.org/ncat )
Ncat: Connected to 172.17.81.44:22.
SSH-2.0-JSSH_4.1
^C
Wassim@linux:~$
habits#
When I create Python scripts, I apply some rules, whenever possible:
- give the owner full permissions,
- give the group only read and execute permissions,
- start the first line of the script with the Shebang.
((venv) ) Wassim@linux:~/private/Juniper/devopstrack$ ls -l mylab03-Setup.py
-rw-r--r--. 1 Wassim Wassim 474 Aug 25 09:30 mylab03-Setup.py
((venv) ) Wassim@linux:~/private/Juniper/devopstrack$
((venv) ) Wassim@linux:~/private/Juniper/devopstrack$ chmod ug+x mylab03-Setup.py
((venv) ) Wassim@linux:~/private/Juniper/devopstrack$ ls -l mylab03-Setup.py
-rwxr-xr--. 1 Wassim Wassim 474 Aug 25 09:30 mylab03-Setup.py
((venv) ) Wassim@linux:~/private/Juniper/devopstrack$
#!/usr/bin/python3
code the IP underlay of a CLOS architecture using Junos#
The IP underlay can be used either for a DC VXLAN-based network design or a seamless VXLAN stitching for DCI design.
I created a python module that performs the following:
- configure the point-to-point interfaces between leaves and spines,
- configure each switch in its own BGP ASN,
- configure BGP groups,
- within the BGP groups, configure BGP neighborships
I used:
- JINJA2 templates to structure the Junos set commands and define the variables,
- CSV files to define the values for the JINJA2 variables,
- python functions that I defined in a module I called myFunctions.py
- the public Juniper PyEZ Cookbook document.
code a VXLAN eBGP-based underlay in a CLOS architecture using Junos#
I am creating a python module that performs the following:
- configure a routing policy for the loopback 0 interface,
- export the routing policy to BGP,
- for each leaf: create a dedicated BGP group for peering with the loopback interfaces of the spines,
- for each spine: create a dedicated BGP group for peering with the loopback interfaces of the leaves,
- use the loopback interface as the BGP local address,
encountered problems#
In the CSV file, I had IP addresses in the form {IP/CIDR}. But I was calling those IP addresses in the JINJA2 template, not aware that the definition of BGP neighbors requires IP addresses that are stripped from their CIDR subnet mask.
Using hyphens in parameter name in JINJA2 templates is not supported. #LessonLearned if ever needed, use underscores in parameter names, not hyphens.
Different locations for JINJA2 and CSV files. I decided to put them all under a templates directory.
I used different naming conventions for the JINJA2 parameters and the CSV headers. #LessonLearned use the same variable names when combining JINJA2 templates with CSV value store.
I was relying too much on explanatory python error messages instead of making sure that my code mentally compiles successfully.
When populating the dictionary that will be used as template values in the pyEZ load() function, I was visually switching between the JINJA2 file and the CSV file which incurred unnecessary I/Os in my concentration. I decided then to fill all the dictionary keys at once while having the JINJA2 file open, then close it, open the CSV file and fill all the dictionary values.
#Observation Using a CSV file as a repository of the values for the JINJA2 template became harder to manage beyond a five-node CLOS architecture.
I received the following error message:
File "/home/Wassim/.local/lib/python3.12/site-packages/jnpr/junos/utils/config.py", line 506, in try_load
raise ConfigLoadError(cmd=err.cmd, rsp=err.rsp, errs=err.errs)
jnpr.junos.exception.ConfigLoadError: ConfigLoadError(severity: error, bad_element: protocol, message: error: syntax error
error: syntax error
error: syntax error
error: syntax error
I reduced the exposed function surface by commenting out some sections. I found out that protocol was used in my JINJA2 template. I corrected it to protocols. #LessonLearned always test the Junos syntax on the CLI before reproducing it in JINJA2 templates.
I coded eBGP neighborships on the loopback interfaces but forgot that IP routing between the loopback interfaces was not established. #LessonLearned Verify that BGP speakers can reach each other source interface in RIB as a requirement for BGP peering.
I was not getting eBGP neighborship between a leaf and a spine, despite that I double-checked the BGP configuration. It happened that the IP address of one loopback interface was wrong:
root@leaf2> show route 192.168.100.12
inet.0: 10 destinations, 10 routes (10 active, 0 holddown, 0 hidden)
Limit/Threshold: 1048576/1048576 destinations
+ = Active Route, - = Last Active, * = Both
0.0.0.0/0 *[Static/5] 1w1d 00:27:51
> to 172.17.81.1 via fxp0.0
root@leaf2>
and the spine fabric interface toward leaf3 was wrong in the CSV template, 172.17. instead of 172.16, which broke the BGP neighborship logic, because the JINJA2 template was using the CSV file:
grep 172.17 dcunderlayspinesscript1.csv
172.17.81.42,spine1,ge-0/0/0,0,172.16.1.0,172.16.1.0/31,ge-0/0/1,0,172.16.1.2,172.16.1.2/31,ge-0/0/2,0,172.17.1.4,172.17.1.4/31,DCinfraUnderlayEBGPgrp,64991,64881,64882,64883,192.168.100.1,172.16.1.1,172.16.1.3,172.16.1.5,192.168.100.11,192.168.100.12,192.168.100.13
172.17.81.43,spine2,ge-0/0/0,0,172.16.1.6,172.16.1.6/31,ge-0/0/1,0,172.16.1.8,172.16.1.8/31,ge-0/0/2,0,172.16.1.10,172.16.1.10/31,DCinfraUnderlayEBGPgrp,64992,64881,64882,64883,192.168.100.2,172.16.1.7,172.16.1.9,172.16.1.11,192.168.100.11,192.168.100.12,192.168.100.13
Wassim@linux:~/privateworksync/templates/mylab03$
cat mylab03dcVxlanUnderlaySpines.jinja2
set policy-options policy-statement advertlo0 term 1 from interface lo0
set policy-options policy-statement advertlo0 term 1 then accept
set policy-options policy-statement advertlo0 term 2 then reject
set protocols bgp group vxlanUnderlay multihop
set protocols bgp group vxlanUnderlay export advertlo0
set protocols bgp group vxlanUnderlay type external
set protocols bgp group vxlanUnderlay neighbor {{ leaf1_lo0ip }} peer-as {{ leaf1ASN }} local-as {{ bgpasn }}
set protocols bgp group vxlanUnderlay neighbor {{ leaf1_lo0ip }} local-address {{ lo0_ip }}
set protocols bgp group vxlanUnderlay neighbor {{ leaf2_lo0ip }} peer-as {{ leaf2ASN }} local-as {{ bgpasn }}
set protocols bgp group vxlanUnderlay neighbor {{ leaf2_lo0ip }} local-address {{ lo0_ip }}
set protocols bgp group vxlanUnderlay neighbor {{ leaf3_lo0ip }} peer-as {{ leaf3ASN }} local-as {{ bgpasn }}
set protocols bgp group vxlanUnderlay neighbor {{ leaf3_lo0ip }} local-address {{ lo0_ip }}
set routing-options static route {{ leaf1_lo0ip }} next-hop {{ leaf1_fip }}
set routing-options static route {{ leaf2_lo0ip }} next-hop {{ leaf2_fip }}
set routing-options static route {{ leaf3_lo0ip }} next-hop {{ leaf3_fip }}
root@spine1> show configuration protocols bgp
group DCinfraUnderlayEBGPgrp {
type external;
multihop;
neighbor 172.16.1.1 {
local-address 172.16.1.0;
peer-as 64881;
local-as 64991;
}
neighbor 172.16.1.3 {
local-address 172.16.1.2;
peer-as 64882;
local-as 64991;
}
neighbor 172.16.1.5 {
local-address 172.17.1.4;
peer-as 64883;
local-as 64991;
}
}
group vxlanUnderlay {
type external;
multihop;
export advertlo0;
root@spine1>
#Observation I wanted to put all parameters in the same CSV template. I ended up with an overloaded file that hindered me from correctly assigning the values to the JINJA template. I was trying to come up with a name of the BGP ASN for the DC VXLAN EVPN overlay for each spine while the JINJA template is generic for all spines, which means that the configuration will be rendered with incorrectly assigned ASNs. #LessonLearned Simplification does not always mean reducing the number of files. I separated the underlay CSV from the overlay CSV. The files look slim and I can read the rows without suffocating.
#Observation I was troubleshooting this error message for hours:
raise ConfigLoadError(cmd=err.cmd, rsp=err.rsp, errs=err.errs)
jnpr.junos.exception.ConfigLoadError: ConfigLoadError(severity: error, bad_element: None, message: error: peer-as: 'local-as': Use format 'x' or 'x.y' to specify number 'x' (Range 1 .. 4294967295) or number with higher 2 bytes 'x' (Range 0 .. 65535) and lower 2 bytes 'y' (Range 0 .. 65535) respectively
error: syntax error, expecting [Enter])
Reviewing CSV and JINJA templates did not help. I found out later that I forgot a letter in the name of a key of the dictionary that is passed in the Config module of pyEZ. #LessonLearned Typing fast and light might cost me hours of unnecessary troubleshooting later.
used files#
mylab03Setup.py mylab03dcUnderlayInfraSpines.jinja2 dcunderlayspinesscript1.csv dcunderlayleavesscript1.csv dcVXLANoverlayleavesscript.csv dcVXLANoverlayspinesscript.csv
TBC#
Problem with big blobs#
I created a new git repo on GitLab and wanted to push some files there. Git threw me an error message. It seemed to me that git blobs of size 100 MB or more collide with the 100 MB file size limit of GitLab. The error message gave a hint on how to find out the file names. They were old Anki files. I tracked them with git lfs track, staged and committed. I repeated the process. I got the same error. I did my research and found a GitLab doc (docs.gitlab.com/user/free_push_limit/) that points to this error message. The suggested solution was to remove the commit that introduced the large files. So I created a dummy file, staged and committed, then rebased with git rebase -i to an earlier commit. It did not help. Then I compressed the triggering files with tar cvzf, made sure no archives were bigger than 95 MB, repeated the git push attempt. No success. Then I moved the triggering files completely out of the git repo and repeated the cycle, with no success either. So, I decided to create a new git repo altogether. I renamed the git database to .git.old and initialized a new repo using the existing files. I executed git lfs track as usual, git add .gitattributes, staged, committed, created a new remote with a fresh empty remote git repo. git push to the remote gave me another error:
Uploading LFS objects: 100% (4408/4408), 4.8 GB | 6.9 MB/s, done.
Enumerating objects: 27269, done.
Counting objects: 100% (27269/27269), done.
Delta compression using up to 18 threads
Compressing objects: 100% (24315/24315), done.
error: RPC failed; HTTP 413 curl 22 The requested URL returned error: 413
send-pack: unexpected disconnect while reading sideband packet
Writing objects: 100% (27269/27269), 5.31 GiB | 14.04 MiB/s, done.
Total 27269 (delta 2785), reused 27269 (delta 2785), pack-reused 0 (from 0)
fatal: the remote end hung up unexpectedly
Everything up-to-date
Wassim@linux:~/MyTechVault$
I found on stackoverflow (https://stackoverflow.com/questions/15240815/git-fatal-the-remote-end-hung-up-unexpectedly) that it could be caused by an inadequate buffer. I gave it a try, with no success. I binged the internet again and found a developer website that suggested switching from git pushing over HTTPS to over SSH. I generated a public-private keypair, copied the public key onto the GitLab project page, tested successful SSH connection from my machine to GitLab and relaunched the git push again, with no success. But this time the error message was different:
remote: fatal: pack exceeds maximum allowed size
I binged the internet again. I learned that the error could be related to Git trying to push files whose size exceeded 2 GiB. So, I created two local copies of my working directory and started careful movement of files from the copy directory to an empty directory that I wiped and initialized with git. This time it seemed to work. So I have extra manual effort to prepare my version control environment before I am able to say that my new Git repo is available on GitLab.
#StoppedAt
host the code on GitLab#
create a project on gitlab: gitlab dashboard -> Projects -> New project -> Create blank project. Select the correct namespace in the Project URL. Give the project a name and a project slug. Keep the visibility level to Private. Uncheck the “Initialize repository with a README” flag. Press “Create Project” button. The new project appears under the dashboard -> Projects after I log out and log in. The main page of the new project contains clear instructions of what to do next.
Under Settings -> Access tokens, create a write token with a “Maintainer” role and write_repository scope, to be able to push from a local git repo. I use Proton Pass to store my access tokens.
Under Settings -> Repository -> Protected branches, deactivate main branch protection or use the newer branch rule method, which is also under Settings -> Repository. I added a rule for main to allow merges and pushes only for Maintainers.
subprocess, type#
In a Python function I wanted to run a curl.exe command to retrieve the public IP address of my host machine. I am using the API endpoint from whatismyip.com with the free tier. That is sufficient to retrieve the public IP a couple of times through the day.
My first attempt was to simply print the complete command as a string, so that I can copy/paste it and run it asynchronously. Then I learned about the subprocess class. Here is my function that calls it out:
def getMyPubipv4Addr():
command = "curl.exe -H 'X-API-KEY: XXXXXXX' 'https://wimi-api-v4.whatismyip.com/ip'"
res = subprocess.run(["curl.exe", "-H", "X-API-KEY: XXXXXXX", "https://wimi-api-v4.whatismyip.com/ip"])
print(type(res))
print(res.stdout)
The output:
{"ip":"82.135.80.153"}<class 'subprocess.CompletedProcess'>
None
<class 'NoneType'>
Then I learned about the capture_output argument. I adapted my module:
def getMyPubipv4Addr():
command = "curl.exe -H 'X-API-KEY: XXXXXXX' 'https://wimi-api-v4.whatismyip.com/ip'"
res = subprocess.run(["curl.exe", "-H", "X-API-KEY: XXXXXXX", "https://wimi-api-v4.whatismyip.com/ip"], capture_output=True)
print(type(res))
print(res.stdout)
The output:
<class 'subprocess.CompletedProcess'>
b'{"ip":"82.135.80.153"}'
<class 'bytes'>
That is slightly better. I want to convert that byte object to a dict or a string, because I want later to manually inject the IP address to my Juniper lab. Then I learned that I could use json.loads() to convert a json-looking object out to a dict object. I updated my function:
def getMyPubipv4Addr():
command = "curl.exe -H 'X-API-KEY: XXXXXXX' 'https://wimi-api-v4.whatismyip.com/ip'"
res = subprocess.run(["curl.exe", "-H", "X-API-KEY: XXXXXXX", "https://wimi-api-v4.whatismyip.com/ip"], capture_output=True)
print(type(res))
print(res.stdout)
print(type(res.stdout))
print('preparing to convert the byte object to a dict object using json.loads')
b = res.stdout
c = json.loads(b.decode('utf-8'))
print("the json object is {} and has the type {}".format(c,type(c)))
print("the IPv4 addr is: {}".format(c['ip']))
The result is better:
<class 'subprocess.CompletedProcess'>
b'{"ip":"82.135.80.153"}'
<class 'bytes'>
preparing to convert the byte object to a dict object using json.loads
the json object is {'ip': '82.135.80.153'} and has the type <class 'dict'>
the IPv4 addr is: 82.135.80.153
the reliability of automation#
One of my scripts pushes a spine-leaf topology configuration. It ran successfully more than once. One day, I ran the script on the same topology but using a different lab environment. The switches were not able to ping each other consistently across the fabric. I verified with CLI that the configuration was correctly displayed. Nonetheless, the ping on an Ethernet segment failed.
jcluser@spine1> show configuration interfaces ge-0/0/0
unit 0 {
family inet {
address 172.16.1.0/31;
}
}
jcluser@spine1>
jcluser@leaf1> show configuration interfaces ge-0/0/0
unit 0 {
family inet {
address 172.16.1.1/31;
}
}
jcluser@leaf1>
jcluser@leaf1> ping 172.16.1.0 source 172.16.1.1
PING 172.16.1.0 (172.16.1.0): 56 data bytes
I thought, I needed to check if my script is addressing the correct devices based on my template. My CSV template contains a list of entries, where each one mentions the IP address and port numbers. I found out that I screwed the port numbers. I corrected them, reran the script. That fixed the issues.
jcluser@spine1> show bgp summary
Warning: License key missing; requires 'bgp' license
Threading mode: BGP I/O
Default eBGP mode: advertise - accept, receive - accept
Groups: 2 Peers: 6 Down peers: 0
Table Tot Paths Act Paths Suppressed History Damp State Pending
inet.0
3 3 0 0 0 0
bgp.evpn.0
0 0 0 0 0 0
Peer AS InPkt OutPkt OutQ Flaps Last Up/Dwn State|#Active/Received/Accepted/Damped...
172.16.1.1 64881 48 49 0 7 20:17 Establ
inet.0: 1/1/1/0
172.16.1.3 64882 209 218 0 0 1:32:06 Establ
inet.0: 1/1/1/0
172.16.1.5 64883 44 45 0 2 18:01 Establ
inet.0: 1/1/1/0
192.168.100.11 64665 4 3 0 0 38 Establ
bgp.evpn.0: 0/0/0/0
192.168.100.12 64665 4 3 0 0 34 Establ
bgp.evpn.0: 0/0/0/0
192.168.100.13 64665 4 3 0 0 30 Establ
bgp.evpn.0: 0/0/0/0
jcluser@spine1>
#LessonLearned Having tested the result of a script once and succeeded in a dynamic environment does not guarantee a deterministic future successful result, if the necessary preliminary work on the new environment does not match the intent of the script.
references#
- docs.gitlab.com/user/project/repository/branches/branch_rules/
- docs.gitlab.com/topics/git/project/
- git-lfs.com/
- docs.gitlab.com/topics/git/undo/#revert-commits-and-modify-history
- docs.gitlab.com/user/free_push_limit/
- geeksforgeeks.org/python/retrieving-the-output-of-subprocesscall-in-python/
- stackoverflow.com/questions/89228/how-do-i-execute-a-program-or-call-a-system-command
- stackoverflow.com/questions/49184578/how-to-convert-bytes-type-to-dictionary
- powershellfaqs.com/powershell-curl/
- whatismyip.com/api/#hosts