Processing NETCONF with Python

In recent years, Python has become one of the de-facto software development languages in the automation and scripting world. Its benefits include an accessible and readable syntax, a just-in-time compilation/interpretation model that allows rapid development cycles, and a batteries included standard library that immediately lends itself to many common situations.

In this recipe, we'll make use of a Python script, netconf.py, to connect to a JUNOS OS device in order to issue CLI-like RPCs, much as in the Expect/TCL example. We'll do this using just the basic standard libraries available out of the box in Python, so there is little fussing about with pip or other package management tools.

Getting ready

In order to complete this recipe, make sure you've got access to a working JUNOS OS device and have completed the JUNOS NETCONF over SSH setup recipe. Additionally, you need a suitable Python development environment. In this case, we made use of macOS X and OpenBSD with Python 2.7.13.

How to do it...

The steps for the following recipe are as follows:

1. Import the necessary standard library modules that we're going to use. In this case, we just need access to basic system functionality, the subprocess module (for managing child processes), and the XML parsing library:

      #!/usr/bin/env python

      import sys
      import subprocess
      import xml.etree.ElementTree as ET

2. Create a Python object class to represent the NETCONF client, making use of the subprocess module in the Python standard library in order to call the underlying operating system's SSH client. Define an appropriate constructor and destructor function as shown:

    class NETCONFClient(object):

      DELIMITER = ']]>]]>\n'

       def __init__(self, hostname):
       self.ssh = subprocess.Popen([
             "/usr/bin/ssh",
             "-q",
             "-i", "JUNOS_auto_id_rsa",
             "-p", "830",
             "-s", 
          hostname,
          "netconf",
           ],
           stdin=subprocess.PIPE,
           stdout=subprocess.PIPE)

       def __del__(self):
          self.ssh.stdin.close()

3. Define a method to read from the NETCONF-over-SSH stream, in a chunked, line-by-line manner, attempting to parse the XML stream.

       def read(self):
         data=""
         for line in iter(self.ssh.stdout.readline,         
       NETCONFClient.DELIMITER):
           if line=='':
           raise IOError("ssh session ended unexpectedly")
       data += line

       return ET.fromstring(data)

4. Define a method to write to the NETCONF-over-SSH stream in order to issue RPCs:

      def cmdrpc(self, cmd):
          e = ET.Element("rpc")
          e.append(ET.Element("command", {'format': "text"}))
          e.find("command").text = cmd;
         self.ssh.stdin.write(ET.tostring(e))
         self.ssh.stdin.write(NETCONFClient.DELIMITER)

5. Write the main code to read the command-line arguments and instantiate the NETCONFClient object:

      if len(sys.argv) < 3:
         print "Usage: netconf.py hostname command"
         sys.exit(1)

         netconf = NETCONFClient("auto@"+str(sys.argv[1]))
         response = netconf.read()
         netconf.cmdrpc(" ".join(sys.argv[2:]))
         response = netconf.read()

         output=response.find(".//{urn:ietf:params:xml:ns:
                   netconf:base:1.0}output")
         config = response.find(".//{urn:ietf:params:xml:ns:
                   netconf:base:1.0}configuration-output")
         error = response.find(".//{urn:ietf:params:xml:ns:
                   netconf:base:1.0}error-message")

6. Output the response:

      if output != None:
        print output.text
      elif config != None:
        print config.text
      elif error != None:
        print error.text
      else:
        print "NETCONF server provided no usable response"

How it works...

Step 1 sets up the dependent standard library modules. In this case, we use only the well-trodden modules included with the standard Python distribution. The sys module provides access to the command-line environment. The subprocess module provides a flexible way of managing child processes. ElementTree is the Python built-in XML parsing environment.

In step 2, we create a Python new-style class with a constructor and a destructor. The constructor invokes the subprocess module in order to manage a child process consisting of an SSH client. We use the typical options of SSH to influence its behavior:

Option	Description
-q	Quiet mode. Typically omits message-of-the-day banners, which are not helpful for machine reading.
-i JUNOS_auto_id_rsa	Specify the private SSH key file.
-p 830	Establish TCP port 830 as the transport endpoint.
-s	Invoke the SSH subsytem specified (netconf).

The destructor attempts to clean up by closing the standard input stream to the SSH client, which will usually result in the SSH client disconnecting from the remote endpoint.

In step 3, we define a method to read data from the SSH client. The data is read line-by-line until we see the special NETCONF delimiter token. When we see that, we know a message has been completed and it is passed to the ElementTree routines for XML decomposition as a Python object.

In step 4, we define the complimenting output method — a function to write a command RPC. The method simply wraps the input parameter — which is the command line to be executed — in the necessary XML decoration in order to invoke the command RPC.

Step 5 is about putting it all together. We read the command-line arguments to determine the hostname and the command to use. Since most commands consist of multiple words, the user is expected to quote the command. For example:

   unix$ ./netconf.py 10.0.201.201 "show route summary"

We call the method to read data from the SSH stream in order to eat the hello message - we've no real need to understand its contents. Then we output a command RPC for the desired command, and call the read method once more in order to receive the response.

As we handle the response from the command RPC, we anticipate receiving one of three types of tag, as shown in the following table:

Tag	Description
<output>	Normal output from a show command or otherwise
<configuration-output>	Output from the show configuration command
<error-message>	An error message when something goes wrong

Note that the response.find() calls in step 5 make use of the so-called fully qualified XML tag name. The braces denote an XML namespace identifier. Namespaces allow the construction of XML documents comprising of multiple tag dictionaries from multiple sources without collision. They are a flexible tool, but they can make for wordy and verbose text.

Finally, in step 6, we print what we've discovered for the user's attention.