Creating Command Line Tools with Python

After spending considerable amount of time on a computer as a developer, the GUI starts to seem amazingly slow and you realize just how awesome command line tools can be. Especially if you follow the Unix philosophy of “one tool for one task”, you can quickly chain together multiple in-built tools and quickly accomplish most tasks without a single line of code.

However, what about when the tool you need doesn’t exist? Wouldn’t it be great to just create it? In this post, I’d like to show you how using Python. My reason for using python instead of the more native C/C++ is simple: easy to read code for even anybody new to programming to understand the basic structure, fast prototyping, a rich set of tools, and the general ease of use. You’re of course welcome to use other languages such as Ruby (I’ve seen a lot of great tools written in Ruby), but then you wouldn’t be reading this post, would you? Without further ado, let’s begin!

Project Setup

I faced a problem a couple of weeks ago where I needed a simple command to nuke a directory in order to rebuild some binaries and unfortunately no tool existed for this at the time. Hence I created nuke, a convenient command line tool that does exactly what it says, nuke directories.

To start, we create a project directory called nuke-tool. Inside nuke-tool, we create a directory called nuke and inside that directory, two files, nuke.py which will house all our main code logic and  __init__.py so that python is able to understand this is a package called nuke.

In the root of the project directory, we should also create a file test_nuke.py to create tests for our code. Don’t want to accidentally nuke something else now, do we? We also create a file called setup.py to aid pip in installing nuke. We’ll flesh these out one at a time.

I did mention a rich set of libraries to help us out in this process. I personally use argparse to help with argument parsing and clint from the awesome Kenneth Reitz to help with output text formatting and confirmation prompts (simple Yes/No prompts). Later on, you’ll see code snippets showing these libraries in action.

Nuke ’em

In any command line tool, the first thing we need is a way to parse command line arguments. It’s the arguments you pass to a command which are separated by spaces. For example, grep takes two arguments, a pattern and a file name grep main nuke.py.

In python, we can easily use the argparse module to help us create a simple argument parser. I wanted to provide two options to the user, an argument specifying the directory to nuke (with the default being the current directory), and a flag -y to override the confirmation prompt. The code to do this is:

import argparse
import os
.
.
.
def _argparse():
    parser = argparse.ArgumentParser("nuke")
    parser.add_argument("directory", nargs='?', default=os.getcwd(),
                        help="Directory to nuke! Default is current directory")
    parser.add_argument("-y", help="Confirm nuking", action="store_true")
                        args = parser.parse_args()
    return args

The code is pretty straightforward. All I am doing is creating an ArgumentParser object with the name nuke, and telling it that it can take two arguments, “directory” which is a positional argument and defaults to os.getcwd() the current directory, and “-y”, an optional argument to override the confirmation prompt I’ll elaborate later. The hyphen in the “-y” is what tells the parser that this is an optional argument, with action telling the parser to store True if it sees the flag else store False. For more information, I highly recommend checking out argparse.

Next we create our main method. This will be the entry point of our tool.

import os.path as osp
from clint.textui import prompt, colored

def main():
    args = _argparse()
    if args.y or \
    prompt.yn("Are you sure you want to nuke directory" +
              colored.blue(args.directory) + "?"):
        nuke(directory)

We call our argument parser to give us back the command line arguments. Next we make sure to confirm with the user that nuking the directory is really what they want, a.k.a. the confirmation prompt. This is important because we are deleting data and that is always dangerous. You don’t want to nuke an important directory. Thankfully, clint provides us with a super simple way to do this, and having the blue coloration with colored.blue lets us highlight the directory name to make it more eye-catching in addition to being really fancy!
The nuke() function is what does the bulk of the work, but I won’t go into much detail about that as you can check it out on the github repo. The idea is that you can write your own code instead so as to customize the tool’s functionality.

Installation

Now that we have a working python program, we still need to install it to our system so we can invoke it via the command line as nuke instead of python nuke. This is where we use the setup.py file we created earlier. The full details of what goes into that file can be found here. What is most useful to us is the line:

entry_points={
        'console_scripts': [
            'nuke = nuke.nuke:main',
        ],
    },

Those lines tell pip (which we use to install packages) that this program will be invoked as a console script, and what the command is and what should it invoke in our code. As you can see, the command is nuke and the entry point is the main function as we have defined it above.

Now we can finally run

pip install -e .

which tells pip to install the package from the local environment. Run a test on a sample directory by calling

mkdir sample && touch sample/random.py
ls sample
nuke sample
ls sample

and voila! You now have a working command line tool that nukes your specified directory.

Testing and Wrapping Up

Now here’s the hard part about command line tools: Since you don’t specify the python interpreter to use, you have to write your code without assuming any python version. This can get annoying to the point of harrowing, especially due to the major differences between Python 2 and Python 3 (at least until 2020 after which Python 2 will be officially retired). For the purposes of testing compatibility with various python versions, we can use a tool called tox. Tox is fantastic in that you can quickly install it, setup a configuration file using the startup script tox-quickstart and run tox to run your tests against the specified python versions. For nuke, I plan to support it for Python 2.7, 3.5 and 3.6+.

Hopefully, you’ve been a good developer and written tests for your code before I even mentioned it here. If not, be sure to take a look at pytest which will make your life really easy. This sort of flexibility and power is what the rich python environment provides us.

And with this you’re done! You should now have a working command line tool that can be installed and used with multiple python versions and is well-tested to provide guarantees of industrial strength. Hopefully this was an enlightening read and you feel a whole lot more confident about creating your own awesome tooling!

Eviva.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s