This module improves Python’s usefulness as a tool for writing shell one-liners. A brief usage example:
# show line numbers python -m oneliner -ne '"%5d: %s" % (NR, _)' < input.txt # convert unix timestamp to something more human readable date +%s | pyl -j ' => ' -line '_, datetime.datetime.fromtimestamp(int(_))' # 1355256353 => 2012-12-11 22:05:53 # triple space a file pyl -ne 'line + "\n"*2' < input.txt # double space only non-blank lines: pyl -ne '_ if re.match("^$", _) else _+"\n"' input.txt
Python is a wonderful general purpose scripting language with a great breadth of excellent libraries. While the simplicity of its syntax is often cited as a major strong point, I believe that the negative attitude towards any kind of syntax magic has prevented Python from becoming a useful tool for writing shell one-liners. When stacked against the likes of Ruby and Perl, Python is ill-equipped to be a part of a shell pipeline. Writing one-liners is possible, but their verbosity makes them impractical. This can be attributed to the following factors:
Lack of a read-print-loop command line option. Ruby and Perl provide the
-pswitches that makes them assume the following loop around code:
for line in sys.stdin: <code> if '-p' in options: sys.stdout.write(line)
No syntax magic and no special variables. Ruby and Perl provide a multitude of cryptic, yet useful variables such as:
$_ last input line $. current input line number $~ the match object returned by the last successful pattern match $& the string matched by last successful pattern match $3 the string matched by the 3rd group of the last successful pattern match $* sys.argv $> sys.stdout (default output file) $< sys.stdint (default input file) $; input field separator (default value to str.split()) $/ record separator (os.linesep) $$ os.getpid()
Lack of a flexible command line import mechanism. For example, Perl has the
perl -MDigest::MD5=md5_hex => from Digest::MD5 import md5_hex perl -MDigest::MD5 => import Digest::MD5
While CPython interpreter has the
-mswitch, it is not suitable for the task at hand. For example, the following one-liner will run random’s test suite, instead of printing a random number:
python -m random -c "print(random.randint(10))"
All these points add up to the verbosity of Python one-liners. The following example demonstrates this point:
# convert a date to a unix timestamp (using strptime) ruby -rdate -pe '$_=Date.strptime($_, "%Y-%m-%d").strftime("%s")' perl -MTime::Piece -pe '$_=Time::Piece->strptime($_, "%Y-%m-%d\n")->epoch()' python -c 'import sys,datetime; [sys.stdout.write(datetime.datetime.strptime("%Y-%m-%d", i).strftime("%s") for i in sys.stdin]'
But why would anyone want to write one-liners in Python, given these shortcomings and the available alternatives? I believe that when doing interactive work on the shell, the first solution that comes to mind is usually good enough. If that solution is a Python one, why not use it?
Python comes with all the building blocks needed to implement a practical method of writing one-liners. This module tries to address the issues outlined above. The command line interface is kept as close as that of Ruby and Perl as reasonable.
To help with the processing of input and output, oneliner provides the the
-lcommand line switches.
-n: assume the following loop around expressions or statements (the distinction will be clarified later):
for line in sys.stdin: ...
-n, but write the value of
lineto stdout at the end of each iteration:
for line in sys.stdin: ... sys.stdout.write(line)
-l: automatic line-ending processing. Roughly equivalent to:
for line in sys.stdin: line = line.strip(os.linesep) ... sys.stdout.write(line) sys.stdout.write(os.linesep)
Makes the following variables available in the local namespace of each one-liner:
_: The current input line. Unless the
-lswitch is given, the line separatator will be a part of this string.
W: Corresponds to the value of
re.split(delimiter, line)where delimiter is the value of the
-doption. Defaults to
wordslist will return an empty string instead of throwing an
IndexErrorwhen a non-existent item is referenced. This behavior is similar to that of arrays in Ruby and field variables in Awk.
NR: Current input line number.
FN: Current input file name. If oneliner is processing input from stdin
FNwill be equal to
<stdin>, otherwise it corresponds to the current input file given on the command line. For example:
echo example | python -m oneliner -ne '"%s:%s\t %s" % (FN, NR, L)' => <stdin>:1 example python -m oneliner -ne '"%s:%s\t %s" % (FN, NR, L)' example.txt => example1.txt:1 line 1
-Moptions and a mini-language for specifying imports. This is best illustrated by the following examples:
-m os,sys,re,pickle => import os, sys, re, pickle -m os -m sys -m re => import os, sys, re -m os sys re pickle => import os, sys, re, pickle -m os.path.[*] => from os.path import * -m os.path.[join,exists] => from os.path import join, exists -m subprocess=sub => import subprocess as sub -m datetime.[datetime=dt] => from datetime import datetime as dt -M os.path => from os.path import *
The latest stable version of python-oneliner is available on pypi and may be installed with pip.
$ pip install oneliner # latest stable version
Alternatively, you may simply put the oneline.py file anywhere in your load path.
- Support one-liners that don’t deal with input/output only. If
-pare not given, python-oneliner should behave mostly like
- Persistent variables in statement one-liners.
- The result of an expression one-liner is always written to stdout
- Define the behaviour of multiple expression/statements specified on the command line.
- Some means of emulating
- Add more examples.