5.1 Input filters

By default, Pyxplot expects data files to be in a simple plaintext format which is described in Section 3.8. However, input filters provide a mechanism by which data files in arbitrary formats can be read.

An input filter is specified to act on all data files that match some filename pattern. For example, a filter could be defined to act on all data files called *.txt or *.dat. The filter itself takes the form of a program which is launched by Pyxplot whenever a matching data file is read. The program is passed the filename of the data file as a command line argument immediately following any arguments specified in the filter’s definition. It is then expected to return the data contained in the file to Pyxplot in plaintext format using its stdout stream. Any errors which such a program returns to stderr are passed to the user as error messages.

Pyxplot has five input filters built-in, as the show filters command reveals:

set filter "*.fits"   "/usr/local/lib/pyxplot/pyxplot_fitshelper"
set filter "*.gz"     "/bin/gunzip -c"
set filter "*.log"    "/usr/local/lib/pyxplot/pyxplot_timehelper"
set filter "ftp://*"  "/usr/bin/wget -O -"
set filter "http://*" "/usr/bin/wget -O -"

The above set of filters allow Pyxplot to read data from gzipped plaintext data files, from data files available over the web via HTTP or FTP, and data tables in FITS format. A filter is also provided for converting textual dates in log files into numbers representing days, months and years. To add to this list of filters, it is necessary to write a short program or shell script; the simple filters provided in Pyxplot’s source code for .log and .fits files may provide a useful model.

The filter can then be installed using the syntax

set filter <filenameWildcard> <shellCommand>