PyNAG – A Python package for interfacing with the NAG C Library

May 13th, 2009 | Categories: NAG Library, programming, python | Tags:

Update (5th November 2009): PyNAG has been updated several times since this post was written. The latest version will always be available from https://www.walkingrandomly.com/?page_id=1662

This is part 6 of a series of articles devoted to demonstrating how to call the Numerical Algorithms Group (NAG) C library from Python.  Click here for the index to this series.

Over the last few articles I have demonstrated that it is relatively straightforward to interface Python to the NAG C library.  The resulting code, however, looks a bit ugly and overly verbose since you have to include a lot of boiler plate code such as telling Python where the C library is, defining pythonic versions of C structures, defining various constants and so on.  I found myself doing a lot of copying and pasting in the early stages of working all this stuff out.

Just recently I made a small step forward by moving all of this boilerplate into a Python package which I have called PyNAG.   It’s a relatively simple package since all it does is define Pythonic versions of various C-structures, link to the NAG library, perform restypes on the function names, define a few often used NAG variables such as NAGERR_DEFAULT and so on.

The practical upshot is that you need to write a lot less code when interfacing with the C library.  For example here is the Python code to minimise a 2D function via the simplex method using e04ccc of the NAG library.

#!/usr/bin/env python
#Mike Croucher - April 2008
#A minimisation problem solved using e04ccc
from PyNAG import e04ccc,NAGERR_DEFAULT,E04_DEFAULT,Nag_Comm
from ctypes import *
from math import *

#Create the communication structure
comm = Nag_Comm() 

#set up the objective function
def py_obj_func(n,xc,objf,comm):
	objf[0] = exp(xc[0]) * (xc[0] * 4.0 * ( xc[0]+xc[1] ) + xc[1] * 2.0 * (xc[1] +1.0) +1.0 )
C_FUNC = CFUNCTYPE(None,c_int,POINTER(c_double),POINTER(c_double),POINTER(Nag_Comm) )
c_obj_func = C_FUNC(py_obj_func)

#Set up the starting point
n=2
x=(c_double*2)()
x[0]=c_double(0.4)
x[1]=c_double(-0.8)

#Set up other variables
objf = c_double()

e04ccc(n, c_obj_func, byref(x), byref(objf),E04_DEFAULT,comm,NAGERR_DEFAULT)

It’s still not as pretty as it would it if it were written in pure python but it’s a (small) step in the right direction I think.

The good

  • PyNAG is free and always will be.  You’ll obviously need to buy a copy of the NAG C library though.
  • It includes a definition of one of the largest C-Structures I have ever seen – Nag_E04_Opt – which has over 130 members.  This took me quite a while to get working and now you don’t need to go through the same pain I did.  All the C structures mentioned in these tutorials have been included to save your typing fingers.
  • PyNAG has been tested on 32bit and 64bit flavours of Linux and seems to work fine.
  • It is relatively easy to add definitions for NAG functions that are not already currently included.  In the best case scenario all you need to do is add two lines of code per NAG function. (They’re not all this easy though!)
  • It comes with a set of fully functional examples (15 so far with more on the way) along with expected output.  This output has been checked against the original C versions of the code to ensure that the results are correct

The bad

  • It only works on Linux at the moment.  Depending on interest levels (mine and yours) I may get it working on Mac OS X and Windows.
  • It only includes all the boilerplate needed to interface with 22 NAG functions at the time of writing.  This is a very small percentage of the library as a whole.
  • It doesn’t include all of the structures you might need to use throughout the entire library at the moment.
  • It doesn’t include any of the enumerations that the NAG C library uses.   I plan to include some of the most commonly used ones but will simply have to define them as variables since ctypes doesn’t support enumerations as far as I know.
  • This is my first ever Python package so things may be a bit rough around the edges. I welcome advice from more experienced Pythonistas concerning things like package deployment.
  • The Python version of the NagError structure has a member called pprint rather than the original print. This is because print is a reserved keyword in Python so I couldn’t use it.
  • It has only been tested on Python versions 2.5.x and 2.6.x.  I have no idea if it will work on other versions at the moment.
  • Documentation is rather sparse at the moment.  I haven’t had time to do a better job.

TheUgly

  • All PyNAG really does is save you from having to type in a load of boilerplate.  It doesn’t properly ‘pythonise’ the NAG C library and so it is painfully obvious that your code is talking to a C library and not a Python library.  Ideally I would like to change this and make the user interface a lot more user-friendly.  Any input on how best to go about this would be gratefully received.

The Installation

The first public release of PyNAG is version 0.12 and is currently available here (Linux only).  Installation instructions are

  • Download the tar gzip file above and run the following commands
  • tar -xvzf ./PyNAG-0.12.tar.gz
    cd PyNAG-0.12/src
  • edit the following line in the file __init__.py so that it points to your installation of the library
    C_libnag= ctypes.cdll.LoadLibrary(“/opt/NAG/cllux08dgl/lib/libnagc_nag.so.8”)
  • Move back into the root PyNAG-0.12 directory:
    cd..
  • run the following command with administrator priviledges
    python setup.py install

The installer will scatter the PyNAG files where ever your machine is set up to put them.  On my machine it was put in the following location but your mileage may vary

/usr/local/lib/python2.6/dist-packages/PyNAG

The Examples

I have written a set of example scripts that show PyNAG in use and these can be found in the src/examples directory of the PyNAG distribution.   Mathematically speaking they are all rather dull but they do show how you can call the routines I have included so far.  More interesting examples will be appearing soon.

The Future

Getting the NAG C library to work with Python has always been a side project for me and I never expected to take it even this far.  Most of this work has been done on the train while commuting to and from work and, although it is far from perfect, I am happy with it so far.  There is much to do though and the rate it gets done depends on user-interest.  Things that spring to mind (in no particular order) are

  • Get PyNAG on a proper CVS system such as github so that other people can easily contribute.
  • Add support for Numpy matrices.
  • Add support for more of the most commonly used NAG functions.
  • Get a windows version up and running.
  • Write some more interesting demos.
  • Write some sort of automated test script that runs all the examples and compares the results with a set of expected results.
  • See if I get any users and take requests :)

Final points.

I do not work for NAG – I just like their products. My professional involvement with them is via the University of Manchester where I am the site-license administrator for NAG products (among many other things).  I have a vested interest in getting the best out of their products as well as getting the best out of the likes of Wolfram, Mathworks and a shed-load of other software vendors.

One of my hobbies is pointing out to NAG where their library could do with extra functionality.  I am aided in this venture by a load of academics who use the library in it’s various guises and who contact me to say ‘I like NAG but it would be much better if it had the ability to do XYZ’.   I am very grateful to NAG for accepting my steady stream of emails with good grace and also for implementing some of our suggestions in the latest version of their library.  Feel free to point out any of your favourite algorithms that are missing from the library and why it should have them.

Finally, I’d like to thank the various people at NAG who helped me get my head around their library and for helping me test my initial tentative steps into the NAG/Python world.  I owe many of you several beverages of your choice.

As always, comments are welcomed.

Update (5th November 2009): PyNAG has been updated several times since this post was written. The latest version will always be available from https://www.walkingrandomly.com/?page_id=1662

  1. May 14th, 2009 at 09:20
    Reply | Quote | #1

    Your link for numpy is considered as obsolete (at least, according from a numpy mail list message), but maybe it will live for a while. It’s better using http://scipy.org/NumPy or wikipedia.org entry for numpy.

    BTW, in your function py_obj_func “return” statement is not required and can be omitted:
    def py_obj_func(n,xc,objf,comm):
    objf[0] = exp(xc[0]) * (xc[0] * 4.0 * ( xc[0]+xc[1] ) + xc[1] * 2.0 * (xc[1] +1.0) +1.0)

  2. Mike Croucher
    May 14th, 2009 at 11:13
    Reply | Quote | #2

    Hello again Dmitrey, good to hear from you.

    Thanks for the advice concerning the numpy link – I have fixed it in the post.

    Thanks also for the ‘return’ information – I was aware that you could leave it out but I tend to forget thanks to my days programming in languages such as C++. Feedback such as yours helps me become more pythonic and so is greatly appreciated. It has also been updated in the post but I’ll leave it in the example bundled with PyNAG until I release the next version.

    Cheers,
    Mike