Saturday, July 09, 2005

MEX and OS X

If it isn't clear by now, half the point of these last few points is to record quirks of architectural quirks as I encounter them while re-building my software under OS X. This is useful for me (it's sometimes easier for me to search old blog entries than it is to search through my old research notebooks); it may or may not also be useful for the people who stumble in here on the guidance of a search engine.

Today's adventure: mixed language MEX (Matlab EXternal interface) files under OS X.

I usually build MEX files by calling a command line utility called mex, which is provided as part of the standard MATLAB installation. mex is a shell script that calls the compiler with flags saying where various MATLAB include files are, then calls the linker with all the options and libraries needed to make a shared library that MATLAB knows how to load.

Unfortunately, the MathWorks often doesn't set up their MEX configuration in the way that I might wish. On my own system, I can modify the build options to my heart's content, either by modifying the system defaults (in or by providing my own options using the -f flag. The problem with this is that, in general, other people will have the default options set if they try to download and compile my software. A natural way around this trouble is to only invoke the mex utility at the last moment, compiling as much as possible by invoking the compiler directly. And that is now what I do with FEAPMEX.

Before the most recent round of patches, the FEAPMEX build system used mex to compile everything that explicitly invoked any of the MATLAB helper functions. But when I tried to compile FEAPMEX on OS X, the system complained that many of the common block variables used by FEAP were multiply defined. Private external definitions, the linker called them. I did a quick search through the linker documentation for the Mach-O object format used by OS X, and found out what a private external symbol is (it seemed like a terrible oxymoron to me when I first saw it). And then I got confused, because it seemed there was no way I could get private external symbols.

And then I realized that the symbols that the linker complained about were only showing up in files that I was handling through mex and not compiling directly. Hmm.

So, lesson the first: if you are building a MEX file under OS X and you get errors involving private external definitions, try invoking the compiler directly to create the object files, and use mex only to invoke the linker. All you will need to do is to specify the location of the MATLAB include paths using the -I compiler command line option. Lesson the second: even if you have no errors in building your MEX file, this might be a good idea. When I recompiled, I found a bunch of places in which I'd forgotten to explicitly declare routines; usually the compiler would warn me about such things, but the mex script suppressed that warning. In one or two places, the lack of an explicit declaration wasn't just a bad idea, but clearly would lead to an error (under certain circumstances). For example, there was one place where a function that returned void (a subroutine) was used without an explicit declaration; that's not a good thing in C code, since implicitly-declared functions are assumed to return type int.

After I fixed the problem with the FEAPMEX build system and corrected the errors that the compiler helped me find, I was able to build the code and run the test suite with no issues. Building the standalone version of FEAP proved even more straightforward, though it took me two tries to discover that under OS X the history library (used in my enhanced version of the command line interface) is build into libreadline rather than being a separate library file.

It took me about an hour to install and compile FEAP and to install, fix, and compile FEAPMEX. And now I'm very pleased, not only because the ports went more smoothly than expected, but also because the code runs fast on this machine. Especially compared to my poor old ThinkPad.