Unit testing compiled code in Maya

There’s a lot of buzz these days about unit testing and test-driven development, for plenty of good reasons. Talk about alternative forms of testing is also abound. I’m not going to discuss the benefits of testing as a philosophy in this post; however you decide to work on your projects, whether for yourself or at scale, ultimately needs to conform to the requirements of the final product and the demands of the end-user(s).

That being said, in my opinion, there is no argument that can support not having a testing process of some sort. But that’s not the point of this post.

The point of this post is to discuss how to implement testing for compiled C/C++ code for your Maya plug-ins and tools.

Why pytest/unittest isn't enough

At work, I constantly see issues arise that, to me, could have been caught extremely quickly before being rolled out to end-users had there been a robust way of running a test suite on things. Most of our code (and my experience!) is in Python, but for certain things, we obviously use C++ for improved performance or where it is impractical to accomplish the end-result through scripting alone. This, of course, means that issues will inevitably arise that cannot be caught with the scripted side of things.

While there are resources aplenty for writing unit tests in Maya, after speaking around with TDs/engineers both at and outside of work, it has become clear to me that most developers do not worry as much about unit testing lower-level code. They might have a scripted test that runs an MPxCommand associated with their plugin, and check the output of that, which is certainly a valid (behaviour-driven) test!

However, it cannot cover everything that a custom plugin would be doing internally. And of course, once your project starts to grow and you start to share libraries across plugins, it becomes a bit of a developer nightmare when one person wants to change functionality in an existing function and has literally no reliable way of knowing what dependencies will break if he changes something in a library function.

Example: if I was working on two plugins that both shared a common function to calculate the axis-aligned bounding box of the given input mesh, and I changed the code mistakenly so that it now returns the actual values of the oriented bounding box instead. As a developer, I might be testing it by literally opening Maya, creating a sphere, and checking that my function simply gives me the right values. And it would!…But then some other plugin, one that perhaps is actually using AABBs for collision detection, would most definitely start to behave erratically, and then I’d be asked to track that down. And assuming it went unnoticed for a while (perhaps no one was using that plugin that relied on collision detection for a week), and lots of code gets committed since that point in time, you might end up having to spend quite a disproportionate amount of time to debug an absurdly simple problem. While BDD tests in Python can help you determine that something is wrong, it will not help you narrow down the scope of what exactly is wrong as your project starts getting more complex, and your behaviour tests start getting more generic.

Building Maya standalone applications

After I came to the conclusion that I needed to implement some form of sane testing process for my own Maya plugin development, I came across the tidbit in the documentation regarding building standalone applications that linked to the Maya libraries. This intrigued me a little, since I never knew that this was possible before. And talking to most people in the industry, it doesn’t seem like this is a commonly-known fact either; you can actually create your own applications that make use of the Maya compiled libraries to run Maya functions. There’s tons of possibilities that this opens up, but today, I’m going to talk about using them to help us unit test our compiled functions that make use of Maya data structures.

Before we begin, however, we’re going to actually need to figure out how to actually get this working. The examples in the devkit are very platform-specific (and marginally outdated), so in favour of getting the build process as portable as possible, I will be using CMake to handle the building of my application.


(The full program is available here.)

The Maya devkit contains a sample application called readAndWrite.cpp. We’re going to update it a little to actually do something other than just read files and append text to their filename at the end.

The important parts of the definition file are the following lines:

#include <maya/MLibrary.h>
... //(inside of main())
MStatus stat = MLibrary::initialize(true, argv[0], true);
... // stuff that the application does goes here

If you’re familiar with maya.standalone in Python, this is similar to basically calling:

import maya.standalone

# 2016 onwards

The difference? Performance is one thing, of course (and it’s pretty significant!), but if you haven’t realized already, you can actually call your C/C++ functions from within the application itself!

The only trouble here, is, of course, that we don’t actually have access to Maya’s source code, so we have to link to those libraries that we’re provided with (.lib on Linux, .dylib on OSX, and DLLs on Windows). You can, of course, opt for doing that through just using batch/shell scripts alone, but I generally prefer to use CMake for this.

Cross-platform building with CMake

(The full CMakeLists.txt file can be found here.)

The CMakeLists.txt file itself contains a lot of procedures, most of which are self-explanatory if you’ve read my post on cross-platform debugging of plugins for Maya previously.

Of particular interest to us, however, are the way linking to the libraries is handled between the different platforms. You will see that on Linux, the only thing required to be done is handle the @rpath that is set in the final built executable through the following lines:


If you want to understand more about how the @rpath works with CMake, I strongly suggest reading this article. The short answer is that it is a more convoluted process than it should be, and those commands tell CMake to basically add runtime path information to both the built/installed executable(s), so that we can run them without having to copy all the .lib files to the application directory for the linker to find them.

On OSX, because the linker works differently (because of course it does), instead of setting the DYLD_LIBRARY_PATH and the DYLD_FRAMEWORK_PATH like the documentation tells you to because of reasons, I opt to copy all the Maya frameworks and .dylib files to the application folder instead for the linker to find. This is arguably stupid, and if you believe it to be so and disregard the reasons stated in that post, you can skip this section and read on to the Windows one where I do link to the libraries dynamically there. If not:

    message(STATUS "Setting MacOSX SDK...")
    set(CMAKE_OSX_SYSROOT "/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.11.sdk")

    set(MAYA_FRAMEWORK_DIR "${MAYA_LOCATION}/Maya.app/Contents/Frameworks")
    message(STATUS "Setting OSX Framework path to: ${MAYA_FRAMEWORK_DIR}")


    message (STATUS "Setting OSX-specific settings...")
    set(CMAKE_INSTALL_NAME_DIR "@rpath")

    foreach(DYLIB ${MAYA_DYLIBS})
        add_custom_command(TARGET ${EXECUTABLE_NAME} POST_BUILD
            COMMAND ${CMAKE_COMMAND} -E 
            copy_if_different "${DYLIB}" "${CMAKE_INSTALL_PREFIX}"
            COMMENT "Copying ${DYLIB} to installation directory...")

    add_custom_command(TARGET ${EXECUTABLE_NAME} POST_BUILD
        COMMAND ${CMAKE_COMMAND} -E copy_directory "${MAYA_LIBRARY_DIR}/plug-ins" "${CMAKE_INSTALL_PREFIX}/plug-ins"
        COMMENT "Copying Maya plug-ins to installation directory...")

    add_custom_command(TARGET ${EXECUTABLE_NAME} POST_BUILD
        COMMAND ${CMAKE_COMMAND} -E copy_directory "${MAYA_LIBRARY_DIR}/rendererDesc" "${CMAKE_INSTALL_PREFIX}/rendererDesc"
        COMMENT "Copying Maya renderer XML configuration files to installation directory...")

    add_custom_command(TARGET ${EXECUTABLE_NAME} POST_BUILD

For the record, it would be nice if the @rpath would just work nicely in OSX, but I have thus far been unsuccessful in my attempts to wrangle the linker to work without having to set those environment variables. Anyone who has any input here (without suggesting running install_name_tool as a post process), I’d be happy to hear it.

On Windows, things are slightly simpler due to the lack of a concept of @rpath on Windows. We simply make a batch file that will be used to set the environment for the standalone application before launching it. This allows the linker to find the necessary DLLs. (You can also do this for OSX with a shell script and using the DYLD_LIBRARY_PATH and DYLD_FRAMEWORK_PATH environment variables to point to the respective directory paths as well, if you don’t want to copy the frameworks and .dylib files to the executable folder as detailed above)

The batch script is as follows:

start "maya_standalone_test" %~dp0\maya_standalone_test.exe

And the relevant commands in CMake:

                   "${CMAKE_INSTALL_PREFIX}/launch_maya_standalone_test.bat" NEWLINE_STYLE CRLF)

This is where CMake at the very least shines a little better than a batch script would have. “configure_file” is an extremely useful macro function for this sort of stuff.

With that, we have a cross-platform solution for building standalone Maya applications.

Catch: A unit test framework for C++

I’m no fan of overly-convoluted frameworks. At best, they do stuff I don’t need, and at worst, they don’t do stuff I do need, and do the stuff that I do need poorly/slowly/obtusely/all of the above.

(Yes, I do realize I just spent some considerable time talking about CMake, but in that case, I feel that it’s better than the alternative of maintaining separate shell/batch scripts for each platform/compiler combination separately. Pick your battles to fight, etc.)

Thus, when I started working out solutions for this problem of lacking testing, I started out by writing my own very simple small tests and assertions. It became clear to me very quickly, however, that in order for this to work at scale, it would require a less hacky framework that I’d cooked up with a couple of #defines.

I’d used Google Test before, and while I wasn’t particularly unhappy with it, I wasn’t particularly impressed with its performance or its overhead. Add to the fact that even setting up a test itself was rather convoluted, and I started looking around for a better, more lightweight framework. Catch caught my eye immediately in that regard; tests were easy to setup, there was a single-header-only version that didn’t seem insanely obtuse to read and understand, and tests ran with (relatively) little overhead. Moreover, the framework seemed stable enough. Perfect.

The source code for the unit test application is available here. It demonstrates how you can use Catch to run a unit test for a compiled C++ function that returns the number of points in a given mesh MObject, something that would not be possible in Python alone.

I’m pretty happy with Catch for now, though as I start writing some code in pure C alone, I’m taking a look at CMocka as well, though I’m still searching for something even more lightweight than that. If you have recommendations, let me know!


The full project is available on Github, and you should be able to build the example applications by following the directions given.

In summary, we’ve gone over how to set up cross-platform building of Maya standalone applications, and I’ve shown one application of that, which is to run unit tests against your compiled code using Catch. If you’re still reading, I hope that this has been informative, and that you’re now, like me, furiously thinking of new ways to leverage this capability of Maya that you might not have known about previously!

0 comments on “Unit testing compiled code in MayaAdd yours →

Leave a Reply