A C++ Lua Binding with Luna

Intro

I didn’t know that much about Lua and how large game engines like CryEngine, Source, and a lot of MMOs with UI Addons actually implemented Lua APIs. What better way to figure it out than to jump right in and try it myself? None, so that’s what I did.

Going in blind isn’t the best idea, so I spent some time reading up on lua as a language and utilizing it a little on its own before trying to make one of my own C++ applications work with a Lua script that is read in at runtime, no recompiles required.

Enter Luna

Luna, as the link above would tell you, is a very lightweight (1 header file, less than 200 lines) C++ binding over the official Lua C API. For my purposes, Luna performed flawlessly. It was integrated into my (already large and growing) physics simulation program with very little effort and in a low amount of time, giving me plenty of time to dive into and explore the ocean of understanding still before me. Luna, by nature, only works with members of C++ classes and requires a little bit of boilerplate code to be placed into the header and source of the class you wish to expose to Luna.

// Warp.h
// Lua boilerplate
static const char className[];
static const Luna<Warp>::PropertyType properties[];
static const Luna<Warp>::FunctionType methods[];

The className char is what you want the exposed class to be called in your Lua script(s), the properties array lists what non-function data members you want to be exposed to Lua, and as you might expect, the methods array is the list of methods you wish to expose to Lua. Lua (and Luna by extension) is picky about the method signature you pass to it, as you’ll see in a moment.

// Warp.cpp
// Method definition for lua
#define method(class, name) {#name, &class::name}
// Lua init
const char Warp::className[] = "Warp";
const Luna<Warp>::PropertyType Warp::properties[] = {0, 0};
const Luna<Warp>::FunctionType Warp::methods[] = {
	method(Warp, OnExecute),
	method(Warp, RegisterSphere),
	method(Warp, RegisterAdditionalForce),
	{0, 0}
};

// -- More code -- \\

int Warp::RegisterSphere(lua_State* L)
{
// Do function code
}

This is the companion boilerplate code defined in the cpp file. Here, I tell Lua that this class will be called Warp, have no directly referenced properties, and expose three methods named OnExecute, RegisterSphere, and RegisterAdditionalForce as well as a custom Lua-specific constructor. All of these functions need to have the signature int foo(lua_State*) in order to work properly with Luna. The reason for this is that Lua utilizes a stack, and data movement between Lua and any other language must be pushed or popped to or from the stack. The aforementioned method signature takes in a pointer to a lua state (an identifier to a previously loaded Lua file) so that you can push or read values from the stack and returns an int which tells Luna how many variables you have pushed to the stack in that function.

Data Movement

From Lua to C++

So now that we understand how we interface with Luna to expose classes to Lua, we can look a little closer at some of the functions I wrote.

int Warp::RegisterSphere(lua_State* L)
{
     // Retrieve a pointer to a Sphere object from Lua
     // L is our Lua state and 1 represents the index on
     // the stack we want to get the value of
     // Lua stack values start at 1, not 0
     Sphere* ptr = Luna<Sphere>::check(L, 1);

     // Add it to my vector of Spheres
     mSphereVec.push_back(ptr);

     // Lua has no default values for arguments
     // so we check if the second index on the
     // stack is nil (Lua's "null")
     if(lua_isnil(L, 2))
     {
          // If this is nil, then the object
          // is not registered with another force.
          // We pushed nothing to the stack, return 0.
          return 0;
     }

     // I don't use this syntax unless I'm in JavaScript.
     // I'm only using it here to keep all text visible.
     // You can see in there, I'm getting another Sphere
     // and passing it to the ParticleGravity class.
     std::shared_ptr<ParticleForceGenerator> gravitationalForce(
          new ParticleGravity( Luna<Sphere>::check(L, 2) )
     );

     mForceRegistry.addRegistration( ptr, gravitationalForce );

     return 0;
}

Not all that elegant code, but it illustrates a few of the things I learned as part of this project. So this function, if I were to write it in C++ would have a method signature more like int Warp::RegisterSphere(Sphere* sphere1, Sphere* sphere2 = NULL); because that’s what I’m effectively giving Lua. Most of the text I would write in this paragraph is in the commented code above, but something to note is that the Luna::check() function will return a reference to my C++ class if the corresponding index’s value on the stack is correct. One of the most important lessons I learned as part of this is that Lua owns the objects created on Lua’s side of the garden. If you explicitly delete them, you will have a bad time. I originally wrote all of my code to use shared pointers instead of raw pointers, and the only reason the above code shows raw pointers is because I thought the shared pointers were causing another unrelated exception during development. You can use shared pointers with objects created from Lua, but you need to define a custom deletion function that, ironically, doesn’t delete your object; Lua’s garbage collector will do that whenever it chooses.

Let’s look at another brief example of Lua and C++’s interactions so I can touch on Lua’s stack functions.

Sphere::Sphere(lua_State* L) :Particle(L)
{
	mType = SPHERE;
	mSphere = gluNewQuadric();
	gluQuadricDrawStyle(mSphere, GLU_FILL);
	gluQuadricTexture(mSphere, GL_TRUE);
	gluQuadricNormals(mSphere, GLU_SMOOTH);

	// Get various constructor arguments from lua state
	mRadius = lua_tonumber(L, 1);
	mSlices = lua_tointeger(L, 2);
	mStacks = lua_tointeger(L, 3);

	setInverseMass( lua_tonumber(L, 4) );

	mPosition = glm::vec3(
             lua_tonumber(L, 5),
             lua_tonumber(L, 6),
             lua_tonumber(L, 7)
        );

	mName = lua_tostring(L, 8);
}

As you might have noticed by now, my Sphere class is also exposed to Lua. This snippet of code shows Lua’s lua_to*() family of functions that are used to pull items from the stack into my C++ program. This family of functions works the same as the Luna<T>::check() function from before, with the first argument the lua state you wish to read from and the second the index on the stack whose value you wish to read. However, these do not check the values that are passed, so unless you do it yourself with the lua_is*() family of functions, you will get undefined behavior. Interesting is the lua_tointeger() function because Lua does not have a defined integer type (All numbers are double precision floating points), so the function exists to do the conversion for you as you bring the value into C++.

From C++ to Lua

I didn’t spend all that much time experimenting with sending data from C++ to Lua, but from all of the resources I’ve looked at, it seems as easy as the other way around. What I did, however, experiment with was calling a Lua function from C++. My program is a nearly-true simulation of the solar system, and in developing it, I needed a way to reset my simulation. Since my simulation is created dynamically from a Lua script, I decided to let Lua reset my simulation, I just needed a way to tell Lua what to do.


function resetSolarSystem()
	makeSolarSystem(warp)
end

function makeSolarSystem(warp)
	local sun     = makeSphere(0.0046, 332946, 0, 0, 0, warp, 0, 0, 0, "Sun", nil)
	local mercury = makeSphere(1.63e-5, 0.05526, 0.4665, 0, 0, warp, 0, 4.5976e-8, 0, "Mercury", sun)
	local venus   = makeSphere(4.04e-5, 0.815, 0.7282, 0, 0, warp, 0, 5.55186e-8, 0, "Venus", sun)
	local earth   = makeSphere(4.26e-5, 1.0, 1.0, 0, 0, warp, 0, 6.77e-8, 0, "Earth", sun)
	local mars    = makeSphere(2.27e-5, 0.107, 1.6659, 0, 0, warp, 0, 8.89533333e-8, 0, "Mars", sun)
	local jupiter = makeSphere(0.000477, 317.83, 5.45876, 0, 0, warp, 0, 1.31562e-7, 0, "Jupiter", sun)
	local saturn  = makeSphere(0.000402, 95.159, 10.1238, 0, 0, warp, 0, 2.162e-7, 0, "Saturn", sun)
	local uranus  = makeSphere(0.000170, 14.536, 20.07795, 0, 0, warp, 0, 3.33829e-7, 0, "Uranus", sun)
	local neptune = makeSphere(0.000165, 17.147, 30.38144, 0, 0, warp, 0, 3.8896e-7, 0, "Neptune", sun)
end

function makeSphere (radius, mass, x, y, z, warp, xVel, yVel, zVel, name, parentBody)
	local sphere = Sphere(radius, 32, 32, mass, x, y, z, name)
	sphere.setVelocity(xVel, yVel, zVel)
	warp.RegisterSphere(sphere, parentBody)
	return sphere
end

-- Warp(int screenWidth, int screenHeight, bool fullscreen)
warp = Warp(1920, 1080, true)

makeSolarSystem(warp)

-- Go into loop
warp.OnExecute()

So the highlighted code above is the function I made that resets my solar system. All it does is make a new solar system by doing the exact same thing it does when the script first runs. In my C++ code, I make sure to remove the sphere references that Lua has already made before calling the reset function. Actually calling the Lua code from C++ is as simple as this:

     lua_getglobal(L, "resetSolarSystem");
     // If pcall had arguments, lua_push*() them here
     lua_pcall(L, 0, 0, 0);

Since almost all values that aren’t local are global (thus, all functions), I can retrieve the reference to my function by name and call it. The lua_pcall() function takes in the state you wish to call a function on, the amount of arguments you’ve pushed on the stack is, the amount of results you expect, and the index on the stack of an error function (you must place it there) in case of an error.

In closing, I found Lua to be an absolute joy to implement into my C++ application; you just need to pay attention to memory. In terms of the pros and cons, I really enjoy the fact that Lua can make my code more dynamic without recompiling and often times be a simpler way to program behaviors so long as there is a robust C++ back-end. On the downside, for projects that aren’t run by only one person, you absolutely must document the API you’re creating. In addition, writing the C++ code that interfaces with the Lua must always have proper and robust error checking (i.e., not like mine) with good feedback for the script writers. I look forward to expanding the small API I have now as this program grows.

My (simple) Planetary Body Simulation

The hardest, but best part of learning is re-inventing the wheel in order to figure out what it truly is and how it works. I was tasked with creating a (simple) simulation of the solar system we live in, using true-to-life values and physical calculations, and I did just that.

Into the Trenches

I began this project only two weeks ago now, and as a milestone for the first week, worked to implement the requisite C++ classes and get some sort of planet being affected by a sun-like object. The values were entirely temporary, and the planet did not have a stable, circular orbit.

Gravity

Besides being a great, award-winning movie, gravity is the single most important part of my simulation and the single most problematic step to get right.

A sense of scale

Space is huge. Mindblowingly huge. Just our local solar system is huge in terms of space, but pales in comparison to the rest of the expanding galaxy or universe. Even within the confines of our solar system, choosing proper units to represent our planets with is a challenge. A single-precision floating point variable, depending on the units chosen, cannot hold the full value of the gravitational constant or represent the position of a planet. I ran into this issue very late into the project attempting to put the moon next to the earth. Around a week in, I chose to utilize Astronomical Units (AU) for my simulation’s unit of distance and Earth Masses for mass because it made the values for the Earth and the Sun very simple, and most definitely within the range of a float. With all that said, we haven’t talked about time; simulating true-to-life values at the time scale that our solar system operates in doesn’t lead to very visually appealing results.

planets

Imagine this was a live version of my simulation. Stare at it for 10 hours and you might be able to see the pixels blitted differently.

Warp

We need a way to speed up time. Take some number (10 for instance) and raise it to the power of a variable (my warp factor), giving us warp speeds of 1, 10, 100, etc.  Things start getting visually appealing around warp speed 1,000,000 and even better at 10 million. However, in my early versions of warp, things got visually appealing, but also horribly inaccurate. The method I used originally was to multiply my delta time (dt) from last frame by my warp speed, but this only approximated a position and abnormally varied dt would throw my planets into far more eccentric orbits. The “real” way to warp forward would be to do your physics calculation warpSpeed times per frame. Unfortunately, this process is obnoxiously taxing on modern hardware, so it starts cutting into your framerate around 1,000 warp. My final solution was suggested to me by my colleague as a compromise between the two. If we’re under a certain number warp (k), we’ll just do the physics calculations warpSpeed times per frame, but otherwise we’ll approximate the calculation with dt * (warpSpeed / k). In my case, k ends up being 1,000 because of performance reasons; thus a warpSpeed of 10,000 would give us a calculation of dt * 10, done 1000 times per frame.

Update

So when creating the physics simulation, I had true-to-life values for the gravitational constant and the masses of the planets, but the velocities for some reason were some magic number that yielded something that looked like orbits, and the stuff you can see in the video above uses those values. Fortunately, I found the problem. I’m in OpenGL land, so I decided to use the GL Math (GLM) library to handle vector and matrix math. It works, as you might expect, really well. GLM defines some vector types, vec2, vec3, and vec4 and overloads the appropriate operators so that you can directly add or subtract them, multiply by scalars and the like. One of the member functions is called length() and if you look no further, you might think it will return to you the length or the magnitude of the vector. length() actually returns the amount of members the vector contains (2 for vec2, 3 for vec3), NOT the actual magnitude of your vector. To get the true magnitude of your vector, you need to use glm::length(vector). After changing this, my magic velocities didn’t work, but plugging in the correct values yielded beautiful results.

.rePRODUCE//

Production has begun anew!

I am excited to be working with a new group of talented individuals who help to form the largest senior team at Champlain College this year. I’m also excited (and scared) to begin working on the new features we’ve set out to boost our game to new heights (literally).

In particular, we want to work to bring new wisps with new abilities to the game, adding diversity and player agency that we’ve lacked. In addition, art asset production should improve by at least a factor of two, allowing for an even more gorgeous game.

Games that moved forward

Yesterday was the mid-year show. Teams worked tirelessly over the weekend to fix remaining issues with their prototypes and create presentations that would show off their game to the school. Today we holed all of the faculty up in one of the labs and the teams cycled through to allow the faculty to directly play our games. Once done, we left to go on with our Tuesday as normal with the knowledge that an announcement of what teams would advance would come between then and Friday. We were all surprised to see an announcement within 3 hours of the end of the play session. In any case, eight teams moved forward and the following are the names and videos of the continuing games.

Heartthrob Studios’ Vaculab

Team Supergeneric’s SunBots

Blue Moose Entertainment’s Wisp (woo us)

The Midnight Tacos’ VXT

Infinitely Cool Industries’ Lucid Delusions

Phlegmatic Studios’ Busy Bots

mODDular Studios’ Quetz

The Networking for Wisp Part One

Let me preface that this blog post will feature a fair portion of code and a less than fair amount of explanation of core concepts, so if something doesn’t make sense to you, I’d encourage you to Google it!

As a challenge to myself, I attempted to create a web server backend for my game in order to record match data that could help the team to make the game consummately better. This initial work was fun, but evolved into far more than I thought it would in short order. So this post details some of my first forays into web server backends, PHP scripting, and PostgreSQL development. While my code and practices are not perfect by any means, they are sufficient for what I was seeking to do and definitely a result of flying by the seat of my pants.

The first step in this process was to create a server that could receive requests, interact with database data, and return useful results. Naturally, this lead me to downloading and installing BitNami’s WAPP stack (Windows, Apache, PostgreSQL, PHP). After installing this onto my home desktop, I read a bunch of tutorials and got to work. I set up my database with a new match_details table in order to take in match details. I then created a PHP script that would take in that data and stuff it in my database.

$db = pg_connect("host=localhost port=5432 dbname=postgres user=postgres password="actuallyapassword");
$query = "INSERT INTO match_details VALUES (DEFAULT, '$_POST[match_time]','$_POST[min_deposited]',
    '$_POST[max_deposited]','$_POST[avg_deposited]','$_POST[min_stolen]',
    '$_POST[max_stolen]','$_POST[avg_stolen]','$_POST[winning_team]',
    '$_POST[build_version]', '$_POST[green_wisp_total]', '$_POST[yellow_wisp_total]')";
$result = pg_query($query);
if (!$result) {
    echo pg_last_error();
} else {
    echo "Update successful;";
}

Keen eyes may spot that I am indeed vulnerable to SQL injection attacks against my database, but the code works and for a proof-of-concept, that’s all that matters. The last part of this entire process is to collect and send the data from the client to the server. Since we are utilizing the Unity engine to make our game, I looked into what the Unity engine provides for web requests: the WWW and WWWForm classes.  I won’t post the whole thing, but the core of what I did is here:

private IEnumerator Send(MatchDetails details)
{
    float avg_deposited = 6.567f;

    WWWForm form = new WWWForm();
    form.AddField("match_id", "DEFAULT");
    form.AddField("match_time", details.totalGameTime.ToString());
    form.AddField("min_deposited", 2);
    form.AddField("max_deposited", 4);
    form.AddField("avg_deposited", avg_deposited.ToString());
    form.AddField("min_stolen", 1);
    form.AddField("max_stolen", 25);
    form.AddField("avg_stolen", "1.234");
    form.AddField("winning_team", details.winningTeam);
    form.AddField("build_version", details.buildVersion);
    form.AddField("green_wisp_total", details.greenWispTotal);
    form.AddField("yellow_wisp_total", details.yellowWispTotal);
    WWW post = new WWW(dbURL + "/match_details.php", form);
    yield return post;

    if (!String.IsNullOrEmpty(post.error)) {
        Debug.Log(post.error);
    } else {
        Debug.Log("Finished Uploading Data to Database");
    }
}

The accepted way to utilize the WWW class is through the use of a coroutine; this allows the web request to happen and wait for a response from the server without blocking the execution of the program until that response actually returns. The code above takes in a struct (MatchDetails) that contains all of the details of the match after it ends to send to my server. It adds all these to an html form to be sent as POST data and throws it at the script above that will put it in my database. Lastly, I check to see if the request succeeded or failed; 4xx or 5xx error codes put data into the post.error variable.

While the server and storage of match data is very interesting, I stumbled upon an idea that piqued my interest far more with more interesting hurdles to surmount.

The Choice

After prototyping the two versions and testing a little, we found that we would be better off moving forward with the multiplayer version of the game. While the networking aspect of the multiplayer prototype is always a hurdle, it is a lesser one for me to tackle as compared to AI in singleplayer. Not only does the multiplayer version of the game give us larger inherent replayability, but it also alleviates the stress of generating a detailed lore and story in such a short time span. Still, a lot of people really liked the singleplayer version of the game for its casual factors (and actually liked the supposed-to-be-scary ominous black blobs).

I met with an adviser for the project before the choice for multiplayer and he told me to find a way to bring some form of networking into the game if the game we moved forward with did end up being offline. He mentioned a project in the past which used no networking for the actual game, but used networking for feedback during their testing sessions. Feeling rejuvenated about what I could do with networking even if the game was offline, I set out to utilize a database on a server.

Prototypes Everywhere!

Since the idea that our team moved forward with was revealed to be the Wisp Game (title still nonexistent) last post, I will talk about what has shaped it and show you the prototype versions all the way until now.

After identifying our main game to be the Wisp Game, we split it into the single and multiplayer versions. I immediately got to working on the singleplayer prototype. After implementing the main mechanic in little time, I spent a total of 5 hours translating the code I wrote to work as a multiplayer game (Thanks Unity). Here are some screenshots of what each game looked like at the time. The goal here was to have a playable character capable of gathering wisps in a dark environment. Both the single- and multi-player version didn’t much have a goal so much as being a proof-of-concept to bring to QA and get feedback on.

The aforementioned lack of goals within the gameplay had to be remedied to see if the gameplay would actually amount to something fun. One of the first parts of our idea while we were coming up with it was to have players in multiplayer fight over a shared, limited resource. Thus, the multiplayer gameplay was updated to show how close the teams are to half-plus-one wisps on the map, and thus victory.

With regards to the singleplayer, the goal was to simply collect all of the wisps on the map with the idea that the game would be an explorable adventure game in nature. The player takes the form of a fairy; fairies in this world are scared of the night, but the wisps they need to light up their village only come out at night, so brave fairies must leave the village, and avoid the shade monsters who steal wisps. The player would move from village to village (level to level) lighting them up and serving as the protagonist in a fairy tale-like narrative. Failing to collect all of the fairies on a particular map would reduce some currency you can use to purchase powerups or upgrades for your fairy throughout the game. Here’s what those looked like.