Converting Lua to JSON in C++

Introduction

I embarked on porting a game from Lua to C++ to refine my C++ skills and tackle the challenges of migrating a large codebase from a garbage-collected, dynamically typed language to a manually managed, statically typed one. Along the way, I gained valuable insights that I’ll share in this article.

One key lesson that significantly reduced bugs was to keep the original (source) and ported (target) software consistent. This is because it ensures the application logic in the target closely mirrors the source. However, this sometimes required adapting to differences between the two environments.

For example, the original graphics system had a Y-axis pointed up with the origin at the center, while the target system’s Y-axis pointed down with the origin at the top-left. To address this, I developed an abstraction layer that converted coordinates just before rendering, enabling the existing logic to be ported with little or no modification.

Another challenge was managing Lua configuration files, which games typically load from disk during initialization. This data can be simple or complex, with nested structures and various data types. While processing Lua files is straightforward in Lua, it’s more complicated in C++:

In Lua, reading a configuration file is easy because the application can directly access the data.
In C++, reading a Lua configuration file requires embedding a Lua Virtual Machine and using its stack API to access the data.

Although Lua files in C++ can also modify or extend a program's behavior after compilation—a feature widely used in the modding community—our focus is on configuration management. While Lua can handle this, simpler formats like JSON, YAML, and TOML are easier for C++ to work with.

This raises the question: should we switch from Lua configuration files to a different data format? The answer depends on several factors:

How complex is the configuration data? Simple data is easy to handle, but complex, nested structures make parsing more challenging.
How much effort will it take to keep the source and target configurations consistent?

In my case, the configuration data is complex, and I believe the target software should use the same inputs as the source to maintain consistency, at least while the port is being developed. A practical solution is to convert the Lua data into a JSON object after reading it. This keeps the inputs consistent between the source and target software while simplifying the parsing logic.

To demonstrate how this can be implemented, I’ll use the following UML class diagram as a guide:

The Database class accepts a loader interface that calls load, passing itself as an argument. This invokes the LoadData method on the ConcreteLoader via virtual dispatch, which reads the Lua configuration file from disk, converts it into a JSON object, and populates the passed in Database instance with the parsed data.

Eventually, once the port is fully complete, all configuration data will be stored in JSON files. This design accommodates that future requirement by allowing the loader to be seamlessly switched out.

In this example, I use C++17 and two key libraries: Lua 5.4.2, a lightweight and embeddable scripting language, and nlohmann/json, a modern C++ library for parsing and manipulating JSON data. You can find the corresponding CMakeLists.txt file here to generate the project files with CMake.

For the sake of brevity, I’ll assume you have a basic understanding of C++ and can refer to the library APIs as needed. I’ll provide detailed explanations where they’re most relevant.

The complete code listing for this article, which has been heavily commented for clarity, can be found here.

Lua Configuration Data (config.lua)

Before we dive into the code, let’s take a look at what the Lua configuration data looks like. Below is the content of config.lua, which our LuaLoader will read and convert into JSON:

-- config.lua
Widgets = {1, 2, 3}

This simple Lua script defines a global table named Widgets containing a list of integers. Our goal is to read this table in C++, convert it into a JSON object, and then use that data to populate the Database with widget IDs.

Implementing the Data Loader

Setup the Data Loader Framework

Let's begin by outlining the data loader framework, which provides a flexible way to load data into the Database class using an interface:

class Database;

class ILoader
{
public:
    virtual ~ILoader() = default;
    virtual bool Load(Database& database) const = 0;
};

class Database
{
public:
    void LoadData(const ILoader& loader)
    {
        loader.Load(*this);
    }

    void AddWidget(int32_t widgetId)
    {
        mWidgets.push_back(widgetId);
    }

    void PrintWidgets() const
    {
        for (const auto& widget : mWidgets)
        {
            std::cout << "Widget ID: " << widget << std::endl;
        }
    }

private:
    std::vector<int32_t> mWidgets;
};

This framework allows us to load data into the Database through the ILoader interface. The ILoader interface defines a Load method that any loader class must implement. This method loads data into the Database and returns a boolean indicating success or failure. The Database class uses this method to populate its list of widgets, and you can use the PrintWidgets method to display the widget IDs.

Implement the Lua Data Loader

Now that the framework is in place, let's implement a loader that reads a Lua file, converts the data to a JSON object, and populates the database.

First, we’ll define some helper functions to handle the conversion from Lua to JSON:

bool IsArray(lua_State* L, int index)
{
    lua_pushnil(L);
    int n = 0;
    while (lua_next(L, index) != 0)
    {
        if (lua_type(L, -2) != LUA_TNUMBER || lua_tointeger(L, -2) != ++n)
        {
            lua_pop(L, 2); // Pop key and value
            return false;
        }
        lua_pop(L, 1); // Pop value, keep key for next iteration
    }
    return true;
}

json LuaValueToJson(lua_State* L)
{
    if (lua_type(L, -1) == LUA_TNUMBER)
    {
        if (lua_isinteger(L, -1))
        {
            return lua_tointeger(L, -1);  // integer
        }
        else
        {
            return lua_tonumber(L, -1);   // double
        }
    }

    switch (lua_type(L, -1))
    {
        case LUA_TSTRING:
            return lua_tostring(L, -1);
        case LUA_TBOOLEAN:
            return lua_toboolean(L, -1);
        case LUA_TTABLE:
            return LuaTableToJson(L);
        default:
            return nullptr;
    }
}

json LuaTableToJson(lua_State* L)
{
    json result;
    bool isArray = IsArray(L, lua_gettop(L));

    lua_pushnil(L);
    while (lua_next(L, -2) != 0)
    {
        if (isArray)
        {
            result.push_back(LuaValueToJson(L));
        }
        else
        {
            if (lua_type(L, -2) == LUA_TSTRING)
            {
                result[lua_tostring(L, -2)] = LuaValueToJson(L);
            }
            else if (lua_type(L, -2) == LUA_TNUMBER)
            {
                result[std::to_string(static_cast<int32_t>(lua_tointeger(L, -2)))] = LuaValueToJson(L);
            }
        }
        lua_pop(L, 1);
    }
    return result;
}

These functions convert data from a Lua script into a JSON object. Here’s a quick overview of what each function does:

IsArray Function: Checks if a Lua table behaves like an array (a list of items).
LuaValueToJson Function: Converts a single Lua value into its equivalent JSON value.
LuaTableToJson Function: Converts a Lua table into a JSON object or array.

Next, we implement the LuaLoader class, which uses these functions to load data from a Lua script:

class LuaLoader : public ILoader
{
public:
    LuaLoader(const fs::path& filepath)
        : mFilepath(filepath)
        , L(luaL_newstate())
    { 
        luaL_openlibs(L);
    }

    ~LuaLoader()
    {
        lua_close(L);
    }

    bool Load(Database& database) const override
    {
        if (luaL_dofile(L, mFilepath.string().c_str()) != LUA_OK)
        {
            std::cerr << "Error: " << lua_tostring(L, -1) << std::endl;
            return false;
        }

        lua_getglobal(L, "Widgets");
        if (!lua_istable(L, -1))
        {
            std::cerr << "Error: 'Widgets' is not a table or not found." << std::endl;
            return false;
        }

        json jsonData = LuaTableToJson(L);

        for (const auto& widget : jsonData)
        {
            database.AddWidget(widget.get<int32_t>());
        }

        return true;
    }

private:
    fs::path mFilepath;
    lua_State* L;
};

The LuaLoader class reads a Lua file, converts the data to JSON, and populates the Database with widget IDs. The process begins by retrieving the Widgets table from the Lua script using lua_getglobal(L, "Widgets");, which pushes the table onto the Lua stack, making it accessible in C++.

Next, the LuaTableToJson(L) function converts the Lua table into a JSON object, allowing the C++ program to easily manipulate and work with the data. If the conversion is successful, the function returns true; otherwise, it logs an error and returns false.

Finally, let’s test the LuaLoader with the following code:

int main()
{
    Database database;
    LuaLoader loader(RESOURCES_PATH "config.lua");

    database.LoadData(loader);  
    database.PrintWidgets();

    return 0;
}

When you run the program, you’ll see the following output:

Widget Value: 1
Widget Value: 2
Widget Value: 3

This output confirms that the LuaLoader successfully loaded the data from the Lua script and populated the Database.

That wraps up this article. Stay tuned for the next installment as we continue this Epic Quest in software development!

social