NOTICE: I originally created this document (and the initial versions of the general and mapgen test documentation) for my Software Development II class. I didn’t do it here nor elsewhere before that nor plagiarize.
Minetest is a virtually infinite block engine whose mechanics and games, on the surface, are generally similar to Minecraft. What may seem ironic, given its name, is that Minetest does not have formal testing procedures. Lua mods, often grouped into “modpacks” or “games,” implement all of the gameplay since Minetest is an engine. The core Lua API of Minetest itself does not make gameplay-related design choices. The APIs in “minetest_game” include those for interacting with materials. The interactions include tiering of gameplay based on tool types, and the registration of unique item types with special behaviors such as buckets. The APIs in the extended set of universally-accepted mods provide additional similar interactions plus more raw materials. All of these mods provide additional APIs at the Lua level beyond the core API, which interacts directly at the C++ level. Therefore, much attention must be devoted to determining scope, which has gone by the wayside. After increasing the extent and supervision of officially-supported mods, a test plan can facilitate the success of Minetest.
Developers must base scope on use cases, not opaque administrative decisions. The core developers of minetest.net ignore a set of universally-accepted APIs in Minetest. The team treats the APIs and mods as ancillary despite the universal level of use and the fact that they implement all gameplay. Due to “minetest_game” being the official game, it has become just as significant to the set of APIs as the engine’s Lua API. However, even the direction of minetest_game jumps around due to a lack of consistent leadership and lack of attention to use cases.
Mods involving mobs, security, and additional realms and materials (ores especially) are also just as significant as APIs and used universally. Even if the specific variations of such features are not universal and different mods implement them differently, they could be officially supported and turned on or off using configuration options. However, the core developers at minetest.net do not support such features. By making changes to the core API, the core team negatively affected such mods. Most significantly, stable mods that no one maintains, such as those by former modders like BlockMen, stujones11, and many others, broke (at various times or, in some cases, permanently). In cases of both maintained and unmaintained mods, server owners had to fend for themselves. They would no longer get fixes for version 0.4 eventually, and even subversions of 0.4 that were parallel to 0.5 (later called 5.0) introduced issues.
Change management has been a significant problem, especially starting around 2017 with changes to the Lua API leading up to version 5.0. To continue getting engine updates, many server owners switched to 5.0 by making self-patched and hence orphaned versions of mods or by discarding them. Adding or removing mods essentially breaks the tiering and world generation, so other server owners and players resort to using older versions of mods or of Minetest itself.
Not everyone agrees with which Minetest product line is valid. One developer, Robert Kiraly (OldCoder), founded Final Minetest (Minetest 6.0), a backward-compatible version, and considers “Old Minetest 5.0” to be an “unstable fork” (Kiraly, n.d.). He has been significant in the development of Minetest and considers his version to be the official version. Like VanessaE, his contributions are more significant than the “core developers” at minetest.net, supporting since 2012 universally-accepted APIs and mods that that minetest.net does not support (Kiraly, 2012). Another developer, only known as “Luke” or his username, created “minetest-luk3yx-4,” “a fork of an older version of Minetest so playing on older servers isn’t a bug-filled nightmare” (Luke, n.d.). Maksym Hamarnyk created MultiCraft, a fork of Minetest 0.4, and MultiCraft 2, a fork of Minetest 5 that is backward compatible even at the protocol level. The core developers at minetest.net have not prioritized change management concerning Lua mods. The Minetest mods repository has not solved the problem, primarily due to the lack of change management. The repository makes mods more officially available but still no more supported than before.
The most significant barrier to Minetest’s development has been a lack of leadership in the areas mistakenly considered ancillary. The lack of leadership is most prominent in the area of decisiveness, most noticeably in the area of scope. Perttu “celeron55″ Ahola is disinterested in leading Minetest and before the split had accepted the idea of Robert Kiraly trademarking Minetest (Ahola, 2017a). He says that the tagline of Minetest is “‘Wait, is it an engine or a game?’” but does not bother to answer the question which seems to disrespect the wide array of server owners and developers who depend on both the engine and universally excepted Lua code that implements gameplay (Ahola, 2017b). Such mods clearly determine the scope of testing. A minimal but effective scope includes:
- areas (“Advanced Areas”)
- basic_materials
- boost_cart
- caverealms
- creatures (“Creatures Mob Engine”)
- hbhunger
- homedecor_modpack
- fake_fire
- farming redo
- loot
- mapfix
- mesecons
- [modpack] mobs redo [mobs] (or [modpack] codermobs [codermobs], which resolves the telescoping constructor anti-pattern from mobs redo and includes its own implementation of many mobs from mobs redo other mob APIs and collections)
- moreblocks
- moreores
- nssm (“Not So Simple Mobs”)
- pipeworks
- pyramids
- sprint
- subterrane
- technic
- travelnet
- tsm_pyramids
- unifieddyes (or a lighter but complete color API)
- vessels
- worldedit (or coderedit which has efficient node and node metadata saving and loading using
//savebothand//loadboth) - wrench
- xdecor
Vanessa “VanessaE” Dannenberg creates and maintains several of the mods or modpacks within scope (Dannenberg, n.d.). Without her reduced maintenance, Minetest 5.0’s relevance and applicability to use cases have declined, as shown by comparing a typical user count on Minetest 5 servers to those of Minetest 0.4 and MultiCraft (Hamarkyk, 2019).
Robert Kiraly and other developers on minetest.org support almost all of the mods within scope, as they are in Bucket_Game. Bucket_Game is a highly configurable game that acts as a basis for creating other games. The stability, in comparison to the same set of mods not maintained centrally, has proven the drastically superior outcome of including these mods in the scope of support. Recognizing that these use cases are crucial to the function of many servers, and the degree of decentralized maintenance, would have prevented the minetest.net team from making changes that broke these mods. The degree of decentralized maintenance is a crucial issue. Many games in use on servers have embedded copies of these mods, sometimes customized and sometimes with no upstream maintainer.
Automated testing is not technically present in Minetest (other than generic continuous integration regarding runnability and style of scripts). However, there is a short-term path to automated testing and a long-term path to more extensive automated testing. Fortunately, since the core Lua API and universally-accepted extended APIs of Minetest focus on registering items, entities, nodes, and events, much of the code of each mod runs when the game or server starts. The “init.lua” file contains or runs most of the code. The engine’s API facilitates some level of automated testing simply by running Minetest, or even more efficiently, running minetestserver using the operating system’s Command-Line Interface (CLI) terminal. A formal automated test could run the command and examine the output and parse or at least display errors. Parsing errors and showing the line number of the error is possible using this author’s public-licensed software, “outputinspector,” available at https://github.com/poikilos/outputinspector.
Though outputinspector is currently scriptable since it operates independently of an IDE, a Lua IDE such as ZeroBrane Studio would be a better platform for such a feature for rapid automated testing of Minetest Lua code. If the functionality were in an IDE plugin, clicking a button to test the code clicking an error to code to the affected code would be possible without additional scripting. Bots or other user simulations could test interactions, triggering events that are not mapgen-related. Another path to more extensive automated tests would be scraping. Listing all of the functions in a Lua file and providing them with dummy data could theoretically provide complete code coverage of mod code.
Volunteers could execute a minimal set of manual tests efficiently. Of course, automating these tests with bots or other user simulations would be a significant step forward. However, the tests are not difficult to perform manually. The primary areas of testing would be world generation, definition regressions, and interaction.
Minetest uses a “seed” to determine how the engine generates the world, manually specifying the seed would provide a predictable backdrop for testing “mapgen” mods such as farming redo (which generates edible plants), moreores, technic (the technic_worldgen mod inside of the modpack), pyramids, and tsm_pyramids.
On my EnlivenMinetest repo, you can view my:
Other tests can follow the pattern in the “Map Generation Test Documentation” and use the guidelines in the “General Test Documentation.” The general documentation will ensure that the coverage of each test document is complete. The general documentation could expand to contain a complete loosely-recommended template (or set of such templates) so that the “Map Generation Test Document” is not a continual reference.
Using a template or example, designing other tests is just as simple as the “Map Generation Test Document.” For example, a definition regression test will have test locations where given generated and manually-placed items. The user would ensure that the “missing node” or “missing entity” texture doesn’t appear at the location. The interaction test document will be similar but ensure that various types of interaction do not cause crashes, for example, using a travelnet box to travel to another travelnet box or clicking on a destination that no longer exists.
Minetest maintainers should begin by accepting the community-defined scope above, providing the set of manual tests to a dedicated testing team, then implementing a ZeroBrane Studio plugin for automated testing. Each user should have a single named server instance and worlds directory. The determination of scope has been successful for Bucket_Game. Both supporting current servers and server preservation has guided the development and prevented Final Minetest from breaking compatibility with mods (Kiraly, 2019). World and mod preservation may seem to be a niche concern, but since the code is still in use, making the scope in this document officially supported in original forms, compatible with Minetest 0.4, satisfies the majority of current use cases.
References
- Ahola, P. (2017a, April 8). IRC log for #minetest-dev. http://irc.minetest.net/minetest-dev/2017-04-08#i_4863656
- Ahola, P. (2017b, September 13). Press release: Minetest 0. 5 has begun development! Electronics, Etc. http://c55.me/blog/?p=1863
- Dannenberg, V. (n.d.). Vanessa Dannenberg. GitLab. Retrieved April 6, 2020, from https://gitlab.com/users/VanessaE/projects
- Hamarkyk, M. (2019, July 11). Modernization of the Multiplayer tab · Issue #8669 · minetest/minetest. GitHub. https://github.com/minetest/minetest/issues/8669
- Kiraly, R. (n.d.). Home. Minetest Is Best. Retrieved April 6, 2020, from https://minetest.org
- Kiraly, R. (2012, September 30). Minetest news. Minetest Is Best. https://minetest.org/news2012.html
- Kiraly, R. (2019, January 6). Minetest news. https://minetest.org/news.html#190106
- Luke. (n.d.). Install [old] minetest 0.4 (Unofficial fork) on Arch Linux using the Snap Store. Snapcraft. Retrieved April 6, 2020, from https://snapcraft.io/install/minetest-luk3yx-4/arch
- Minetest-mods/moreores. (2020, December 31). GitHub. https://github.com/minetest-mods/moreores
- Minetest-mods/technic. (2019, January 26). GitHub. https://github.com/minetest-mods/technic
- wintermute. (2015, July 27). Working & wonderfull Mintest IDE, based on ZeroBrane Studio [sic] [Original Post]. https://forum.minetest.org/viewtopic.php?f=14&t=12923