confusing dependency graph

Use animalmaterials and basic_materials for Minetest mods: low-level dependencies for better architecture

Have you ever wondered why there are so many circular dependencies in your project? Maybe you’ve made reusable code in your project and made everything into components (or at least a good number of them, like Drupal 9 tried to do better than previous versions), but the order in which dependencies should be determined is unclear. This is not as important in true object-oriented programming (OOP) languages like C#, but in Lua or even C++ there is still an issue of order-of-definition. I started to write a long “readme” for why I forked animalmaterials in 2021 (about 4 years after it was abandoned) and it became this article. The animalmaterials modpack is an excellent case study on the general Computer Science topic of why definitions should be in a module that is separate from the behaviors.

The purpose of a low-level module (such as a low-level minetest mod that primarily provides registrations) is to define things that multiple other modules can use (In game modding, mod is short for modification rather than module, but the principle applies to both). For example, the Minetest cooking mod depends on animalmaterials and doesn’t have to depend on any specific mobs mod. My fork is a fork of AntumMT’s fork of sapier’s animalmaterials modpack. The dependency on mobs was removed. AntumMT’s antum branch had added the “mobs” dependency. I commented on the related commits (f6fc55b and 00fd6a5) as follows, which is also the TLDR version of this article:

I don’t know if you care about this mod anymore, but this commit and 00fd6a5 defeat the single advantage of animalmaterials: to be a low-level dependency that other mobs mods and various mods can utilize to help them integrate with each other and not register duplicate craftitems. Every time you add a dependency, that mod and mods that mod requires can no longer utilize animalmaterials, because they must load before it. Therefore, the better way to integrate the mods listed in this commit are to make them utilize a common low-level mod (animalmaterials). minetest.override_item is useful in those cases from their end since those high-level mods personalize behavior as players and server owners wish. With minetest_game planned to not ship with minetest anymore and @rubenwardy encouraging people to fork it, mods such as basic_materials and possibly even animalmaterials will become more important for the reasons above. Core devs are even encouraging mods not to depend on default. My fork based on this antum branch will be useful to what I’m doing. I can’t guarantee anyone else will care about this modpack in particular due to being tied to mobf, but I also plan to make mobf optional in other mods in this modpack (even making it optional forces it to be a high-level mod, but that is probably ok for mods in the animalmaterials modpack other than the animalmaterials mod).

https://github.com/AntumMT/mp-animalmaterials/commit/f6fc55bc54d327307b01c6144c751cbe8a2e1e0a

What can existing mods do better?

You can change your mobs mod to depend on the animalmaterials mod or a mod like it (mods like default and basic_materials use the same idea, being low-level mods which primarily define materials for use by other mods). Then mods don’t have to depend on your specific mobs mod and assume which materials are present. The headings below show why this is better.

1. Let people choose how their project behaves.

If this article applies to you, someone is already going to use your mod. You don’t have to make them use behaviors from some other mod. You can give them free choice by depending on a mod that offers definitions (registrations) rather than behaviors. Even then, they may not want all of the behaviors in your mod, so if you put your registrations in a separate mod, users can choose whether to use the behaviors your programmed or just your nodes/craftitems.

2. Reduce the number of assumptions regarding names of accessors.

What I’ve explained so far indicates that a high-level mod like mobs redo assumes many things by the time it is loaded. A worse aspect of the situation is that if you changed mobs redo or another high-level mob like it, the code would become very divergent (you couldn’t reuse as much code). For example, even just the fact that mobs redo defines chicken meat instead of using animalmaterials means that every mod that affects chicken must use the “mobs” namespace to access it, or add checks to see whether that or a different chicken is present–none of that would have to happen if mobs redo instead simply used animalmaterials and improved it if necessary (and for compatibility like my animalmaterials fork does, kept the exact same set of craftitems).

3. Allow more components to reuse your code.

The number of assumptions isn’t the worst part about placing low-level items in a high-level mod like mobs. The worst part is that every lower-level mod in the tree of dependencies including the things that those depend on and so on have no access to whatever mobs defines–such as meats in the case of mobs_redo–no mod already loaded (due to being in the dependency tree) can use them! There are probably cases other than mobs redo where this problem is even more disruptive. Even if mod B risked using the materials from the high-level mod A, there are at least two problems with that:

  • Mod B can’t add the mod as a dependency nor an optional dependency because that would create a circular dependency. Not adding the mod as a dependency breaks the concept of dependencies: Mod B would assume things are there without listing requirements.
  • Mod B can’t check for registered craftitems nor nodes: Mod B can’t reliably check if there is one or another version of the mod if the mod changes or a different fork or replacement version of the mod is used. For example, if a mod depends on “farming”, the mod still has to check whether certain craftitems or nodes are defined in order to determine whether minetest_game/farming or farming redo is installed. In that case, farming is a low-level mod and doesn’t have any problems. However, if the mod that registered the items was a high-level mod with several dependencies like mobs redo, the circular dependency problem would occur: The mod wouldn’t be loaded yet and mod B would have no way to check if something was registered. The solution is for both mod B and mobs redo to depend on a low-level mod like animalmaterials. Mod B can check for registrations after that, but doing so still isn’t ideal: See “Adding everything to the core doesn’t reduce work–It creates technological debt” below regarding why not to keep expanding mod A (default or some other mod) continuously.

4. Adding everything to the core doesn’t reduce work–It creates technological debt.

The main disadvantage to a monolithic solution is that you and those using your product have to deal with not just a moving target, but a target that acquires parts from other things. The other things still exist. Whether you use them or not, someone else does or their user-generated content such as a Minetest world may use them. Therefore, the downstream mod maintainer is left to support both your version and the componentized version.

Having a low-level mod is a better solution than adding everything to the core (or the Minetest default mod, which functions as a core component for perhaps the majority of mods). If the registrations are in a separate mod, mod B and the high-level mod can require the low-level mod and have a reasonable assumption of what is in there, and if there is a known variant, the two mods can check what is registered. Checking for what is registered by default can become a problem if low-level items used for recipes, treasure/prizes mods, minigames, etc. may use them. Each of those uses are entire categories of mods. Having boilerplate such as:

if minetest.registered_nodes["default:permafrost"] then

…in every mod that depends on default is already technically necessary (if the node is used). That node isn’t very important for recipes, treasure/prizes mods, minigames, etc., but it is already used by some mods and maybe a mob mod will want the creature to spawn there or a plantlife mapgen mod will want a certain plant to spawn there.

  • If the core or default succumbs to feature creep (or scope creep) to encompass nodes or craftitems those categories of mods use, then the registration checks will have to creep into mods in those or similar categories, in other words, such ever-expanding boilerplate will be necessary to maintain in a virtually unlimited and ever-expanding number of mods.
  • The problem can get more complex if either default or another mod may define it. Then each high-level mod must optionally depend on both low-level mods, check for the registration in both namespaces, and store the id string of the one that is available in a variable for later use in the mod (The namespace is unknown so the ID can’t be used directly).

5. Cooperate with others for better results.

As a positive solution, create or reuse low-level registrations as a separate dependency. Doing registrations in a low-level mod helps you cooperate with others by not creating duplicate materials such as ingredients or mapgen nodes in mods where you add features.

Examples

cooking

This mod can solve the most frequent problems.

It is a mod which provides recipes for raw meat(s): This is a frequent issue since several advanced cooking mods exist which use specific meats or meat groups.

animalmaterials

The animalmaterials mod in (my fork of) animalmaterials modpack can be used by:

  • Any mod which provides specific mob(s)
  • Any mod which provides recipes for raw meat(s) (such as cooking)

nether (my fork)

My fork of nether can be used by:

  • a mod which defines the nether * (this would be a fork of nether or a new nether or nether-like mapgen mod)
  • some odd mod which lets you obtain nether materials some other way *, such as a prizes, loot, or minigame mod.
  • a mod which defines nether fences but doesn’t require a nether realm mod (Maybe someone wants a museum or creative world with the materials but not the maintenance and storage involved in having a nether realm). *

basic_materials

This mod solves the most disruptive problems.

The basic_materials mod can be used by:

  • chains
  • homedecor
  • mesecons but isn’t yet… See “mesecons should use my basic_materials mod” mesecons issue by VanessaE
  • technic
  • any mod that wants the materials for recipes or mapgen but doesn’t want to require mesecons (or only wants mesecons or technic but not both)

*: Examples marked with ‘*’ are only hypothetical and don’t represent any mod(s) that I know to exist.

What if I want different features?

You may ask, “What if I want different features than these simplistic items offer?” Depend on animalmaterials then use minetest.override_item. The Minetest lua API has the function for one reason: to solve your issue! Not everyone wants your specific lasso, but you can still have all the features you want when you or they are using your specific lasso mod–problem solved. Then multiple mods can utilize the same lasso or make new recipes for it and they only ever need animalmaterials not some lasso in some specific namespace.

If you make animalmaterials an optional dependency rather than a required one, simply check for it in your code (such as via rawget (_G, "animalmaterials")), then if animalmaterials isn’t present, call minetest.register_craftitem instead of minetest.override_item. Then in the same case you can optionally make an alias from yours to the animalmaterials, such as if you want to use an old world that contains items from an old version of your mod that was monolithic.

Other Applications

I welcome comments if they are courteous critique with logic or examples–should we use data-oriented design, or functional programming? You can decide for yourself whether my explanation defeats the concept of OOP, which includes behaviors in definitions! Probably not, since OOP (C++ etc) usually involves abstract classes, interfaces, along with inheritance. Using separate registration mods along with minetest.override_item is analogous to OOP patterns, validating my points.


Posted

in

by

Comments

2 responses to “Use animalmaterials and basic_materials for Minetest mods: low-level dependencies for better architecture”

  1. Герхард "PICCORO" Lenz McKAY Avatar

    hi poikilos, reading you after you comment in the tenplus1 issue..

    i m not finished of reading this.. but do you know that minetest it has a type of sabotage… related to multicraft… that is to say…

    although they have contributed mutually… I feel that there is really no great contribution… and that it is really multicraft that benefits… it is even a company that promotes this attitude… of a multitude of games that impoverish the real effort root

    1. poikilos Avatar
      poikilos

      Piccoro, apparently people attacked MultiCraft’s servers but I do not see any evidence MultiCraft sabotaged Minetest. I worked for Maksym (MoNTE48) and contributed a significant amount of Lua code and C++ code, as well as several models. As far as I know, all of the models are proprietary and haven’t been released, so I’ll eventually write an article here with screenshots of the project files and works in progress to demonstrate my involvement. I told Maksym that I would agree to keep discussions with him private unless he described illegal activity or issues in public-licensed projects. Nonetheless, he discussed illegal activity and I told Celeron55 about it.

      The first thing that happened in the back and forth, to my knowledge, is that Maksym was attacked for discussing joint development with OldCoder:

      About my relationship with .net. Lejo has been trying to break my server for a long time. I have evidence, but no evidence.

      . . .

      Imagine – the server worked 12 days in a row. Tens of thousands of new players! When Robert wrote about me on Twitter, someone found a very old vulnerability and started crashing my server every time a lot of players visited it. It was a very old mistake.

      . . .

      When I fixed it, the server began to crash due to a builtin error. Tenplus is a bad coder. I rewrote this part of the code. Now the server is overloafs (big lag). He cannot overload it too much (MultiCraft has an exclusive playerstep function. The more players, the more difficult it is to reboot the server with one mod.) But he tries. This happens only during his waking hours.

      As I said on the forum. My server is only for testing and debugs. Therefore, I allowed access to 123 players. If someone decides to do attack to my servers from the multicraft server list, I will destroy minetest

      I asked him for the commit that fixed the security issue and he provided the following (I edited the URL here to add the -legacy part): https://github.com/MultiCraft/MultiCraft-legacy/commit/b757cdb6e2dc0afa8e7668a9f40e7d2b28772aae#diff-7b4242351dd3cf5b2a70c8aa9d12149b

      We discussed it further then had the following exchange:

      16:23 Maksym Hamarnyk “I fixed it already. The biggest MultiCraft bug is mobs_redo. I will find a person who will transfer mobs to the engine (C++)”

      “The server is stable. Then someone starts attacking something and crashing it with one bug.”

      “I fix it and everything works perfectly for several weeks. Then he finds a bug. He really has nothing to do. He needs a woman”

      16:24 Jake Gustafson “Sad”

      16:25 “Not for me 😁”

      16:25 Jake Gustafson “Yes”

      16:25 Maksym Hamarnyk “Imagine how much I would have to pay a team of professional testers? He does it for free.”

      16:26 Jake Gustafson “Lol”

      September 7, 2019 Maksym said that if he is attacked he will retaliate and “DDoS Minetest.net if they decide to harm me.” September 9, 2019 Maksym said,

      I can guarantee that I am involved in interruptions of minetest net only 1 time. Problems were observed for about 10-15 minutes to check my capacities. I’ve never done DDoS anymore

      As I said, I am ready to take this step if the net community decides to sue me, cause physical harm, publish my personal data and threats, and the like.
      I see no reason to spend a lot of money on DDoS games, the source code of which I have been using for many years, and whose development I would like to help.

      In fact, the infrastructure of Minetest is so terrible, few of them could do it.
      Lack of backups, RAID disks, only one server, lack of protection against ddos ​​(even the base one), only one centralized server-list. I am sure that someday some idiot will definitely turn off the site for several weeks.

      I’ve now published his threats (above), so if my servers go down, he’d be the first suspect, and the attacker who attacks OldCoder’s friends, the next. OldCoder and I will be mostly using a MultiCraft fork for servers to be compatible with virtually any version of the Minetest/MultiCraft client, and I’ll make sure the security fixes and the change for logging the user involved in a crash are included.

      I provided the last quote to Celeron55 April 4, 2020 because the forum was timing out that day. Celeron55 replied that “the VPS is just being crappy”, and that the logs didn’t seem indicate suspicious activity.

      I can say I contributed C++ code to MultiCraft to identify the user causing a crash, but don’t know if Maksym retaliated against such a user or against Minetest.net. He described multiple exploits done on the same day to his servers. Knowing all of the exploits would have required extensive and detailed knowledge of Minetest. Therefore, the attacker is likely someone that had been in Minetest for a long time. Also, I reviewed and helped Maksym add code for rate limiting to mitigate attacks.

      As far as code he’s contributed, the only problems I’ve seen were related to Lua code, and the only problems were related to Lua code calling functions that were only in MultiCraft. I didn’t personally see him carry out threats in the form of DDoS nor in the form of code. Note that some of his assets and probably some of the code in https://github.com/MultiCraft/MultiCraft_game is incomplete and old, and that some of what he’s added as far as assets (and potentially code) is proprietary and closed. He says he did so because there were Chinese clones of MultiCraft appearing. Though I discovered some Minetest or MultiCraft clones that appeared to be based on “Minecraft in Unity” or other open source demos, he specifically said there were clients that would log into his server and they all had the same first name but with a number after them and that they were from China and couldn’t type English so the client would fill in a random username. If you could point to C++ or Lua code that you think it malicious, I’d be happy to review it if that is what you are saying is your concern. If that is not your concern, I would need clarification.

      I’m not sure why you said it is a “company”: He doesn’t have a company per se, and has stolen the name of multicraft.org and isn’t a part of that site. As for MultiCraft impoverishing the root effort of Minetest, I don’t see the problem. All he expressed is a that he wanted to make money, wanted to make a better game, and would retaliate if attacked. He said that he’d contribute code back to MT5 because he wanted it to stay alive with support from his own popularity, so that he would be using code that was maintained so he wouldn’t have to maintain it all himself. That is a fair position for a business to take. The only negative things he said about Minetest were about the abilities of the maintainers and the low popularity, and various shortcomings (such as compared to Minecraft). Our main difference was that: He kept saying, on many occasions, that anything not popular was “dead” and that MultiCraft was the only thing that mattered because of its popularity. He offered other insults of mods and mod compatibility and even said he hates mods.