setBlock is too slow

[Naohiro2g]

Hi, I have been using MCPI with my students for a long time and was tied to 1.16.5 because I was using a modified MCPI with the Forge mod. Last week I noticed the possibility of using newer versions of Minecraft on the plugin server with plugins out there.

First I tried Spigot, then PaperMC, then PurpurMC with success with the 1.18.2 compatible websocket plugin. At that time I was thinking that there was still a lot of work left to do to make the mods as well as the plugins compatible with the latest Minecraft versions.

Then yesterday I happened to find your repository and decided to give it a try, and was surprised to see that it says it would work with 1.18.2+. So I tried using the 1.18.2 compatible websocket plugin with PurpurMC 1.21.1 and confirmed that at least postToChat and setBlock work. Excuse me???

And as you mentioned, the release version of the MCPQ-plugin also worked fine. Amazing.

Then I wrote a little MCPQ code and tried it, and noticed that setBlock is very slow. Slowing down is sometimes very good to show off the process of building but ... Is there any limit to how fast setBlock can work? Or about the communication speed?

Here is my code:

from mcpq import Minecraft, Vec3

mc = Minecraft()  # connect to server on localhost

mc.postToChat("Hello Minecraft!")

def setBlocks(block_id, start, end):
    for x in range(min(start.x, end.x), max(start.x, end.x + 1)):
        for y in range(min(start.y, end.y), max(start.y, end.y + 1)):
            for z in range(min(start.z, end.z), max(start.z, end.z + 1)):
                mc.setBlock(block_id, Vec3(x, y, z))


setBlocks("air", Vec3(-40, 63, -40), Vec3(40, 70, 40))
setBlocks("gold_block", Vec3(-40, 61, -40), Vec3(40, 61, 40))
setBlocks("grass_block", Vec3(-40, 62, -40), Vec3(40, 62, 40))

[icezyclon]

Hi there 👋

First off, I am very happy that you found the repository and decided to use it. 😁
I also use it for teaching purposes so if you have any questions don't hesitate to ask 👍

Regarding the version/compatibility, the plugin is written using a certain Bukkit API, which means it is compatible with Minecraft 1.18.2, but as Bukkit takes great care to allow plugins to remain backwards compatible (in most cases), the plugin automatically works for newer versions as well (as long as Bukkit does not introduce any breaking changes in the API).
However, some methods that use commands under the hood, aka. non-natively implemented commands, e.g., world.setBed, entity.replaceItem or entity.replaceHelmet, might not work (fully) for newer versions!

Regarding the speed of the methods, generally there are two types of function calls, synchronous and asynchronous ones:
The synchronous ones basically wait until the call finishes on the server and return then (potentially with response information), these ones are pretty slow because they have to wait at least one tick on the server side, spawnEntity is one of them for example.

Most methods that do not return a specific response are asynchronous, such as setBlock, but asynchronous in this case still means, that they send a request to the server, the request is verified (for example, if you selected a valid block) and return at the same time when the block setting request is set on the server.
That means that most of the time for these is spent in flight, aka. for the direct communication back and from again (= remote procedure call). This is pretty much the fastest it could be for each individual call (it could maybe be a bit faster with a stream request, which I might look into in the future).

But as I said, the cost is for each individual call, and because you often want to set many blocks of the same type, there is the setBlockList function, which takes an entire list of positions and a single blocktype. This requires a single verification of the block types (for every 50k blocks set) and is thus pretty much as fast as possible while still setting individual blocks (so no chunk or stream operations or similar).

Your code could be made significantly faster with this method, for example:

from mcpq import Minecraft, Vec3

mc = Minecraft()  # connect to server on localhost

mc.postToChat("Hello Minecraft!")

def setBlocks(block_id, start, end):
    positions = []
    for x in range(min(start.x, end.x), max(start.x, end.x + 1)):
        for y in range(min(start.y, end.y), max(start.y, end.y + 1)):
            for z in range(min(start.z, end.z), max(start.z, end.z + 1)):
                positions.append(Vec3(x, y, z))
    mc.setBlockList(block_id, positions)


setBlocks("air", Vec3(-40, 63, -40), Vec3(40, 70, 40))
setBlocks("gold_block", Vec3(-40, 61, -40), Vec3(40, 61, 40))
setBlocks("grass_block", Vec3(-40, 62, -40), Vec3(40, 62, 40))

In comparison to MCPI, which limits the number of operations per tick to 9000 I believe, MCPQ allows an unlimited number of operations and you should - theoretically - be able to use as much of the server resources as you want.
I hope this helps you, check out the documentation to see all the functions available 👋

PS: If you want to set a cube of one blocktype use the setBlockCube call.

[Naohiro2g]

Excellent! Thanks for the detailed explanation.

Regarding plugin servers and APIs, I am still confused about the relationship between Bukkit, CraftBukkit, Spigot, Paper, and Purpur. Bukkit is the API, CraftBukkit is the original plugin server using Bukkit, and Spigot, Paper and Purpur are derived from CraftBukkit. Is this correct?

Then which API documentation exactly are you referring to?? The Bukkit and CraftBukkit projects have already been shut down, and I assume the API is maintained by the Spigot people, on the other hand, Spigot's GitHub repository has already been archived. Anyway the efforts to maintain backward compatibility are just great, indeed.

I have a general understanding of the elements related to protocol.
I confirmed it's much faster using setBlockList and setBlockCube in place of setBlock. Rewriting the sample code for a different API would also be a good exercise for young leaners of Python.

I also tried the digital clock with LCD matrix font which was originally written for Raspberry Pi 3. I used double buffer and setBlock to update limited blocks that only changed from the past second. The reason is to make it possible to complete the update during one second regarding the processing power of Raspberry Pi.

But now, flipping the buffer like pygame with setBlockList should be much better with MCPQ.

The MCPQ 1.0.1 documentation is great, too. I will make another issue regarding build instructions for plugin, and one more issue regarfing the setup for this repo.

https://github.com/user-attachments/assets/9c76911b-8c7c-4a02-b9bc-d9f0f39b2193
https://github.com/user-attachments/assets/6455c244-591a-4f2b-b98f-68736032065f

[icezyclon]

Your welcome 👋

Regarding the API/Servers:
Pretty much. As far as I am aware the relationship goes something like this:

• There is an API simply called Bukkit that is supported by most server implementations. You cannot run Bukkit on its own, because it is just interfaces and enums, basically no implementations on its own.
• Then there is CraftBukkit, which is/was a server implementation. Its philosophy was basically to not change the vanilla Minecraft behaviour but simply implement the Bukkit API for plugins. The CraftBukkit project is maintained by the Spigot team.
• Then there is Spigot, which is a fork of CraftBukkit. They change the vanilla behaviour (for example of redstone and spawning) to allow for more features and more powerful plugins. (Also, they host their own repositories, not on Github)
• Then there is PaperMC, which is a fork of Spigot. They focus on performance over everything else, patching even more Spigot behaviour (like async chunks) and also have their own API. (I always use PaperMC servers).
• I've never heard of PurpurMC before, but it seems to be a fork/drop-in-replacement of/for Tuinity (also never heard before) which is in turn a fork of PaperMC. It seems to perform even more patches than Paper already does and seems to come with more features like a lighting engine (but it is missing other optimizations?) To be honest not to sure about that one.

As far as I can tell, all of these projects are still supported/maintained, the Bukkit, CraftBukkit and Spigot projects moved to the Spigot team's Bitbucket instance.

For MCPQ, I am using the Bukkit API version 1.18.2 here and otherwise no other dependencies. This has advantages and disadvantages:

Pros:

• The plugin is compatible with the base Spigot server implementation and all of its forks that support the API, aka. all listed above.
• The plugin is much more compatible with newer versions, as the Bukkit API is very stable and great care is taken to keep it backwards compatible.

Cons:

• I am somewhat limited in which features I can support as the Bukkit API supports only "basic" operations while some of the other APIs, like the PaperAPI, support many more features.
  • For example, I wanted to implement a feature (currently on dev branch) where commands can be executed in a blocking way and return the output on the server side back to the client. This is only possible in a limited capacity as I would have to modify the CraftBukkit functions directly. They can change/break much more often, which would potentially be bad for compatibility with newer versions. I still have not decided if I will "upgrade" the API requirements or not.

Regarding the speed of setBlock operations:
I actually ran some tests and found an easy fix that provides a substantial speedup (about 50% with setBlockList and 33% with setBlock operations) TLDR; the transformation of Vec3 into the pb.Vec3 classes was extremely slow.
The patch is on the dev branch already and will be in the next release 👍

Regarding the digital clock:
That is a super cool example, where can I find the code for that? I would like to take a look at it, your example videos look very cool.

Regarding the new issues:
You are of course welcome to open new issues if you have any questions, but build instructions/setup should be available in the readme files of this repo and the plugin. If you run into trouble just open an issue.

[Naohiro2g]

Wow, that was most clear. I will use PaperMC along with you for a while. They seem to have great documentations:
https://docs.papermc.io/paper/dev/api

And I'm happy to wait to see the improvements in speed of setBlock operations to come.

The original version of the digital clock is here: https://github.com/Naohiro2g/minecraft_remote
MCPQ version is here: https://github.com/Naohiro2g/mcpq-python

The plugin build:

• I could build the plugin from your source but was wondering what is the one without "all".
• Is it possible to make it using Java21? Kotlin is using 17.

The python package:
Because I'm using pyenv/poetry, am not familiar with the setuptools.

For my repo Naohiro2g/minecraft_remote, the intention was to create an integrated framework compatible with MCPI-MCJE1.12.2 / MCJE1.13+. (Almost) same code runs everywhere. We can use many legacy MCPI samples and our future codes will run on MCPI, too. But it could be much smarter. Apart from that, I will update the repo with the recent insights about utilizing plugin servers to enjoy remote controll over the latest Minecraft as a breaking news.

Another idea would be to implement a websocket client plugin to communicate with the proxy server via a websocket tunnel. This plugin might allow the use of inexpensive rental servers that do not give me root admin.

[icezyclon]

Wow, that was most clear. I will use PaperMC along with you for a while. They seem to have great documentations: https://docs.papermc.io/paper/dev/api

Agreed. I have used Paper with much success in the past, as it works well w.r.t. performance and runs efficiently even on low end systems, like the student's laptops.

And I'm happy to wait to see the improvements in speed of setBlock operations to come.

Great 😁 Yes the entire library is still very much in development and I have plans for more features.

The original version of the digital clock is here: https://github.com/Naohiro2g/minecraft_remote MCPQ version is here: https://github.com/Naohiro2g/mcpq-python

Thanks, you have quite the nice collection of tools there. You probably want to integrate MCPQ into your remote repo?

The plugin build:

• I could build the plugin from your source but was wondering what is the one without "all".

The all package bundles all requirements/dependencies in one big jar file (such as the Kotlin runtime).
If you provide these dependencies for many packages it is possible to provide them separatly.

• Is it possible to make it using Java21? Kotlin is using 17.

It's likely that the version can be upgraded, but you might also have to upgrade all dependence-libraries and check for compatibility issues. I might upgrade to Java21 in the future but it's not a priority.

The python package: Because I'm using pyenv/poetry, am not familiar with the setuptools.

You don't have to know anything. The entire build process is described in metadata in the pyproject.toml file. If you pip install setuptools build (venv is also required), you can then build the Python library with python3 -m build, which will/should install everything you need. (Again, all requirements and build settings are in pyproject.toml.)

[Naohiro2g]

Hi, for the Java switching, I've installed the SDKMAN and it's working.

Just now, PaperMC with the JuicyRaspberryPie plug-in is running on one of my computer at home and is available on the Internet for my students. Students can now remotely control the Minecraft world.

I've been trying to figure out how to protect that world from vandalism or a baggy code, and I realized that this plugin has no user authentication mechanism, so it's impossible.

Furthermore, I am guessing there is no user authentication even at the API level, what do you think?

My concept was to allocate an area of about 2000x400x2000 to each player, with no one allowed to destroy/build except their own area, and to allocate a shared area to each group to make it a place for collaborative work.

If the destruction/building works are done by hand or by WorldEdit, etc., it could be managed by the WorldGuard plugin.

Oh, If the player injects Python code and the plugin executes it, it might be manageable, but that sounds like a Trojan horse. Besides, it's a one-way operation, which we don't need; we need interaction, like playing reversi in a Minecreaft world. Or like playing rock-paper-scissors with the CPU using image recognition.

[Naohiro2g]

demo movie. Sorry this was not done by MCPQ, I'll make it later.
https://github.com/user-attachments/assets/1e112908-f12c-4e54-97de-274ef6eff4ac

[icezyclon]

Hi, so first let me just say that MCPQ is designed with the idea that having access the MCPQ gives you access to the entire server. Having full control of the server was always the goal and I do not believe I will add any user-separated authentification.

However, gRPC is fully operable - and even designed for - use over the network and supports a large number of secure connection features using SSL/HTTPS or token-based authentification. Python Auth Example.
I think that at least adding a secure channel (HTTPS + self signed certificates) would be a good feature to have in the future. 👍
I am pretty sure that implementing certificate encryption is pretty easy and token-based auth might also not be too complicated.

For your use case, you might want to implement the authentification via a proxy, as then you could also control not only who can access what part of the world but also which methods they can use. The protocol defines all allowed methods/access to the Minecraft world.
The idea with a certain area sound like a good idea in general, but I think it will prove very difficult to actually "protect" a server from a malicious (or mischievous) users. For example, they could build (with or without MCPQ) giant TNT explosions, entity lag or redstone lag, just to name a few - and while WorldGuard would protect against users just building (some) of these, it would be harder to prevent MCPQ in its current form from doing the same.

Again, your best bet is probably a proxy, where you check the commands sent by the students and allow certain ones, in certain quantities or certain areas. This would give you a lot of fine control and also allow you to have a uniform interface for the students while using multiple plugins in the background.

Regarding allowing Python code to be injected, that sounds like an even harder problem to correctly sandbox. I looked into that briefly (there are some projects that would help) but again, its sounds like too much of a hassle.

Let me know how you decide to tackle this problem 👋

PS: The only feature that the plugin currently has is the host option in the config.yml file that allows only certain host (ranges) to connect to MCPQ (localhost by default).

[icezyclon]

Hi there, I wanted to give an update on some of the topics we talked about.

First things first, version 2.0.0 was released, which comes with a lot of updates, improvements and more. 🚀

Regarding the Bukkit API and different servers/fork, there has been a development: Paper is hard forking Spigot, meaning that starting from version 1.21.4 they will slowly diverge and won't be the same project anymore (this has not resulted in incompatibilities yet as far as I can tell). They announce it in this blog post.
So, a lot of what I explained above does not apply anymore. At some point MCPQ will have to either support the Spigot API, the Paper API or both (both would mean having separate code bases).
I am testing everything with Paper at the moment as that is what I use most. The library might be built against the Paper API in the future, we'll see.
I will track compatibility here.

Let me know if you have any more questions 👍

[icezyclon]

I will close this issue for now!
If you want to continue our discussion or have new questions, feel free to open a new issue, discussion is always welcome 😉 👍