Headless renderer #14

neverix · 2022-12-08T04:26:24Z

Add compact, short information about your PR for easier understanding:

Add headless renderer
Changes a few render targets

To do

This PR is a Work in Progress

Most of the work
Hide the window

How to test

make -j8 && python hacking_testing/test_loop.py

This reverts commit 7358d6b.

This reverts commit 4b70f36. Should be squashed before merging.

Fix memleak in render core

Fix crash caused by deleting buffer we do not own

The symbol will be defined as 1 as long as the BUILD_HEADLESS CMake var is a truthy value.

Merge develop into headless renderer

Decouple BUILD_HEADLESS CMake option from preprocessor symbol

Cmake sdl2 target fix

Merge develop

rk1a · 2023-02-09T22:45:35Z

hacking_testing/minetest_env.py

@@ -392,6 +392,8 @@ def _write_config(self):
            config_file.write("mute_sound = true\n")
            config_file.write("show_debug = false\n")
            config_file.write("enable_client_modding = true\n")
+            config_file.write("video_driver = null\n")


I think these config changes shouldn't be included because I was just adding them to get the test loop to run in colab. but then it turned out that only empty frames were sent from minetest.

rk1a · 2023-02-09T22:54:48Z

Overall I am not quite sure why we would want to merge this now. We already have the semi headless mode using Xvfb.
The code in this branch hides the window, but it turned out this is only possible with a running XServer again requiring Xvfb (see #46).
So from what I know this would not add any useful functionality while adding the additional SDL dependency.

What would be good to know is if there is a performance difference between the current state of develop and this branch.
Also maybe this somehow does work without Xvfb on the TPU server? But seems like #46 is a general problem.

JosiahWI · 2023-02-10T01:30:45Z

src/CMakeLists.txt

 	${LUA_INCLUDE_DIR}
 	${GMP_INCLUDE_DIR}
 	${JSON_INCLUDE_DIR}
 	${LUA_BIT_INCLUDE_DIR}
+	${SDL2_INCLUDE_DIR}


Not needed.

JosiahWI · 2023-02-10T01:30:51Z

src/CMakeLists.txt

@@ -528,10 +538,12 @@ include_directories(SYSTEM
 	${ZMQ_INCLUDE_DIR}
 	${ZMQPP_INCLUDE_DIR}
 	${SQLITE3_INCLUDE_DIR}
+	${SDL2_INCLUDE_DIR}


Not needed.

JosiahWI · 2023-02-10T01:32:26Z

minetest.conf

@@ -1,4 +1,5 @@
 name = MinetestAgent
+update_last_checked = 1670529488


Probably shouldn't be committed.

JosiahWI · 2023-02-10T01:32:47Z

src/CMakeLists.txt

 		${ZSTD_LIBRARY}
 		${ZMQ_LIBRARY}
 		${ZMQPP_LIBRARY}
 		${X11_LIBRARIES}
 		${SOUND_LIBRARIES}
 		${SQLITE3_LIBRARY}
+		${SDL2_LIBRARY}


Not needed.

JosiahWI · 2023-02-10T01:33:51Z

.gitignore

@@ -133,7 +133,8 @@ compile_commands.json
 .vs/

 # Optional user provided library folder
-lib/irrlichtmt
+# TODO: don't add irrlichtmt, apply patch from CMakeLists.txt


Don't merge TODO comments.

JosiahWI · 2023-02-10T01:36:40Z

CMakeLists.txt

@@ -68,6 +69,19 @@ set(ENABLE_UPDATE_CHECKER (NOT ${DEVELOPMENT_BUILD}) CACHE BOOL
 set(CMAKE_MODULE_PATH ${CMAKE_MODULE_PATH} "${CMAKE_SOURCE_DIR}/cmake/Modules/")


+
+if(BUILD_CLIENT AND BUILD_HEADLESS)


This should not be in both CMakeLists.txt and src/CMakeLists.txt.

JosiahWI · 2023-02-10T01:36:56Z

src/CMakeLists.txt

@@ -36,6 +36,17 @@ set(CMAKE_BUILD_TYPE "${CMAKE_BUILD_TYPE}" CACHE STRING
 # Set some random things default to not being visible in the GUI
 mark_as_advanced(EXECUTABLE_OUTPUT_PATH LIBRARY_OUTPUT_PATH)

+if(BUILD_CLIENT AND BUILD_HEADLESS)


This should not be in both CMakeLists.txt and src/CMakeLists.txt.

JosiahWI · 2023-02-10T01:38:18Z

src/client/client.cpp

@@ -1834,6 +1836,7 @@ void Client::makeScreenshot()
 	if (!raw_image)
 		return;

+	// warningstream << "got data" << raw_image->getDimension().Width << std::endl;


Don't merge commented out code.

merge develop into headless renderer

Rabbidon and others added 30 commits December 3, 2022 20:36

merge commit

8ad2d3f

Fixed all issues

6173102

Architecture for screenshotting

ef83b83

Final additions

8c3723a

Fix color bug again

49be8b5

One weird trick

59a83a5

..debugging

10af879

Remove headless code

772ae82

Make everything work

3fa414c

Remove debugging code

43f69bc

Changes

37f4282

Merge branch 'headless-renderer' into develop

67ba452

Test adding irrlicht

467ce1f

Make it build

724bb7c

Recreate setup

c2ede77

Merged

0c9e65b

Add irrlichtmt

07ad613

Everything works

f76587f

Change submodule path

f1882c9

Update renderingengine.cpp

610c7f0

Try to fix memory leak

7358d6b

Revert "Try to fix memory leak"

25e0cce

This reverts commit 7358d6b.

Remove rendering

65b624e

Make headlessness a flag

b3a7477

Make it partially work

b33b8ec

include no SDL headers if no headless build

0a9a351

Add IMPORTED target for SDL if none exported

4b70f36

Fix memory leak in render core

b11d2d8

Revert commit from other branch

564b4ba

This reverts commit 4b70f36. Should be squashed before merging.

Merge pull request #26 from JosiahWI/fix/render-core-memleak

078108a

Fix memleak in render core

neverix and others added 15 commits January 13, 2023 01:54

Revert revert commit

7c3effd

Do not delete buffer we do not own

691eb6f

Merge pull request #28 from JosiahWI/fix/render-core-bad-delete

e615079

Fix crash caused by deleting buffer we do not own

Define BUILD_HEADLESS symbol in cmake_config.h.in

ad86c4c

The symbol will be defined as 1 as long as the BUILD_HEADLESS CMake var is a truthy value.

Remove duplicate find_package call for SDL2

2c33126

it compiles

7b01b1b

fix SDL2 dependencies

3ffe1ad

Merge pull request #38 from AI-WAIFU/zmqpp-headless

7ff403c

Merge develop into headless renderer

Fix SDL2 target not found

d21895c

Set video_driver=null in minetest.conf

fc5b597

Disable shaders in minetest.conf

e93b26a

Merge pull request #29 from JosiahWI/fix/build-headless-symbol

2f3ce0f

Decouple BUILD_HEADLESS CMake option from preprocessor symbol

Merge pull request #41 from EleutherAI/cmake-SDL2-target-fix

8161954

Cmake sdl2 target fix

merge develop into headless renderer

53ca098

Merge pull request #52 from AI-WAIFU/merge-develop

95fa15a

Merge develop

AI-WAIFU marked this pull request as ready for review February 9, 2023 16:46

rk1a requested changes Feb 9, 2023

View reviewed changes

JosiahWI requested changes Feb 10, 2023

View reviewed changes

AI-WAIFU and others added 5 commits March 19, 2023 17:47

add build_instructions

b8913a3

merge develop into headless render

191dec2

mod build instructions after merge

1d0bf70

Merge pull request #60 from AI-WAIFU/merge-headless-renderer

ac11d8c

merge develop into headless renderer

incorporate feedback

6d25eb5

AI-WAIFU merged commit 34a44b2 into develop Mar 19, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Headless renderer #14

Headless renderer #14

neverix commented Dec 8, 2022 •

edited

Loading

rk1a Feb 9, 2023

rk1a commented Feb 9, 2023

JosiahWI Feb 10, 2023

JosiahWI Feb 10, 2023

JosiahWI Feb 10, 2023

JosiahWI Feb 10, 2023

JosiahWI Feb 10, 2023

JosiahWI Feb 10, 2023

JosiahWI Feb 10, 2023

JosiahWI Feb 10, 2023

		@@ -1,4 +1,5 @@
		name = MinetestAgent
		update_last_checked = 1670529488

		@@ -68,6 +69,19 @@ set(ENABLE_UPDATE_CHECKER (NOT ${DEVELOPMENT_BUILD}) CACHE BOOL
		set(CMAKE_MODULE_PATH ${CMAKE_MODULE_PATH} "${CMAKE_SOURCE_DIR}/cmake/Modules/")



		if(BUILD_CLIENT AND BUILD_HEADLESS)

Headless renderer #14

Headless renderer #14

Conversation

neverix commented Dec 8, 2022 • edited Loading

To do

How to test

Choose a reason for hiding this comment

rk1a commented Feb 9, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

neverix commented Dec 8, 2022 •

edited

Loading