Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

joltphysics: add version 5.2.0 #24670

Merged
merged 20 commits into from
Jan 30, 2025
Merged

joltphysics: add version 5.2.0 #24670

merged 20 commits into from
Jan 30, 2025

Conversation

cuppajoeman
Copy link
Contributor

@cuppajoeman cuppajoeman commented Jul 19, 2024

Summary

Changes to recipe: joltphysics/[5.0.0]
These changes allow for users to install jolt5.0.0 with conan

Motivation

Many new features were added in jolt 5.0.0, these pertain to soft body, vehicles, character, constraint, collision detection, and can be read more about here: https://github.com/jrouwe/JoltPhysics/releases/tag/v5.0.0

Details

I am still not an authorized user, so I don't think this will be able to be merged in until I am, but I thought I would start the merge request a bit earlier, because it's my first time doing this and thought that there might be things that I have to change after review, by the time it's correct I'd probably have authorization.

WARNING: This merge request is most likely incorrect because I don't fully understand what the previous patch was doing and so I didn't include a patch in this commit, even though running conan . create --version=5.0.0 works just fine.

So far I have:

Questions

  • I am the least familiar with patching, notice that in my commit I have added the following line to the CMakeLists.txt in test_package:
add_definitions(-DJPH_PROFILE_ENABLED=0 -DJPH_DEBUG_RENDERER=0)

I added this because when it was not there and I tried to run conan create . --version=5.0.0 I got the error:

======== Testing the package: Executing test ========
joltphysics/5.0.0 (test package): Running test()
joltphysics/5.0.0 (test package): RUN: ./test_package
Version mismatch, make sure you compile the client code with the same Jolt version and compiler definitions!
Mismatching define JPH_PROFILE_ENABLED.
Mismatching define JPH_DEBUG_RENDERER.

From what I understand this is because we build the joltphysics package in isolation from the test package, since they both use jolt, they must have matching compiler definitions or else the code is not compatible. When I inspect the previous version patches they have lines like this:

-target_precompile_headers(Jolt PRIVATE ${JOLT_PHYSICS_ROOT}/Jolt.h)
-target_compile_definitions(Jolt PUBLIC "$<$<CONFIG:Debug>:_DEBUG;JPH_PROFILE_ENABLED;JPH_DEBUG_RENDERER>")
-target_compile_definitions(Jolt PUBLIC "$<$<CONFIG:Release>:NDEBUG;JPH_PROFILE_ENABLED;JPH_DEBUG_RENDERER>")
+target_compile_definitions(Jolt PUBLIC "$<$<CONFIG:Debug>:_DEBUG>")
+target_compile_definitions(Jolt PUBLIC "$<$<CONFIG:Release>:NDEBUG>")

Is my method of changing the CMakeLists.txt incorrect? Should I be using patching for this? [Also I have read the corresponding section on patches]

  • Another question I had comes from the previous patch that looks like this:

+add_library(Jolt ${JOLT_PHYSICS_SRC_FILES})
+target_compile_features(Jolt PUBLIC cxx_std_17)
+include(GNUInstallDirs)
+install(
+ TARGETS Jolt
+ ARCHIVE DESTINATION ${CMAKE_INSTALL_LIBDIR}
+ LIBRARY DESTINATION ${CMAKE_INSTALL_LIBDIR}
+ RUNTIME DESTINATION ${CMAKE_INSTALL_BINDIR}
+)
+install(DIRECTORY ${JOLT_PHYSICS_ROOT} DESTINATION ${CMAKE_INSTALL_INCLUDEDIR} FILES_MATCHING REGEX "(.*).(h|inl)$")
+if(JPH_DEBUG_RENDERER)
+ target_compile_definitions(Jolt PUBLIC JPH_DEBUG_RENDERER)
+endif()
+if(JPH_PROFILE_ENABLED)
+ target_compile_definitions(Jolt PUBLIC JPH_PROFILE_ENABLED)
+endif()

What is the purpose of this modification? Do I need create a patch that does that in my version here?

@toge I know that you created the previous patches, would you be able to help answer a few of the above questions?

  • I am also curious if I am supposed to add an option in a one-to-one manner for each available option in the new jolt version (aside from build_testing), the options are laid out here. Note that I have read the corresponding section about the conanfile options attribute

Apologies for asking so many questions, but I really want to help contribute to conan center index. I think once I complete my first merge request I will have far less questions.

Closes: #25558


@conan-center-bot

This comment has been minimized.

@conan-center-bot

This comment has been minimized.

@ErniGH ErniGH self-assigned this Aug 21, 2024
@conan-center-bot

This comment has been minimized.

@conan-center-bot

This comment has been minimized.

@cuppajoeman
Copy link
Contributor Author

Hey @ErniGH (or @toge who created the original files) Hopefully you can help me out with this, the main problem I'm having now is that I'm having trouble understanding why patching is necessary, what the current patches do for this recipe, and how to generate my own patches.

I read this: https://github.com/conan-io/conan-center-index/blob/master/docs/adding_packages/sources_and_patches.md, but I'm still having trouble seeing why we need a patch in our specific case of this recipe, my specific questions are for the entire patch, but I'll focus on this one first:

+add_library(Jolt ${JOLT_PHYSICS_SRC_FILES})
+target_compile_features(Jolt PUBLIC cxx_std_17)
+include(GNUInstallDirs)
+install(
+	TARGETS Jolt
+	ARCHIVE DESTINATION ${CMAKE_INSTALL_LIBDIR}
+	LIBRARY DESTINATION ${CMAKE_INSTALL_LIBDIR}
+	RUNTIME DESTINATION ${CMAKE_INSTALL_BINDIR}
+)
+install(DIRECTORY ${JOLT_PHYSICS_ROOT} DESTINATION ${CMAKE_INSTALL_INCLUDEDIR} FILES_MATCHING REGEX "(.*).(h|inl)$")
+if(JPH_DEBUG_RENDERER)
+    target_compile_definitions(Jolt PUBLIC JPH_DEBUG_RENDERER)
+endif()
+if(JPH_PROFILE_ENABLED)
+    target_compile_definitions(Jolt PUBLIC JPH_PROFILE_ENABLED)
+endif()
  • Why are we adding the compile features for cxx_std_17?
  • Why do we define a way to install jolt?
  • Why are we adding the JPH_DEBUG_RENDERER definition to the compile definitions of the Jolt library?

@AbrilRBS AbrilRBS assigned AbrilRBS and unassigned ErniGH Sep 5, 2024
@AbrilRBS
Copy link
Member

AbrilRBS commented Sep 5, 2024

Hi @cuppajoeman thanks for taking the time to add the new version. To be able to answer your questions, we'll need to check the previous PRs, see if some info was discussed there. I'll try to take a look tomorrow, see if we can come up with some answers! :)

@shiena
Copy link
Contributor

shiena commented Sep 14, 2024

@cuppajoeman
It looks like that patch was first made over here.
#14573 (@SpaceIm )

@shiena
Copy link
Contributor

shiena commented Sep 14, 2024

@cuppajoeman @AbrilRBS
I would like to add the following changes for mac.
When building with build_type=Release in apple-clang, INTERPROCEDURAL_OPTIMIZATION is turned on and binaries with bitcode enabled are generated.
Since bitcode has been deprecated since Xcode14 and is now causing trouble, I want to disable it by default for mac.
https://developer.apple.com/documentation/xcode-release-notes/xcode-14-release-notes#Deprecations

diff --git a/recipes/joltphysics/all/conanfile.py b/recipes/joltphysics/all/conanfile.py
index ef2cbdfc7..bdd5f9e77 100644
--- a/recipes/joltphysics/all/conanfile.py
+++ b/recipes/joltphysics/all/conanfile.py
@@ -1,5 +1,6 @@
 from conan import ConanFile
 from conan.errors import ConanInvalidConfiguration
+from conan.tools.apple import is_apple_os
 from conan.tools.build import check_min_cppstd
 from conan.tools.cmake import CMake, CMakeToolchain, cmake_layout
 from conan.tools.files import apply_conandata_patches, copy, export_conandata_patches, get
@@ -29,6 +30,7 @@ class JoltPhysicsConan(ConanFile):
         "simd": ["sse", "sse41", "sse42", "avx", "avx2", "avx512"],
         "debug_renderer": [True, False],
         "profile": [True, False],
+        "lto": [True, False],
     }
     default_options = {
         "shared": False,
@@ -36,6 +38,7 @@ class JoltPhysicsConan(ConanFile):
         "simd": "sse42",
         "debug_renderer": False,
         "profile": False,
+        "lto": True,
     }
 
     @property
@@ -80,6 +83,8 @@ class JoltPhysicsConan(ConanFile):
             del self.options.fPIC
         if self.settings.arch not in ("x86", "x86_64"):
             del self.options.simd
+        if is_apple_os(self):
+            self.options.lto = False
 
     def configure(self):
         if self.options.shared:
@@ -131,6 +136,7 @@ class JoltPhysicsConan(ConanFile):
             tc.variables["USE_STATIC_MSVC_RUNTIME_LIBRARY"] = is_msvc_static_runtime(self)
         tc.variables["JPH_DEBUG_RENDERER"] = self.options.debug_renderer
         tc.variables["JPH_PROFILE_ENABLED"] = self.options.profile
+        tc.variables["INTERPROCEDURAL_OPTIMIZATION"] = self.options.lto
         if Version(self.version) >= "3.0.0":
             tc.variables["ENABLE_ALL_WARNINGS"] = False
         tc.generate()

@SpaceIm
Copy link
Contributor

SpaceIm commented Sep 14, 2024

@shiena Please disable lto if it's manually managed in upstream CMakeList, and don't add such option in conancenter recipe. Lto can be enabled by users through conan config. It's always disabled by default for consistency across libraries provided by conancenter.

@shiena
Copy link
Contributor

shiena commented Sep 15, 2024

@SpaceIm
Perhaps because of the upstream definition, if you build for mac with build_type=Release , lto is enabled.
Since the purpose is to disable it, it is sufficient to add tc.variables[“INTERPROCEDURAL_OPTIMIZATION”] = False.

@uilianries
Copy link
Member

uilianries commented Sep 15, 2024

@shiena Hello and thank you for commenting. Have you discussed with the upstream this situation involving lto, instead? It look pretty particular, and it's configured already to be always ON: https://github.com/jrouwe/JoltPhysics/blob/f95ad217f2c2797081cbc33b6d3463dd0ef65f84/Build/CMakeLists.txt#L29 Maybe the upstream could change the behavior when using OSX to avoid the current error.

Otherwise, I would just enforce INTERPROCEDURAL_OPTIMIZATION always to False

@SpaceIm
Copy link
Contributor

SpaceIm commented Sep 15, 2024

@cuppajoeman It looks like that patch was first made over here. #14573 (@SpaceIm )

I've made this patch for joltphysics 2.0.1 (first version of joltphysics packaged in conancenter), I don't know if the logic is still relevant for other versions.

For 2.0.1, patch was fixing this:

  • C++17 was hardcoded (raw injection of -std=c++17 flag), so it was removed and replaced by target_compile_defintions cxx_std_17, which allows to honor compiler.cppstd from conan profile if above C++17.
  • hardcoded LTO was removed (conancenter recipes provide non-LTO build by default, otherwise it can lead to binary compatibility issues across minor versions of some compilers).
  • Werror/WX removed (warnings as errors can lead to compilation errors for recent compilers)
  • Upstream CMakeLists was only creating static lib, not shared, so STATIC was removed from add_library call, allowing to honor shared option from recipe.
  • There was no install target in upstream CMakeLists, so it has been added
  • In Upstream CMakeLists, JPH_DEBUG_RENDERER and JPH_PROFILE_ENABLED were injected based on build type. Patch replaced this behavior by an option, so that users can decide to enable these features based on recipe option.

@uilianries
Copy link
Member

Just read the issue jrouwe/JoltPhysics#418. I would pass tc.variables[“INTERPROCEDURAL_OPTIMIZATION”] = False by default. In case someone want that 10% of extra performance, is still possible to override it via the configuration tools.cmake.cmaketoolchain:extra_variables.

@shiena
Copy link
Contributor

shiena commented Sep 16, 2024

@uilianries Thank you for all your advice. I will withdraw my request and investigate the upstream topic.

@uilianries uilianries self-assigned this Nov 21, 2024
@uilianries
Copy link
Member

@cuppajoeman Hello! We are going to review this PR on this Friday (22/11) and check possible solution involving the build. Thank you for your PR again!

@uilianries uilianries removed the Failed label Nov 22, 2024
Signed-off-by: Uilian Ries <[email protected]>
@uilianries
Copy link
Member

First, @cuppajoeman Thank for bringing us this new major version of joltphysics in this PR 💚

@TariqYoussef @noizex @shiena Thank for your comments in this PR, it's clear the project has several options and can result in different configurations, as those options are dependent of each other.

After discussing about this PR with Conan team, we idealized of reducing this PR, following the minimal of options possible, as the CI can not test all those combinations and may result in future issues. The current configured options for 5.x are to disable examples, tests, profiler and the renderer.

In case you really need a specific option, we could discuss in a future issue after this PR, so we would avoid new delays.


Still, Conan 2.x offers a safe way to inject cmake definitions without changing any line of the recipe, the configuration tools.cmake.cmaketoolchain:extra_variables, so you can enabled/disable features, for instance:

-c tools.cmake.cmaketoolchain:extra_variables='{"PROFILER_IN_DEBUG_AND_RELEASE": "ON"}'

This override the value for PROFILER_IN_DEBUG_AND_RELEASE. It can be extended to any variable present in the upstream's CMakeLists.txt.

This configuration does not affect the package ID, so it will override your local cache with a new package that does not follow the recipe in Conan Center. In order affect the package ID, by any configuration used, you can use configuration tools.info.package_id:confs:

-c tools.info.package_id:confs='["tools.cmake.cmaketoolchain:extra_variables"]'

Reference: https://docs.conan.io/2/reference/config_files/global_conf.html

These configuration can be added to your profile as well, so you do not need to copy/paste for every command line. That's a fast way to unblock you in case needing a new option not available in the recipe yet.

@uilianries
Copy link
Member

The CI was not able to process the current state of this PR. It's an internal bug and is listed to be fixed. Sorry for the inconvenience.

@cuppajoeman
Copy link
Contributor Author

The CI was not able to process the current state of this PR. It's an internal bug and is listed to be fixed. Sorry for the inconvenience.

Is there any idea on the progress of the internal bug or an expected time the bug blocking this will get fixed? Is there an issue we can look at to see the progress of it?

@cuppajoeman
Copy link
Contributor Author

@uilianries Hi, I'm just asking again about any more information or a timeline we can expect the internal bug which is blocking this to be fixed

@jcar87
Copy link
Contributor

jcar87 commented Jan 28, 2025

Hi @cuppajoeman - the bug has been fixed for a while, apologies the job for this has not been relaunched. Please see the Checks tab for CI build results (as there are some failures)

@uilianries
Copy link
Member

The current error seems to be related to Apple Clang 13 because locally I can build using Apple Clang 16.

As I didn't find a related issue on the upstream, so I opened an issue asking for help: jrouwe/JoltPhysics#1482

Once we have the minimal Apple Clang version required, we can add it validate(), so the CI will skip Apple builds in case not fitting the required compiler version, but will build for Linux and Windows.

@uilianries
Copy link
Member

The upstream kindly provided a workaround for Apple Clang 13, and the same is merged in the master branch already. I pushed the very same patch to this new recipe version, on the commit 4c278ee

@noizex
Copy link

noizex commented Jan 29, 2025

Thanks for all the effort to get this package closer to merging 🙇

uilianries
uilianries previously approved these changes Jan 30, 2025
Copy link
Member

@uilianries uilianries left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jcar87 jcar87 changed the title joltphysics: add version 5.0.0 joltphysics: add version 5.2.0 Jan 30, 2025
@jcar87 jcar87 merged commit 1bfbf97 into conan-io:master Jan 30, 2025
9 checks passed
@cuppajoeman
Copy link
Contributor Author

@uilianries I don't believe that the supplied test_package.cpp is sufficient, as when we expand the package test to the full HelloWorld.cpp then the package fails to work properly on windows.

@uilianries
Copy link
Member

@cuppajoeman Thank you for reporting the case in a new issue! It should be investigated during next week!

@noizex
Copy link

noizex commented Feb 13, 2025

Yeah I also ran into this issue. The suggested fix works for now!

What I wonder is how we can change the different options for this recipe like:

        tc.cache_variables["TARGET_UNIT_TESTS"] = False
        tc.cache_variables["TARGET_HELLO_WORLD"] = False
        tc.cache_variables["TARGET_PERFORMANCE_TEST"] = False
        tc.cache_variables["TARGET_SAMPLES"] = False
        tc.cache_variables["TARGET_VIEWER"] = False
        tc.cache_variables["CROSS_PLATFORM_DETERMINISTIC"] = False
        tc.cache_variables["INTERPROCEDURAL_OPTIMIZATION"] = False
        tc.cache_variables["GENERATE_DEBUG_SYMBOLS"] = False
        tc.cache_variables["ENABLE_ALL_WARNINGS"] = False
        tc.cache_variables["OVERRIDE_CXX_FLAGS"] = False
        tc.cache_variables["DEBUG_RENDERER_IN_DEBUG_AND_RELEASE"] = False
        tc.cache_variables["PROFILER_IN_DEBUG_AND_RELEASE"] = False

via conanfile.txt? It seems like these options are sort of hardcoded and some we may want to have changed for our builds? I noticed I had this

joltphysics/*:disable_custom_allocator=True

in my [options] section of my older Jolt recipe but not sure this ever worked.

@uilianries
Copy link
Member

Yeah I also ran into this issue. The suggested fix works for now!

What I wonder is how we can change the different options for this recipe like:

        tc.cache_variables["TARGET_UNIT_TESTS"] = False
        tc.cache_variables["TARGET_HELLO_WORLD"] = False
        tc.cache_variables["TARGET_PERFORMANCE_TEST"] = False
        tc.cache_variables["TARGET_SAMPLES"] = False
        tc.cache_variables["TARGET_VIEWER"] = False
        tc.cache_variables["CROSS_PLATFORM_DETERMINISTIC"] = False
        tc.cache_variables["INTERPROCEDURAL_OPTIMIZATION"] = False
        tc.cache_variables["GENERATE_DEBUG_SYMBOLS"] = False
        tc.cache_variables["ENABLE_ALL_WARNINGS"] = False
        tc.cache_variables["OVERRIDE_CXX_FLAGS"] = False
        tc.cache_variables["DEBUG_RENDERER_IN_DEBUG_AND_RELEASE"] = False
        tc.cache_variables["PROFILER_IN_DEBUG_AND_RELEASE"] = False

via conanfile.txt? It seems like these options are sort of hardcoded and some we may want to have changed for our builds? I noticed I had this

joltphysics/*:disable_custom_allocator=True

in my [options] section of my older Jolt recipe but not sure this ever worked.

Hello, @noizex! For those exposed options, you can use the Conan 2.x config feature to inject variables directly to CMake when running Conan. For instance:

-c 'tools.cmake.cmaketoolchain:extra_variables={"DISABLE_CUSTOM_ALLOCATOR": "OFF"}'

You can add this configuration to your Conan command line, as represented here, or you can add it to your Conan profile. It will not work in case it is being added to your conanfile.txt.

Still, this configuration alone does not affect your package ID, so you will need to add it in another configuration, so Conan will consider any change in extra_variables as part of the package ID:

-c 'tools.info.package_id:confs=["tools.cmake.cmaketoolchain:extra_variables"]'

The same can be added to your profile as well. Regards.

@cuppajoeman
Copy link
Contributor Author

Yeah I also ran into this issue. The suggested fix works for now!
What I wonder is how we can change the different options for this recipe like:

        tc.cache_variables["TARGET_UNIT_TESTS"] = False
        tc.cache_variables["TARGET_HELLO_WORLD"] = False
        tc.cache_variables["TARGET_PERFORMANCE_TEST"] = False
        tc.cache_variables["TARGET_SAMPLES"] = False
        tc.cache_variables["TARGET_VIEWER"] = False
        tc.cache_variables["CROSS_PLATFORM_DETERMINISTIC"] = False
        tc.cache_variables["INTERPROCEDURAL_OPTIMIZATION"] = False
        tc.cache_variables["GENERATE_DEBUG_SYMBOLS"] = False
        tc.cache_variables["ENABLE_ALL_WARNINGS"] = False
        tc.cache_variables["OVERRIDE_CXX_FLAGS"] = False
        tc.cache_variables["DEBUG_RENDERER_IN_DEBUG_AND_RELEASE"] = False
        tc.cache_variables["PROFILER_IN_DEBUG_AND_RELEASE"] = False

via conanfile.txt? It seems like these options are sort of hardcoded and some we may want to have changed for our builds? I noticed I had this

joltphysics/*:disable_custom_allocator=True

in my [options] section of my older Jolt recipe but not sure this ever worked.

Hello, @noizex! For those exposed options, you can use the Conan 2.x config feature to inject variables directly to CMake when running Conan. For instance:

-c 'tools.cmake.cmaketoolchain:extra_variables={"DISABLE_CUSTOM_ALLOCATOR": "OFF"}'

You can add this configuration to your Conan command line, as represented here, or you can add it to your Conan profile. It will not work in case it is being added to your conanfile.txt.

Still, this configuration alone does not affect your package ID, so you will need to add it in another configuration, so Conan will consider any change in extra_variables as part of the package ID:

-c 'tools.info.package_id:confs=["tools.cmake.cmaketoolchain:extra_variables"]'

The same can be added to your profile as well. Regards.

This might already be obvious but something I didn't realize for a while was that if you specify the extra config options, it doesn't by default cause the library to rebuild, you have to pass the --build=jolt... option as well to rebuild it with the new option.

I think when you allow for extra variables go be taken into account for the package id that it will automatically trigger the rebuild, and I think once I had to delete my build folder to get it to work. Things to keep in mind for anyone else who visits this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[request] joltphysics/5.1.0