Skip to content

Switch build system to pixi and rattler-build#398

Open
lohedges wants to merge 8 commits intodevelfrom
feature_rattler
Open

Switch build system to pixi and rattler-build#398
lohedges wants to merge 8 commits intodevelfrom
feature_rattler

Conversation

@lohedges
Copy link
Contributor

@lohedges lohedges commented Feb 5, 2026

This PR implements a major overhaul of the Sire build system by modernising the software stack to make use of pixi and rattler-build. This resolves multiple issues with our CI, i.e. we once again have a working Windows build and environment resolution no longer hangs for Python 3.10 builds on Linux and macOS.

Our build is made more complex as result of Sire being the foundational package for other packges within the OpenBioSim ecosystem. This leads to a complex dependency resolution when the Sire build is treated as the "host" environment for other tools and features. Previously, this was handled via a set of requirements files for specific features, which were resolved and injected into the conda recipe at build time. This approach doesn't work for rattler, so we instead use the pixi.toml as the "source-of-truth", where features and their required dependencies are stated explicitly. This can be used with pixi to install an development environment for source install, or for a rattler build by first running python actions/generate_recipe.py to generate the recipe from the information in the pixi.toml file.

Following refactoring, I have discovered that the SireGemmi interface is currently broken. Going back through the CI logs it is apparent that this hasn't been tested for some time, since gemmi wasn't correctly detected in the test environment, so the appropriate tests were skipped. The issue is due to an issue with the Boost::Python / pybind11 interface, which I have been unable to resolve locally. For now, I have commented out the SireGemmi build in the appropriate CMake file. (gemmi has always been extremely brittle and both Boost:Python and pybind11 are unstable in the parts of the API required for the interface.)

A local build on my workstation discovered that OpenMM requires a more recent version of sysroot_linux-64 in the host section of its recipe (2.34 rather than 2.17). This means that we need to match this when doing a source build, since we are effectively creating a "host" environment. This is handled via the pixi.toml. (The standard rattler build is unaffected.)

Note that there are a few wrinkles with using rattler-build, which will be discussed further in my PRs for our other packages. The main issue is that it only does a shallow clone, so standard Python versioning tools, such as versioningit are unable to access the information needed at build time. This can be resolved by installing from a path rather than a git URL, but this also leads to other issues. (Everything is can be fixed, but it's painful that this is needed for standard pure-Python packages.) Using rattler also exposes some packaging issues with some of our BioSimSpace dependencies. This is because it reads dependency constraints from the pip metadata as well as the conda recipe, so if a package specifies a constraint in the pip one but not the conda one, then the dependency resolution will fail. This is true for nglview, where it requires numpy<2.3, but this isn't added to the conda recipe. (They are currently building a new version with support for 2.4+, so this will be resolved soon.)

Closes #268
Closes #339
Closes #392
Closes #396

  • I confirm that I have merged the latest version of devel into this branch before issuing this pull request (e.g. by running git pull origin devel): [y]
  • I confirm that I have added a test for any new functionality in this pull request: [y]
  • I confirm that I have added documentation (e.g. a new tutorial page or detailed guide) for any new functionality in this pull request: [y]
  • I confirm that I have added a changelog entry to the changelog (we will add a link to this PR as part of the review): [y]
  • I confirm that I have permission to release this code under the GPL3 license: [y]

@lohedges
Copy link
Contributor Author

lohedges commented Feb 5, 2026

I decided to add a feature for all downstream OpenBioSim dependencies to avoid issues with dependency resolution in future. I've not tested this locally, so might need to make it less restrictive tomorrow if there are issues. (The standard BSS deps approach works.)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working enhancement New feature or request

Projects

None yet

1 participant