Frequent issues such broken builds or inability to make a release have led us to freeze as many of our build dependencies as possible. Note that this does not imply that runtime dependencies of the final packages are frozen. There are a number of non-obvious dependencies that have led to surpising issues, so we attempt to document this below.
Places that define dependencies#
GitHub workflows define
runs-on, i.e., VM image versions. We are not using the
*-latestimages but instead rely on specific ones such as
winddows-2019. Unfortunately GitHub appears to make changes even to the non-latest images, e.g., we once experienced build breaks from a compiler update in the
windows-2019images. In other words, the VM image “versions” are not truly frozen.
Actions for GitHub workflows are frozen to specific versions. Dependabot is configured to update these regularely. Note one known issue:
pypa/cibuildwheelupdates may add support for new Python versions, but our
pybind11may not be supporting it. Make sure to run the
release.ymlworkflow before accepting an update, or consider adding future Python versions to the
skiplist to make support for new Python versions a process fully under our control.
pyproject.toml defines build requirements for Python Wheels.
setup.py defines runtime requirements and “extras” for the
All Python dependencies for static analysis, tests, documentation, and more are defined in
pip-compile-multiis used to generate the corresponding
requirements/*.txtfiles. This should be run regularly, e.g., before a release. Create a PR with the results and verify the changes. This must include manual verification that the documentation is ok (download the artifact and inspect locally), since we have several Sphinx-related dependencies that have previously led to silently “broken” documentation pages.
C++ library dependencies are installed using
conan. Versions are pinned in lib/cmake/scipp-conan.cmake in the
conan_cmake_configurecall. Known problem: We have not frozen the recipe versions. New recipe versions can be incompatible with older conan versions, such as the one we have frozen. We are therefore considering freezing these as well, see #2770.
C++ conda build dependencies used in CI (non-release) builds are defined in .buildconfig/ in files such as
ci-linux.yml. These need to be updated manually.
conda/meta.yaml defines conda package build and runtime dependencies. Files in conda/variants/ complement this and set specific versions. These are currently not frozen explicitly. The goal is to move to the global conda-forge pin once they move to
numpy-1.20. See #2571.
Environment files in docs/environments/ list optional requirements. These enviroments are not used for builds but for users.