You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

88 lines
3.3 KiB

Nix shell updates for `develop` (#18131) * shell.nix: Update `tomlkit` to 0.11.4 using a Nixpkgs overlay The used Nixpkgs snapshot contains `tomlkit` version 0.7.0, which is affected by https://www.github.com/sdispater/tomlkit/issues/148; that bug is triggered by `pyproject.toml` from `jsonschema` >= 4.11.0, preventing the build of that module. Just adding `tomlkit = "*"` to the `[tool.poetry.dev-dependencies]` section of `nix/pyproject.toml` does not fix the `jsonschema` build, because `makeRemoveSpecialDependenciesHook` inside `poetry2nix` is not affected by `nix/pyproject.toml`. Add a Nixpkgs overlay which updates the `tomlkit` Python module globally, so that `poetry2nix` would also use the updated version internally. * shell.nix: Bump `poetry2nix` to the most recent version The new `poetry2nix` version includes overrides which are required for recent versions of some Python packages (in particular, `jsonschema` and `dotty-dict`). * shell.nix: Bump QMK CLI to 1.1.1; update other Python deps Update `pyproject.toml` to match `requirements*.txt`: - add `pyserial = "*"` - replace `qmk-dotty-dict = "*"` with `dotty-dict = "*"` (#18117, also required for compatibility with `qmk` 1.1.1, where this replacement had already been performed) Add build dependencies of various Python modules to `pyproject.toml`: - `hatchling`, `hatch-vcs`, `hatch-fancy-pypi-readme` (required by `jsonschema` >= 4.11.0) - `pytest` (a newer version is required to solve the dependency conflict with the `hatchling` module due to the upper bound on `pluggy`) - `flit-core` (a more recent version is required to build `tomli`) - `poetry-core` (required for `dotty-dict` >= 1.3.1, and the version from Nixpkgs does not build on Darwin due to NixOS/nix#4758) Update `poetry.lock` to use the most recent versions of Python modules. The complete list of Python module updates as listed in `poetry.lock` (note that other modules might be present in the Python environment, e.g., if they come from Nixpkgs): - `atomicwrites`: none -> 1.4.1 (but this module is not actually used, because the corresponding dependency of `pytest` is win32-only) - `attrs`: 21.4.0 -> 22.1.0 - `colorama`: 0.4.4 -> 0.4.5 - `coverage`: 6.4 -> none - `dotty-dict`: none -> 1.3.1 (used instead of `qmk-dotty-dict`) - `editables`: none -> 0.3 - `flake8`: 4.0.1 -> 5.0.4 - `flake8-polyfill`: 1.0.2 -> none - `flit-core`: none -> 3.7.1 - `hatch-fancy-pypi-readme`: none -> 22.3.0 - `hatch-vcs`: none -> 0.2.0 - `hatchling`: none -> 1.8.0 - `hjson`: 3.0.2 -> 3.1.0 - `importlib-resources`: 5.7.1 -> 5.9.0 - `iniconfig`: none -> 1.1.1 - `jsonschema`: 4.5.1 -> 4.14.0 - `mccabe`: 0.6.1 -> 0.7.0 - `nose2`: 0.11.0 -> 0.12.0 - `packaging`: none -> 21.3 - `pathspec`: none -> 0.9.0 - `pep8-naming`: 0.12.1 -> 0.13.2 - `pillow`: 9.1.1 -> 9.2.0 - `pkgutil-resolve-name`: none -> 1.3.10 - `pluggy`: none -> 1.0.0 - `poetry-core`: none -> 1.0.8 - `py`: none -> 1.11.0 - `pycodestyle`: 2.8.0 -> 2.9.1 - `pyflakes`: 2.4.0 -> 2.5.0 - `pygments`: 2.12.0 -> 2.13.0 - `pyparsing`: none -> 3.0.9 - `pyserial`: none -> 3.5 - `pytest`: none -> 7.1.2 - `qmk`: 1.1.0 -> 1.1.1 - `qmk-dotty-dict`: 1.3.0.post1 -> none (replaced by `dotty-dict`) - `setuptools-scm`: none -> 7.0.5 - `tomli`: none -> 2.0.1 - `typing-extensions`: none -> 4.3.0 - `zipp`: 3.8.0 -> 3.8.1
1 year ago
Nix shell updates for `develop` (#18131) * shell.nix: Update `tomlkit` to 0.11.4 using a Nixpkgs overlay The used Nixpkgs snapshot contains `tomlkit` version 0.7.0, which is affected by https://www.github.com/sdispater/tomlkit/issues/148; that bug is triggered by `pyproject.toml` from `jsonschema` >= 4.11.0, preventing the build of that module. Just adding `tomlkit = "*"` to the `[tool.poetry.dev-dependencies]` section of `nix/pyproject.toml` does not fix the `jsonschema` build, because `makeRemoveSpecialDependenciesHook` inside `poetry2nix` is not affected by `nix/pyproject.toml`. Add a Nixpkgs overlay which updates the `tomlkit` Python module globally, so that `poetry2nix` would also use the updated version internally. * shell.nix: Bump `poetry2nix` to the most recent version The new `poetry2nix` version includes overrides which are required for recent versions of some Python packages (in particular, `jsonschema` and `dotty-dict`). * shell.nix: Bump QMK CLI to 1.1.1; update other Python deps Update `pyproject.toml` to match `requirements*.txt`: - add `pyserial = "*"` - replace `qmk-dotty-dict = "*"` with `dotty-dict = "*"` (#18117, also required for compatibility with `qmk` 1.1.1, where this replacement had already been performed) Add build dependencies of various Python modules to `pyproject.toml`: - `hatchling`, `hatch-vcs`, `hatch-fancy-pypi-readme` (required by `jsonschema` >= 4.11.0) - `pytest` (a newer version is required to solve the dependency conflict with the `hatchling` module due to the upper bound on `pluggy`) - `flit-core` (a more recent version is required to build `tomli`) - `poetry-core` (required for `dotty-dict` >= 1.3.1, and the version from Nixpkgs does not build on Darwin due to NixOS/nix#4758) Update `poetry.lock` to use the most recent versions of Python modules. The complete list of Python module updates as listed in `poetry.lock` (note that other modules might be present in the Python environment, e.g., if they come from Nixpkgs): - `atomicwrites`: none -> 1.4.1 (but this module is not actually used, because the corresponding dependency of `pytest` is win32-only) - `attrs`: 21.4.0 -> 22.1.0 - `colorama`: 0.4.4 -> 0.4.5 - `coverage`: 6.4 -> none - `dotty-dict`: none -> 1.3.1 (used instead of `qmk-dotty-dict`) - `editables`: none -> 0.3 - `flake8`: 4.0.1 -> 5.0.4 - `flake8-polyfill`: 1.0.2 -> none - `flit-core`: none -> 3.7.1 - `hatch-fancy-pypi-readme`: none -> 22.3.0 - `hatch-vcs`: none -> 0.2.0 - `hatchling`: none -> 1.8.0 - `hjson`: 3.0.2 -> 3.1.0 - `importlib-resources`: 5.7.1 -> 5.9.0 - `iniconfig`: none -> 1.1.1 - `jsonschema`: 4.5.1 -> 4.14.0 - `mccabe`: 0.6.1 -> 0.7.0 - `nose2`: 0.11.0 -> 0.12.0 - `packaging`: none -> 21.3 - `pathspec`: none -> 0.9.0 - `pep8-naming`: 0.12.1 -> 0.13.2 - `pillow`: 9.1.1 -> 9.2.0 - `pkgutil-resolve-name`: none -> 1.3.10 - `pluggy`: none -> 1.0.0 - `poetry-core`: none -> 1.0.8 - `py`: none -> 1.11.0 - `pycodestyle`: 2.8.0 -> 2.9.1 - `pyflakes`: 2.4.0 -> 2.5.0 - `pygments`: 2.12.0 -> 2.13.0 - `pyparsing`: none -> 3.0.9 - `pyserial`: none -> 3.5 - `pytest`: none -> 7.1.2 - `qmk`: 1.1.0 -> 1.1.1 - `qmk-dotty-dict`: 1.3.0.post1 -> none (replaced by `dotty-dict`) - `setuptools-scm`: none -> 7.0.5 - `tomli`: none -> 2.0.1 - `typing-extensions`: none -> 4.3.0 - `zipp`: 3.8.0 -> 3.8.1
1 year ago
Update the nix-shell environment (#13316) * Nix: Allow calls to `bin/qmk` even when the build was started by `qmk` The `$PATH` modifications performed by the Nix wrapper for the `qmk` executable prevent `bin/qmk` from working properly (the changed `$PATH` contains a wrong `python3` executable which does not have the needed Python modules in its module path). As a workaround, disable the generation of that wrapper for the `qmk` Python package (there is yet another wrapper generated while building the Python environment, which would still set the Python module path properly when running `qmk`). Although `bin/qmk` is officially deprecated, QMK CLI still invokes it in some cases (at least `qmk doctor` and `qmk pytest`), therefore keeping these invocations working is useful. * Nix: Update `util/nix/pyproject.toml` to match `requirements*.txt` Update the Python dependency information used by Poetry to match the current state of the qmk_firmware code. * Nix: Bump QMK CLI dependency to 1.0.0; bump other Python deps Update Python dependencies for nix-shell to the most recent releases: - dotty-dict: 1.3.0 -> no longer used - milc: 1.4.2 -> 1.6.2 - pep8-naming: 0.11.1 -> 0.12.1 - pygments: 2.9.0 -> 2.10.0 - pyrsistent: 0.17.3 -> 0.18.0 - pyusb: 1.1.1 -> 1.2.1 - setuptools-scm: 6.0.1 -> no longer used - qmk: 0.1.0 -> 1.0.0 - qmk-dotty-dict: not used -> 1.3.0.post1 - yapf: 0.30.0 -> 0.31.0 Note to self: The command to update Python dependencies changed to: ( cd util/nix && nix run 'nixpkgs#poetry' -- update --lock )
2 years ago
Update the nix-shell environment (#13316) * Nix: Allow calls to `bin/qmk` even when the build was started by `qmk` The `$PATH` modifications performed by the Nix wrapper for the `qmk` executable prevent `bin/qmk` from working properly (the changed `$PATH` contains a wrong `python3` executable which does not have the needed Python modules in its module path). As a workaround, disable the generation of that wrapper for the `qmk` Python package (there is yet another wrapper generated while building the Python environment, which would still set the Python module path properly when running `qmk`). Although `bin/qmk` is officially deprecated, QMK CLI still invokes it in some cases (at least `qmk doctor` and `qmk pytest`), therefore keeping these invocations working is useful. * Nix: Update `util/nix/pyproject.toml` to match `requirements*.txt` Update the Python dependency information used by Poetry to match the current state of the qmk_firmware code. * Nix: Bump QMK CLI dependency to 1.0.0; bump other Python deps Update Python dependencies for nix-shell to the most recent releases: - dotty-dict: 1.3.0 -> no longer used - milc: 1.4.2 -> 1.6.2 - pep8-naming: 0.11.1 -> 0.12.1 - pygments: 2.9.0 -> 2.10.0 - pyrsistent: 0.17.3 -> 0.18.0 - pyusb: 1.1.1 -> 1.2.1 - setuptools-scm: 6.0.1 -> no longer used - qmk: 0.1.0 -> 1.0.0 - qmk-dotty-dict: not used -> 1.3.0.post1 - yapf: 0.30.0 -> 0.31.0 Note to self: The command to update Python dependencies changed to: ( cd util/nix && nix run 'nixpkgs#poetry' -- update --lock )
2 years ago
Update the nix-shell environment (#13316) * Nix: Allow calls to `bin/qmk` even when the build was started by `qmk` The `$PATH` modifications performed by the Nix wrapper for the `qmk` executable prevent `bin/qmk` from working properly (the changed `$PATH` contains a wrong `python3` executable which does not have the needed Python modules in its module path). As a workaround, disable the generation of that wrapper for the `qmk` Python package (there is yet another wrapper generated while building the Python environment, which would still set the Python module path properly when running `qmk`). Although `bin/qmk` is officially deprecated, QMK CLI still invokes it in some cases (at least `qmk doctor` and `qmk pytest`), therefore keeping these invocations working is useful. * Nix: Update `util/nix/pyproject.toml` to match `requirements*.txt` Update the Python dependency information used by Poetry to match the current state of the qmk_firmware code. * Nix: Bump QMK CLI dependency to 1.0.0; bump other Python deps Update Python dependencies for nix-shell to the most recent releases: - dotty-dict: 1.3.0 -> no longer used - milc: 1.4.2 -> 1.6.2 - pep8-naming: 0.11.1 -> 0.12.1 - pygments: 2.9.0 -> 2.10.0 - pyrsistent: 0.17.3 -> 0.18.0 - pyusb: 1.1.1 -> 1.2.1 - setuptools-scm: 6.0.1 -> no longer used - qmk: 0.1.0 -> 1.0.0 - qmk-dotty-dict: not used -> 1.3.0.post1 - yapf: 0.30.0 -> 0.31.0 Note to self: The command to update Python dependencies changed to: ( cd util/nix && nix run 'nixpkgs#poetry' -- update --lock )
2 years ago
3 years ago
  1. let
  2. # We specify sources via Niv: use "niv update nixpkgs" to update nixpkgs, for example.
  3. sources = import ./util/nix/sources.nix { };
  4. # `tomlkit` >= 0.8.0 is required to build `jsonschema` >= 4.11.0 (older
  5. # version do not support some valid TOML syntax: sdispater/tomlkit#148). The
  6. # updated `tomlkit` must be used by `makeRemoveSpecialDependenciesHook`
  7. # inside `poetry2nix`, therefore just providing the updated version through
  8. # our `nix/pyproject.toml` does not work, and using an overlay is required.
  9. pythonOverlay = final: prev: {
  10. python3 = prev.python3.override {
  11. packageOverrides = self: super: {
  12. tomlkit = super.tomlkit.overridePythonAttrs(old: rec {
  13. version = "0.11.4";
  14. src = super.fetchPypi {
  15. inherit (old) pname;
  16. inherit version;
  17. sha256 = "sha256-MjWpAQ+uVDI+cnw6wG+3IHUv5mNbNCbjedrsYPvUSoM=";
  18. };
  19. });
  20. };
  21. };
  22. };
  23. in
  24. # However, if you want to override Niv's inputs, this will let you do that.
  25. { pkgs ? import sources.nixpkgs { overlays = [ pythonOverlay ]; }
  26. , poetry2nix ? pkgs.callPackage (import sources.poetry2nix) { }
  27. , avr ? true
  28. , arm ? true
  29. , teensy ? true }:
  30. with pkgs;
  31. let
  32. avrlibc = pkgsCross.avr.libcCross;
  33. avr_incflags = [
  34. "-isystem ${avrlibc}/avr/include"
  35. "-B${avrlibc}/avr/lib/avr5"
  36. "-L${avrlibc}/avr/lib/avr5"
  37. "-B${avrlibc}/avr/lib/avr35"
  38. "-L${avrlibc}/avr/lib/avr35"
  39. "-B${avrlibc}/avr/lib/avr51"
  40. "-L${avrlibc}/avr/lib/avr51"
  41. ];
  42. # Builds the python env based on nix/pyproject.toml and
  43. # nix/poetry.lock Use the "poetry update --lock", "poetry add
  44. # --lock" etc. in the nix folder to adjust the contents of those
  45. # files if the requirements*.txt files change
  46. pythonEnv = poetry2nix.mkPoetryEnv {
  47. projectDir = ./util/nix;
  48. overrides = poetry2nix.overrides.withDefaults (self: super: {
  49. pillow = super.pillow.overridePythonAttrs(old: {
  50. # Use preConfigure from nixpkgs to fix library detection issues and
  51. # impurities which can break the build process; this also requires
  52. # adding propagatedBuildInputs and buildInputs from the same source.
  53. propagatedBuildInputs = (old.buildInputs or []) ++ pkgs.python3.pkgs.pillow.propagatedBuildInputs;
  54. buildInputs = (old.buildInputs or []) ++ pkgs.python3.pkgs.pillow.buildInputs;
  55. preConfigure = (old.preConfigure or "") + pkgs.python3.pkgs.pillow.preConfigure;
  56. });
  57. qmk = super.qmk.overridePythonAttrs(old: {
  58. # Allow QMK CLI to run "qmk" as a subprocess (the wrapper changes
  59. # $PATH and breaks these invocations).
  60. dontWrapPythonPrograms = true;
  61. });
  62. });
  63. };
  64. in
  65. mkShell {
  66. name = "qmk-firmware";
  67. buildInputs = [ clang-tools dfu-programmer dfu-util diffutils git pythonEnv niv ]
  68. ++ lib.optional avr [
  69. pkgsCross.avr.buildPackages.binutils
  70. pkgsCross.avr.buildPackages.gcc8
  71. avrlibc
  72. avrdude
  73. ]
  74. ++ lib.optional arm [ gcc-arm-embedded ]
  75. ++ lib.optional teensy [ teensy-loader-cli ];
  76. AVR_CFLAGS = lib.optional avr avr_incflags;
  77. AVR_ASFLAGS = lib.optional avr avr_incflags;
  78. shellHook = ''
  79. # Prevent the avr-gcc wrapper from picking up host GCC flags
  80. # like -iframework, which is problematic on Darwin
  81. unset NIX_CFLAGS_COMPILE_FOR_TARGET
  82. '';
  83. }