Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow packages to point outside their directory #64

Open
philiptaron opened this issue May 14, 2024 · 5 comments
Open

Allow packages to point outside their directory #64

philiptaron opened this issue May 14, 2024 · 5 comments
Labels
enhancement New feature or request

Comments

@philiptaron
Copy link
Contributor

philiptaron commented May 14, 2024

For derivations that wrap things in nixpkgs itself -- for instance, the NixOS manual and the Nixpkgs manual -- the rule that paths are found only in the by-name directory itself is too restrictive. It's a good default, but there should be an opt-out mechanism.

I ran into this implementing NixOS/nixpkgs#311459.

@philiptaron philiptaron added the enhancement New feature or request label May 14, 2024
@infinisil
Copy link
Member

infinisil commented Jun 4, 2024

A big motivation for this is avoiding action at a distance. Having this restriction makes it very clear that nothing outside the package directory can influence its result.

And specifically in the case of the manuals, it hints at a deeper problem: The fact that the top-level Nixpkgs value (import <nixpkgs> {}) contains the packages directly, which gives no fitting space for non-package attributes. Nixpkgs doesn't follow the rule that the top-level value should be a record, and that's a problem.

The only solution for that is to create a separate top-level value that doesn't have this issue. Perhaps something like:

import <nixpkgs> {
  apiVersion = 1;
}

@teto
Copy link
Member

teto commented Aug 24, 2024

I also get the error with NixOS/nixpkgs#337033, see output at:
https://github.com/NixOS/nixpkgs/actions/runs/10539567151/job/29203277988?pr=337033

I dont know if action at distance is a valid concern here, the dependency is explicit.
cp ${../../../../maintainers/scripts/pluginupdate.py} $out/lib/pluginupdate.py.

What's your advice ?

Note: I wish the nixpkgs CI would give more pointers to help understand the message like sharing the link to this repo. Each package directory must not refer to files outside itself using symlinks or Nix path expressions. is also a bit frustrating to read without a rationale behind it (that you provide here, and to which I can agree).

@infinisil
Copy link
Member

Oh I actually talked about this at NixCon: https://media.ccc.de/v/nixcon-2023-35713-not-all-packages-anymore-nix#t=742

Summarising and expanding:

  • The package directories should be independent, so we can move them around later (e.g. when we change the hashing schema, move to independent repositories, or move them to another directory)
  • Accessing arbitrary files is not great because it discourages us from restructuring Nixpkgs, since moving any file could break a package in an arbitrary other place
  • Instead of accessing such files, I recommend adding an extra callPackage argument to pass in the file that you need explicitly, therefore also allowing it to be overridden with .override.

For your specific case @teto, it sounds like pluginupdate.py could benefit from being packaged as pkgs.pluginupdate or perhaps pkgs.internal.pluginupdate to indicate that this is not a package meant to be installed by users. Then you can access it in your package with the standard callPackage arguments. Alternatively, do callPackage .../package.nix { pluginupdate = .../pluginupdate.py; } in all-packages.nix.

@infinisil
Copy link
Member

Opened #97 to improve the error :)

@teto
Copy link
Member

teto commented Aug 25, 2024

thanks for the quick feedback and the pointers. I've used your last trick: Alternatively, do callPackage .../package.nix { pluginupdate = .../pluginupdate.py; } in all-packages.nix..

I had tried beforehand to add a symlink towards pluginupdate.py in the same folder but this tool caught the hack so good job ^^''

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

3 participants