lib.fileset.fileFilter: Restrict second argument to paths

While this change is backwards-incompatible, I think it's okay because:
- The `fileFilter` function is not yet in a stable NixOS release, it was only merged about [a month ago](https://github.com/NixOS/nixpkgs/pull/257356).
  - All public uses of the function on GitHub only pass a path
- Any `fileFilter pred fileset` can also be expressed as `intersection fileset (fileFilter pred path)` without loss of functionality.
  - This is furthermore pointed out in the new error message when a file set is passed
This commit is contained in:
Silvan Mosberger 2023-11-14 07:42:59 +01:00 committed by Silvan Mosberger
parent b5953ae45d
commit 1c3eb9eff1
4 changed files with 52 additions and 31 deletions

View File

@ -238,6 +238,21 @@ Arguments:
And it would be unclear how the library should behave if the one file wouldn't be added to the store: And it would be unclear how the library should behave if the one file wouldn't be added to the store:
`toSource { root = ./file.nix; fileset = <empty>; }` has no reasonable result because returing an empty store path wouldn't match the file type, and there's no way to have an empty file store path, whatever that would mean. `toSource { root = ./file.nix; fileset = <empty>; }` has no reasonable result because returing an empty store path wouldn't match the file type, and there's no way to have an empty file store path, whatever that would mean.
### `fileFilter` takes a path
The `fileFilter` function takes a path, and not a file set, as its second argument.
- (-) Makes it harder to compose functions, since the file set type, the return value, can't be passed to the function itself like `fileFilter predicate fileset`
- (+) It's still possible to use `intersection` to filter on file sets: `intersection fileset (fileFilter predicate ./.)`
- (-) This does need an extra `./.` argument that's not obvious
- (+) This could always be `/.` or the project directory, `intersection` will make it lazy
- (+) In the future this will allow `fileFilter` to support a predicate property like `subpath` and/or `components` in a reproducible way.
This wouldn't be possible if it took a file set, because file sets don't have a predictable absolute path.
- (-) What about the base path?
- (+) That can change depending on which files are included, so if it's used for `fileFilter`
it would change the `subpath`/`components` value depending on which files are included.
- (+) If necessary, this restriction can be relaxed later, the opposite wouldn't be possible
## To update in the future ## To update in the future
Here's a list of places in the library that need to be updated in the future: Here's a list of places in the library that need to be updated in the future:

View File

@ -366,7 +366,7 @@ in {
type :: String, type :: String,
... ...
} -> Bool) } -> Bool)
-> FileSet -> Path
-> FileSet -> FileSet
Example: Example:
@ -397,14 +397,24 @@ in {
Other attributes may be added in the future. Other attributes may be added in the future.
*/ */
predicate: predicate:
# The file set to filter based on the predicate function # The path whose files to filter
fileset: path:
if ! isFunction predicate then if ! isFunction predicate then
throw '' throw ''
lib.fileset.fileFilter: First argument is of type ${typeOf predicate}, but it should be a function instead.'' lib.fileset.fileFilter: First argument is of type ${typeOf predicate}, but it should be a function instead.''
else if ! isPath path then
if path._type or "" == "fileset" then
throw ''
lib.fileset.fileFilter: Second argument is a file set, but it should be a path instead.
If you need to filter files in a file set, use `intersection fileset (fileFilter pred ./.)` instead.''
else else
_fileFilter predicate throw ''
(_coerce "lib.fileset.fileFilter: Second argument" fileset); lib.fileset.fileFilter: Second argument is of type ${typeOf path}, but it should be a path instead.''
else if ! pathExists path then
throw ''
lib.fileset.fileFilter: Second argument (${toString path}) is a path that does not exist.''
else
_fileFilter predicate path;
/* /*
The file set containing all files that are in both of two given file sets. The file set containing all files that are in both of two given file sets.

View File

@ -786,9 +786,9 @@ rec {
_differenceTree (path + "/${name}") lhsValue (rhs.${name} or null) _differenceTree (path + "/${name}") lhsValue (rhs.${name} or null)
) (_directoryEntries path lhs); ) (_directoryEntries path lhs);
# Filters all files in a file set based on a predicate # Filters all files in a path based on a predicate
# Type: ({ name, type, ... } -> Bool) -> FileSet -> FileSet # Type: ({ name, type, ... } -> Bool) -> Path -> FileSet
_fileFilter = predicate: fileset: _fileFilter = predicate: root:
let let
# Check the predicate for a single file # Check the predicate for a single file
# Type: String -> String -> filesetTree # Type: String -> String -> filesetTree
@ -807,19 +807,22 @@ rec {
# Check the predicate for all files in a directory # Check the predicate for all files in a directory
# Type: Path -> filesetTree # Type: Path -> filesetTree
fromDir = path: tree: fromDir = path:
mapAttrs (name: subtree: mapAttrs (name: type:
if isAttrs subtree || subtree == "directory" then if type == "directory" then
fromDir (path + "/${name}") subtree fromDir (path + "/${name}")
else if subtree == null then
null
else else
fromFile name subtree fromFile name type
) (_directoryEntries path tree); ) (readDir path);
rootType = pathType root;
in in
if fileset._internalIsEmptyWithoutBase then if rootType == "directory" then
_emptyWithoutBase _create root (fromDir root)
else else
_create fileset._internalBase # Single files are turned into a directory containing that file or nothing.
(fromDir fileset._internalBase fileset._internalTree); _create (dirOf root) {
${baseNameOf root} =
fromFile (baseNameOf root) rootType;
};
} }

View File

@ -813,14 +813,15 @@ checkFileset 'difference ./. ./b'
# The first argument needs to be a function # The first argument needs to be a function
expectFailure 'fileFilter null (abort "this is not needed")' 'lib.fileset.fileFilter: First argument is of type null, but it should be a function instead.' expectFailure 'fileFilter null (abort "this is not needed")' 'lib.fileset.fileFilter: First argument is of type null, but it should be a function instead.'
# The second argument can be a file set or an existing path # The second argument needs to be an existing path
expectFailure 'fileFilter (file: abort "this is not needed") null' 'lib.fileset.fileFilter: Second argument is of type null, but it should be a file set or a path instead.' expectFailure 'fileFilter (file: abort "this is not needed") _emptyWithoutBase' 'lib.fileset.fileFilter: Second argument is a file set, but it should be a path instead.
\s*If you need to filter files in a file set, use `intersection fileset \(fileFilter pred \./\.\)` instead.'
expectFailure 'fileFilter (file: abort "this is not needed") null' 'lib.fileset.fileFilter: Second argument is of type null, but it should be a path instead.'
expectFailure 'fileFilter (file: abort "this is not needed") ./a' 'lib.fileset.fileFilter: Second argument \('"$work"'/a\) is a path that does not exist.' expectFailure 'fileFilter (file: abort "this is not needed") ./a' 'lib.fileset.fileFilter: Second argument \('"$work"'/a\) is a path that does not exist.'
# The predicate is not called when there's no files # The predicate is not called when there's no files
tree=() tree=()
checkFileset 'fileFilter (file: abort "this is not needed") ./.' checkFileset 'fileFilter (file: abort "this is not needed") ./.'
checkFileset 'fileFilter (file: abort "this is not needed") _emptyWithoutBase'
# The predicate must be able to handle extra attributes # The predicate must be able to handle extra attributes
touch a touch a
@ -882,14 +883,6 @@ checkFileset 'union ./c/a (fileFilter (file: assert file.name != "a"; true) ./.)
# but here we need to use ./c # but here we need to use ./c
checkFileset 'union (fileFilter (file: assert file.name != "a"; true) ./.) ./c' checkFileset 'union (fileFilter (file: assert file.name != "a"; true) ./.) ./c'
# Also lazy, the filter isn't called on a filtered out path
tree=(
[a]=1
[b]=0
[c]=0
)
checkFileset 'fileFilter (file: assert file.name != "c"; file.name == "a") (difference ./. ./c)'
# Make sure single files are filtered correctly # Make sure single files are filtered correctly
tree=( tree=(
[a]=1 [a]=1