Rationale:
With the advent of RFC 140, Nixpkgs becomes to promote self-contained package
directories and strongly discourages scattering along the old and deprecated
filesystem hierarchy.
Further, `build-support/flutter/*` is not referenced by any other expressions in
Nixpkgs besides flutter itself.
Further, concentrating all Flutter-related toolset below a same directory
hierarchy (arguably?) eases maintenance and future migrations.
Seeing the following new warnings pop up on stderr when cargo was bumped
to 1.78:
```
warning: `/build/.cargo/config` is deprecated in favor of `config.toml`
note: if you need to support cargo 1.38 or earlier, you can symlink `config` to `config.toml`
```
which happens to break commitmsgfmt builds in nix (#320294).
closes#320294
Tested the following packages with and without structuredAttrs:
- rust-analyzer: cargo-build-hook, cargo-check-hook
- jujutsu: cargo-nextest-hook
- kornia-rs: maturin-build-hook
For a long time, we've had `crossLibcStdenv`, `*Cross` libc attributes,
and `*bsdCross` pre-libc package sets. This was always bad because
having "cross" things is "not declarative": the naming doesn't reflect
what packages *need* but rather how we *provide* something. This is
ugly, and creates needless friction between cross and native building.
Now, almost all of these `*Cross` attributes are gone: just these are
kept:
- Glibc's and Musl's are kept, because those packages are widely used
and I didn't want to risk changing the native builds of those at this
time.
- generic `libcCross`, `theadsCross`, and friends, because these relate
to the convolulted GCC bootstrap which still needs to be redone.
The BSD and obscure Linux or freestnanding libcs have conversely all
been made to use a new `stdenvNoLibc`, which is like the old
`crossLibcStdenv` except:
1. It usable for native and cross alike
2. It named according to what it *is* ("a standard environment without
libc but with a C compiler"), rather than some non-compositional
jargon ("the stdenv used for building libc when cross compiling",
yuck).
I should have done this change long ago, but I was stymied because of
"infinite recursions". The problem was that in too many cases we are
overriding `stdenv` to *remove* things we don't need, and this risks
cyles since those more minimal stdenvs are used to build things in the
more maximal stdenvs.
The solution is to pass `stage.nix` `stdenvNoCC`, so we can override to
*build up* rather than *tear down*. For now, the full `stdenv` is also
passed, so I don't need to change the native bootstraps, but I can see
this changing as we make things more uniform and clean those up.
(adapted from commit 51f1ecaa59)
(adapted from commit 1743662e55)
I broke the usage of buildRustCrate with a toolchain from rust-overlay
when I added support for wasm32-unknown-unknown, this change adds
additional conditionals to restore the usage.
The toolchain can now be overriden either through
```
buildRustCrate {
rust = toolchain;
cargo = toolchain
}
```
or
```
buildRustCrate.override {
rustc = toolchain;
cargo = toolchain;
}
```
They should now be consistent with each other.
- stop binding attributes we don't care about (e.g. name, doCheck)
- remove attributes we handle in nix (e.g. useAppHost)
- inherit attributes with default values (e.g. packNupkg)
Unpacking to the build root was a bad idea. stdenv uses dumpVars() to
create a file env-vars containing the entire environment. This was
being installed in the derivation output, and since it contains lots of
store paths, it was bloating the closure for every nuget package.
After this change, enableParallelBuilding defaults to true for packages
using buildDotnetModule. The argument value already defaults to true,
but we did not use it. Note that default values in Nix are not present
in the arguments attribute set, e.g. `args` does not contain `def` for
`{ def ? true }@args: …` function unless this argument is explicitly
passed.
Previously, it wasn't possible to access the list of platforms we can
build Rust programs for outside of buildRustPackage. This was a
problem for packages that have optional Rust components, like
gstreamer or Meson, as there was no way to only build the Rust parts
for supported platforms. Now it's possible to get that information
from rustc's passthru.
Always specify the postPhases attribute as a list instead of a string.
Append elements to the postPhases Bash variable using appendToVar
instead of string or Bash array concatenation.
Always specify the preDistPhases attribute as a list instead of a string.
Append elements to the preDistPhases Bash variable using appendToVar
instead of string or Bash array concatenation.
Handle element insertion before a specific element using string
substitution as before, but handle both structured and unstructured
attributes.
Always specify the preConfigurePhases attribute as a list instead of a
string.
Append elements to the preConfigurePhases Bash variable using
appendToVar instead of string or Bash array concatenation.
Always specify the prePhases attribute as a list instead of a string.
Append elements to the prePhases Bash variable using appendToVar
instead of string or Bash array concatenation.
Before an overlay in the form of:
package.overrideAttrs (old: {
passthru = {};
})
would fail evaluation like:
error: attribute 'overrideModAttrs' missing
at /nix/store/afwc3m1sm49qq57xjv2hmd7iy4x0j33h-source/pkgs/build-support/go/module.nix:179:20:
178| outputHashAlgo = if finalAttrs.vendorHash == "" then "sha256" else null;
179| }).overrideAttrs finalAttrs.passthru.overrideModAttrs;
| ^
180|
Now instead we take the fallback default.
We should constrain the set of supported platforms to the platforms
supported by the compiler. Otherwise we run into the unfortunate
situation where setting meta.platforms in a Rust package to
platforms.unix or platforms.linux will reintroduce CPU architectures
unsupported by the compiler.
`importNpmLock.buildNodeModules` returns a derivation with a pre-built `node_modules` directory, as imported by `importNpmLock`.
This is to be used together with `importNpmLock.hooks.linkNodeModulesHook` to facilitate `nix-shell`/`nix develop` based development workflows:
```nix
pkgs.mkShell {
packages = [
importNpmLock.hooks.linkNodeModulesHook
nodejs
];
npmDeps = importNpmLock.buildNodeModules {
npmRoot = ./.;
inherit nodejs;
};
}
```
will create a development shell where a `node_modules` directory is created & packages symlinked to the Nix store when activated.
This code is adapted from https://github.com/adisbladis/buildNodeModules
Rationale: Since RFCs 140 and 146, the old category-based hierarchy is
deprecated and obsolete, and a new approach took place: packages should be as
self-contained as possible.
This paradigm is reflected in many new strict checks that prohibit a package to
refer to files outside its directory tree.
Following this spirit, this commit essentially moves nixpkgs
pkgs/test/default.nix to ./tests/default.nix.
Further, to keep the top-level `tests` attribute, a green alias is kept in the
place of older file.
There's no need to use a Nix file in the path here. By using a different
file we won't cause rebuilds when we change the Nix file, in particular
also when the Nix file is reformatted.
The previously used pattern was introduced in #318614, but technically
leaked the default flags into the global scope. While this would
probably not make much of a practical difference, making concatTo
support default values is a much cleaner approach.
This adds a new `imageStream` option that can be used in conjunction
with `pkgs.dockerTools.streamLayeredImage` so that the image archive
never needs to be materialized in the `/nix/store`. This greatly
improves the disk utilization for systems that use container images
built using Nix because they only need to store image layers instead of
the full image. Additionally, when deploying the new system and only
new layers need to be built/copied.
Currently we cannot debug rust binaries on darwin (via lldb).
The debug symbols seem to be provided by default in a number of files
in `target/debug/deps/*.rcgu.o`. As far as I can tell these have
hardcoded paths referring to the ephemeral build directory. However,
`split-debuginfo=packed` conveniently produces a `.dSYM` file that can
be copied to `$out/bin/` and immediately provide debugging information.
Fixes https://github.com/NixOS/nixpkgs/issues/262131
This envvar is also added to lib.proxyImpureEnvVars since it's
typically required for https proxies.
This change also updates fetchgit and go module fetching to use this
envvar. NIX_GIT_SSL_CAINFO is still supported for backwards
compatibility in fetchgit.
Since version 2.0.0, proc-macro-crate has assumed it can exec()
`env::var("CARGO")` in order to run `cargo locate-project`.
This commit adds a crate override to proc-macro-crate which simply writes the
path to buildPlatform.cargo into the proc-macro-crate sources.
This way we don't need to set `env.CARGO` for every build that depends on
proc-macro-crate -- if we do that, the $CARGO environment variable would be
visible to the entire build. This could potentially lead to incredibly
hard-to-troubleshoot heisenbugs if there is some other crate that expects
`env::var("CARGO")` to exist -- that other crate would mysteriously work only
in projects that use proc-macro-crate but not anywhere else!
Fixes two bugs:
- pass a forgotten `{}` to `compressDrv`.
- remove incorrect usage of `lndir` in `compressDrv`. I added a brief
comment on why, see [this comment][1] for more details.
Tested with:
```
$ nix build .#legacyPackages.x86_64-linux.gitea.passthru.data-compressed
$ ls -lh result/public/assets/licenses.txt*
lrwxrwxrwx 1 root root 90 Jan 1 1970 result/public/assets/licenses.txt -> /nix/store/p21irsr57hckd3x3ym18aa0cr9zmm3an-gitea-1.22.1-data/./public/assets/licenses.txt
-r--r--r-- 1 root root 30K Jan 1 1970 result/public/assets/licenses.txt.br
-r--r--r-- 1 root root 82K Jan 1 1970 result/public/assets/licenses.txt.gz
```
[1]: https://github.com/NixOS/nixpkgs/pull/332752#discussion_r1721043286
String-interpolation converts path objects inside `contents` into store
paths to ensure they are properly included in the result image.
See tests.trivial-builders.references for the necessity of
string-interpolation.
Quote each string-interpolated content member to accomodates spaces
inside.
Was trying out the recommended example for `writePython3Bin` and it
failed with:
```
Traceback (most recent call last):
File "/nix/store/gcmhfm7mslpndjasfhvs66f1ca24vxim-test_py3/bin/test_py3", line 4, in <module>
y = yaml.load("""
TypeError: load() missing 1 required positional argument: 'Loader'
```
Looks like `yaml.load(input)` was deprecated in 5.1: https://msg.pyyaml.org/load
`nixos-24.05` uses `6.0.1` so we're in the clear.
* autoPatchelfHook: add keep_libc flag
- Add keep_libc flag to disable the default libc handling. Intended
to be used by systemd.
- Add autoPatchelfFlags to autoPatchelfHook for passing arguments to
the autoPatchelf script
This reverts part of the change made in #307068 / 80be926.
Fixes#332533
Fix overriding of vendorHash and various attributes via the fixed point
attribute support of stdenv.mkDerivation.
Pass as derivation attributes
goModules, modRoot, vendorHash, deleteVendor, and proxyVendor.
Move goModules and vendorHash out of passthru.
Co-authored-by: Doron Behar <doron.behar@gmail.com>
Place the VM disk image in a local directory "disk-image" instead of
"$out", so that we don't have to delete it to reserve "$out" for the
container image.
Deprecate singularity-tools.mkLayer and singularity-tools.shellScript,
for they are no longer related to image building.
Use writers.writeBash instead of singularity-tools.shellScript.
Allow users to pass arguments to `buildDotnetModule` in the form:
```nix
buildDotnetModule (finalAttrs: {
# Args
})
```
Exposing the behaviour of the underlying `mkDerivation` and allowing
packages to be defined in a recursive way that works correctly even when
the package is overridden, e.g. using `overrideAttrs`.
Added some simple test cases that piggyback on the existing
`structured-attrs` test.
*compressDrv* compresses files in a given derivation.
*compressDrvWeb* compresses a derivation for a loosely-defined
pre-compressed "web server" usage.
This intends to replace the `passthru.data-compressed` derivations that
have accumulated in nixpkgs with something more reusable.
The current error message is hard to debug because the error is in the
nix store:
> Error: failed to parse JSON in
"/build/.attr-1s42g1c76fxb77skzq0b4wdhcrg8jmzb54czmxvh1qm7psgsbcni"
>
> Caused by:
> missing field `source` at line 1 column 102
>
> Location:
> src/main.rs:329:10
* streamLayeredImage: self-document the script
'podman load' doesn't let me override the name/tag from the image.
This name and tag is dynamic and in a CI environment so I would like to
be able to adjust the tag dynamically.
Since the image is streamed by stream_layered_image.py, there is no need
to stick with the nix-hardcoded image name/tag: the python script can
accept another name
I've added argparse to expose the feature. It has the added benefit of
adding `--help` support to the script, which makes its usage
self-explanatory.
* blackified file
'command -v builtin' returns 'builtin', which doesn't suit us since
we're looking for program in the given PATH. This could give us shebangs
like this:
#!builtin
which is surprising.
Switch to 'type -P command' which always returns a path, even if command
is both a builtin and an executable (for example 'test'), or fail is
'command' is just a builtin.
The docker-tools test, where this originates, was not run on aarch64-linux, but this is an artifact of its age more so than anything else.
Co-authored-by: Ivan Trubach <mr.trubach@icloud.com>
This removes redundant inline docs, because
- users should consult the better docs in the manual,
- contributors should add to the manual, not the inline comments
Before this commit, `pkgs/build-support/cc-wrapper/add-flags.sh`
was using `-B@out@/bin` instead of `-B@bintools@/bin`
to force `cc` to use `ld-wrapper.sh` when calling `ld`.
That was confusing `cc` when asked to print
the location of a library precisely named `c++`
because `-B` prefixes are also used by `cc` to find libraries,
see https://gcc.gnu.org/onlinedocs/gcc/Directory-Options.html#index-B
Indeed, instead of having `cc --print-file-name c++`
failing to found a `c++` library and just returning the given `c++` string
to let a linker resolve it thereafter,
it was finding that `@out@/bin/c++` executable,
mistaking it for a library and returning its absolute path,
forcing the linker to load an executable as a library.
Before this commit:
```console
$ nix run -f . stdenv.cc -- --print-file-name c++
/nix/store/9bv7dcvmfcjnmg5mnqwqlq2wxfn8d7yi-gcc-wrapper-13.2.0/bin/c++
```
After this commit:
```console
$ nix run -f . stdenv.cc -- --print-file-name c++
c++
```
Fixes https://gitlab.haskell.org/ghc/ghc/-/issues/23138#note_567034
where this behavior was breaking GHC on Darwin.
[Confirmed by @414owen](https://github.com/NixOS/nixpkgs/pull/317224#issuecomment-2171276177):
> This fixed all our haskell builds on Arm64 darwin, which were trying
> to link in clang++...
When the hash of an url being fetched does not match the expected value, this
commit will cause fetch-yarn-deps to include the url in the error message to
assist debugging.
When a dependency references a github *release* URL, that dependency must be
fetched using https rather than git, since github does not require that
release tarballs have any relationship whatsoever to the git history.
This commit causes them to be fetched using https, not git.
A test case (which fails prior to this commit, and passes afterwards) is included.
There's no point generating debug info if the compiler immediately
strips it before we get a chance to do anything with it.
This is especially important since Cargo 1.77, which asks rustc to
strip by default.
Updates tests for testers.testEqualContents with diffoscope and fixes
some bugs in tests (e.g. fileDiff always succeed because subshell does
not inherit errexit option).
Before this change, testers.testEqualContents implementation had several
bugs (e.g. executables at different paths were not considered equal). So
we switch to diffoscope that that is designed to handle exactly these
kinds of comparisons and gives more insights into the differences in the
output.
modules-closure.sh seems to consider everything under
/lib/firmware to be a kernel module,
this change adds a special handling of `edid` directory,
which does not contain kernel modules
fixes#279739
When the linker signs a Mach-O binary, it sets a flag in the signature’s code directory indicating that the signature was generated by a linker. Tools such as `strip` and `install_name_tool` read this flag and will
update ad hoc signatures after they perform their modifications.
The updated l64 supports signing binaries automatically. Both the updated cctools and LLVM will check for the linker-signed flag and resign binaries they modify automatically when it’s present. Given that, use of postLinkSignHook is unnecessary and potentially harmful.
In particular, if the hook is used and an unwrapped `strip` or `install_name_tool` is on the user’s path, they will not automatically update an ad hoc signature. Instead, they will issue a warning and create a binary with a broken signature.
It is more robust to let the tools handled this since the only time a signature would not be linker-signed is when the user is manually invoking `codesign` (or another tool such as `sigtool` or `rcodesign`), which by nature of the invocation updates the signature to a valid one.
Since `strip` no longer needs to be wrapped for code-signing, binutils-wrapper now uses the GNU strip wrapper on Darwin.
Fixes https://github.com/NixOS/nixpkgs/issues/208951.
Darwin and its bootstrap tools both use LLVM 16, which is sufficient for
arm64e support in LLVM. Thsi change removes an unnecessary LLVM 13 build
from the stdenv bootstrap.
This changes ld-wrapper to use a temporary file for the response file
passed to ld instead of using process substitution.
ld64 does not handle long command-lines when reading from the response
file, which defeats the point of using a response file to handle long
command-lines. cctools-port was patched to work around this, but nixpkgs
is now using Apple’s source release directly instead of the port.
Since it’s preferable not to patch Apple’s release heavily (to reduce
the difficulty of updating to new versions and to match upstream’s
behavior), use the approach that was adopted in cc-wrapper to work
around issues with response files in newer versions of clang.
Related PRs (cctools-port):
- https://github.com/NixOS/nixpkgs/pull/213831
- https://github.com/tpoechtrager/cctools-port/pull/132
Related PRs (cc-wrapper):
- https://github.com/NixOS/nixpkgs/pull/245282
- https://github.com/NixOS/nixpkgs/pull/258608
This makes the minimal change to put the script in a new file.
It does not fix the resulting ShellCheck warnings (which would
have been present in the previous version; I've just exposed
them).
As a consequence of restrictions imposed by RFC 140 - Simple Package Paths [1]
-, files related to a package should be confined on the package directory.
Certainly this restriction does not apply to packages outside by-name hierarchy.
Nonetheless, this is an interesting organization heuristics: things that affect
Emacs should be confined inside Emacs directory. Besides a future migration, the
"debuggability" of a framework is way more enhanced when we know how to find all
its files.
A similar task was done before, when RFC 140 was not a thging yet - namely, the
migration of emacs-modes to elisp-packages [2].
[1] https://github.com/NixOS/rfcs/pull/140
[2] https://github.com/NixOS/nixpkgs/pull/123859
This is executed curing the build, and usually quite helpful in telling
you how the build failed, if it failed.
For example, when linking against system libraries, it shows which
system lib could not be discovered, the pkg-config path used etc.
Not showing it simply removes all this, and trying to parse out the
"important" bits and only show these is probably also brittle.
The output generally isn't that spammy normally, so let's just `tee` it.
