Commit Graph

200 Commits

Author SHA1 Message Date
nicoo
87c22100a6 stdenv.mkDerivation: Reject MD5 hashes
While there is no fetcher or builder (in nixpkgs) that takes an `md5` parameter,
for some inscrutable reason the nix interpreter accepts the following:
```nix
fetchurl {
  url = "https://www.perdu.com";
  hash = "md5-rrdBU2a35b2PM2ZO+n/zGw==";
}
```

Note that neither MD5 nor SHA1 are allowed by the syntax of SRI hashes.
2023-10-25 23:14:15 +00:00
Artturin
c25f7f894f stdenv: Add hack to fix cmake canExecute cross-compilation
Fixes `pkgsCross.musl64.llvmPackages_16.clang.cc` on `x86_64-linux`,
which used to fail with `/bin/sh: clang-tblgen: not found`.

Same hack is used in other projects:
https://github.com/search?q=%2FCMAKE_CROSSCOMPILING_EMULATOR.%2B%5C%2Fusr%5C%2Fbin%5C%2Fenv%2F+NOT+is%3Afork&type=code

Comment from 30435a9d0f/build/cmake/HostLinuxToolchain.cmake (L64)

> Required to run host Linux executables during the build itself.
> An example would be https://gitub.com/KhronosGroup/Vulkan-Loader and
> its "asm_offset" program.
>
> NOTE: Alternatives have been tried unsuccessfully, i.e.:
>
>  With $(set CMAKE_CROSSCOMPILING_EMULATOR), the build fails because
>  the CMake ninja/Make script tries to find the executable in the current
>  path, as in:
>
>    [3/16] Generating gen_defines.asm
>    FAILED: loader/gen_defines.asm
>    cd /tmp/cc/build-Vulkan-Loader/loader && asm_offset GAS
>    /bin/sh: asm_offset: command not found
>    ninja: build stopped: subcommand failed.
>
> With $(set CMAKE_CROSSCOMPILING_EMULATOR ""), the build fails because
> the shell cannot find the "" program as in:
>
>    [3/16] Generating gen_defines.asm
>    FAILED: loader/gen_defines.asm
>    cd /tmp/cc/build-Vulkan-Loader/loader && "" /tmp/cc/build-Vulkan-Loader/loader/asm_offset GAS
>    /bin/sh: : command not found
>    ninja: build stopped: subcommand failed.
>
> It seems that the root of the problem comes from how the CMake function
> cmCustomCommandGenerator::GetArgc0Location() computes the target
> executable's location. At this point it's unclear whether this is a CMake
> bug or a feature.

Risicle discovered this hack.

Co-authored-by: Robert Scott <code@humanleg.org.uk>
2023-08-29 22:12:06 +03:00
Jan Malakhovski
85d4248a4f
treewide: fix some comments (#247365)
* pkgs/top-level/splice.nix: fix broken comment

* stdenv: use a value that actually works in the comment there
2023-08-05 23:01:26 +02:00
Vladimír Čunát
58392652f0
Merge #224822: hardening flags: enable fortify3 by default
...into staging
2023-07-06 10:33:24 +02:00
Artturin
42c94d06fd stdenv: fix overriding with attrset when finalAttrs isn't used
```
nix-repl> (pkgs.htop.overrideAttrs { pname = "hello-overriden"; }).pname
error:
       … while evaluating a branch condition

         at /nix/store/phn5cahwacv9wjgalygw62x8l4xbl6x3-source/lib/customisation.nix:86:7:

           85|     in
           86|       if builtins.isAttrs result then
             |       ^
           87|         result // {

       … while calling the 'isAttrs' builtin

         at /nix/store/phn5cahwacv9wjgalygw62x8l4xbl6x3-source/lib/customisation.nix:86:10:

           85|     in
           86|       if builtins.isAttrs result then
             |          ^
           87|         result // {

       (stack trace truncated; use '--show-trace' to show the full trace)

       error: attempt to call something which is not a function but a set

       at /nix/store/phn5cahwacv9wjgalygw62x8l4xbl6x3-source/pkgs/stdenv/generic/make-derivation.nix:58:21:

           57|           f = self: super:
           58|             let x = f0 super;
             |                     ^
           59|             in
```
2023-07-03 21:34:50 +03:00
Robert Hensing
1f5a0b0135
Merge pull request #239005 from Artturin/setorfunoverr
stdenv: let overrideAttrs accept attrset OR function
2023-07-01 15:27:23 +02:00
Artturin
0fdae31531 stdenv: let overrideAttrs accept attrset OR function
Makes overrideAttrs usable in the same way that `override` can be used.
It allows the first argument of `overrideAttrs` to be either a function
or an attrset, instead of only a function:

hello.overrideAttrs (old: { postBuild = "echo hello"; })
hello.overrideAttrs { postBuild = "echo hello"; }

Previously only the first example was possible.

Co-authored-by: adisbladis <adisbladis@gmail.com>
Co-authored-by: matthewcroughan <matt@croughan.sh>
2023-06-30 23:28:00 +03:00
Robert Scott
db3e94c3b2 hardening flags: enable fortify3 by default 2023-06-25 11:50:46 +01:00
Artturin
d88b362808 stdenv: finish removal of deprecated string/null Flags
forgot to do it before 23.05 but it's ok
2023-06-22 20:37:06 +03:00
Artturin
ac031478cb stdenv: use lib.isX instead of typeOf Y == X
more readable
2023-06-22 12:31:52 +03:00
Robert Hensing
9524f57dd5
Merge pull request #230666 from alyssais/inputDerivation-passAsFile
stdenv: fix inputDerivation with passAsFile
2023-05-09 20:36:29 +02:00
Alyssa Ross
a295c4566e
stdenv: fix inputDerivation with passAsFile
passAsFile passes the values of Nix bindings to the builder as
files, so if those values contained references, they wouldn't end up
in the inputDerivation output.  To fix that, append the contents of
every such passed file to the output.

We only have shell builtins in this derivation, so we can't use cat.
The only way I know of appending the contents of one file to another
using only shell builtins is as I've done here, but it requires
putting the contents of the file on echo's argv.  This might end up
causing problems with large files.  Regardless, I think we should try
this, as a failure is better than silently producing an incorrect
result like the previous behavior.
2023-05-09 14:00:57 +00:00
Alyssa Ross
cd0bccbbe0
stdenv: fix inputDerivation with allowed refs
Same motivation as for the disallowed references.
2023-05-08 12:02:59 +00:00
Sergei Trofimovich
72d71dd641 stdenv: do not pass __contentAddressed = false as environment variable
`nix-2.4+` automatically filters `__contentAddressed` out of the
environment. But not `nix-2.3`. This make `.drv` to differ between
unset and `__contentAddressed = false` derivations.

This change makes them equal by filtering out `__contentAddressed`
unless it's set to `true`.
2023-05-02 22:57:54 +01:00
Emily Trau
2ad68a7d90 stdenv: factor out meta attr augmentation for reusability
stdenv: check if separating commonMeta from assertValidity fixes laziness issue

```
error: cannot coerce null to a string

at /var/lib/ofborg/checkout/repo/38dca4e3aa6bca43ea96d2fcc04e8229/mr-est/ofborg-evaluator-3/pkgs/development/coq-modules/coq-lsp/default.nix:34:60:

    33|     homepage = "https://github.com/ejgallego/coq-lsp";
    34|     changelog = "https://github.com/ejgallego/coq-lsp/blob/${defaultVersion}/CHANGES.md";
```
2023-05-01 20:18:11 +03:00
Sandro
b04d4bad27
Merge pull request #216992 from SuperSandro2000/stdenvNative-fix-eval
{bintools,cc}-wrapper: don't fallback to version = null
2023-04-14 11:22:20 +02:00
Sandro Jäckel
b0ee79f9fd
mkDerivation: show meaningful error when version is set to null
instead of `Cannot coerce null to string`
2023-04-12 22:08:36 +02:00
github-actions[bot]
455127ad5e
Merge master into staging-next 2023-03-16 18:01:20 +00:00
Bernardo Meurer
6e55733359
Merge pull request #219747 from Stunkymonkey/deprecate-isNull 2023-03-16 11:10:22 -03:00
Martin Weinelt
19680e9902
Merge pull request #217568 from trofi/stdenv-parallel-install
stdenv/generic/setup.sh: enable parallel installs for parallel builds
2023-03-15 17:10:19 +00:00
Felix Buehler
d10e69c86b treewide: deprecate isNull
https://nixos.org/manual/nix/stable/language/builtins.html#builtins-isNull
2023-03-06 22:40:04 +01:00
Sergei Trofimovich
69cf5181c3 stdenv/generic/setup.sh: enable parallel installs by default
The primary motivating example is openssl:

Before the change full package build took 1m54s minutes.
After the change full package build takes 59s.

About a 2x speedup.

The difference is visible because openssl builds hundreds of manpages
spawning a perl process per manual in `install` phase. Such a workload
is very easy to parallelize.

Another example would be `autotools`+`libtool` based build system where
install step requires relinking. The more binaries there are to relink
the more gain it will be to do it in parallel.

The change enables parallel installs by default only for buiilds that
already have parallel builds enabled. There is a high chance those build
systems already handle parallelism well but some packages will fail.

Consistently propagated the enableParallelBuilding to:
- cmake (enabled by default, similar to builds)
- ninja (set parallelism explicitly, don't rely on default)
- bmake (enable when requested)
- scons (enable when requested)
- meson (set parallelism explicitly, don't rely on default)
- waf (set parallelism explicitly, don't rely on default)
- qmake-4/5/6 (enable by default, similar to builds)
- xorg (always enable, similar to builds)
2023-02-26 22:02:09 +00:00
Rahul Butani
d364ee8d13
mkDerivation: do not disable separateDebugInfo on LLVM stdenvs
This was disabled here: b86e62d30d (diff-282a02cc3871874f16401347d8fadc90d59d7ab11f6a99eaa5173c3867e1a160)

h/t to @teh: b86e62d30d (commitcomment-77916294)
for pointing out that the failure that @matthewbauer was
seeing was caused by the `separate-debug-info.sh` `build-id` length
requirement that #146275 will relax

`lld` has had `--build-id` support dating back to LLVM4: https://reviews.llvm.org/D18091

This predates every `llvmPackages_` version currently in nixpkgs (and
certainly every version actually still used in `useLLVM` stdenvs) so
with the previous commit (asking `ld` for sufficiently long SHA1 hashes)
I think we can safely enable `separateDebugInfo` when using LLVM
bintools.
2023-02-25 12:49:40 -06:00
Robert Scott
0eedcfc3f4
Merge pull request #212498 from risicle/ris-fortify3
hardening flags: add `FORTIFY_SOURCE=3` support
2023-02-16 21:19:30 +00:00
github-actions[bot]
29c2637dda
Merge staging-next into staging 2023-02-13 12:02:05 +00:00
github-actions[bot]
fd8daee7c2
Merge master into staging-next 2023-02-13 12:01:30 +00:00
Robert Hensing
059bd43546 make-derivation.nix: Support inputDerivation on disallowedReferences 2023-02-07 18:58:13 +01:00
github-actions[bot]
dfee1a3150
Merge staging-next into staging 2023-02-05 00:03:15 +00:00
github-actions[bot]
bc833a50cc
Merge master into staging-next 2023-02-05 00:02:31 +00:00
Artturi
dcc7df7fe6
Merge pull request #211685 from Artturin/splicingstuff1-split 2023-02-03 12:49:08 +02:00
Alyssa Ross
b6304bf11d
stdenv.mkDerivation: fix meson for some archs
platform.uname.processor seems to be what we want in many more cases
than what we were using before — it does the right thing for aarch64,
x86_64, riscv32, riscv64, mips, mips64, powerpc, and powerpc64 (the
latter three of which were broken before).

This fixes cross-compilation of systemd for PowerPC/POWER platforms.
2023-01-30 19:42:19 +00:00
R-VdP
51acc6245e
stdenv: don't include drvs in disallowedRefs as build-time deps.
Derivations listed as disallowedReferences or disallowedRequisites,
currently end up as build-time dependencies.
This is problematic since the disallowed derivations will be built by nix as
build-time dependencies, while those derivations might take a very long time
to build, or might not even build successfully on the platform used.
However, in order to scan for disallowed references in the final output,
knowing the out path is sufficient, and the out path can be calculated from
the derivation without needing to build it, saving time and resources.

While the problem is less severe for allowedReferences and allowedRequisites,
since we want the derivation to be built eventually, we would still like to
get the error early and without having to wait while nix builds a derivation
that might not be used (e.g. if we listed the wrong one).
2023-01-30 17:58:05 +01:00
Robert Scott
3d453e2aee mkDerivation: add support for fortify3 hardening flag 2023-01-24 21:52:10 +00:00
Guillaume Girol
f39abbc350 mkDerivation: introduce native checkInputs and nativeCheckInputs
When strictDeps is set, only nativeCheckInputs are added to PATH and
only checkInputs can be linked against. See https://github.com/NixOS/nixpkgs/issues/161570
2023-01-21 16:42:09 +01:00
Robert Hensing
bff126ffbc stdenv.mkDerivation: Make overrideAttrs overridable
(cherry picked from commit 43c8b43f808f48fd5600afcad5503eaeaf6d71b7)
2023-01-15 04:37:14 +02:00
Sergei Trofimovich
428107f837 stdenv: set enableParallelBuilding explicitly if enableParallelBuildingByDefault is set
Without the change we don't propagate `enableParallelBuilding = true`
and leave most builds sequential.

Noticed on `mythtv` package which did not specify parallelism and
`config.enableParallelBuildingByDefault = true` had no effect.
2022-12-17 22:23:24 +00:00
Naïm Favier
0b067316d4
stdenv: use intersectAttrs instead of intersectLists
Better complexity.
2022-12-17 12:55:44 +01:00
Naïm Favier
e14de22618
stdenv: handle env gracefully
Derivations not using `__structuredAttrs` should not attempt to set
environment variables from `env`.

Derivations using `__structuredAttrs` should fail if `env` is not
exportable.
2022-12-15 13:27:11 +01:00
Artturin
734d7df235 allow derivation attributes in env
derivations can be coerced to their output paths
2022-12-08 06:13:19 +02:00
Artturin
8ad0103a34 config.structuredAttrsByDefault: add option 2022-12-08 06:13:19 +02:00
Artturin
238a6053c4 stdenv: support opt-in __structuredAttrs
Co-authored-by: Robin Gloster <mail@glob.in>

stdenv: print message if structuredAttrs is enabled

stdenv: add _append

reduces the chance of a user doing it wrong

fix nix develop issue

output hooks don't work yet in nix develop though

making $outputs be the same on non-structuredAttrs and structuredAttrs
is too much trouble.

lets instead make a function that gets the output names

reading environment file '/nix/store/2x7m69a2sm2kh0r6v0q5s9z1dh41m4xf-xz-5.2.5-env-bin'
nix: src/nix/develop.cc:299: std::string Common::makeRcScript(nix::ref<nix::Store>, const BuildEnvironment&, const Path&): Assertion `outputs != buildEnvironment.vars.end()' failed.

use a function to get all output names instead of using $outputs

copy env functionality from https://github.com/NixOS/nixpkgs/pull/76732/commits
2022-12-08 06:13:19 +02:00
Artturin
341e6fd558 splice.nix: start deprecating nativeDrv and crossDrv 2022-11-19 00:04:54 +02:00
Sandro
09cb5e8416
Merge pull request #183507 from Artturin/stdenvmeson1 2022-08-01 23:06:51 +02:00
Artturin
9f309c8792 stdenv: mesonFlags: use canExecute in needs_exe_wrapper
this is more accurate
2022-08-01 20:03:47 +03:00
Alyssa Ross
5330c0a1af
treewide: use isAarch where appropriate 2022-07-30 16:18:27 +00:00
Artturin
ca2ca8b194 stdenv: start deprecating non-list mesonFlags
the motivation for this is to simplify stdenv and ease the job of
reviewers due to them needing to tell contributors about the defacto
rule that mesonFlags should be a list of strings
2022-07-06 15:23:20 +03:00
Artturin
3173c3b6b6 stdenv: start deprecating non-list cmakeFlags
the motivation for this is to simplify stdenv and ease the job of
reviewers due to them needing to tell contributors about the defacto
rule that cmakeFlags should be a list of strings
2022-07-05 04:05:30 +03:00
Sergei Trofimovich
b0f5f5cc96
Merge pull request #178408 from trofi/add-config.enableParallelBuildingByDefault
config.configurePlatformsByDefault: init option
2022-06-23 21:32:59 +00:00
Sergei Trofimovich
c534d056fe pkgs/stdenv/generic/make-derivation.nix: add a bug reference to strictDepsByDefault TODO 2022-06-21 19:03:43 +01:00
Sergei Trofimovich
db09d70d33 config.configurePlatformsByDefault: init option
Useful to enable tree-wide occasionally to have incremental progress
towards https://github.com/NixOS/nixpkgs/pull/87909 resolution.
2022-06-21 09:23:13 +01:00