dev/nixpkgs
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity
7553d0fe29
nixpkgs/pkgs/build-support
History
Adam Joseph 7553d0fe29 stdenv: Nix-driven bootstrap of gcc 
#### Summary

By default, when you type `make`, GCC will compile itself three
times.  This PR inhibits that behavior by configuring GCC with
`--disable-bootstrap`, and reimplements the triple-rebuild using
Nix rather than `make`/`sh`.

 #### Immediate Benefits

- Allow `gcc11` and `gcc12` on `aarch64` (without needing new
  `bootstrapFiles`)
- Faster stdenv rebuilds: the third compilation of gcc
  (i.e. stageCompare) is no longer a `drvInput` of the final stdenv.
  This allows Nix to build stageCompare in parallel with the rest of
  nixpkgs instead of in series.
- No more copying `libgcc_s` out of the bootstrap-files or other
  derivations
- No more Frankenstein compiler: the final gcc and the libraries it
  links against (mpfr, mpc, isl, glibc) are all built by the same
  compiler (xgcc) instead of a mixture of the bootstrapFiles'
  compiler and xgcc.
- No more [static lib{mpfr,mpc,gmp,isl}.a hack]
- Many other small `stdenv` hacks eliminated
- `gcc` and `clang` share the same codepath for more of `cc-wrapper`.

 #### Future Benefits

- This should allow using a [foreign] `bootstrap-files` so long as
  `hostPlatform.canExecute bootstrapFiles`.
- This should allow each of the libraries that ship with `gcc`
  (lib{backtrace, atomic, cc1, decnumber, ffi, gomp, iberty,
  offloadatomic, quadmath, sanitizer, ssp, stdc++-v3, vtv}) to be
  built in separate (one-liner) derivations which `inherit src;`
  from `gcc`, much like https://github.com/NixOS/nixpkgs/pull/132343

 #### Incorporates

- https://github.com/NixOS/nixpkgs/pull/210004
- https://github.com/NixOS/nixpkgs/pull/36948 (unreverted)
- https://github.com/NixOS/nixpkgs/pull/210325
- https://github.com/NixOS/nixpkgs/pull/210118
- https://github.com/NixOS/nixpkgs/pull/210132
- https://github.com/NixOS/nixpkgs/pull/210109
- https://github.com/NixOS/nixpkgs/pull/213909
- https://github.com/NixOS/nixpkgs/pull/216136
- https://github.com/NixOS/nixpkgs/pull/216237
- https://github.com/NixOS/nixpkgs/pull/210019
- https://github.com/NixOS/nixpkgs/pull/216232
- https://github.com/NixOS/nixpkgs/pull/216016
- https://github.com/NixOS/nixpkgs/pull/217977
- https://github.com/NixOS/nixpkgs/pull/217995

 #### Closes

- Closes #108305
- Closes #108111
- Closes #201254
- Closes #208412

 #### Credits

This project was made possible by three important insights, none of
which were mine:

1. @ericson2314 was the first to advocate for this change, and
   probably the first to appreciate its advantages.  Nix-driven
   (external) bootstrap is "cross by default".

2. @trofi has figured out a lot about how to get gcc to not mix up
   the copy of `libstdc++` that it depends on with the copy that it
   builds, by moving the `bootstrapFiles`' `libstdc++` into a
   [versioned directory].  This allows a Nix-driven bootstrap of gcc
   without the final gcc would still having references to the
   `bootstrapFiles`.

3. Using the undocumented variable [`user-defined-trusted-dirs`]
   when building glibc.  When glibc `dlopen()`s `libgcc_s.so`, it
   uses a completely different and totally special set of rules for
   finding `libgcc_s.so`.  This trick is the only way we can put
   `libgcc_s.so` in its own separate outpath without creating
   circular dependencies or dependencies on the bootstrapFiles.  I
   would never have guessed to use this (or that it existed!) if it
   were not for a [comment in guix] which @Mic92 [mentioned].

My own role in this PR was basically: being available to go on a
coding binge at an opportune moment, so we wouldn't waste a
[crisis].

[aarch64-compare-ofborg]: https://github.com/NixOS/nixpkgs/pull/209870/checks?check_run_id=10662822938
[amd64-compare-ofborg]: https://github.com/NixOS/nixpkgs/pull/209870/checks?check_run_id=10662825857
[nonexistent sysroot]: https://github.com/NixOS/nixpkgs/pull/210004
[versioned directory]: https://github.com/NixOS/nixpkgs/pull/209054
[`user-defined-trusted-dirs`]: https://sourceware.org/legacy-ml/libc-help/2013-11/msg00026.html
[comment in guix]: 5e4ec82181/gnu/packages/gcc.scm (L253)
[mentioned]: https://github.com/NixOS/nixpkgs/pull/210112#issuecomment-1379608483
[crisis]: https://github.com/NixOS/nixpkgs/issues/108305
[foreign]: https://github.com/NixOS/nixpkgs/pull/170857#issuecomment-1170558348
[static lib{mpfr,mpc,gmp,isl}.a hack]: 2f1948af9c/pkgs/stdenv/linux/default.nix (L380)
2023-04-02 13:49:41 -07:00
..
add-opengl-runpath
agda Merge remote-tracking branch 'origin/master' into haskell-updates 2023-02-18 21:56:57 +01:00
alternatives treewide: use optionalString 2023-02-13 21:52:34 +01:00
appimage Merge pull request #203598 from symphorien/appimage-owd 2022-12-02 20:12:49 +00:00
binary-cache Introduce mkBinaryCache function 2023-02-07 16:16:07 -08:00
bintools-wrapper bintools-wrapper: specify SHA1 as the build-id hash style explicitly 2023-02-25 12:49:40 -06:00
build-bazel-package treewide: use optionalString 2023-02-13 21:52:34 +01:00
build-fhs-userenv treewide: use optionalString 2023-02-13 21:52:34 +01:00
build-fhs-userenv-bubblewrap Merge pull request #215837 from K900/fhsenv-x11-socket-permissions 2023-02-16 12:25:04 +03:00
build-graalvm-native-image buildGraalvmNativeImage: allow overriding attributes 2023-02-18 12:38:09 +00:00
build-setupcfg treewide: use lib.optionals 2023-02-14 19:11:59 +01:00
buildenv
cc-wrapper stdenv: Nix-driven bootstrap of gcc 2023-04-02 13:49:41 -07:00
coq mkCoqDerivation: findlib is an optional input 2023-02-28 10:59:42 +00:00
deterministic-uname deterministic-uname: add missing whitespace 2023-01-31 14:55:11 +02:00
dhall
docker dockerTools: Preprocess layers list before unpack to handle repeated layers 2023-02-06 12:19:29 -06:00
dotnet Merge pull request #218849 from Atemu/buildDotnetModule-put-dep-file-path-in-fetch-script 2023-03-06 17:46:32 +01:00
emacs emacs: Add basic tree-sitter support (#219559) 2023-03-15 16:51:29 +13:00
expand-response-params
fake-nss fake-nss: Add support for extra passwd and group lines 2022-10-04 00:05:34 +02:00
fetchbitbucket
fetchbower
fetchbzr fetchbzr: set cache directory to tmpdir 2023-03-01 04:20:00 +00:00
fetchcvs treewide: remove usages of header and stopNest 2023-01-16 00:08:12 +02:00
fetchdarcs Merge pull request #205115 from hraban/fetchdarcs-by-hash 2023-03-14 01:25:22 +01:00
fetchdocker treewide: use optionalString 2023-02-13 21:52:34 +01:00
fetchfirefoxaddon treewide: remove usages of header and stopNest 2023-01-16 00:08:12 +02:00
fetchfossil treewide: use optionalString 2023-02-13 21:52:34 +01:00
fetchgit treewide: use optionalString 2023-02-13 21:52:34 +01:00
fetchgitea
fetchgithub fetchGitHub: inherit owner and repo for use with rocmUpdateScript 2022-12-10 13:35:47 -06:00
fetchgitiles
fetchgitlab fetchFromGitLab: don't passthruAttrs fetchGit specific attrs to fetchUrl 2022-10-29 19:12:07 +03:00
fetchgitlocal
fetchgx
fetchhg treewide: use optionalString 2023-02-13 21:52:34 +01:00
fetchipfs treewide: source .attrs in builders 2022-12-08 21:09:02 +02:00
fetchmavenartifact fetchMavenArtifact: add classifier to filename 2023-02-02 19:23:58 -08:00
fetchmtn treewide: remove usages of header and stopNest 2023-01-16 00:08:12 +02:00
fetchnextcloudapp
fetchpatch fetchpatch: add decode test 2023-03-11 21:39:29 +00:00
fetchpypi fetchPypi: move to top level 2023-01-21 08:21:33 +01:00
fetchrepoorcz
fetchrepoproject
fetchs3
fetchsavannah
fetchsourcehut
fetchsvn treewide: remove usages of header and stopNest 2023-01-16 00:08:12 +02:00
fetchsvnrevision
fetchsvnssh treewide: remove usages of header and stopNest 2023-01-16 00:08:12 +02:00
fetchurl treewide: remove usages of header and stopNest 2023-01-16 00:08:12 +02:00
fetchzip Merge pull request #191355 from GenericNerdyUsername/fetchzip-include-hidden 2022-12-16 18:08:36 +01:00
flutter treewide: move NIX_CFLAGS_COMPILE to the env attrset 2023-02-22 21:23:04 +02:00
go Merge staging-next into staging 2023-02-07 00:02:46 +00:00
icon-conv-tools
install-shell-files
java
kernel make-initrd-ng: document wrapped file behavior 2023-02-20 07:02:55 -05:00
libredirect libredirect: fix build on musl libc 2023-02-06 10:18:52 -08:00
make-darwin-bundle
make-desktopitem
make-hardcode-gsettings-patch makeHardcodeGsettingsPatch: Support other constructors 2022-12-19 17:38:57 +01:00
make-pkgconfigitem makePkgconfigItem: fix cross 2022-12-30 14:51:17 +02:00
make-startupitem
mkshell mkShell: set preferLocalBuild by default 2023-02-22 11:53:51 +01:00
mono-dll-fixer
nix-gitignore
node Merge master into staging-next 2023-02-20 12:01:32 +00:00
nuke-references treewide: use optionalString 2023-02-13 21:52:34 +01:00
ocaml [OCaml] tell dune where to install man 2023-03-14 14:37:57 +01:00
oci-tools
pkg-config-wrapper Merge pull request #207294 from happyalu/staging 2023-01-24 00:02:25 +01:00
portable-service
prefer-remote-fetch
references-by-popularity dockerTools: prefer local builds 2022-11-18 09:31:53 -05:00
release treewide: use optionalString 2023-02-13 21:52:34 +01:00
remove-references-to
replace-secret
rust Merge remote-tracking branch 'origin/master' into staging-next 2023-03-15 02:01:15 +01:00
setup-hooks Merge pull request #219683 from symphorien/separatedebuginfo_static 2023-03-12 17:02:03 +00:00
singularity-tools apptainer, singularity: fix defaultPath and reflect upstream changes 2023-02-08 18:03:11 +08:00
snap
src-only srcOnly: prevent phases being skipped 2023-01-06 19:02:50 +01:00
substitute
substitute-files
templaterpm
testers meta.pkgConfigModules: Init convention 2023-02-03 09:37:31 -05:00
trivial-builders trivial-builders/test/references.nix: fix eval 2023-04-02 03:04:20 -07:00
vm build-support/vm/deb/deb-closure: quote generated urls 2023-02-20 01:38:31 +01:00
wrapper-common build-support: Quote expansions inside ${…} 2022-12-02 04:59:43 -05:00
writers writers: make codesign_allocate available in PATH on aarch64-darwin 2023-03-14 19:45:53 +01:00
build-maven.nix
build-pecl.nix feat: add passthru flags in PHP extension builder 2022-12-29 18:19:12 +01:00
closure-info.nix
make-impure-test.nix makeImpureTest: init function for hardware tests 2022-11-22 16:54:32 +01:00
plugins.nix
replace-dependency.nix replaceDependency: use runCommandLocal 2023-03-01 14:36:49 +02:00
setup-systemd-units.nix
source-from-head-fun.nix
trivial-builders.nix trivial-builders.writeShellApplication: use unwrapped shellcheck 2023-03-07 21:18:48 +02:00