I made a mistake merge. Reverting it in c778945806 undid the state
on master, but now I realize it crippled the git merge mechanism.
As the merge contained a mix of commits from `master..staging-next`
and other commits from `staging-next..staging`, it got the
`staging-next` branch into a state that was difficult to recover.
I reconstructed the "desired" state of staging-next tree by:
- checking out the last commit of the problematic range: 4effe769e2
- `git rebase -i --preserve-merges a8a018ddc0` - dropping the mistaken
merge commit and its revert from that range (while keeping
reapplication from 4effe769e2)
- merging the last unaffected staging-next commit (803ca85c20)
- fortunately no other commits have been pushed to staging-next yet
- applying a diff on staging-next to get it into that state
This adds a warning to the top of each “boot” package that reads:
Note: this package is used for bootstrapping fetchurl, and thus cannot
use fetchpatch! All mutable patches (generated by GitHub or cgit) that
are needed here should be included directly in Nixpkgs as files.
This makes it clear to maintainer that they may need to treat this
package a little differently than others. Importantly, we can’t use
fetchpatch here due to using <nix/fetchurl.nix>. To avoid having stale
hashes, we need to include patches that are subject to changing
overtime (for instance, gitweb’s patches contain a version number at
the bottom).
In the sandbox built for https://nixbuild.net, the coreutils build fails
because a failure in the df skip-rootfs test. The test failure is triggered by
the existance of a rootfs file system. However, I think that the test is faulty,
and I have reported it upstream in
https://lists.gnu.org/archive/html/bug-coreutils/2019-12/msg00000.html.
Disabling the test makes the coreutils build work in the nixbuild.net sandbox,
and I can't think of any negative impact disabling it can have. In normal nix
setups and in the normal nix sandbox, this test is not exercised anyway, since
there is no rootfs visible.
Some target platforms, especially when building inside a container
have issues with the inotify test. I also saw issues related to that
test under macOS so I would suggest in skipping that for now.
Release notes:
https://savannah.gnu.org/forum/forum.php?forum_id=9394
Touchups:
* enable tests w/musl (disable 2 gnulib tests)
* improve tests we modify: use exit code 77 to
indicate test skipped (causing it to be reportd as SKIP)
Note: Not yet on mirrors, but can be (manually) fetched from primary
In usernamespaces some POSIX assumptions break, which
make some coreutils fail when running with sandbox but without a nix-deamon.
With this pull request it is possible to bootstrap stdenv without root-permission,
which is quiet useful in HPC environments.
- default coreutils is stripped of /share/ (11 -> 2 MiB)
- coreutils-full retains /share/ and adds openssl for faster *sum tools
- NixOS systemPackages contains coreutils-full
- *Support parameter defaults are moved inside
(it seemed confusing to have `? false` and "at once" with `? isLinux`)
Closure considerations:
+ typical build-time closure will get lighter by ~9 MiB
- typical closure of NixOS installation will grow by ~2 MiB,
due to referring to both versions. I think it would be possible to
re-use most of the utils between the two versions, but the expression
would get much more complex.
I considered having stdenv with minimal coreutils and the default
`coreutils` attribute being full, but it turned out there were too many
trivial references in nixpkgs, so it didn't seem easy to keep rebuild
impact of openssl from growing significantly.
coreutils is part of stdenv, which doesn't allow openssl currently.
It's unclear that adding openssl to stdenv was intended,
but if it was it was not discussed or mentioned.
To unbreak "all the things", reverting until this
has been discussed and a proper fix has been put together.
This reverts commit df9f76c62d, reversing
changes made to 585ded7329.
Originally this was introduced in 055e646b ('coreutils: Guard against compiler not supporting __builtin_stpncpy_chk') four years ago. Right now this doesn't seem to serve any purpose and it conflicts with the hardening flags:
<command line>:2:9: warning: '_FORTIFY_SOURCE' macro redefined [-Wmacro-redefined]
#define _FORTIFY_SOURCE 2
^
<command line>:1:9: note: previous definition is here
#define _FORTIFY_SOURCE 0
^
1 warning generated.
The main changes are in libSystem, which lost the coretls component in 10.13
and some hardening changes that quietly crash any program that uses %n in
a non-constant format string, so we've needed to patch a lot of programs that
use gnulib.
`make install` was failing presumably due to an upstream cross-compilation bug:
Makefile:3440: *** Recursive variable 'INSTALL' references itself (eventually). Stop.
While at it, remove the old ugly cross buildPhase. It was seemingly
trying to work around the fact that the man pages are generated by
piping each command's --help output to help2man. That obviously doesn't
work while cross-compiling.
Instead, set PERL=missing in the environment to make the build system
think there's no perl available to run help2man. (This approach was
stolen from buildroot.)
With the symlinked build it's only renaming 'coreutils' to
'gcoreutils' and leaving all the actual command symlinks alone.
Instead of custom renaming scripts, let's use --program-prefix
from the autotools kitchen sink. This has the nice(?) bonus that
the manpages also get prefixed with 'g' now.
The build system refuses to enable both single-binary build and
--program-prefix - but the symlink-based single-binary build couldn't
possibly work either (as it will be looking at "$0" for which command
to execute).
Our coreutils now uses single-binary-build mode where, by default,
simple shebang scripts are used for all the binaries. That doesn't work
e.g. with the Linux unpacker which only handles standard binaries and
symlinks. Let's use the symlinked mode instead for boostrapping.
This does NOT change any stdenv hashes.
I only tested the case most important to me:
$ nix-build pkgs/top-level/release.nix -A stdenvBootstrapTools.x86_64-linux.test
The following parameters are now available:
* hardeningDisable
To disable specific hardening flags
* hardeningEnable
To enable specific hardening flags
Only the cc-wrapper supports this right now, but these may be reused by
other wrappers, builders or setup hooks.
cc-wrapper supports the following flags:
* fortify
* stackprotector
* pie (disabled by default)
* pic
* strictoverflow
* format
* relro
* bindnow
The patch no longer applies as it's applied in the 8.25 release.
While at it don't use FORCE_UNSAFE_CONFIGURE on ARM as I don't see why
it's needed and it works for me without it.
Build-tested GNU hello on ARMv7 and verified that this affect its hash
on i686.
The most complex problems were from dealing with switches reverted in
the meantime (gcc5, gmp6, ncurses6).
It's likely that darwin is (still) broken nontrivially.
It was using 'null' instead of '[]' in case of no patches. So I change
it to provide 'null'.
(cherry picked from commit 56ff659e5d1fffd371786a3b8237129ee365ca5c)
I pick the patch from upstream, and I apply only for ARM to avoid rebuilding
stdenv.
(cherry picked from commit 1b36f489b6fb2bde3b241d2763abfaa860a4fe55)
- there were many easy merge conflicts
- cc-wrapper needed nontrivial changes
Many other problems might've been created by interaction of the branches,
but stdenv and a few other packages build fine now.
