dev/nixpkgs
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity

Merge branch 'master' into mjolnir-init

Browse Source
This commit is contained in:
Graham Christensen 2021-11-12 20:56:14 -05:00 committed by GitHub
commit fd1f8e1498
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
6900 changed files with 290741 additions and 190611 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

12
.github/CODEOWNERS vendored
View File

@ -65,6 +65,7 @@
/nixos/doc/manual/development/writing-modules.xml @nbp
/nixos/doc/manual/man-nixos-option.xml @nbp
/nixos/modules/installer/tools/nixos-option.sh @nbp
/nixos/modules/system @dasJ
# NixOS integration test driver
/nixos/lib/test-driver @tfc
@ -96,9 +97,9 @@
/pkgs/top-level/haskell-packages.nix @cdepillabout @sternenseemann @maralorn @expipiplus1
# Perl
/pkgs/development/interpreters/perl @volth @stigtsp
/pkgs/top-level/perl-packages.nix @volth @stigtsp
/pkgs/development/perl-modules @volth @stigtsp
/pkgs/development/interpreters/perl @volth @stigtsp @zakame
/pkgs/top-level/perl-packages.nix @volth @stigtsp @zakame
/pkgs/development/perl-modules @volth @stigtsp @zakame
# R
/pkgs/applications/science/math/R @jbedo @bcdarwin
@ -231,3 +232,8 @@
# Cinnamon
/pkgs/desktops/cinnamon @mkg20001
#nim
/pkgs/development/compilers/nim @ehmry
/pkgs/development/nim-packages @ehmry
/pkgs/top-level/nim-packages.nix @ehmry

8
.github/PULL_REQUEST_TEMPLATE.md vendored
View File

@ -21,9 +21,13 @@ Reviewing guidelines: https://nixos.org/manual/nixpkgs/unstable/#chap-reviewing-
- [ ] x86_64-darwin
- [ ] aarch64-darwin
- [ ] For non-Linux: Is `sandbox = true` set in `nix.conf`? (See [Nix manual](https://nixos.org/manual/nix/stable/#sec-conf-file))
- [ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside [nixos/tests](https://github.com/NixOS/nixpkgs/blob/master/nixos/tests))
- [ ] Tested, as applicable:
- [NixOS test(s)](https://nixos.org/manual/nixos/unstable/index.html#sec-nixos-tests) (look inside [nixos/tests](https://github.com/NixOS/nixpkgs/blob/master/nixos/tests))
- and/or [package tests](https://nixos.org/manual/nixpkgs/unstable/#sec-package-tests)
- or, for functions and "core" functionality, tests in [lib/tests](https://github.com/NixOS/nixpkgs/blob/master/lib/tests) or [pkgs/test](https://github.com/NixOS/nixpkgs/blob/master/pkgs/test)
- made sure NixOS tests are [linked](https://nixos.org/manual/nixpkgs/unstable/#ssec-nixos-tests-linking) to the relevant packages
- [ ] Tested compilation of all packages that depend on this change using `nix-shell -p nixpkgs-review --run "nixpkgs-review wip"`
- [ ] Tested execution of all binary files (usually in `./result/bin/`)
- [ ] Tested basic functionality of all binary files (usually in `./result/bin/`)
- [21.11 Release Notes (or backporting 21.05 Release notes)](https://github.com/NixOS/nixpkgs/blob/master/CONTRIBUTING.md#generating-2111-release-notes)
- [ ] (Package updates) Added a release notes entry if the change is major or breaking
- [ ] (Module updates) Added a release notes entry if the change is significant

6
.github/labeler.yml vendored
View File

@ -72,6 +72,12 @@
- nixos/**/*
- pkgs/os-specific/linux/nixos-rebuild/**/*
"6.topic: nim":
- doc/languages-frameworks/nim.section.md
- pkgs/development/compilers/nim/*
- pkgs/development/nim-packages/**/*
- pkgs/top-level/nim-packages.nix
"6.topic: ocaml":
- doc/languages-frameworks/ocaml.section.md
- pkgs/development/compilers/ocaml/**/*

4
.github/workflows/periodic-merge-24h.yml vendored
View File

@ -28,6 +28,10 @@ jobs:
pairs:
- from: master
into: haskell-updates
- from: release-21.05
into: staging-next-21.05
- from: staging-next-21.05
into: staging-21.05
name: ${{ matrix.pairs.from }} → ${{ matrix.pairs.into }}
steps:
- uses: actions/checkout@v2

4
.github/workflows/periodic-merge-6h.yml vendored
View File

@ -30,10 +30,6 @@ jobs:
into: staging-next
- from: staging-next
into: staging
- from: release-21.05
into: staging-next-21.05
- from: staging-next-21.05
into: staging-21.05
name: ${{ matrix.pairs.from }} → ${{ matrix.pairs.into }}
steps:
- uses: actions/checkout@v2

4
README.md
View File

@ -8,7 +8,7 @@
</p>
[Nixpkgs](https://github.com/nixos/nixpkgs) is a collection of over
60,000 software packages that can be installed with the
80,000 software packages that can be installed with the
[Nix](https://nixos.org/nix/) package manager. It also implements
[NixOS](https://nixos.org/nixos/), a purely-functional Linux distribution.
@ -53,7 +53,7 @@ system, [Hydra](https://hydra.nixos.org/).
Artifacts successfully built with Hydra are published to cache at
https://cache.nixos.org/. When successful build and test criteria are
met, the Nixpkgs expressions are distributed via [Nix
channels](https://nixos.org/nix/manual/#sec-channels).
channels](https://nixos.org/manual/nix/stable/package-management/channels.html).
# Contributing

14
doc/builders/fetchers.chapter.md
View File

@ -1,8 +1,16 @@
# Fetchers {#chap-pkgs-fetchers}
When using Nix, you will frequently need to download source code and other files from the internet. Nixpkgs comes with a few helper functions that allow you to fetch fixed-output derivations in a structured way.
When using Nix, you will frequently need to download source code and other files from the internet. For this purpose, Nix provides the [_fixed output derivation_](https://nixos.org/manual/nix/stable/#fixed-output-drvs) feature and Nixpkgs provides various functions that implement the actual fetching from various protocols and services.
The two fetcher primitives are `fetchurl` and `fetchzip`. Both of these have two required arguments, a URL and a hash. The hash is typically `sha256`, although many more hash algorithms are supported. Nixpkgs contributors are currently recommended to use `sha256`. This hash will be used by Nix to identify your source. A typical usage of fetchurl is provided below.
## Caveats
Because fixed output derivations are _identified_ by their hash, a common mistake is to update a fetcher's URL or a version parameter, without updating the hash. **This will cause the old contents to be used.** So remember to always invalidate the hash argument.
For those who develop and maintain fetchers, a similar problem arises with changes to the implementation of a fetcher. These may cause a fixed output derivation to fail, but won't normally be caught by tests because the supposed output is already in the store or cache. For the purpose of testing, you can use a trick that is embodied by the [`invalidateFetcherByDrvHash`](#sec-pkgs-invalidateFetcherByDrvHash) function. It uses the derivation `name` to create a unique output path per fetcher implementation, defeating the caching precisely where it would be harmful.
## `fetchurl` and `fetchzip` {#fetchurl}
Two basic fetchers are `fetchurl` and `fetchzip`. Both of these have two required arguments, a URL and a hash. The hash is typically `sha256`, although many more hash algorithms are supported. Nixpkgs contributors are currently recommended to use `sha256`. This hash will be used by Nix to identify your source. A typical usage of fetchurl is provided below.
```nix
{ stdenv, fetchurl }:
@ -20,7 +28,7 @@ The main difference between `fetchurl` and `fetchzip` is in how they store the c
`fetchpatch` works very similarly to `fetchurl` with the same arguments expected. It expects patch files as a source and performs normalization on them before computing the checksum. For example it will remove comments or other unstable parts that are sometimes added by version control systems and can change over time.
Other fetcher functions allow you to add source code directly from a VCS such as subversion or git. These are mostly straightforward nambes based on the name of the command used with the VCS system. Because they give you a working repository, they act most like `fetchzip`.
Most other fetchers return a directory rather than a single file.
## `fetchsvn` {#fetchsvn}

18
doc/builders/packages/etc-files.section.md Normal file
View File

@ -0,0 +1,18 @@
# /etc files {#etc}
Certain calls in glibc require access to runtime files found in /etc such as `/etc/protocols` or `/etc/services` -- [getprotobyname](https://linux.die.net/man/3/getprotobyname) is one such function.
On non-NixOS distributions these files are typically provided by packages (i.e. [netbase](https://packages.debian.org/sid/netbase)) if not already pre-installed in your distribution. This can cause non-reproducibility for code if they rely on these files being present.
If [iana-etc](https://hydra.nixos.org/job/nixos/trunk-combined/nixpkgs.iana-etc.x86_64-linux) is part of your _buildInputs_ then it will set the environment varaibles `NIX_ETC_PROTOCOLS` and `NIX_ETC_SERVICES` to the corresponding files in the package through a _setup-hook_.
```bash
> nix-shell -p iana-etc
[nix-shell:~]$ env | grep NIX_ETC
NIX_ETC_SERVICES=/nix/store/aj866hr8fad8flnggwdhrldm0g799ccz-iana-etc-20210225/etc/services
NIX_ETC_PROTOCOLS=/nix/store/aj866hr8fad8flnggwdhrldm0g799ccz-iana-etc-20210225/etc/protocols
```
Nixpkg's version of [glibc](https://github.com/NixOS/nixpkgs/blob/master/pkgs/development/libraries/glibc/default.nix) has been patched to check for the existence of these environment variables. If the environment variable are *not set*, then it will attempt to find the files at the default location within _/etc_.

1
doc/builders/packages/index.xml
View File

@ -17,6 +17,7 @@
<xi:include href="kakoune.section.xml" />
<xi:include href="linux.section.xml" />
<xi:include href="locales.section.xml" />
<xi:include href="etc-files.section.xml" />
<xi:include href="nginx.section.xml" />
<xi:include href="opengl.section.xml" />
<xi:include href="shell-helpers.section.xml" />

1
doc/builders/special.xml
View File

@ -7,4 +7,5 @@
</para>
<xi:include href="special/fhs-environments.section.xml" />
<xi:include href="special/mkshell.section.xml" />
<xi:include href="special/invalidateFetcherByDrvHash.section.xml" />
</chapter>

31
doc/builders/special/invalidateFetcherByDrvHash.section.md Normal file
View File

@ -0,0 +1,31 @@
## `invalidateFetcherByDrvHash` {#sec-pkgs-invalidateFetcherByDrvHash}
Use the derivation hash to invalidate the output via name, for testing.
Type: `(a@{ name, ... } -> Derivation) -> a -> Derivation`
Normally, fixed output derivations can and should be cached by their output
hash only, but for testing we want to re-fetch everytime the fetcher changes.
Changes to the fetcher become apparent in the drvPath, which is a hash of
how to fetch, rather than a fixed store path.
By inserting this hash into the name, we can make sure to re-run the fetcher
every time the fetcher changes.
This relies on the assumption that Nix isn't clever enough to reuse its
database of local store contents to optimize fetching.
You might notice that the "salted" name derives from the normal invocation,
not the final derivation. `invalidateFetcherByDrvHash` has to invoke the fetcher
function twice: once to get a derivation hash, and again to produce the final
fixed output derivation.
Example:
tests.fetchgit = invalidateFetcherByDrvHash fetchgit {
name = "nix-source";
url = "https://github.com/NixOS/nix";
rev = "9d9dbe6ed05854e03811c361a3380e09183f4f4a";
sha256 = "sha256-7DszvbCNTjpzGRmpIVAWXk20P0/XTrWZ79KSOGLrUWY=";
};

22
doc/builders/trivial-builders.chapter.md
View File

@ -47,6 +47,28 @@ These functions write `text` to the Nix store. This is useful for creating scrip
Many more commands wrap `writeTextFile` including `writeText`, `writeTextDir`, `writeScript`, and `writeScriptBin`. These are convenience functions over `writeTextFile`.
## `writeShellApplication` {#trivial-builder-writeShellApplication}
This can be used to easily produce a shell script that has some dependencies (`runtimeInputs`). It automatically sets the `PATH` of the script to contain all of the listed inputs, sets some sanity shellopts (`errexit`, `nounset`, `pipefail`), and checks the resulting script with [`shellcheck`](https://github.com/koalaman/shellcheck).
For example, look at the following code:
```nix
writeShellApplication {
name = "show-nixos-org";
runtimeInputs = [ curl w3m ];
text = ''
curl -s 'https://nixos.org' | w3m -dump -T text/html
'';
}
```
Unlike with normal `writeShellScriptBin`, there is no need to manually write out `${curl}/bin/curl`, setting the PATH
was handled by `writeShellApplication`. Moreover, the script is being checked with `shellcheck` for more strict
validation.
## `symlinkJoin` {#trivial-builder-symlinkJoin}
This can be used to put many derivations into the same directory structure. It works by creating a new derivation and adding symlinks to each of the paths listed. It expects two arguments, `name`, and `paths`. `name` is the name used in the Nix store path for the created derivation. `paths` is a list of paths that will be symlinked. These paths can be to Nix store derivations or any other subdirectory contained within.

15
doc/contributing/coding-conventions.chapter.md
View File

@ -181,6 +181,21 @@
rev = "${version}";
```
- Building lists conditionally _should_ be done with `lib.optional(s)` instead of using `if cond then [ ... ] else null` or `if cond then [ ... ] else [ ]`.
```nix
buildInputs = lib.optional stdenv.isDarwin iconv;
```
instead of
```nix
buildInputs = if stdenv.isDarwin then [ iconv ] else null;
```
As an exception, an explicit conditional expression with null can be used when fixing a important bug without triggering a mass rebuild.
If this is done a follow up pull request _should_ be created to change the code to `lib.optional(s)`.
- Arguments should be listed in the order they are used, with the exception of `lib`, which always goes first.
## Package naming {#sec-package-naming}

64
doc/languages-frameworks/agda.section.md
View File

@ -158,7 +158,23 @@ This can be overridden.
By default, Agda sources are files ending on `.agda`, or literate Agda files ending on `.lagda`, `.lagda.tex`, `.lagda.org`, `.lagda.md`, `.lagda.rst`. The list of recognised Agda source extensions can be extended by setting the `extraExtensions` config variable.
## Adding Agda packages to Nixpkgs {#adding-agda-packages-to-nixpkgs}
## Maintaining the Agda package set on Nixpkgs {#maintaining-the-agda-package-set-on-nixpkgs}
We are aiming at providing all common Agda libraries as packages on `nixpkgs`,
and keeping them up to date.
Contributions and maintenance help is always appreciated,
but the maintenance effort is typically low since the Agda ecosystem is quite small.
The `nixpkgs` Agda package set tries to take up a role similar to that of [Stackage](https://www.stackage.org/) in the Haskell world.
It is a curated set of libraries that:
1. Always work together.
2. Are as up-to-date as possible.
While the Haskell ecosystem is huge, and Stackage is highly automatised,
the Agda package set is small and can (still) be maintained by hand.
### Adding Agda packages to Nixpkgs {#adding-agda-packages-to-nixpkgs}
To add an Agda package to `nixpkgs`, the derivation should be written to `pkgs/development/libraries/agda/${library-name}/` and an entry should be added to `pkgs/top-level/agda-packages.nix`. Here it is called in a scope with access to all other Agda libraries, so the top line of the `default.nix` can look like:
@ -192,3 +208,49 @@ mkDerivation {
This library has a file called `.agda-lib`, and so we give an empty string to `libraryFile` as nothing precedes `.agda-lib` in the filename. This file contains `name: IAL-1.3`, and so we let `libraryName = "IAL-1.3"`. This library does not use an `Everything.agda` file and instead has a Makefile, so there is no need to set `everythingFile` and we set a custom `buildPhase`.
When writing an Agda package it is essential to make sure that no `.agda-lib` file gets added to the store as a single file (for example by using `writeText`). This causes Agda to think that the nix store is a Agda library and it will attempt to write to it whenever it typechecks something. See [https://github.com/agda/agda/issues/4613](https://github.com/agda/agda/issues/4613).
In the pull request adding this library,
you can test whether it builds correctly by writing in a comment:
```
@ofborg build agdaPackages.iowa-stdlib
```
### Maintaining Agda packages
As mentioned before, the aim is to have a compatible, and up-to-date package set.
These two conditions sometimes exclude each other:
For example, if we update `agdaPackages.standard-library` because there was an upstream release,
this will typically break many reverse dependencies,
i.e. downstream Agda libraries that depend on the standard library.
In `nixpkgs` we are typically among the first to notice this,
since we have build tests in place to check this.
In a pull request updating e.g. the standard library, you should write the following comment:
```
@ofborg build agdaPackages.standard-library.passthru.tests
```
This will build all reverse dependencies of the standard library,
for example `agdaPackages.agda-categories`, or `agdaPackages.generic`.
In some cases it is useful to build _all_ Agda packages.
This can be done with the following Github comment:
```
@ofborg build agda.passthru.tests.allPackages
```
Sometimes, the builds of the reverse dependencies fail because they have not yet been updated and released.
You should drop the maintainers a quick issue notifying them of the breakage,
citing the build error (which you can get from the ofborg logs).
If you are motivated, you might even send a pull request that fixes it.
Usually, the maintainers will answer within a week or two with a new release.
Bumping the version of that reverse dependency should be a further commit on your PR.
In the rare case that a new release is not to be expected within an acceptable time,
simply mark the broken package as broken by setting `meta.broken = true;`.
This will exclude it from the build test.
It can be added later when it is fixed,
and does not hinder the advancement of the whole package set in the meantime.

4
doc/languages-frameworks/crystal.section.md
View File

@ -4,12 +4,12 @@
This section uses [Mint](https://github.com/mint-lang/mint) as an example for how to build a Crystal package.
If the Crystal project has any dependencies, the first step is to get a `shards.nix` file encoding those. Get a copy of the project and go to its root directory such that its `shard.lock` file is in the current directory, then run `crystal2nix` in it
If the Crystal project has any dependencies, the first step is to get a `shards.nix` file encoding those. Get a copy of the project and go to its root directory such that its `shard.lock` file is in the current directory. Executable projects should usually commit the `shard.lock` file, but sometimes that's not the case, which means you need to generate it yourself. With an existing `shard.lock` file, `crystal2nix` can be run.
```bash
$ git clone https://github.com/mint-lang/mint
$ cd mint
$ git checkout 0.5.0
$ if [ ! -f shard.lock ]; then nix-shell -p shards --run "shards lock"; fi
$ nix-shell -p crystal2nix --run crystal2nix
```

61
doc/languages-frameworks/dhall.section.md
View File

@ -50,7 +50,7 @@ expression does not protect the Prelude import with a semantic integrity
check, so the first step is to freeze the expression using `dhall freeze`,
like this:
```bash
```ShellSession
$ dhall freeze --inplace ./true.dhall
```
@ -113,7 +113,7 @@ in
… which we can then build using this command:
```bash
```ShellSession
$ nix build --file ./example.nix dhallPackages.true
```
@ -121,7 +121,7 @@ $ nix build --file ./example.nix dhallPackages.true
The above package produces the following directory tree:
```bash
```ShellSession
$ tree -a ./result
result
├── .cache
@ -135,7 +135,7 @@ result
* `source.dhall` contains the result of interpreting our Dhall package:
```bash
```ShellSession
$ cat ./result/source.dhall
True
```
@ -143,7 +143,7 @@ result
* The `.cache` subdirectory contains one binary cache product encoding the
same result as `source.dhall`:
```bash
```ShellSession
$ dhall decode < ./result/.cache/dhall/122027abdeddfe8503496adeb623466caa47da5f63abd2bc6fa19f6cfcb73ecfed70
True
```
@ -151,7 +151,7 @@ result
* `binary.dhall` contains a Dhall expression which handles fetching and decoding
the same cache product:
```bash
```ShellSession
$ cat ./result/binary.dhall
missing sha256:27abdeddfe8503496adeb623466caa47da5f63abd2bc6fa19f6cfcb73ecfed70
$ cp -r ./result/.cache .cache
@ -168,7 +168,7 @@ to conserve disk space when they are used exclusively as dependencies. For
example, if we build the Prelude package it will only contain the binary
encoding of the expression:
```bash
```ShellSession
$ nix build --file ./example.nix dhallPackages.Prelude
$ tree -a result
@ -199,7 +199,7 @@ Dhall overlay like this:
… and now the Prelude will contain the fully decoded result of interpreting
the Prelude:
```bash
```ShellSession
$ nix build --file ./example.nix dhallPackages.Prelude
$ tree -a result
@ -302,7 +302,7 @@ Additionally, `buildDhallGitHubPackage` accepts the same arguments as
You can use the `dhall-to-nixpkgs` command-line utility to automate
packaging Dhall code. For example:
```bash
```ShellSession
$ nix-env --install --attr haskellPackages.dhall-nixpkgs
$ nix-env --install --attr nix-prefetch-git # Used by dhall-to-nixpkgs
@ -329,12 +329,12 @@ The utility takes care of automatically detecting remote imports and converting
them to package dependencies. You can also use the utility on local
Dhall directories, too:
```bash
```ShellSession
$ dhall-to-nixpkgs directory ~/proj/dhall-semver
{ buildDhallDirectoryPackage, Prelude }:
buildDhallDirectoryPackage {
name = "proj";
src = /Users/gabriel/proj/dhall-semver;
src = ~/proj/dhall-semver;
file = "package.dhall";
source = false;
document = false;
@ -342,6 +342,37 @@ $ dhall-to-nixpkgs directory ~/proj/dhall-semver
}
```
### Remote imports as fixed-output derivations {#ssec-dhall-remote-imports-as-fod}
`dhall-to-nixpkgs` has the ability to fetch and build remote imports as
fixed-output derivations by using their Dhall integrity check. This is
sometimes easier than manually packaging all remote imports.
This can be used like the following:
```ShellSession
$ dhall-to-nixpkgs directory --fixed-output-derivations ~/proj/dhall-semver
{ buildDhallDirectoryPackage, buildDhallUrl }:
buildDhallDirectoryPackage {
name = "proj";
src = ~/proj/dhall-semver;
file = "package.dhall";
source = false;
document = false;
dependencies = [
(buildDhallUrl {
url = "https://prelude.dhall-lang.org/v17.0.0/package.dhall";
hash = "sha256-ENs8kZwl6QRoM9+Jeo/+JwHcOQ+giT2VjDQwUkvlpD4=";
dhallHash = "sha256:10db3c919c25e9046833df897a8ffe2701dc390fa0893d958c3430524be5a43e";
})
];
}
```
Here, `dhall-semver`'s `Prelude` dependency is fetched and built with the
`buildDhallUrl` helper function, instead of being passed in as a function
argument.
## Overriding dependency versions {#ssec-dhall-overriding-dependency-versions}
Suppose that we change our `true.dhall` example expression to depend on an older
@ -359,7 +390,7 @@ in Prelude.Bool.not False
If we try to rebuild that expression the build will fail:
```
```ShellSession
$ nix build --file ./example.nix dhallPackages.true
builder for '/nix/store/0f1hla7ff1wiaqyk1r2ky4wnhnw114fi-true.drv' failed with exit code 1; last 10 log lines:
@ -385,7 +416,7 @@ importing the URL.
However, we can override the default Prelude version by using `dhall-to-nixpkgs`
to create a Dhall package for our desired Prelude:
```bash
```ShellSession
$ dhall-to-nixpkgs github https://github.com/dhall-lang/dhall-lang.git \
--name Prelude \
--directory Prelude \
@ -396,7 +427,7 @@ $ dhall-to-nixpkgs github https://github.com/dhall-lang/dhall-lang.git \
… and then referencing that package in our Dhall overlay, by either overriding
the Prelude globally for all packages, like this:
```bash
```nix
dhallOverrides = self: super: {
true = self.callPackage ./true.nix { };
@ -407,7 +438,7 @@ the Prelude globally for all packages, like this:
… or selectively overriding the Prelude dependency for just the `true` package,
like this:
```bash
```nix
dhallOverrides = self: super: {
true = self.callPackage ./true.nix {
Prelude = self.callPackage ./Prelude.nix { };

49
doc/languages-frameworks/dotnet.section.md
View File

@ -28,8 +28,7 @@ mkShell {
packages = [
(with dotnetCorePackages; combinePackages [
sdk_3_1
sdk_3_0
sdk_2_1
sdk_5_0
])
];
}
@ -64,12 +63,50 @@ $ dotnet --info
The `dotnetCorePackages.sdk_X_Y` is preferred over the old dotnet-sdk as both major and minor version are very important for a dotnet environment. If a given minor version isn't present (or was changed), then this will likely break your ability to build a project.
## dotnetCorePackages.sdk vs dotnetCorePackages.net vs dotnetCorePackages.netcore vs dotnetCorePackages.aspnetcore {#dotnetcorepackages.sdk-vs-dotnetcorepackages.net-vs-dotnetcorepackages.netcore-vs-dotnetcorepackages.aspnetcore}
## dotnetCorePackages.sdk vs dotnetCorePackages.runtime vs dotnetCorePackages.aspnetcore {#dotnetcorepackages.sdk-vs-dotnetcorepackages.runtime-vs-dotnetcorepackages.aspnetcore}
The `dotnetCorePackages.sdk` contains both a runtime and the full sdk of a given version. The `net`, `netcore` and `aspnetcore` packages are meant to serve as minimal runtimes to deploy alongside already built applications. For runtime versions >= .NET 5 `net` is used while `netcore` is used for older .NET Core runtime version.
The `dotnetCorePackages.sdk` contains both a runtime and the full sdk of a given version. The `runtime` and `aspnetcore` packages are meant to serve as minimal runtimes to deploy alongside already built applications.
## Packaging a Dotnet Application {#packaging-a-dotnet-application}
Ideally, we would like to build against the sdk, then only have the dotnet runtime available in the runtime closure.
To package Dotnet applications, you can use `buildDotnetModule`. This has similar arguments to `stdenv.mkDerivation`, with the following additions:
TODO: Create closure-friendly way to package dotnet applications
* `projectFile` has to be used for specifying the dotnet project file relative to the source root. These usually have `.sln` or `.csproj` file extensions.
* `nugetDeps` has to be used to specify the NuGet dependency file. Unfortunately, these cannot be deterministically fetched without a lockfile. This file should be generated using `nuget-to-nix` tool, which is available in nixpkgs.
* `executables` is used to specify which executables get wrapped to `$out/bin`, relative to `$out/lib/$pname`. If this is unset, all executables generated will get installed. If you do not want to install any, set this to `[]`.
* `runtimeDeps` is used to wrap libraries into `LD_LIBRARY_PATH`. This is how dotnet usually handles runtime dependencies.
* `buildType` is used to change the type of build. Possible values are `Release`, `Debug`, etc. By default, this is set to `Release`.
* `dotnet-sdk` is useful in cases where you need to change what dotnet SDK is being used.
* `dotnet-runtime` is useful in cases where you need to change what dotnet runtime is being used. This can be either a regular dotnet runtime, or an aspnetcore.
* `dotnet-test-sdk` is useful in cases where unit tests expect a different dotnet SDK. By default, this is set to the `dotnet-sdk` attribute.
* `testProjectFile` is useful in cases where the regular project file does not contain the unit tests. By default, this is set to the `projectFile` attribute.
* `disabledTests` is used to disable running specific unit tests. This gets passed as: `dotnet test --filter "FullyQualifiedName!={}"`, to ensure compatibility with all unit test frameworks.
* `dotnetRestoreFlags` can be used to pass flags to `dotnet restore`.
* `dotnetBuildFlags` can be used to pass flags to `dotnet build`.
* `dotnetTestFlags` can be used to pass flags to `dotnet test`.
* `dotnetInstallFlags` can be used to pass flags to `dotnet install`.
* `dotnetFlags` can be used to pass flags to all of the above phases.
Here is an example `default.nix`, using some of the previously discussed arguments:
```nix
{ lib, buildDotnetModule, dotnetCorePackages, ffmpeg }:
buildDotnetModule rec {
pname = "someDotnetApplication";
version = "0.1";
src = ./.;
projectFile = "src/project.sln";
nugetDeps = ./deps.nix; # File generated with `nuget-to-nix path/to/src > deps.nix`.
dotnet-sdk = dotnetCorePackages.sdk_3_1;
dotnet-runtime = dotnetCorePackages.net_5_0;
dotnetFlags = [ "--runtime linux-x64" ];
executables = [ "foo" ]; # This wraps "$out/lib/$pname/foo" to `$out/bin/foo`.
executables = []; # Don't install any executables.
runtimeDeps = [ ffmpeg ]; # This will wrap ffmpeg's library path into `LD_LIBRARY_PATH`.
}
```

2
doc/languages-frameworks/index.xml
View File

@ -12,6 +12,7 @@
<xi:include href="coq.section.xml" />
<xi:include href="crystal.section.xml" />
<xi:include href="dhall.section.xml" />
<xi:include href="dotnet.section.xml" />
<xi:include href="emscripten.section.xml" />
<xi:include href="gnome.section.xml" />
<xi:include href="go.section.xml" />
@ -23,6 +24,7 @@
<xi:include href="javascript.section.xml" />
<xi:include href="lua.section.xml" />
<xi:include href="maven.section.xml" />
<xi:include href="nim.section.xml" />
<xi:include href="ocaml.section.xml" />
<xi:include href="octave.section.xml" />
<xi:include href="perl.section.xml" />

9
doc/languages-frameworks/java.section.md
View File

@ -72,6 +72,15 @@ in
...
```
You can also specify what JDK your JRE should be based on, for example
selecting a 'headless' build to avoid including a link to GTK+:
```nix
my_jre = pkgs.jre_minimal.override {
jdk = jdk11_headless;
};
```
Note all JDKs passthru `home`, so if your application requires
environment variables like `JAVA_HOME` being set, that can be done in a
generic fashion with the `--set` argument of `makeWrapper`:

91
doc/languages-frameworks/nim.section.md Normal file
View File

@ -0,0 +1,91 @@
# Nim {#nim}
## Overview {#nim-overview}
The Nim compiler, a builder function, and some packaged libraries are available
in Nixpkgs. Until now each compiler release has been effectively backwards
compatible so only the latest version is available.
## Nim program packages in Nixpkgs {#nim-program-packages-in-nixpkgs}
Nim programs can be built using `nimPackages.buildNimPackage`. In the
case of packages not containing exported library code the attribute
`nimBinOnly` should be set to `true`.
The following example shows a Nim program that depends only on Nim libraries:
```nix
{ lib, nimPackages, fetchurl }:
nimPackages.buildNimPackage rec {
pname = "hottext";
version = "1.4";
nimBinOnly = true;
src = fetchurl {
url = "https://git.sr.ht/~ehmry/hottext/archive/v${version}.tar.gz";
sha256 = "sha256-hIUofi81zowSMbt1lUsxCnVzfJGN3FEiTtN8CEFpwzY=";
};
buildInputs = with nimPackages; [
bumpy
chroma
flatty
nimsimd
pixie
sdl2
typography
vmath
zippy
];
}
```
## Nim library packages in Nixpkgs {#nim-library-packages-in-nixpkgs}
Nim libraries can also be built using `nimPackages.buildNimPackage`, but
often the product of a fetcher is sufficient to satisfy a dependency.
The `fetchgit`, `fetchFromGitHub`, and `fetchNimble` functions yield an
output that can be discovered during the `configurePhase` of `buildNimPackage`.
Nim library packages are listed in
[pkgs/top-level/nim-packages.nix](https://github.com/NixOS/nixpkgs/blob/master/pkgs/top-level/nim-packages.nix) and implemented at
[pkgs/development/nim-packages](https://github.com/NixOS/nixpkgs/tree/master/pkgs/development/nim-packages).
The following example shows a Nim library that propagates a dependency on a
non-Nim package:
```nix
{ lib, buildNimPackage, fetchNimble, SDL2 }:
buildNimPackage rec {
pname = "sdl2";
version = "2.0.4";
src = fetchNimble {
inherit pname version;
hash = "sha256-Vtcj8goI4zZPQs2TbFoBFlcR5UqDtOldaXSH/+/xULk=";
};
propagatedBuildInputs = [ SDL2 ];
}
```
## `buildNimPackage` parameters {#buildnimpackage-parameters}
All parameters from `stdenv.mkDerivation` function are still supported. The
following are specific to `buildNimPackage`:
* `nimBinOnly ? false`: If `true` then build only the programs listed in
the Nimble file in the packages sources.
* `nimbleFile`: Specify the Nimble file location of the package being built
rather than discover the file at build-time.
* `nimRelease ? true`: Build the package in *release* mode.
* `nimDefines ? []`: A list of Nim defines. Key-value tuples are not supported.
* `nimFlags ? []`: A list of command line arguments to pass to the Nim compiler.
Use this to specify defines with arguments in the form of `-d:${name}=${value}`.
* `nimDoc` ? false`: Build and install HTML documentation.
* `buildInputs` ? []: The packages listed here will be searched for `*.nimble`
files which are used to populate the Nim library path. Otherwise the standard
behavior is in effect.

4
doc/languages-frameworks/python.section.md
View File

@ -765,7 +765,7 @@ and in this case the `python38` interpreter is automatically used.
### Interpreters {#interpreters}
Versions 2.7, 3.6, 3.7, 3.8 and 3.9 of the CPython interpreter are available as
respectively `python27`, `python36`, `python37`, `python38` and `python39`. The
respectively `python27`, `python37`, `python38` and `python39`. The
aliases `python2` and `python3` correspond to respectively `python27` and
`python39`. The attribute `python` maps to `python2`. The PyPy interpreters
compatible with Python 2.7 and 3 are available as `pypy27` and `pypy3`, with
@ -830,10 +830,10 @@ attribute set is created for each available Python interpreter. The available
sets are
* `pkgs.python27Packages`
* `pkgs.python36Packages`
* `pkgs.python37Packages`
* `pkgs.python38Packages`
* `pkgs.python39Packages`
* `pkgs.python310Packages`
* `pkgs.pypyPackages`
and the aliases

20
doc/languages-frameworks/r.section.md
View File

@ -96,6 +96,11 @@ re-enter the shell.
## Updating the package set {#updating-the-package-set}
There is a script and associated environment for regenerating the package
sets and synchronising the rPackages tree to the current CRAN and matching
BIOC release. These scripts are found in the `pkgs/development/r-modules`
directory and executed as follows:
```bash
nix-shell generate-shell.nix
@ -112,12 +117,11 @@ Rscript generate-r-packages.R bioc-experiment > bioc-experiment-packages.nix.new
mv bioc-experiment-packages.nix.new bioc-experiment-packages.nix
```
`generate-r-packages.R <repo>` reads `<repo>-packages.nix`, therefor the renaming.
`generate-r-packages.R <repo>` reads `<repo>-packages.nix`, therefore
the renaming.
## Testing if the Nix-expression could be evaluated {#testing-if-the-nix-expression-could-be-evaluated}
```bash
nix-build test-evaluation.nix --dry-run
```
If this exits fine, the expression is ok. If not, you have to edit `default.nix`
Some packages require overrides to specify external dependencies or other
patches and special requirements. These overrides are specified in the
`pkgs/development/r-modules/default.nix` file. As the `*-packages.nix`
contents are automatically generated it should not be edited and broken
builds should be addressed using overrides.

2
doc/languages-frameworks/ruby.section.md
View File

@ -8,7 +8,7 @@ In the Nixpkgs tree, Ruby packages can be found throughout, depending on what th
There are two main approaches for using Ruby with gems. One is to use a specifically locked `Gemfile` for an application that has very strict dependencies. The other is to depend on the common gems, which we'll explain further down, and rely on them being updated regularly.
The interpreters have common attributes, namely `gems`, and `withPackages`. So you can refer to `ruby.gems.nokogiri`, or `ruby_2_6.gems.nokogiri` to get the Nokogiri gem already compiled and ready to use.
The interpreters have common attributes, namely `gems`, and `withPackages`. So you can refer to `ruby.gems.nokogiri`, or `ruby_2_7.gems.nokogiri` to get the Nokogiri gem already compiled and ready to use.
Since not all gems have executables like `nokogiri`, it's usually more convenient to use the `withPackages` function like this: `ruby.withPackages (p: with p; [ nokogiri ])`. This will also make sure that the Ruby in your environment will be able to find the gem and it can be used in your Ruby code (for example via `ruby` or `irb` executables) via `require "nokogiri"` as usual.

197
doc/languages-frameworks/rust.section.md
View File

@ -13,7 +13,7 @@ into your `configuration.nix` or bring them into scope with `nix-shell -p rustc
For other versions such as daily builds (beta and nightly),
use either `rustup` from nixpkgs (which will manage the rust installation in your home directory),
or use Mozilla's [Rust nightlies overlay](#using-the-rust-nightlies-overlay).
or use a community maintained [Rust overlay](#using-community-rust-overlays).
## Compiling Rust applications with Cargo {#compiling-rust-applications-with-cargo}
@ -116,22 +116,44 @@ is updated after every change to `Cargo.lock`. Therefore,
a `Cargo.lock` file using the `cargoLock` argument. For example:
```nix
rustPlatform.buildRustPackage rec {
rustPlatform.buildRustPackage {
pname = "myproject";
version = "1.0.0";
cargoLock = {
lockFile = ./Cargo.lock;
}
};
# ...
}
```
This will retrieve the dependencies using fixed-output derivations from
the specified lockfile. Note that setting `cargoLock.lockFile` doesn't
add a `Cargo.lock` to your `src`, and a `Cargo.lock` is still required
to build a rust package. A simple fix is to use:
the specified lockfile.
One caveat is that `Cargo.lock` cannot be patched in the `patchPhase`
because it runs after the dependencies have already been fetched. If
you need to patch or generate the lockfile you can alternatively set
`cargoLock.lockFileContents` to a string of its contents:
```nix
rustPlatform.buildRustPackage {
pname = "myproject";
version = "1.0.0";
cargoLock = let
fixupLockFile = path: f (builtins.readFile path);
in {
lockFileContents = fixupLockFile ./Cargo.lock;
};
# ...
}
```
Note that setting `cargoLock.lockFile` or `cargoLock.lockFileContents`
doesn't add a `Cargo.lock` to your `src`, and a `Cargo.lock` is still
required to build a rust package. A simple fix is to use:
```nix
postPatch = ''
@ -215,22 +237,6 @@ where they are known to differ. But there are ways to customize the argument:
--target /nix/store/asdfasdfsadf-thumb-crazy.json # contains {"foo":"","bar":""}
```
Finally, as an ad-hoc escape hatch, a computed target (string or JSON file
path) can be passed directly to `buildRustPackage`:
```nix
pkgs.rustPlatform.buildRustPackage {
/* ... */
target = "x86_64-fortanix-unknown-sgx";
}
```
This is useful to avoid rebuilding Rust tools, since they are actually target
agnostic and don't need to be rebuilt. But in the future, we should always
build the Rust tools and standard library crates separately so there is no
reason not to take the `stdenv.hostPlatform.rustc`-modifying approach, and the
ad-hoc escape hatch to `buildRustPackage` can be removed.
Note that currently custom targets aren't compiled with `std`, so `cargo test`
will fail. This can be ignored by adding `doCheck = false;` to your derivation.
@ -405,7 +411,7 @@ you of the correct hash.
`rustPlatform` provides the following hooks to automate Cargo builds:
* `cargoSetupHook`: configure Cargo to use depenencies vendored
* `cargoSetupHook`: configure Cargo to use dependencies vendored
through `fetchCargoTarball`. This hook uses the `cargoDeps`
environment variable to find the vendored dependencies. If a project
already vendors its dependencies, the variable `cargoVendorDir` can
@ -420,7 +426,7 @@ you of the correct hash.
* `maturinBuildHook`: use [Maturin](https://github.com/PyO3/maturin)
to build a Python wheel. Similar to `cargoBuildHook`, the optional
variable `buildAndTestSubdir` can be used to build a crate in a
Cargo workspace. Additional maturin flags can be passed through
Cargo workspace. Additional Maturin flags can be passed through
`maturinBuildFlags`.
* `cargoCheckHook`: run tests using Cargo. The build type for checks
can be set using `cargoCheckType`. Additional flags can be passed to
@ -441,7 +447,7 @@ dependencies. The build itself is then performed by
The following example outlines how the `tokenizers` Python package is
built. Since the Python package is in the `source/bindings/python`
directory of the *tokenizers* project's source archive, we use
directory of the `tokenizers` project's source archive, we use
`sourceRoot` to point the tooling to this directory:
```nix
@ -666,7 +672,7 @@ Some crates require external libraries. For crates from
`defaultCrateOverrides` package in nixpkgs itself.
Starting from that file, one can add more overrides, to add features
or build inputs by overriding the hello crate in a seperate file.
or build inputs by overriding the hello crate in a separate file.
```nix
with import <nixpkgs> {};
@ -723,7 +729,7 @@ with import <nixpkgs> {};
Actually, the overrides introduced in the previous section are more
general. A number of other parameters can be overridden:
- The version of rustc used to compile the crate:
- The version of `rustc` used to compile the crate:
```nix
(hello {}).override { rust = pkgs.rust; };
@ -736,7 +742,7 @@ general. A number of other parameters can be overridden:
(hello {}).override { release = false; };
```
- Whether to print the commands sent to rustc when building
- Whether to print the commands sent to `rustc` when building
(equivalent to `--verbose` in cargo:
```nix
@ -865,11 +871,87 @@ rustc 1.26.0-nightly (188e693b3 2018-03-26)
To see that you are using nightly.
## Using the Rust nightlies overlay {#using-the-rust-nightlies-overlay}
## Using community Rust overlays {#using-community-rust-overlays}
Mozilla provides an overlay for nixpkgs to bring a nightly version of Rust into scope.
This overlay can _also_ be used to install recent unstable or stable versions
of Rust, if desired.
There are two community maintained approaches to Rust toolchain management:
- [oxalica's Rust overlay](https://github.com/oxalica/rust-overlay)
- [fenix](https://github.com/nix-community/fenix)
Oxalica's overlay allows you to select a particular Rust version and components.
See [their documentation](https://github.com/oxalica/rust-overlay#rust-overlay) for more
detailed usage.
Fenix is an alternative to `rustup` and can also be used as an overlay.
Both oxalica's overlay and fenix better integrate with nix and cache optimizations.
Because of this and ergonomics, either of those community projects
should be preferred to the Mozilla's Rust overlay (`nixpkgs-mozilla`).
### How to select a specific `rustc` and toolchain version {#how-to-select-a-specific-rustc-and-toolchain-version}
You can consume the oxalica overlay and use it to grab a specific Rust toolchain version.
Here is an example `shell.nix` showing how to grab the current stable toolchain:
```nix
{ pkgs ? import <nixpkgs> {
overlays = [
(import (fetchTarball "https://github.com/oxalica/rust-overlay/archive/master.tar.gz"))
];
}
}:
pkgs.mkShell {
nativeBuildInputs = with pkgs; [
pkg-config
rust-bin.stable.latest.minimal
];
}
```
You can try this out by:
1. Saving that to `shell.nix`
2. Executing `nix-shell --pure --command 'rustc --version'`
As of writing, this prints out `rustc 1.56.0 (09c42c458 2021-10-18)`.
### How to use an overlay toolchain in a derivation {#how-to-use-an-overlay-toolchain-in-a-derivation}
You can also use an overlay's Rust toolchain with `buildRustPackage`.
The below snippet demonstrates invoking `buildRustPackage` with an oxalica overlay selected Rust toolchain:
```nix
with import <nixpkgs> {
overlays = [
(import (fetchTarball "https://github.com/oxalica/rust-overlay/archive/master.tar.gz"))
];
};
rustPlatform.buildRustPackage rec {
pname = "ripgrep";
version = "12.1.1";
nativeBuildInputs = [
rust-bin.stable.latest.minimal
];
src = fetchFromGitHub {
owner = "BurntSushi";
repo = "ripgrep";
rev = version;
sha256 = "1hqps7l5qrjh9f914r5i6kmcz6f1yb951nv4lby0cjnp5l253kps";
};
cargoSha256 = "03wf9r2csi6jpa7v5sw5lpxkrk4wfzwmzx7k3991q3bdjzcwnnwp";
meta = with lib; {
description = "A fast line-oriented regex search tool, similar to ag and ack";
homepage = "https://github.com/BurntSushi/ripgrep";
license = licenses.unlicense;
maintainers = [ maintainers.tailhook ];
};
}
```
Follow the below steps to try that snippet.
1. create a new directory
1. save the above snippet as `default.nix` in that directory
1. cd into that directory and run `nix-build`
### Rust overlay installation {#rust-overlay-installation}
@ -877,27 +959,15 @@ You can use this overlay by either changing your local nixpkgs configuration,
or by adding the overlay declaratively in a nix expression, e.g. in `configuration.nix`.
For more information see [the manual on installing overlays](#sec-overlays-install).
#### Imperative rust overlay installation {#imperative-rust-overlay-installation}
Clone [nixpkgs-mozilla](https://github.com/mozilla/nixpkgs-mozilla),
and create a symbolic link to the file
[rust-overlay.nix](https://github.com/mozilla/nixpkgs-mozilla/blob/master/rust-overlay.nix)
in the `~/.config/nixpkgs/overlays` directory.
```ShellSession
$ git clone https://github.com/mozilla/nixpkgs-mozilla.git
$ mkdir -p ~/.config/nixpkgs/overlays
$ ln -s $(pwd)/nixpkgs-mozilla/rust-overlay.nix ~/.config/nixpkgs/overlays/rust-overlay.nix
```
### Declarative rust overlay installation {#declarative-rust-overlay-installation}
### Declarative Rust overlay installation {#declarative-rust-overlay-installation}
This snippet shows how to use oxalica's Rust overlay.
Add the following to your `configuration.nix`, `home-configuration.nix`, `shell.nix`, or similar:
```nix
{ pkgs ? import <nixpkgs> {
overlays = [
(import (builtins.fetchTarball https://github.com/mozilla/nixpkgs-mozilla/archive/master.tar.gz))
(import (builtins.fetchTarball "https://github.com/oxalica/rust-overlay/archive/master.tar.gz"))
# Further overlays go here
];
};
@ -905,36 +975,3 @@ Add the following to your `configuration.nix`, `home-configuration.nix`, `shell.
```
Note that this will fetch the latest overlay version when rebuilding your system.
### Rust overlay usage {#rust-overlay-usage}
The overlay contains attribute sets corresponding to different versions of the rust toolchain, such as:
* `latest.rustChannels.stable`
* `latest.rustChannels.nightly`
* a function `rustChannelOf`, called as `(rustChannelOf { date = "2018-04-11"; channel = "nightly"; })`, or...
* `(nixpkgs.rustChannelOf { rustToolchain = ./rust-toolchain; })` if you have a local `rust-toolchain` file (see https://github.com/mozilla/nixpkgs-mozilla#using-in-nix-expressions for an example)
Each of these contain packages such as `rust`, which contains your usual rust development tools with the respective toolchain chosen.
For example, you might want to add `latest.rustChannels.stable.rust` to the list of packages in your configuration.
Imperatively, the latest stable version can be installed with the following command:
```ShellSession
$ nix-env -Ai nixpkgs.latest.rustChannels.stable.rust
```
Or using the attribute with nix-shell:
```ShellSession
$ nix-shell -p nixpkgs.latest.rustChannels.stable.rust
```
Substitute the `nixpkgs` prefix with `nixos` on NixOS.
To install the beta or nightly channel, "stable" should be substituted by
"nightly" or "beta", or
use the function provided by this overlay to pull a version based on a
build date.
The overlay automatically updates itself as it uses the same source as
[rustup](https://www.rustup.rs/).

4
doc/stdenv/cross-compilation.chapter.md
View File

@ -158,9 +158,9 @@ One would think that `localSystem` and `crossSystem` overlap horribly with the t
### Implementation of dependencies {#ssec-cross-dependency-implementation}
The categories of dependencies developed in [](#ssec-cross-dependency-categorization) are specified as lists of derivations given to `mkDerivation`, as documented in [](#ssec-stdenv-dependencies). In short, each list of dependencies for "host → target" of "foo → bar" is called `depsFooBar`, with exceptions for backwards compatibility that `depsBuildHost` is instead called `nativeBuildInputs` and `depsHostTarget` is instead called `buildInputs`. Nixpkgs is now structured so that each `depsFooBar` is automatically taken from `pkgsFooBar`. (These `pkgsFooBar`s are quite new, so there is no special case for `nativeBuildInputs` and `buildInputs`.) For example, `pkgsBuildHost.gcc` should be used at build-time, while `pkgsHostTarget.gcc` should be used at run-time.
The categories of dependencies developed in [](#ssec-cross-dependency-categorization) are specified as lists of derivations given to `mkDerivation`, as documented in [](#ssec-stdenv-dependencies). In short, each list of dependencies for "host → target" is called `deps<host><target>` (where `host`, and `target` values are either `build`, `host`, or `target`), with exceptions for backwards compatibility that `depsBuildHost` is instead called `nativeBuildInputs` and `depsHostTarget` is instead called `buildInputs`. Nixpkgs is now structured so that each `deps<host><target>` is automatically taken from `pkgs<host><target>`. (These `pkgs<host><target>`s are quite new, so there is no special case for `nativeBuildInputs` and `buildInputs`.) For example, `pkgsBuildHost.gcc` should be used at build-time, while `pkgsHostTarget.gcc` should be used at run-time.
Now, for most of Nixpkgs's history, there were no `pkgsFooBar` attributes, and most packages have not been refactored to use it explicitly. Prior to those, there were just `buildPackages`, `pkgs`, and `targetPackages`. Those are now redefined as aliases to `pkgsBuildHost`, `pkgsHostTarget`, and `pkgsTargetTarget`. It is acceptable, even recommended, to use them for libraries to show that the host platform is irrelevant.
Now, for most of Nixpkgs's history, there were no `pkgs<host><target>` attributes, and most packages have not been refactored to use it explicitly. Prior to those, there were just `buildPackages`, `pkgs`, and `targetPackages`. Those are now redefined as aliases to `pkgsBuildHost`, `pkgsHostTarget`, and `pkgsTargetTarget`. It is acceptable, even recommended, to use them for libraries to show that the host platform is irrelevant.
But before that, there was just `pkgs`, even though both `buildInputs` and `nativeBuildInputs` existed. \[Cross barely worked, and those were implemented with some hacks on `mkDerivation` to override dependencies.\] What this means is the vast majority of packages do not use any explicit package set to populate their dependencies, just using whatever `callPackage` gives them even if they do correctly sort their dependencies into the multiple lists described above. And indeed, asking that users both sort their dependencies, _and_ take them from the right attribute set, is both too onerous and redundant, so the recommended approach (for now) is to continue just categorizing by list and not using an explicit package set.

2
doc/stdenv/multiple-output.chapter.md
View File

@ -79,7 +79,7 @@ A commonly adopted convention in `nixpkgs` is that executables provided by the p
The `glibc` package is a deliberate single exception to the “binaries first” convention. The `glibc` has `libs` as its first output allowing the libraries provided by `glibc` to be referenced directly (e.g. `${stdenv.glibc}/lib/ld-linux-x86-64.so.2`). The executables provided by `glibc` can be accessed via its `bin` attribute (e.g. `${stdenv.glibc.bin}/bin/ldd`).
The reason for why `glibc` deviates from the convention is because referencing a library provided by `glibc` is a very common operation among Nix packages. For instance, third-party executables packaged by Nix are typically patched and relinked with the relevant version of `glibc` libraries from Nix packages (please see the documentation on [patchelf](https://github.com/NixOS/patchelf/blob/master/README) for more details).
The reason for why `glibc` deviates from the convention is because referencing a library provided by `glibc` is a very common operation among Nix packages. For instance, third-party executables packaged by Nix are typically patched and relinked with the relevant version of `glibc` libraries from Nix packages (please see the documentation on [patchelf](https://github.com/NixOS/patchelf) for more details).
### File type groups {#multiple-output-file-type-groups}

37
doc/stdenv/stdenv.chapter.md
View File

@ -116,15 +116,27 @@ On Linux, `stdenv` also includes the `patchelf` utility.
## Specifying dependencies {#ssec-stdenv-dependencies}
As described in the Nix manual, almost any `*.drv` store path in a derivation’s attribute set will induce a dependency on that derivation. `mkDerivation`, however, takes a few attributes intended to, between them, include all the dependencies of a package. This is done both for structure and consistency, but also so that certain other setup can take place. For example, certain dependencies need their bin directories added to the `PATH`. That is built-in, but other setup is done via a pluggable mechanism that works in conjunction with these dependency attributes. See [](#ssec-setup-hooks) for details.
As described in the Nix manual, almost any `*.drv` store path in a derivation’s attribute set will induce a dependency on that derivation. `mkDerivation`, however, takes a few attributes intended to include all the dependencies of a package. This is done both for structure and consistency, but also so that certain other setup can take place. For example, certain dependencies need their bin directories added to the `PATH`. That is built-in, but other setup is done via a pluggable mechanism that works in conjunction with these dependency attributes. See [](#ssec-setup-hooks) for details.
Dependencies can be broken down along three axes: their host and target platforms relative to the new derivation’s, and whether they are propagated. The platform distinctions are motivated by cross compilation; see [](#chap-cross) for exactly what each platform means. [^footnote-stdenv-ignored-build-platform] But even if one is not cross compiling, the platforms imply whether or not the dependency is needed at run-time or build-time, a concept that makes perfect sense outside of cross compilation. By default, the run-time/build-time distinction is just a hint for mental clarity, but with `strictDeps` set it is mostly enforced even in the native case.
The extension of `PATH` with dependencies, alluded to above, proceeds according to the relative platforms alone. The process is carried out only for dependencies whose host platform matches the new derivation’s build platform i.e. dependencies which run on the platform where the new derivation will be built. [^footnote-stdenv-native-dependencies-in-path] For each dependency \<dep\> of those dependencies, `dep/bin`, if present, is added to the `PATH` environment variable.
The dependency is propagated when it forces some of its other-transitive (non-immediate) downstream dependencies to also take it on as an immediate dependency. Nix itself already takes a package’s transitive dependencies into account, but this propagation ensures nixpkgs-specific infrastructure like setup hooks (mentioned above) also are run as if the propagated dependency.
A dependency is said to be **propagated** when some of its other-transitive (non-immediate) downstream dependencies also need it as an immediate dependency.
[^footnote-stdenv-propagated-dependencies]
It is important to note that dependencies are not necessarily propagated as the same sort of dependency that they were before, but rather as the corresponding sort so that the platform rules still line up. The exact rules for dependency propagation can be given by assigning to each dependency two integers based one how its host and target platforms are offset from the depending derivation’s platforms. Those offsets are given below in the descriptions of each dependency list attribute. Algorithmically, we traverse propagated inputs, accumulating every propagated dependency’s propagated dependencies and adjusting them to account for the “shift in perspective” described by the current dependency’s platform offsets. This results in sort a transitive closure of the dependency relation, with the offsets being approximately summed when two dependency links are combined. We also prune transitive dependencies whose combined offsets go out-of-bounds, which can be viewed as a filter over that transitive closure removing dependencies that are blatantly absurd.
It is important to note that dependencies are not necessarily propagated as the same sort of dependency that they were before, but rather as the corresponding sort so that the platform rules still line up. To determine the exact rules for dependency propagation, we start by assigning to each dependency a couple of ternary numbers (`-1` for `build`, `0` for `host`, and `1` for `target`), representing how respectively its host and target platforms are "offset" from the depending derivation’s platforms. The following table summarize the different combinations that can be obtained:
| `host → target` | attribute name | offset |
| ------------------- | ------------------- | -------- |
| `build --> build` | `depsBuildBuild` | `-1, -1` |
| `build --> host` | `nativeBuildInputs` | `-1, 0` |
| `build --> target` | `depsBuildTarget` | `-1, 1` |
| `host --> host` | `depsHostHost` | `0, 0` |
| `host --> target` | `buildInputs` | `0, 1` |
| `target --> target` | `depsTargetTarget` | `1, 1` |
Algorithmically, we traverse propagated inputs, accumulating every propagated dependency’s propagated dependencies and adjusting them to account for the “shift in perspective” described by the current dependency’s platform offsets. This results is sort of a transitive closure of the dependency relation, with the offsets being approximately summed when two dependency links are combined. We also prune transitive dependencies whose combined offsets go out-of-bounds, which can be viewed as a filter over that transitive closure removing dependencies that are blatantly absurd.
We can define the process precisely with [Natural Deduction](https://en.wikipedia.org/wiki/Natural_deduction) using the inference rules. This probably seems a bit obtuse, but so is the bash code that actually implements it! [^footnote-stdenv-find-inputs-location] They’re confusing in very different ways so… hopefully if something doesn’t make sense in one presentation, it will in the other!
@ -179,37 +191,37 @@ Overall, the unifying theme here is that propagation shouldn’t be introducing
#### `depsBuildBuild` {#var-stdenv-depsBuildBuild}
A list of dependencies whose host and target platforms are the new derivation’s build platform. This means a `-1` host and `-1` target offset from the new derivation’s platforms. These are programs and libraries used at build time that produce programs and libraries also used at build time. If the dependency doesn’t care about the target platform (i.e. isn’t a compiler or similar tool), put it in `nativeBuildInputs` instead. The most common use of this `buildPackages.stdenv.cc`, the default C compiler for this role. That example crops up more than one might think in old commonly used C libraries.
A list of dependencies whose host and target platforms are the new derivation’s build platform. These are programs and libraries used at build time that produce programs and libraries also used at build time. If the dependency doesn’t care about the target platform (i.e. isn’t a compiler or similar tool), put it in `nativeBuildInputs` instead. The most common use of this `buildPackages.stdenv.cc`, the default C compiler for this role. That example crops up more than one might think in old commonly used C libraries.
Since these packages are able to be run at build-time, they are always added to the `PATH`, as described above. But since these packages are only guaranteed to be able to run then, they shouldn’t persist as run-time dependencies. This isn’t currently enforced, but could be in the future.
#### `nativeBuildInputs` {#var-stdenv-nativeBuildInputs}
A list of dependencies whose host platform is the new derivation’s build platform, and target platform is the new derivation’s host platform. This means a `-1` host offset and `0` target offset from the new derivation’s platforms. These are programs and libraries used at build-time that, if they are a compiler or similar tool, produce code to run at run-time—i.e. tools used to build the new derivation. If the dependency doesn’t care about the target platform (i.e. isn’t a compiler or similar tool), put it here, rather than in `depsBuildBuild` or `depsBuildTarget`. This could be called `depsBuildHost` but `nativeBuildInputs` is used for historical continuity.
A list of dependencies whose host platform is the new derivation’s build platform, and target platform is the new derivation’s host platform. These are programs and libraries used at build-time that, if they are a compiler or similar tool, produce code to run at run-time—i.e. tools used to build the new derivation. If the dependency doesn’t care about the target platform (i.e. isn’t a compiler or similar tool), put it here, rather than in `depsBuildBuild` or `depsBuildTarget`. This could be called `depsBuildHost` but `nativeBuildInputs` is used for historical continuity.
Since these packages are able to be run at build-time, they are added to the `PATH`, as described above. But since these packages are only guaranteed to be able to run then, they shouldn’t persist as run-time dependencies. This isn’t currently enforced, but could be in the future.
#### `depsBuildTarget` {#var-stdenv-depsBuildTarget}
A list of dependencies whose host platform is the new derivation’s build platform, and target platform is the new derivation’s target platform. This means a `-1` host offset and `1` target offset from the new derivation’s platforms. These are programs used at build time that produce code to run with code produced by the depending package. Most commonly, these are tools used to build the runtime or standard library that the currently-being-built compiler will inject into any code it compiles. In many cases, the currently-being-built-compiler is itself employed for that task, but when that compiler won’t run (i.e. its build and host platform differ) this is not possible. Other times, the compiler relies on some other tool, like binutils, that is always built separately so that the dependency is unconditional.
A list of dependencies whose host platform is the new derivation’s build platform, and target platform is the new derivation’s target platform. These are programs used at build time that produce code to run with code produced by the depending package. Most commonly, these are tools used to build the runtime or standard library that the currently-being-built compiler will inject into any code it compiles. In many cases, the currently-being-built-compiler is itself employed for that task, but when that compiler won’t run (i.e. its build and host platform differ) this is not possible. Other times, the compiler relies on some other tool, like binutils, that is always built separately so that the dependency is unconditional.
This is a somewhat confusing concept to wrap one’s head around, and for good reason. As the only dependency type where the platform offsets are not adjacent integers, it requires thinking of a bootstrapping stage *two* away from the current one. It and its use-case go hand in hand and are both considered poor form: try to not need this sort of dependency, and try to avoid building standard libraries and runtimes in the same derivation as the compiler produces code using them. Instead strive to build those like a normal library, using the newly-built compiler just as a normal library would. In short, do not use this attribute unless you are packaging a compiler and are sure it is needed.
This is a somewhat confusing concept to wrap one’s head around, and for good reason. As the only dependency type where the platform offsets, `-1` and `1`, are not adjacent integers, it requires thinking of a bootstrapping stage *two* away from the current one. It and its use-case go hand in hand and are both considered poor form: try to not need this sort of dependency, and try to avoid building standard libraries and runtimes in the same derivation as the compiler produces code using them. Instead strive to build those like a normal library, using the newly-built compiler just as a normal library would. In short, do not use this attribute unless you are packaging a compiler and are sure it is needed.
Since these packages are able to run at build time, they are added to the `PATH`, as described above. But since these packages are only guaranteed to be able to run then, they shouldn’t persist as run-time dependencies. This isn’t currently enforced, but could be in the future.
#### `depsHostHost` {#var-stdenv-depsHostHost}
A list of dependencies whose host and target platforms match the new derivation’s host platform. This means a `0` host offset and `0` target offset from the new derivation’s host platform. These are packages used at run-time to generate code also used at run-time. In practice, this would usually be tools used by compilers for macros or a metaprogramming system, or libraries used by the macros or metaprogramming code itself. It’s always preferable to use a `depsBuildBuild` dependency in the derivation being built over a `depsHostHost` on the tool doing the building for this purpose.
A list of dependencies whose host and target platforms match the new derivation’s host platform. In practice, this would usually be tools used by compilers for macros or a metaprogramming system, or libraries used by the macros or metaprogramming code itself. It’s always preferable to use a `depsBuildBuild` dependency in the derivation being built over a `depsHostHost` on the tool doing the building for this purpose.
#### `buildInputs` {#var-stdenv-buildInputs}
A list of dependencies whose host platform and target platform match the new derivation’s. This means a `0` host offset and a `1` target offset from the new derivation’s host platform. This would be called `depsHostTarget` but for historical continuity. If the dependency doesn’t care about the target platform (i.e. isn’t a compiler or similar tool), put it here, rather than in `depsBuildBuild`.
A list of dependencies whose host platform and target platform match the new derivation’s. This would be called `depsHostTarget` but for historical continuity. If the dependency doesn’t care about the target platform (i.e. isn’t a compiler or similar tool), put it here, rather than in `depsBuildBuild`.
These are often programs and libraries used by the new derivation at *run*-time, but that isn’t always the case. For example, the machine code in a statically-linked library is only used at run-time, but the derivation containing the library is only needed at build-time. Even in the dynamic case, the library may also be needed at build-time to appease the linker.
#### `depsTargetTarget` {#var-stdenv-depsTargetTarget}
A list of dependencies whose host platform matches the new derivation’s target platform. This means a `1` offset from the new derivation’s platforms. These are packages that run on the target platform, e.g. the standard library or run-time deps of standard library that a compiler insists on knowing about. It’s poor form in almost all cases for a package to depend on another from a future stage \[future stage corresponding to positive offset\]. Do not use this attribute unless you are packaging a compiler and are sure it is needed.
A list of dependencies whose host platform matches the new derivation’s target platform. These are packages that run on the target platform, e.g. the standard library or run-time deps of standard library that a compiler insists on knowing about. It’s poor form in almost all cases for a package to depend on another from a future stage \[future stage corresponding to positive offset\]. Do not use this attribute unless you are packaging a compiler and are sure it is needed.
#### `depsBuildBuildPropagated` {#var-stdenv-depsBuildBuildPropagated}
@ -373,11 +385,11 @@ Additional file types can be supported by setting the `unpackCmd` variable (see
##### `srcs` / `src` {#var-stdenv-src}
The list of source files or directories to be unpacked or copied. One of these must be set.
The list of source files or directories to be unpacked or copied. One of these must be set. Note that if you use `srcs`, you should also set `sourceRoot` or `setSourceRoot`.
##### `sourceRoot` {#var-stdenv-sourceRoot}
After running `unpackPhase`, the generic builder changes the current directory to the directory created by unpacking the sources. If there are multiple source directories, you should set `sourceRoot` to the name of the intended directory.
After running `unpackPhase`, the generic builder changes the current directory to the directory created by unpacking the sources. If there are multiple source directories, you should set `sourceRoot` to the name of the intended directory. Set `sourceRoot = ".";` if you use `srcs` and control the unpack phase yourself.
##### `setSourceRoot` {#var-stdenv-setSourceRoot}
@ -1228,6 +1240,7 @@ If the libraries lack `-fPIE`, you will get the error `recompile with -fPIE`.
[^footnote-stdenv-ignored-build-platform]: The build platform is ignored because it is a mere implementation detail of the package satisfying the dependency: As a general programming principle, dependencies are always *specified* as interfaces, not concrete implementation.
[^footnote-stdenv-native-dependencies-in-path]: Currently, this means for native builds all dependencies are put on the `PATH`. But in the future that may not be the case for sake of matching cross: the platforms would be assumed to be unique for native and cross builds alike, so only the `depsBuild*` and `nativeBuildInputs` would be added to the `PATH`.
[^footnote-stdenv-propagated-dependencies]: Nix itself already takes a package’s transitive dependencies into account, but this propagation ensures nixpkgs-specific infrastructure like setup hooks (mentioned above) also are run as if the propagated dependency.
[^footnote-stdenv-find-inputs-location]: The `findInputs` function, currently residing in `pkgs/stdenv/generic/setup.sh`, implements the propagation logic.
[^footnote-stdenv-sys-lib-search-path]: It clears the `sys_lib_*search_path` variables in the Libtool script to prevent Libtool from using libraries in `/usr/lib` and such.
[^footnote-stdenv-build-time-guessing-impurity]: Eventually these will be passed building natively as well, to improve determinism: build-time guessing, as is done today, is a risk of impurity.

2
doc/using/overlays.chapter.md
View File

@ -112,7 +112,7 @@ self: super:
This overlay uses Intel's MKL library for both BLAS and LAPACK interfaces. Note that the same can be accomplished at runtime using `LD_LIBRARY_PATH` of `libblas.so.3` and `liblapack.so.3`. For instance:
```ShellSession
$ LD_LIBRARY_PATH=$(nix-build -A mkl)/lib:$LD_LIBRARY_PATH nix-shell -p octave --run octave
$ LD_LIBRARY_PATH=$(nix-build -A mkl)/lib${LD_LIBRARY_PATH:+:}$LD_LIBRARY_PATH nix-shell -p octave --run octave
```
Intel MKL requires an `openmp` implementation when running with multiple processors. By default, `mkl` will use Intel's `iomp` implementation if no other is specified, but this is a runtime-only dependency and binary compatible with the LLVM implementation. To use that one instead, Intel recommends users set it with `LD_PRELOAD`. Note that `mkl` is only available on `x86_64-linux` and `x86_64-darwin`. Moreover, Hydra is not building and distributing pre-compiled binaries using it.

10
flake.nix
View File

@ -11,15 +11,7 @@
lib = import ./lib;
systems = [
"x86_64-linux"
"i686-linux"
"x86_64-darwin"
"aarch64-linux"
"armv6l-linux"
"armv7l-linux"
"aarch64-darwin"
];
systems = lib.systems.supported.hydra;
forAllSystems = f: lib.genAttrs systems (system: f system);

2
lib/attrsets.nix
View File

@ -487,7 +487,7 @@ rec {
=> "/nix/store/9rz8gxhzf8sw4kf2j2f1grr49w8zx5vj-openssl-1.0.1r-dev"
*/
getOutput = output: pkg:
if pkg.outputUnspecified or false
if ! pkg ? outputSpecified || ! pkg.outputSpecified
then pkg.${output} or pkg.out or pkg
else pkg;

2
lib/customisation.nix
View File

@ -152,6 +152,7 @@ rec {
{ name = outputName;
value = commonAttrs // {
inherit (drv.${outputName}) type outputName;
outputSpecified = true;
drvPath = assert condition; drv.${outputName}.drvPath;
outPath = assert condition; drv.${outputName}.outPath;
};
@ -159,7 +160,6 @@ rec {
outputsList = map outputToAttrListElement outputs;
in commonAttrs // {
outputUnspecified = true;
drvPath = assert condition; drv.drvPath;
outPath = assert condition; drv.outPath;
};

6
lib/default.nix
View File

@ -91,7 +91,7 @@ let
concatImapStringsSep makeSearchPath makeSearchPathOutput
makeLibraryPath makeBinPath optionalString
hasInfix hasPrefix hasSuffix stringToCharacters stringAsChars escape
escapeShellArg escapeShellArgs escapeRegex replaceChars lowerChars
escapeShellArg escapeShellArgs escapeRegex escapeXML replaceChars lowerChars
upperChars toLower toUpper addContextFrom splitString
removePrefix removeSuffix versionOlder versionAtLeast
getName getVersion
@ -123,8 +123,8 @@ let
inherit (self.options) isOption mkEnableOption mkSinkUndeclaredOptions
mergeDefaultOption mergeOneOption mergeEqualOption getValues
getFiles optionAttrSetToDocList optionAttrSetToDocList'
scrubOptionValue literalExample showOption showFiles
unknownModule mkOption;
scrubOptionValue literalExpression literalExample literalDocBook
showOption showFiles unknownModule mkOption;
inherit (self.types) isType setType defaultTypeMerge defaultFunctor
isOptionType mkOptionType;
inherit (self.asserts)

27
lib/generators.nix
View File

@ -197,6 +197,30 @@ rec {
*/
toYAML = {}@args: toJSON args;
withRecursion =
args@{
/* If this option is not null, the given value will stop evaluating at a certain depth */
depthLimit
/* If this option is true, an error will be thrown, if a certain given depth is exceeded */
, throwOnDepthLimit ? true
}:
assert builtins.isInt depthLimit;
let
transform = depth:
if depthLimit != null && depth > depthLimit then
if throwOnDepthLimit
then throw "Exceeded maximum eval-depth limit of ${toString depthLimit} while trying to evaluate with `generators.withRecursion'!"
else const "<unevaluated>"
else id;
mapAny = with builtins; depth: v:
let
evalNext = x: mapAny (depth + 1) (transform (depth + 1) x);
in
if isAttrs v then mapAttrs (const evalNext) v
else if isList v then map evalNext v
else transform (depth + 1) v;
in
mapAny 0;
/* Pretty print a value, akin to `builtins.trace`.
* Should probably be a builtin as well.
@ -208,7 +232,8 @@ rec {
allowPrettyValues ? false,
/* If this option is true, the output is indented with newlines for attribute sets and lists */
multiline ? true
}@args: let
}@args:
let
go = indent: v: with builtins;
let isPath = v: typeOf v == "path";
introSpace = if multiline then "\n${indent} " else " ";

10
lib/licenses.nix
View File

@ -153,6 +153,11 @@ in mkLicense lset) ({
free = false;
};
capec = {
fullName = "Common Attack Pattern Enumeration and Classification";
url = "https://capec.mitre.org/about/termsofuse.html";
};
clArtistic = {
spdxId = "ClArtistic";
fullName = "Clarified Artistic License";
@ -240,6 +245,11 @@ in mkLicense lset) ({
fullName = "CeCILL Free Software License Agreement v2.0";
};
cecill21 = {
spdxId = "CECILL-2.1";
fullName = "CeCILL Free Software License Agreement v2.1";
};
cecill-b = {
spdxId = "CECILL-B";
fullName = "CeCILL-B Free Software License Agreement";

7
lib/lists.nix
View File

@ -642,7 +642,7 @@ rec {
unique [ 3 2 3 4 ]
=> [ 3 2 4 ]
*/
unique = foldl' (acc: e: if elem e acc then acc else acc ++ [ e ]) [];
unique = foldl' (acc: e: if elem e acc then acc else acc ++ [ e ]) [];
/* Intersects list 'e' and another list. O(nm) complexity.
@ -663,9 +663,6 @@ rec {
/* Test if two lists have no common element.
It should be slightly more efficient than (intersectLists a b == [])
*/
mutuallyExclusive = a: b:
(builtins.length a) == 0 ||
(!(builtins.elem (builtins.head a) b) &&
mutuallyExclusive (builtins.tail a) b);
mutuallyExclusive = a: b: length a == 0 || !(any (x: elem x a) b);
}

85
lib/modules.nix
View File

@ -52,15 +52,39 @@ in
rec {
/* Evaluate a set of modules. The result is a set of two
attributes: ‘options’: the nested set of all option declarations,
and ‘config’: the nested set of all option values.
/*
Evaluate a set of modules. The result is a set with the attributes:
‘options’: The nested set of all option declarations,
‘config’: The nested set of all option values.
‘type’: A module system type representing the module set as a submodule,
to be extended by configuration from the containing module set.
‘extendModules’: A function similar to ‘evalModules’ but building on top
of the module set. Its arguments, ‘modules’ and ‘specialArgs’ are
added to the existing values.
Using ‘extendModules’ a few times has no performance impact as long
as you only reference the final ‘options’ and ‘config’.
If you do reference multiple ‘config’ (or ‘options’) from before and
after ‘extendModules’, performance is the same as with multiple
‘evalModules’ invocations, because the new modules' ability to
override existing configuration fundamentally requires a new
fixpoint to be constructed.
‘_module’: A portion of the configuration tree which is elided from
‘config’. It contains some values that are mostly internal to the
module system implementation.
!!! Please think twice before adding to this argument list! The more
that is specified here instead of in the modules themselves the harder
it is to transparently move a set of modules to be a submodule of another
config (as the proper arguments need to be replicated at each call to
evalModules) and the less declarative the module set is. */
evalModules = { modules
evalModules = evalModulesArgs@
{ modules
, prefix ? []
, # This should only be used for special arguments that need to be evaluated
# when resolving module structure (like in imports). For everything else,
@ -120,7 +144,9 @@ rec {
};
config = {
_module.args = args;
_module.args = {
inherit extendModules;
} // args;
};
};
@ -162,20 +188,49 @@ rec {
baseMsg = "The option `${showOption (prefix ++ firstDef.prefix)}' does not exist. Definition values:${showDefs [ firstDef ]}";
in
if attrNames options == [ "_module" ]
then throw ''
${baseMsg}
then
let
optionName = showOption prefix;
in
if optionName == ""
then throw ''
${baseMsg}
However there are no options defined in `${showOption prefix}'. Are you sure you've
declared your options properly? This can happen if you e.g. declared your options in `types.submodule'
under `config' rather than `options'.
''
It seems as if you're trying to declare an option by placing it into `config' rather than `options'!
''
else
throw ''
${baseMsg}
However there are no options defined in `${showOption prefix}'. Are you sure you've
declared your options properly? This can happen if you e.g. declared your options in `types.submodule'
under `config' rather than `options'.
''
else throw baseMsg
else null;
result = builtins.seq checkUnmatched {
inherit options;
config = removeAttrs config [ "_module" ];
inherit (config) _module;
checked = builtins.seq checkUnmatched;
extendModules = extendArgs@{
modules ? [],
specialArgs ? {},
prefix ? [],
}:
evalModules (evalModulesArgs // {
modules = evalModulesArgs.modules ++ modules;
specialArgs = evalModulesArgs.specialArgs or {} // specialArgs;
prefix = extendArgs.prefix or evalModulesArgs.prefix;
});
type = lib.types.submoduleWith {
inherit modules specialArgs;
};
result = {
options = checked options;
config = checked (removeAttrs config [ "_module" ]);
_module = checked (config._module);
inherit extendModules type;
};
in result;

38
lib/options.nix
View File

@ -54,7 +54,7 @@ rec {
Example:
mkOption { } // => { _type = "option"; }
mkOption { defaultText = "foo"; } // => { _type = "option"; defaultText = "foo"; }
mkOption { default = "foo"; } // => { _type = "option"; default = "foo"; }
*/
mkOption =
{
@ -74,7 +74,7 @@ rec {
apply ? null,
# Whether the option is for NixOS developers only.
internal ? null,
# Whether the option shows up in the manual.
# Whether the option shows up in the manual. Default: true. Use false to hide the option and any sub-options from submodules. Use "shallow" to hide only sub-options.
visible ? null,
# Whether the option can be set only once
readOnly ? null,
@ -180,7 +180,10 @@ rec {
description = opt.description or (lib.warn "Option `${name}' has no description." "This option has no description.");
declarations = filter (x: x != unknownModule) opt.declarations;
internal = opt.internal or false;
visible = opt.visible or true;
visible =
if (opt?visible && opt.visible == "shallow")
then true
else opt.visible or true;
readOnly = opt.readOnly or false;
type = opt.type.description or null;
}
@ -192,8 +195,9 @@ rec {
subOptions =
let ss = opt.type.getSubOptions opt.loc;
in if ss != {} then optionAttrSetToDocList' opt.loc ss else [];
subOptionsVisible = docOption.visible && opt.visible or null != "shallow";
in
[ docOption ] ++ optionals docOption.visible subOptions) (collect isOption options);
[ docOption ] ++ optionals subOptionsVisible subOptions) (collect isOption options);
/* This function recursively removes all derivation attributes from
@ -212,11 +216,25 @@ rec {
else x;
/* For use in the `example` option attribute. It causes the given
text to be included verbatim in documentation. This is necessary
for example values that are not simple values, e.g., functions.
/* For use in the `defaultText` and `example` option attributes. Causes the
given string to be rendered verbatim in the documentation as Nix code. This
is necessary for complex values, e.g. functions, or values that depend on
other values or packages.
*/
literalExample = text: { _type = "literalExample"; inherit text; };
literalExpression = text:
if ! isString text then throw "literalExpression expects a string."
else { _type = "literalExpression"; inherit text; };
literalExample = lib.warn "literalExample is deprecated, use literalExpression instead, or use literalDocBook for a non-Nix description." literalExpression;
/* For use in the `defaultText` and `example` option attributes. Causes the
given DocBook text to be inserted verbatim in the documentation, for when
a `literalExpression` would be too hard to read.
*/
literalDocBook = text:
if ! isString text then throw "literalDocBook expects a string."
else { _type = "literalDocBook"; inherit text; };
# Helper functions.
@ -247,7 +265,9 @@ rec {
showDefs = defs: concatMapStrings (def:
let
# Pretty print the value for display, if successful
prettyEval = builtins.tryEval (lib.generators.toPretty {} def.value);
prettyEval = builtins.tryEval
(lib.generators.toPretty { }
(lib.generators.withRecursion { depthLimit = 10; throwOnDepthLimit = false; } def.value));
# Split it into its lines
lines = filter (v: ! isList v) (builtins.split "\n" prettyEval.value);
# Only display the first 5 lines, and indent them for better visibility

13
lib/strings.nix
View File

@ -362,6 +362,19 @@ rec {
if match "[a-zA-Z_][a-zA-Z0-9_'-]*" s != null
then s else escapeNixString s;
/* Escapes a string such that it is safe to include verbatim in an XML
document.
Type: string -> string
Example:
escapeXML ''"test" 'test' < & >''
=> "&quot;test&quot; &apos;test&apos; &lt; &amp; &gt;"
*/
escapeXML = builtins.replaceStrings
["\"" "'" "<" ">" "&"]
["&quot;" "&apos;" "&lt;" "&gt;" "&amp;"];
# Obsolete - use replaceStrings instead.
replaceChars = builtins.replaceStrings or (
del: new: s:

1
lib/systems/default.nix
View File

@ -8,6 +8,7 @@ rec {
platforms = import ./platforms.nix { inherit lib; };
examples = import ./examples.nix { inherit lib; };
architectures = import ./architectures.nix { inherit lib; };
supported = import ./supported.nix { inherit lib; };
# Elaborate a `localSystem` or `crossSystem` so that it contains everything
# necessary.

6
lib/systems/examples.nix
View File

@ -258,6 +258,12 @@ rec {
platform = {};
};
x86_64-darwin = {
config = "x86_64-apple-darwin";
xcodePlatform = "MacOSX";
platform = {};
};
#
# Windows
#

8
lib/systems/platforms.nix
View File

@ -20,15 +20,17 @@ rec {
name = "PowerNV";
baseConfig = "powernv_defconfig";
target = "zImage";
installTarget = "install";
file = "vmlinux";
target = "vmlinux";
autoModules = true;
# avoid driver/FS trouble arising from unusual page size
extraConfig = ''
PPC_64K_PAGES n
PPC_4K_PAGES y
IPV6 y
ATA_BMDMA y
ATA_SFF y
VIRTIO_MENU y
'';
};
};

25
lib/systems/supported.nix Normal file
View File

@ -0,0 +1,25 @@
# Supported systems according to RFC0046's definition.
#
# https://github.com/NixOS/rfcs/blob/master/rfcs/0046-platform-support-tiers.md
{ lib }:
rec {
# List of systems that are built by Hydra.
hydra = tier1 ++ tier2 ++ tier3;
tier1 = [
"x86_64-linux"
];
tier2 = [
"aarch64-linux"
"x86_64-darwin"
];
tier3 = [
"aarch64-darwin"
"armv6l-linux"
"armv7l-linux"
"i686-linux"
"mipsel-linux"
];
}

4
lib/tests/maintainers.nix
View File

@ -16,6 +16,10 @@ let
email = lib.mkOption {
type = types.str;
};
matrix = lib.mkOption {
type = types.nullOr types.str;
default = null;
};
github = lib.mkOption {
type = types.nullOr types.str;
default = null;

24
lib/tests/misc.nix
View File

@ -246,6 +246,11 @@ runTests {
};
};
testEscapeXML = {
expr = escapeXML ''"test" 'test' < & >'';
expected = "&quot;test&quot; &apos;test&apos; &lt; &amp; &gt;";
};
# LISTS
testFilter = {
@ -529,6 +534,25 @@ runTests {
};
};
testToPrettyLimit =
let
a.b = 1;
a.c = a;
in {
expr = generators.toPretty { } (generators.withRecursion { throwOnDepthLimit = false; depthLimit = 2; } a);
expected = "{\n b = 1;\n c = {\n b = \"<unevaluated>\";\n c = {\n b = \"<unevaluated>\";\n c = \"<unevaluated>\";\n };\n };\n}";
};
testToPrettyLimitThrow =
let
a.b = 1;
a.c = a;
in {
expr = (builtins.tryEval
(generators.toPretty { } (generators.withRecursion { depthLimit = 2; } a))).success;
expected = false;
};
testToPrettyMultiline = {
expr = mapAttrs (const (generators.toPretty { })) rec {
list = [ 3 4 [ false ] ];

35
lib/tests/modules.sh
View File

@ -62,17 +62,17 @@ checkConfigError() {
# Check boolean option.
checkConfigOutput "false" config.enable ./declare-enable.nix
checkConfigError 'The option .* does not exist. Definition values:\n- In .*: true' config.enable ./define-enable.nix
checkConfigError 'The option .* does not exist. Definition values:\n\s*- In .*: true' config.enable ./define-enable.nix
# Check integer types.
# unsigned
checkConfigOutput "42" config.value ./declare-int-unsigned-value.nix ./define-value-int-positive.nix
checkConfigError 'A definition for option .* is not of type.*unsigned integer.*. Definition values:\n- In .*: -23' config.value ./declare-int-unsigned-value.nix ./define-value-int-negative.nix
checkConfigError 'A definition for option .* is not of type.*unsigned integer.*. Definition values:\n\s*- In .*: -23' config.value ./declare-int-unsigned-value.nix ./define-value-int-negative.nix
# positive
checkConfigError 'A definition for option .* is not of type.*positive integer.*. Definition values:\n- In .*: 0' config.value ./declare-int-positive-value.nix ./define-value-int-zero.nix
checkConfigError 'A definition for option .* is not of type.*positive integer.*. Definition values:\n\s*- In .*: 0' config.value ./declare-int-positive-value.nix ./define-value-int-zero.nix
# between
checkConfigOutput "42" config.value ./declare-int-between-value.nix ./define-value-int-positive.nix
checkConfigError 'A definition for option .* is not of type.*between.*-21 and 43.*inclusive.*. Definition values:\n- In .*: -23' config.value ./declare-int-between-value.nix ./define-value-int-negative.nix
checkConfigError 'A definition for option .* is not of type.*between.*-21 and 43.*inclusive.*. Definition values:\n\s*- In .*: -23' config.value ./declare-int-between-value.nix ./define-value-int-negative.nix
# Check either types
# types.either
@ -125,7 +125,7 @@ checkConfigOutput 'true' "$@" ./define-enable.nix ./define-attrsOfSub-foo-enable
set -- config.enable ./define-enable.nix ./declare-enable.nix
checkConfigOutput "true" "$@"
checkConfigOutput "false" "$@" ./disable-define-enable.nix
checkConfigError "The option .*enable.* does not exist. Definition values:\n- In .*: true" "$@" ./disable-declare-enable.nix
checkConfigError "The option .*enable.* does not exist. Definition values:\n\s*- In .*: true" "$@" ./disable-declare-enable.nix
checkConfigError "attribute .*enable.* in selection path .*config.enable.* not found" "$@" ./disable-define-enable.nix ./disable-declare-enable.nix
checkConfigError "attribute .*enable.* in selection path .*config.enable.* not found" "$@" ./disable-enable-modules.nix
@ -142,18 +142,18 @@ checkConfigError 'infinite recursion encountered' "$@"
# Check _module.check.
set -- config.enable ./declare-enable.nix ./define-enable.nix ./define-attrsOfSub-foo.nix
checkConfigError 'The option .* does not exist. Definition values:\n- In .*' "$@"
checkConfigError 'The option .* does not exist. Definition values:\n\s*- In .*' "$@"
checkConfigOutput "true" "$@" ./define-module-check.nix
# Check coerced value.
checkConfigOutput "\"42\"" config.value ./declare-coerced-value.nix
checkConfigOutput "\"24\"" config.value ./declare-coerced-value.nix ./define-value-string.nix
checkConfigError 'A definition for option .* is not.*string or signed integer convertible to it.*. Definition values:\n- In .*: \[ \]' config.value ./declare-coerced-value.nix ./define-value-list.nix
checkConfigError 'A definition for option .* is not.*string or signed integer convertible to it.*. Definition values:\n\s*- In .*: \[ \]' config.value ./declare-coerced-value.nix ./define-value-list.nix
# Check coerced value with unsound coercion
checkConfigOutput "12" config.value ./declare-coerced-value-unsound.nix
checkConfigError 'A definition for option .* is not of type .*. Definition values:\n- In .*: "1000"' config.value ./declare-coerced-value-unsound.nix ./define-value-string-bigint.nix
checkConfigError 'unrecognised JSON value' config.value ./declare-coerced-value-unsound.nix ./define-value-string-arbitrary.nix
checkConfigError 'A definition for option .* is not of type .*. Definition values:\n\s*- In .*: "1000"' config.value ./declare-coerced-value-unsound.nix ./define-value-string-bigint.nix
checkConfigError 'json.exception.parse_error' config.value ./declare-coerced-value-unsound.nix ./define-value-string-arbitrary.nix
# Check mkAliasOptionModule.
checkConfigOutput "true" config.enable ./alias-with-priority.nix
@ -169,7 +169,7 @@ checkConfigOutput "foo" config.submodule.foo ./declare-submoduleWith-special.nix
## shorthandOnlyDefines config behaves as expected
checkConfigOutput "true" config.submodule.config ./declare-submoduleWith-shorthand.nix ./define-submoduleWith-shorthand.nix
checkConfigError 'is not of type `boolean' config.submodule.config ./declare-submoduleWith-shorthand.nix ./define-submoduleWith-noshorthand.nix
checkConfigError "You're trying to declare a value of type \`bool'\nrather than an attribute-set for the option" config.submodule.config ./declare-submoduleWith-noshorthand.nix ./define-submoduleWith-shorthand.nix
checkConfigError "You're trying to declare a value of type \`bool'\n\s*rather than an attribute-set for the option" config.submodule.config ./declare-submoduleWith-noshorthand.nix ./define-submoduleWith-shorthand.nix
checkConfigOutput "true" config.submodule.config ./declare-submoduleWith-noshorthand.nix ./define-submoduleWith-noshorthand.nix
## submoduleWith should merge all modules in one swoop
@ -179,6 +179,13 @@ checkConfigOutput "true" config.submodule.outer ./declare-submoduleWith-modules.
# which evaluates all the modules defined by the type)
checkConfigOutput "submodule" options.submodule.type.description ./declare-submoduleWith-modules.nix
## submodules can be declared using (evalModules {...}).type
checkConfigOutput "true" config.submodule.inner ./declare-submodule-via-evalModules.nix
checkConfigOutput "true" config.submodule.outer ./declare-submodule-via-evalModules.nix
# Should also be able to evaluate the type name (which evaluates freeformType,
# which evaluates all the modules defined by the type)
checkConfigOutput "submodule" options.submodule.type.description ./declare-submodule-via-evalModules.nix
## Paths should be allowed as values and work as expected
checkConfigOutput "true" config.submodule.enable ./declare-submoduleWith-path.nix
@ -186,7 +193,7 @@ checkConfigOutput "true" config.submodule.enable ./declare-submoduleWith-path.ni
checkConfigOutput "true" config.enable ./disable-recursive/main.nix
checkConfigOutput "true" config.enable ./disable-recursive/{main.nix,disable-foo.nix}
checkConfigOutput "true" config.enable ./disable-recursive/{main.nix,disable-bar.nix}
checkConfigError 'The option .* does not exist. Definition values:\n- In .*: true' config.enable ./disable-recursive/{main.nix,disable-foo.nix,disable-bar.nix}
checkConfigError 'The option .* does not exist. Definition values:\n\s*- In .*: true' config.enable ./disable-recursive/{main.nix,disable-foo.nix,disable-bar.nix}
# Check that imports can depend on derivations
checkConfigOutput "true" config.enable ./import-from-store.nix
@ -254,8 +261,10 @@ checkConfigOutput / config.value.path ./types-anything/equal-atoms.nix
checkConfigOutput null config.value.null ./types-anything/equal-atoms.nix
checkConfigOutput 0.1 config.value.float ./types-anything/equal-atoms.nix
# Functions can't be merged together
checkConfigError "The option .* has conflicting definition values" config.value.multiple-lambdas ./types-anything/functions.nix
checkConfigError "The option .value.multiple-lambdas.<function body>. has conflicting option types" config.applied.multiple-lambdas ./types-anything/functions.nix
checkConfigOutput '<LAMBDA>' config.value.single-lambda ./types-anything/functions.nix
checkConfigOutput 'null' config.applied.merging-lambdas.x ./types-anything/functions.nix
checkConfigOutput 'null' config.applied.merging-lambdas.y ./types-anything/functions.nix
# Check that all mk* modifiers are applied
checkConfigError 'attribute .* not found' config.value.mkiffalse ./types-anything/mk-mods.nix
checkConfigOutput '{ }' config.value.mkiftrue ./types-anything/mk-mods.nix
@ -268,7 +277,7 @@ checkConfigOutput baz config.value.nested.bar.baz ./types-anything/mk-mods.nix
## types.functionTo
checkConfigOutput "input is input" config.result ./functionTo/trivial.nix
checkConfigOutput "a b" config.result ./functionTo/merging-list.nix
checkConfigError 'A definition for option .fun.\[function body\]. is not of type .string.. Definition values:\n- In .*wrong-type.nix' config.result ./functionTo/wrong-type.nix
checkConfigError 'A definition for option .fun.\[function body\]. is not of type .string.. Definition values:\n\s*- In .*wrong-type.nix' config.result ./functionTo/wrong-type.nix
checkConfigOutput "b a" config.result ./functionTo/list-order.nix
checkConfigOutput "a c" config.result ./functionTo/merging-attrs.nix

28
lib/tests/modules/declare-submodule-via-evalModules.nix Normal file
View File

@ -0,0 +1,28 @@
{ lib, ... }: {
options.submodule = lib.mkOption {
inherit (lib.evalModules {
modules = [
{
options.inner = lib.mkOption {
type = lib.types.bool;
default = false;
};
}
];
}) type;
default = {};
};
config.submodule = lib.mkMerge [
({ lib, ... }: {
options.outer = lib.mkOption {
type = lib.types.bool;
default = false;
};
})
{
inner = true;
outer = true;
}
];
}

12
lib/tests/modules/types-anything/functions.nix
View File

@ -1,16 +1,22 @@
{ lib, ... }: {
{ lib, config, ... }: {
options.value = lib.mkOption {
type = lib.types.anything;
};
options.applied = lib.mkOption {
default = lib.mapAttrs (name: fun: fun null) config.value;
};
config = lib.mkMerge [
{
value.single-lambda = x: x;
value.multiple-lambdas = x: x;
value.multiple-lambdas = x: { inherit x; };
value.merging-lambdas = x: { inherit x; };
}
{
value.multiple-lambdas = x: x;
value.multiple-lambdas = x: [ x ];
value.merging-lambdas = y: { inherit y; };
}
];

4
lib/tests/release.nix
View File

@ -23,6 +23,10 @@ pkgs.runCommand "nixpkgs-lib-tests" {
export NIX_STORE_DIR=$TEST_ROOT/store
export PAGER=cat
cacheDir=$TEST_ROOT/binary-cache
mkdir -p $NIX_CONF_DIR
echo "experimental-features = nix-command" >> $NIX_CONF_DIR/nix.conf
nix-store --init
cp -r ${../.} lib

6
lib/tests/sources.sh
View File

@ -26,7 +26,7 @@ touch {README.md,module.o,foo.bar}
# nix-instantiate doesn't write out the source, only computing the hash, so
# this uses the experimental nix command instead.
dir="$(nix eval --raw '(with import <nixpkgs/lib>; "${
dir="$(nix eval --impure --raw --expr '(with import <nixpkgs/lib>; "${
cleanSource ./.
}")')"
(cd $dir; find) | sort -f | diff -U10 - <(cat <<EOF
@ -37,7 +37,7 @@ EOF
) || die "cleanSource 1"
dir="$(nix eval --raw '(with import <nixpkgs/lib>; "${
dir="$(nix eval --impure --raw --expr '(with import <nixpkgs/lib>; "${
cleanSourceWith { src = '"$work"'; filter = path: type: ! hasSuffix ".bar" path; }
}")')"
(cd $dir; find) | sort -f | diff -U10 - <(cat <<EOF
@ -47,7 +47,7 @@ dir="$(nix eval --raw '(with import <nixpkgs/lib>; "${
EOF
) || die "cleanSourceWith 1"
dir="$(nix eval --raw '(with import <nixpkgs/lib>; "${
dir="$(nix eval --impure --raw --expr '(with import <nixpkgs/lib>; "${
cleanSourceWith { src = cleanSource '"$work"'; filter = path: type: ! hasSuffix ".bar" path; }
}")')"
(cd $dir; find) | sort -f | diff -U10 - <(cat <<EOF

21
lib/trivial.nix
View File

@ -303,7 +303,26 @@ rec {
# TODO: figure out a clever way to integrate location information from
# something like __unsafeGetAttrPos.
warn = msg: builtins.trace "warning: ${msg}";
/*
Print a warning before returning the second argument. This function behaves
like `builtins.trace`, but requires a string message and formats it as a
warning, including the `warning: ` prefix.
To get a call stack trace and abort evaluation, set the environment variable
`NIX_ABORT_ON_WARN=true` and set the Nix options `--option pure-eval false --show-trace`
Type: string -> a -> a
*/
warn =
if lib.elem (builtins.getEnv "NIX_ABORT_ON_WARN") ["1" "true" "yes"]
then msg: builtins.trace "warning: ${msg}" (abort "NIX_ABORT_ON_WARN=true; warnings are treated as unrecoverable errors.")
else msg: builtins.trace "warning: ${msg}";
/*
Like warn, but only warn when the first argument is `true`.
Type: bool -> string -> a -> a
*/
warnIf = cond: msg: if cond then warn msg else id;
info = msg: builtins.trace "INFO: ${msg}";

62
lib/types.nix
View File

@ -192,6 +192,12 @@ rec {
else (listOf anything).merge;
# This is the type of packages, only accept a single definition
stringCoercibleSet = mergeOneOption;
lambda = loc: defs: arg: anything.merge
(loc ++ [ "<function body>" ])
(map (def: {
file = def.file;
value = def.value arg;
}) defs);
# Otherwise fall back to only allowing all equal definitions
}.${commonType} or mergeEqualOption;
in mergeFunction loc defs;
@ -499,17 +505,36 @@ rec {
then setFunctionArgs (args: unify (value args)) (functionArgs value)
else unify (if shorthandOnlyDefinesConfig then { config = value; } else value);
allModules = defs: modules ++ imap1 (n: { value, file }:
allModules = defs: imap1 (n: { value, file }:
if isAttrs value || isFunction value then
# Annotate the value with the location of its definition for better error messages
coerce (lib.modules.unifyModuleSyntax file "${toString file}-${toString n}") value
else value
) defs;
freeformType = (evalModules {
inherit modules specialArgs;
args.name = "‹name›";
})._module.freeformType;
base = evalModules {
inherit specialArgs;
modules = [{
# This is a work-around for the fact that some sub-modules,
# such as the one included in an attribute set, expects an "args"
# attribute to be given to the sub-module. As the option
# evaluation does not have any specific attribute name yet, we
# provide a default for the documentation and the freeform type.
#
# This is necessary as some option declaration might use the
# "name" attribute given as argument of the submodule and use it
# as the default of option declarations.
#
# We use lookalike unicode single angle quotation marks because
# of the docbook transformation the options receive. In all uses
# &gt; and &lt; wouldn't be encoded correctly so the encoded values
# would be used, and use of `<` and `>` would break the XML document.
# It shouldn't cause an issue since this is cosmetic for the manual.
_module.args.name = lib.mkOptionDefault "‹name›";
}] ++ modules;
};
freeformType = base._module.freeformType;
in
mkOptionType rec {
@ -517,32 +542,13 @@ rec {
description = freeformType.description or name;
check = x: isAttrs x || isFunction x || path.check x;
merge = loc: defs:
(evalModules {
modules = allModules defs;
inherit specialArgs;
args.name = last loc;
(base.extendModules {
modules = [ { _module.args.name = last loc; } ] ++ allModules defs;
prefix = loc;
}).config;
emptyValue = { value = {}; };
getSubOptions = prefix: (evalModules
{ inherit modules prefix specialArgs;
# This is a work-around due to the fact that some sub-modules,
# such as the one included in an attribute set, expects a "args"
# attribute to be given to the sub-module. As the option
# evaluation does not have any specific attribute name, we
# provide a default one for the documentation.
#
# This is mandatory as some option declaration might use the
# "name" attribute given as argument of the submodule and use it
# as the default of option declarations.
#
# Using lookalike unicode single angle quotation marks because
# of the docbook transformation the options receive. In all uses
# &gt; and &lt; wouldn't be encoded correctly so the encoded values
# would be used, and use of `<` and `>` would break the XML document.
# It shouldn't cause an issue since this is cosmetic for the manual.
args.name = "‹name›";
}).options // optionalAttrs (freeformType != null) {
getSubOptions = prefix: (base.extendModules
{ inherit prefix; }).options // optionalAttrs (freeformType != null) {
# Expose the sub options of the freeform type. Note that the option
# discovery doesn't care about the attribute name used here, so this
# is just to avoid conflicts with potential options from the submodule

634
maintainers/maintainer-list.nix
View File

File diff suppressed because it is too large Load Diff

2
maintainers/scripts/find-tarballs.nix
View File

@ -37,7 +37,7 @@ let
keyDrv = drv: if canEval drv.drvPath then { key = drv.drvPath; value = drv; } else { };
immediateDependenciesOf = drv:
concatLists (mapAttrsToList (n: v: derivationsIn v) (removeAttrs drv ["meta" "passthru"]));
concatLists (mapAttrsToList (n: v: derivationsIn v) (removeAttrs drv (["meta" "passthru"] ++ optionals (drv?passthru) (attrNames drv.passthru))));
derivationsIn = x:
if !canEval x then []

6
maintainers/scripts/haskell/hydra-report.hs
View File

@ -437,7 +437,7 @@ printBuildSummary
<> Text.pack (formatTime defaultTimeLocale "%Y-%m-%d %H:%M UTC" fetchTime)
<> "*"
]
brokenLine (name, rdeps) = "[" <> name <> "](https://search.nixos.org/packages?channel=unstable&show=haskellPackages." <> name <> "&query=haskellPackages." <> name <> ") :arrow_heading_up: " <> Text.pack (show rdeps)
brokenLine (name, rdeps) = "[" <> name <> "](https://packdeps.haskellers.com/reverse/" <> name <> ") :arrow_heading_up: " <> Text.pack (show rdeps) <> " "
numSummary = statusToNumSummary summary
jobsByState predicate = Map.filter (predicate . worstState) summary
worstState = foldl' min Success . fmap state . summaryBuilds
@ -464,8 +464,8 @@ printBuildSummary
if' (isNothing mergeableJob) "No `mergeable` job found." <>
if' (isNothing maintainedJob) "No `maintained` job found." <>
if' (Unfinished > maybe Success worstState mergeableJob) "`mergeable` jobset failed." <>
if' (outstandingJobs (Platform "x86_64-linux") > 100) "Too much outstanding jobs on x86_64-linux." <>
if' (outstandingJobs (Platform "aarch64-linux") > 100) "Too much outstanding jobs on aarch64-linux."
if' (outstandingJobs (Platform "x86_64-linux") > 100) "Too many outstanding jobs on x86_64-linux." <>
if' (outstandingJobs (Platform "aarch64-linux") > 100) "Too many outstanding jobs on aarch64-linux."
if' p e = if p then [e] else mempty
outstandingJobs platform | Table m <- numSummary = Map.findWithDefault 0 (platform, Unfinished) m
maintainedJob = Map.lookup "maintained" summary

10
maintainers/scripts/haskell/merge-and-open-pr.sh
View File

@ -75,6 +75,10 @@ fi
echo "Merging https://github.com/NixOS/nixpkgs/pull/${curr_haskell_updates_pr_num}..."
gh pr merge --repo NixOS/nixpkgs --merge "$curr_haskell_updates_pr_num"
# Update the list of Haskell package versions in NixOS on Hackage.
echo "Updating list of Haskell package versions in NixOS on Hackage..."
./maintainers/scripts/haskell/upload-nixos-package-list-to-hackage.sh
# Update stackage, Hackage hashes, and regenerate Haskell package set
echo "Updating Stackage..."
./maintainers/scripts/haskell/update-stackage.sh --do-commit
@ -84,7 +88,7 @@ echo "Regenerating Hackage packages..."
./maintainers/scripts/haskell/regenerate-hackage-packages.sh --do-commit
# Push these new commits to the haskell-updates branch
echo "Pushing commits just created to the haskell-updates branch"
echo "Pushing commits just created to the remote haskell-updates branch..."
git push
# Open new PR
@ -93,7 +97,7 @@ new_pr_body=$(cat <<EOF
This PR is the regular merge of the \`haskell-updates\` branch into \`master\`.
This branch is being continually built and tested by hydra at https://hydra.nixos.org/jobset/nixpkgs/haskell-updates.
This branch is being continually built and tested by hydra at https://hydra.nixos.org/jobset/nixpkgs/haskell-updates. You may be able to find an up-to-date Hydra build report at [cdepillabout/nix-haskell-updates-status](https://github.com/cdepillabout/nix-haskell-updates-status).
We roughly aim to merge these \`haskell-updates\` PRs at least once every two weeks. See the @NixOS/haskell [team calendar](https://cloud.maralorn.de/apps/calendar/p/Mw5WLnzsP7fC4Zky) for who is currently in charge of this branch.
@ -114,5 +118,5 @@ This is the follow-up to #${curr_haskell_updates_pr_num}. Come to [#haskell:nixo
EOF
)
echo "Opening a PR for the next haskell-updates merge cycle"
echo "Opening a PR for the next haskell-updates merge cycle..."
gh pr create --repo NixOS/nixpkgs --base master --head haskell-updates --title "haskellPackages: update stackage and hackage" --body "$new_pr_body"

1
maintainers/scripts/haskell/upload-nixos-package-list-to-hackage.sh
View File

@ -19,3 +19,4 @@ package_list="$(nix-build -A haskell.package-list)/nixos-hackage-packages.csv"
username=$(grep "^username:" ~/.cabal/config | sed "s/^username: //")
password_command=$(grep "^password-command:" ~/.cabal/config | sed "s/^password-command: //")
curl -u "$username:$($password_command | head -n1)" --digest -H "Content-type: text/csv" -T "$package_list" http://hackage.haskell.org/distro/NixOS/packages.csv
echo

2
maintainers/scripts/luarocks-packages.csv
View File

@ -68,7 +68,7 @@ luautf8,,,,,,pstn
luazip,,,,,,
lua-yajl,,,,,,pstn
luuid,,,,,,
luv,,,,1.30.0-0,,
luv,,,,1.42.0-0,,
lyaml,,,,,,lblasc
markdown,,,,,,
mediator_lua,,,,,,

1 name src ref server version luaversion maintainers
68 luazip
69 lua-yajl pstn
70 luuid
71 luv 1.30.0-0 1.42.0-0
72 lyaml lblasc
73 markdown
74 mediator_lua

147
maintainers/scripts/rebuild-amount.sh
View File

@ -4,123 +4,130 @@ set -e
# --print: avoid dependency on environment
optPrint=
if [ "$1" == "--print" ]; then
optPrint=true
shift
optPrint=true
shift
fi
if [ "$#" != 1 ] && [ "$#" != 2 ]; then
cat <<-EOF
Usage: $0 [--print] commit-spec [commit-spec]
You need to be in a git-controlled nixpkgs tree.
The current state of the tree will be used if the second commit is missing.
EOF
exit 1
cat <<EOF
Usage: $0 [--print] from-commit-spec [to-commit-spec]
You need to be in a git-controlled nixpkgs tree.
The current state of the tree will be used if the second commit is missing.
Examples:
effect of latest commit:
$ $0 HEAD^
$ $0 --print HEAD^
effect of the whole patch series for 'staging' branch:
$ $0 origin/staging staging
EOF
exit 1
fi
# A slightly hacky way to get the config.
parallel="$(echo 'config.rebuild-amount.parallel or false' | nix-repl . 2>/dev/null \
| grep -v '^\(nix-repl.*\)\?$' | tail -n 1 || true)"
| grep -v '^\(nix-repl.*\)\?$' | tail -n 1 || true)"
echo "Estimating rebuild amount by counting changed Hydra jobs."
echo "Estimating rebuild amount by counting changed Hydra jobs (parallel=${parallel:-unset})."
toRemove=()
cleanup() {
rm -rf "${toRemove[@]}"
rm -rf "${toRemove[@]}"
}
trap cleanup EXIT SIGINT SIGQUIT ERR
MKTEMP='mktemp --tmpdir nix-rebuild-amount-XXXXXXXX'
nixexpr() {
cat <<-EONIX
let
lib = import $1/lib;
hydraJobs = import $1/pkgs/top-level/release.nix
# Compromise: accuracy vs. resources needed for evaluation.
{ supportedSystems = cfg.systems or [ "x86_64-linux" "x86_64-darwin" ]; };
cfg = (import $1 {}).config.rebuild-amount or {};
cat <<EONIX
let
lib = import $1/lib;
hydraJobs = import $1/pkgs/top-level/release.nix
# Compromise: accuracy vs. resources needed for evaluation.
{ supportedSystems = cfg.systems or [ "x86_64-linux" "x86_64-darwin" ]; };
cfg = (import $1 {}).config.rebuild-amount or {};
recurseIntoAttrs = attrs: attrs // { recurseForDerivations = true; };
recurseIntoAttrs = attrs: attrs // { recurseForDerivations = true; };
# hydraJobs leaves recurseForDerivations as empty attrmaps;
# that would break nix-env and we also need to recurse everywhere.
tweak = lib.mapAttrs
(name: val:
if name == "recurseForDerivations" then true
else if lib.isAttrs val && val.type or null != "derivation"
then recurseIntoAttrs (tweak val)
else val
);
# hydraJobs leaves recurseForDerivations as empty attrmaps;
# that would break nix-env and we also need to recurse everywhere.
tweak = lib.mapAttrs
(name: val:
if name == "recurseForDerivations" then true
else if lib.isAttrs val && val.type or null != "derivation"
then recurseIntoAttrs (tweak val)
else val
);
# Some of these contain explicit references to platform(s) we want to avoid;
# some even (transitively) depend on ~/.nixpkgs/config.nix (!)
blacklist = [
"tarball" "metrics" "manual"
"darwin-tested" "unstable" "stdenvBootstrapTools"
"moduleSystem" "lib-tests" # these just confuse the output
];
# Some of these contain explicit references to platform(s) we want to avoid;
# some even (transitively) depend on ~/.nixpkgs/config.nix (!)
blacklist = [
"tarball" "metrics" "manual"
"darwin-tested" "unstable" "stdenvBootstrapTools"
"moduleSystem" "lib-tests" # these just confuse the output
];
in
tweak (builtins.removeAttrs hydraJobs blacklist)
EONIX
in
tweak (builtins.removeAttrs hydraJobs blacklist)
EONIX
}
# Output packages in tree $2 that weren't in $1.
# Changing the output hash or name is taken as a change.
# Extra nix-env parameters can be in $3
newPkgs() {
# We use files instead of pipes, as running multiple nix-env processes
# could eat too much memory for a standard 4GiB machine.
local -a list
for i in 1 2; do
local l="$($MKTEMP)"
list[$i]="$l"
toRemove+=("$l")
# We use files instead of pipes, as running multiple nix-env processes
# could eat too much memory for a standard 4GiB machine.
local -a list
for i in 1 2; do
local l="$($MKTEMP)"
list[$i]="$l"
toRemove+=("$l")
local expr="$($MKTEMP)"
toRemove+=("$expr")
nixexpr "${!i}" > "$expr"
local expr="$($MKTEMP)"
toRemove+=("$expr")
nixexpr "${!i}" > "$expr"
nix-env -f "$expr" -qaP --no-name --out-path --show-trace $3 \
| sort > "${list[$i]}" &
nix-env -f "$expr" -qaP --no-name --out-path --show-trace $3 \
| sort > "${list[$i]}" &
if [ "$parallel" != "true" ]; then
wait
fi
done
if [ "$parallel" != "true" ]; then
wait
fi
done
wait
comm -13 "${list[@]}"
wait
comm -13 "${list[@]}"
}
# Prepare nixpkgs trees.
declare -a tree
for i in 1 2; do
if [ -n "${!i}" ]; then # use the given commit
dir="$($MKTEMP -d)"
tree[$i]="$dir"
toRemove+=("$dir")
if [ -n "${!i}" ]; then # use the given commit
dir="$($MKTEMP -d)"
tree[$i]="$dir"
toRemove+=("$dir")
git clone --shared --no-checkout --quiet . "${tree[$i]}"
(cd "${tree[$i]}" && git checkout --quiet "${!i}")
else #use the current tree
tree[$i]="$(pwd)"
fi
git clone --shared --no-checkout --quiet . "${tree[$i]}"
(cd "${tree[$i]}" && git checkout --quiet "${!i}")
else #use the current tree
tree[$i]="$(pwd)"
fi
done
newlist="$($MKTEMP)"
toRemove+=("$newlist")
# Notes:
# - the evaluation is done on x86_64-linux, like on Hydra.
# - using $newlist file so that newPkgs() isn't in a sub-shell (because of toRemove)
# - the evaluation is done on x86_64-linux, like on Hydra.
# - using $newlist file so that newPkgs() isn't in a sub-shell (because of toRemove)
newPkgs "${tree[1]}" "${tree[2]}" '--argstr system "x86_64-linux"' > "$newlist"
# Hacky: keep only the last word of each attribute path and sort.
sed -n 's/\([^. ]*\.\)*\([^. ]*\) .*$/\2/p' < "$newlist" \
| sort | uniq -c
| sort | uniq -c
if [ -n "$optPrint" ]; then
echo
cat "$newlist"
echo
cat "$newlist"
fi

9
maintainers/scripts/update-luarocks-packages
View File

@ -33,8 +33,7 @@ TMP_FILE="$(mktemp)"
GENERATED_NIXFILE="pkgs/development/lua-modules/generated-packages.nix"
LUAROCKS_CONFIG="$NIXPKGS_PATH/maintainers/scripts/luarocks-config.lua"
HEADER = """
/* {GENERATED_NIXFILE} is an auto-generated file -- DO NOT EDIT!
HEADER = """/* {GENERATED_NIXFILE} is an auto-generated file -- DO NOT EDIT!
Regenerate it with:
nixpkgs$ ./maintainers/scripts/update-luarocks-packages
@ -99,9 +98,8 @@ class LuaEditor(Editor):
header2 = textwrap.dedent(
# header2 = inspect.cleandoc(
"""
{ self, stdenv, lib, fetchurl, fetchgit, ... } @ args:
self: super:
with self;
{ self, stdenv, lib, fetchurl, fetchgit, callPackage, ... } @ args:
final: prev:
{
""")
f.write(header2)
@ -199,6 +197,7 @@ def generate_pkg_nix(plug: LuaPlugin):
log.debug("running %s", ' '.join(cmd))
output = subprocess.check_output(cmd, text=True)
output = "callPackage(" + output.strip() + ") {};\n\n"
return (plug, output)
def main():

2
maintainers/scripts/update.py
View File

@ -114,7 +114,7 @@ async def check_changes(package: Dict, worktree: str, update_info: str):
changes[0]['newVersion'] = json.loads((await obtain_new_version_process.stdout.read()).decode('utf-8'))
if 'files' not in changes[0]:
changed_files_process = await check_subprocess('git', 'diff', '--name-only', stdout=asyncio.subprocess.PIPE, cwd=worktree)
changed_files_process = await check_subprocess('git', 'diff', '--name-only', 'HEAD', stdout=asyncio.subprocess.PIPE, cwd=worktree)
changed_files = (await changed_files_process.stdout.read()).splitlines()
changes[0]['files'] = changed_files

22
maintainers/team-list.nix
View File

@ -137,7 +137,7 @@ with lib.maintainers; {
cleverca22
disassembler
jonringer
maveru
manveru
nrdxp
];
scope = "Input-Output Global employees, which maintain critical software";
@ -145,6 +145,7 @@ with lib.maintainers; {
jitsi = {
members = [
cleeyv
petabyteboy
ryantm
yuka
@ -164,6 +165,24 @@ with lib.maintainers; {
scope = "Maintain Kodi and related packages.";
};
linux-kernel = {
members = [
TredwellGit
ma27
nequissimus
qyliss
];
scope = "Maintain the Linux kernel.";
};
mate = {
members = [
j03
romildo
];
scope = "Maintain Mate desktop environment and related packages.";
};
matrix = {
members = [
ma27
@ -181,7 +200,6 @@ with lib.maintainers; {
openstack = {
members = [
angustrau
superherointj
SuperSandro2000
];
scope = "Maintain the ecosystem around OpenStack";

4
nixos/doc/manual/administration/cleaning-store.chapter.md
View File

@ -58,5 +58,5 @@ a while to finish.
## NixOS Boot Entries {#sect-nixos-gc-boot-entries}
If your `/boot` partition runs out of space, after clearing old profiles
you must rebuild your system with `nixos-rebuild` to update the `/boot`
partition and clear space.
you must rebuild your system with `nixos-rebuild boot` or `nixos-rebuild
switch` to update the `/boot` partition and clear space.

10
nixos/doc/manual/development/option-declarations.section.md
View File

@ -38,9 +38,19 @@ The function `mkOption` accepts the following arguments.
of the module will have to define the value of the option, otherwise
an error will be thrown.
`defaultText`
: A textual representation of the default value to be rendered verbatim in
the manual. Useful if the default value is a complex expression or depends
on other values or packages.
Use `lib.literalExpression` for a Nix expression, `lib.literalDocBook` for
a plain English description in DocBook format.
`example`
: An example value that will be shown in the NixOS manual.
You can use `lib.literalExpression` and `lib.literalDocBook` in the same way
as in `defaultText`.
`description`

14
nixos/doc/manual/development/writing-nixos-tests.section.md
View File

@ -159,6 +159,17 @@ The following methods are available on machine objects:
`execute`
: Execute a shell command, returning a list `(status, stdout)`.
If the command detaches, it must close stdout, as `execute` will wait
for this to consume all output reliably. This can be achieved by
redirecting stdout to stderr `>&2`, to `/dev/console`, `/dev/null` or
a file. Examples of detaching commands are `sleep 365d &`, where the
shell forks a new process that can write to stdout and `xclip -i`, where
the `xclip` command itself forks without closing stdout.
Takes an optional parameter `check_return` that defaults to `True`.
Setting this parameter to `False` will not check for the return code
and return -1 instead. This can be used for commands that shut down
the VM and would therefore break the pipe that would be used for
retrieving the return code.
`succeed`
@ -174,6 +185,9 @@ The following methods are available on machine objects:
- Dereferencing unset variables fail the command.
- It will wait for stdout to be closed. See `execute` for the
implications.
`fail`
: Like `succeed`, but raising an exception if the command returns a zero

3
nixos/doc/manual/from_md/administration/cleaning-store.chapter.xml
View File

@ -64,7 +64,8 @@ $ nix-store --optimise
<para>
If your <literal>/boot</literal> partition runs out of space,
after clearing old profiles you must rebuild your system with
<literal>nixos-rebuild</literal> to update the
<literal>nixos-rebuild boot</literal> or
<literal>nixos-rebuild switch</literal> to update the
<literal>/boot</literal> partition and clear space.
</para>
</section>

20
nixos/doc/manual/from_md/development/option-declarations.section.xml
View File

@ -57,13 +57,31 @@ options = {
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>
<literal>defaultText</literal>
</term>
<listitem>
<para>
A textual representation of the default value to be rendered
verbatim in the manual. Useful if the default value is a
complex expression or depends on other values or packages. Use
<literal>lib.literalExpression</literal> for a Nix expression,
<literal>lib.literalDocBook</literal> for a plain English
description in DocBook format.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>
<literal>example</literal>
</term>
<listitem>
<para>
An example value that will be shown in the NixOS manual.
An example value that will be shown in the NixOS manual. You
can use <literal>lib.literalExpression</literal> and
<literal>lib.literalDocBook</literal> in the same way as in
<literal>defaultText</literal>.
</para>
</listitem>
</varlistentry>

24
nixos/doc/manual/from_md/development/writing-nixos-tests.section.xml
View File

@ -266,7 +266,23 @@ start_all()
<listitem>
<para>
Execute a shell command, returning a list
<literal>(status, stdout)</literal>.
<literal>(status, stdout)</literal>. If the command detaches,
it must close stdout, as <literal>execute</literal> will wait
for this to consume all output reliably. This can be achieved
by redirecting stdout to stderr <literal>&gt;&amp;2</literal>,
to <literal>/dev/console</literal>,
<literal>/dev/null</literal> or a file. Examples of detaching
commands are <literal>sleep 365d &amp;</literal>, where the
shell forks a new process that can write to stdout and
<literal>xclip -i</literal>, where the
<literal>xclip</literal> command itself forks without closing
stdout. Takes an optional parameter
<literal>check_return</literal> that defaults to
<literal>True</literal>. Setting this parameter to
<literal>False</literal> will not check for the return code
and return -1 instead. This can be used for commands that shut
down the VM and would therefore break the pipe that would be
used for retrieving the return code.
</para>
</listitem>
</varlistentry>
@ -300,6 +316,12 @@ start_all()
Dereferencing unset variables fail the command.
</para>
</listitem>
<listitem>
<para>
It will wait for stdout to be closed. See
<literal>execute</literal> for the implications.
</para>
</listitem>
</itemizedlist>
</listitem>
</varlistentry>

5
nixos/doc/manual/from_md/installation/installing.chapter.xml
View File

@ -25,8 +25,11 @@
<para>
You are logged-in automatically as <literal>nixos</literal>. The
<literal>nixos</literal> user account has an empty password so you
can use <literal>sudo</literal> without a password.
can use <literal>sudo</literal> without a password:
</para>
<programlisting>
$ sudo -i
</programlisting>
<para>
If you downloaded the graphical ISO image, you can run
<literal>systemctl start display-manager</literal> to start the

622
nixos/doc/manual/from_md/release-notes/rl-2111.section.xml
View File

@ -1,5 +1,5 @@
<section xmlns="http://docbook.org/ns/docbook" xmlns:xlink="http://www.w3.org/1999/xlink" xml:id="sec-release-21.11">
<title>Release 21.11 (“?”, 2021.11/??)</title>
<title>Release 21.11 (“Porcupine”, 2021.11/??)</title>
<para>
In addition to numerous new and upgraded packages, this release has
the following highlights:
@ -15,6 +15,21 @@
<section xml:id="sec-release-21.11-highlights">
<title>Highlights</title>
<itemizedlist>
<listitem>
<para>
Nix has been updated to version 2.4, reference its
<link xlink:href="https://discourse.nixos.org/t/nix-2-4-released/15822">release
notes</link> for more information on what has changed. The
previous version of Nix, 2.3.16, remains available for the
time being in the <literal>nix_2_3</literal> package.
</para>
</listitem>
<listitem>
<para>
<literal>iptables</literal> now uses
<literal>nf_tables</literal> backend.
</para>
</listitem>
<listitem>
<para>
PHP now defaults to PHP 8.0, updated from 7.4.
@ -44,6 +59,29 @@
guide</link> is available.
</para>
</listitem>
<listitem>
<para>
Improvements have been made to the Hadoop module and package:
</para>
<itemizedlist spacing="compact">
<listitem>
<para>
HDFS and YARN now support production-ready highly
available deployments with automatic failover.
</para>
</listitem>
<listitem>
<para>
Hadoop now defaults to Hadoop 3, updated from 2.
</para>
</listitem>
<listitem>
<para>
JournalNode, ZKFS and HTTPFS services have been added.
</para>
</listitem>
</itemizedlist>
</listitem>
<listitem>
<para>
Activation scripts can now opt int to be run when running
@ -55,6 +93,21 @@
actions.
</para>
</listitem>
<listitem>
<para>
KDE Plasma now finally works on Wayland.
</para>
</listitem>
<listitem>
<para>
bash now defaults to major version 5.
</para>
</listitem>
<listitem>
<para>
Systemd was updated to version 249 (from 247).
</para>
</listitem>
<listitem>
<para>
Pantheon desktop has been updated to version 6. Due to changes
@ -62,6 +115,63 @@
<literal>gsettings set org.gnome.desktop.lockdown disable-lock-screen false</literal>.
</para>
</listitem>
<listitem>
<para>
<literal>kubernetes-helm</literal> now defaults to 3.7.0,
which introduced some breaking changes to the experimental OCI
manifest format. See
<link xlink:href="https://github.com/helm/community/blob/main/hips/hip-0006.md">HIP
6</link> for more details. <literal>helmfile</literal> also
defaults to 0.141.0, which is the minimum compatible version.
</para>
</listitem>
<listitem>
<para>
GNOME has been upgraded to 41. Please take a look at their
<link xlink:href="https://help.gnome.org/misc/release-notes/41.0/">Release
Notes</link> for details.
</para>
</listitem>
<listitem>
<para>
LXD support was greatly improved:
</para>
<itemizedlist spacing="compact">
<listitem>
<para>
building LXD images from configurations is now directly
possible with just nixpkgs
</para>
</listitem>
<listitem>
<para>
hydra is now building nixOS LXD images that can be used
standalone with full nixos-rebuild support
</para>
</listitem>
</itemizedlist>
</listitem>
<listitem>
<para>
OpenSSH was updated to version 8.8p1
</para>
<itemizedlist spacing="compact">
<listitem>
<para>
This breaks connections to old SSH daemons as ssh-rsa host
keys and ssh-rsa public keys that were signed with SHA-1
are disabled by default now
</para>
</listitem>
<listitem>
<para>
These can be re-enabled, see the
<link xlink:href="https://www.openssh.com/txt/release-8.8">OpenSSH
changelog</link> for details
</para>
</listitem>
</itemizedlist>
</listitem>
</itemizedlist>
</section>
<section xml:id="sec-release-21.11-new-services">
@ -80,7 +190,15 @@
<para>
<link xlink:href="https://github.com/xrelkd/clipcat/">clipcat</link>,
an X11 clipboard manager written in Rust. Available at
[services.clipcat](options.html#o pt-services.clipcat.enable).
<link xlink:href="options.html#opt-services.clipcat.enable">services.clipcat</link>.
</para>
</listitem>
<listitem>
<para>
<link xlink:href="https://github.com/dexidp/dex">dex</link>,
an OpenID Connect (OIDC) identity and OAuth 2.0 provider.
Available at
<link xlink:href="options.html#opt-services.dex.enable">services.dex</link>.
</para>
</listitem>
<listitem>
@ -90,6 +208,14 @@
<link xlink:href="options.html#opt-services.geoipupdate.enable">services.geoipupdate</link>.
</para>
</listitem>
<listitem>
<para>
<link xlink:href="https://github.com/jitsi/jibri">Jibri</link>,
a service for recording or streaming a Jitsi Meet conference.
Available as
<link xlink:href="options.html#opt-services.jibri.enable">services.jibri</link>.
</para>
</listitem>
<listitem>
<para>
<link xlink:href="https://www.isc.org/kea/">Kea</link>, ISCs
@ -97,6 +223,21 @@
<link xlink:href="options.html#opt-services.kea">services.kea</link>.
</para>
</listitem>
<listitem>
<para>
<link xlink:href="https://owncast.online/">owncast</link>,
self-hosted video live streaming solution. Available at
<link xlink:href="options.html#opt-services.owncast">services.owncast</link>.
</para>
</listitem>
<listitem>
<para>
<link xlink:href="https://joinpeertube.org/">PeerTube</link>,
developed by Framasoft, is the free and decentralized
alternative to video platforms. Available at
<link xlink:href="options.html#opt-services.peertube">services.peertube</link>.
</para>
</listitem>
<listitem>
<para>
<link xlink:href="https://sr.ht">sourcehut</link>, a
@ -235,6 +376,14 @@
<link linkend="opt-services.postfixadmin.enable">postfixadmin</link>.
</para>
</listitem>
<listitem>
<para>
<link xlink:href="https://wiki.servarr.com/prowlarr">prowlarr</link>,
an indexer manager/proxy built on the popular arr .net/reactjs
base stack
<link linkend="opt-services.prowlarr.enable">services.prowlarr</link>.
</para>
</listitem>
<listitem>
<para>
<link xlink:href="https://sr.ht/~emersion/soju">soju</link>, a
@ -287,11 +436,89 @@
<link linkend="opt-programs.pantheon-tweaks.enable">programs.pantheon-tweaks</link>.
</para>
</listitem>
<listitem>
<para>
<link xlink:href="https://github.com/DanielOgorchock/joycond">joycond</link>,
a service that uses <literal>hid-nintendo</literal> to provide
nintendo joycond pairing and better nintendo switch pro
controller support.
</para>
</listitem>
<listitem>
<para>
<link xlink:href="https://github.com/opensvc/multipath-tools">multipath</link>,
the device mapper multipath (DM-MP) daemon. Available as
<link linkend="opt-services.multipath.enable">services.multipath</link>.
</para>
</listitem>
<listitem>
<para>
<link xlink:href="https://www.seafile.com/en/home/">seafile</link>,
an open source file syncing &amp; sharing software. Available
as
<link xlink:href="options.html#opt-services.seafile.enable">services.seafile</link>.
</para>
</listitem>
<listitem>
<para>
<link xlink:href="https://github.com/mchehab/rasdaemon">rasdaemon</link>,
a hardware error logging daemon. Available as
<link linkend="opt-hardware.rasdaemon.enable">hardware.rasdaemon</link>.
</para>
</listitem>
<listitem>
<para>
<literal>code-server</literal>-module now available
</para>
</listitem>
<listitem>
<para>
<link xlink:href="https://github.com/xmrig/xmrig">xmrig</link>,
a high performance, open source, cross platform RandomX,
KawPow, CryptoNight and AstroBWT unified CPU/GPU miner and
RandomX benchmark.
</para>
</listitem>
<listitem>
<para>
Auto nice daemons
<link xlink:href="https://github.com/Nefelim4ag/Ananicy">ananicy</link>
and
<link xlink:href="https://gitlab.com/ananicy-cpp/ananicy-cpp/">ananicy-cpp</link>.
Available as
<link linkend="opt-services.ananicy.enable">services.ananicy</link>.
</para>
</listitem>
</itemizedlist>
</section>
<section xml:id="sec-release-21.11-incompatibilities">
<title>Backward Incompatibilities</title>
<itemizedlist>
<listitem>
<para>
The NixOS VM test framework,
<literal>pkgs.nixosTest</literal>/<literal>make-test-python.nix</literal>,
now requires detaching commands such as
<literal>succeed(&quot;foo &amp;&quot;)</literal> and
<literal>succeed(&quot;foo | xclip -i&quot;)</literal> to
close stdout. This can be done with a redirect such as
<literal>succeed(&quot;foo &gt;&amp;2 &amp;&quot;)</literal>.
This breaking change was necessitated by a race condition
causing tests to fail or hang. It applies to all methods that
invoke commands on the nodes, including
<literal>execute</literal>, <literal>succeed</literal>,
<literal>fail</literal>,
<literal>wait_until_succeeds</literal>,
<literal>wait_until_fails</literal>.
</para>
</listitem>
<listitem>
<para>
The <literal>services.wakeonlan</literal> option was removed,
and replaced with
<literal>networking.interfaces.&lt;name&gt;.wakeOnLan</literal>.
</para>
</listitem>
<listitem>
<para>
The <literal>security.wrappers</literal> option now requires
@ -302,6 +529,14 @@
nobody/nogroup, which is unsafe.
</para>
</listitem>
<listitem>
<para>
Since <literal>iptables</literal> now uses
<literal>nf_tables</literal> backend and
<literal>ipset</literal> doesn’t support it, some applications
(ferm, shorewall, firehol) may have limited functionality.
</para>
</listitem>
<listitem>
<para>
The <literal>paperless</literal> module and package have been
@ -389,7 +624,7 @@ Superuser created successfully.
<listitem>
<para>
The <literal>staticjinja</literal> package has been upgraded
from 1.0.4 to 4.1.0
from 1.0.4 to 4.1.1
</para>
</listitem>
<listitem>
@ -448,6 +683,17 @@ Superuser created successfully.
<link xlink:href="options.html#opt-services.geoipupdate.enable">services.geoipupdate</link>.
</para>
</listitem>
<listitem>
<para>
<literal>ihatemoney</literal> has been updated to version
5.1.1
(<link xlink:href="https://github.com/spiral-project/ihatemoney/blob/5.1.1/CHANGELOG.rst">release
notes</link>). If you serve ihatemoney by HTTP rather than
HTTPS, you must set
<link xlink:href="options.html#opt-services.ihatemoney.secureCookie">services.ihatemoney.secureCookie</link>
to <literal>false</literal>.
</para>
</listitem>
<listitem>
<para>
PHP 7.3 is no longer supported due to upstream not supporting
@ -968,8 +1214,8 @@ Superuser created successfully.
<listitem>
<para>
The <literal>varnish</literal> package was upgraded from 6.3.x
to 6.5.x. <literal>varnish60</literal> for the last LTS
release is also still available.
to 7.x. <literal>varnish60</literal> for the last LTS release
is also still available.
</para>
</listitem>
<listitem>
@ -1025,6 +1271,70 @@ Superuser created successfully.
changelog</link>.
</para>
</listitem>
<listitem>
<para>
<literal>opencv2</literal> no longer includes the non-free
libraries by default, and consequently
<literal>pfstools</literal> no longer includes OpenCV support
by default. Both packages now support an
<literal>enableUnfree</literal> option to re-enable this
functionality.
</para>
</listitem>
<listitem>
<para>
<literal>services.xserver.displayManager.defaultSession = &quot;plasma5&quot;</literal>
does not work anymore, instead use either
<literal>&quot;plasma&quot;</literal> for the Plasma X11
session or <literal>&quot;plasmawayland&quot;</literal> for
the Plasma Wayland sesison.
</para>
</listitem>
<listitem>
<para>
<literal>boot.kernelParams</literal> now only accepts one
command line parameter per string. This change is aimed to
reduce common mistakes like <quote>param = 12</quote>, which
would be parsed as 3 parameters.
</para>
</listitem>
<listitem>
<para>
The <literal>coursier</literal> package’s binary was renamed
from <literal>coursier</literal> to <literal>cs</literal>.
Completions which haven’t worked for a while should now work
with the renamed binary. To keep using
<literal>coursier</literal>, you can create a shell alias.
</para>
</listitem>
<listitem>
<para>
The <literal>services.mosquitto</literal> module has been
rewritten to support multiple listeners and per-listener
configuration. Module configurations from previous releases
will no longer work and must be updated.
</para>
</listitem>
<listitem>
<para>
Nextcloud 20 (<literal>pkgs.nextcloud20</literal>) has been
dropped because it was EOLed by upstream in 2021-10.
</para>
</listitem>
<listitem>
<para>
The <literal>virtualisation.pathsInNixDB</literal> option was
renamed
<link xlink:href="options.html#opt-virtualisation.additionalPaths"><literal>virtualisation.additionalPaths</literal></link>.
</para>
</listitem>
<listitem>
<para>
The <literal>services.ddclient.password</literal> option was
removed, and replaced with
<literal>services.ddclient.passwordFile</literal>.
</para>
</listitem>
</itemizedlist>
</section>
<section xml:id="sec-release-21.11-notable-changes">
@ -1044,6 +1354,46 @@ Superuser created successfully.
<literal>linuxPackages_latest</literal>) remain untouched.
</para>
</listitem>
<listitem>
<para>
In NixOS virtual machines (QEMU), the
<literal>virtualisation</literal> module has been updated with
new options:
</para>
<itemizedlist spacing="compact">
<listitem>
<para>
<link xlink:href="options.html#opt-virtualisation.forwardPorts"><literal>forwardPorts</literal></link>
to configure IPv4 port forwarding,
</para>
</listitem>
<listitem>
<para>
<link xlink:href="options.html#opt-virtualisation.sharedDirectories"><literal>sharedDirectories</literal></link>
to set up shared host directories,
</para>
</listitem>
<listitem>
<para>
<link xlink:href="options.html#opt-virtualisation.resolution"><literal>resolution</literal></link>
to set the screen resolution,
</para>
</listitem>
<listitem>
<para>
<link xlink:href="options.html#opt-virtualisation.useNixStoreImage"><literal>useNixStoreImage</literal></link>
to use a disk image for the Nix store instead of 9P.
</para>
</listitem>
</itemizedlist>
<para>
In addition, the default
<link xlink:href="options.html#opt-virtualisation.msize"><literal>msize</literal></link>
parameter in 9P filesystems (including /nix/store and all
shared directories) has been increased to 16K for improved
performance.
</para>
</listitem>
<listitem>
<para>
The setting
@ -1209,6 +1559,73 @@ Superuser created successfully.
</listitem>
</itemizedlist>
</listitem>
<listitem>
<para>
The
<link xlink:href="options.html#opt-networking.wireless.enable">networking.wireless</link>
module (based on wpa_supplicant) has been heavily reworked,
solving a number of issues and adding useful features:
</para>
<itemizedlist spacing="compact">
<listitem>
<para>
The automatic discovery of wireless interfaces at boot has
been made reliable again (issues
<link xlink:href="https://github.com/NixOS/nixpkgs/issues/101963">#101963</link>,
<link xlink:href="https://github.com/NixOS/nixpkgs/issues/23196">#23196</link>).
</para>
</listitem>
<listitem>
<para>
WPA3 and Fast BSS Transition (802.11r) are now enabled by
default for all networks.
</para>
</listitem>
<listitem>
<para>
Secrets like pre-shared keys and passwords can now be
handled safely, meaning without including them in a
world-readable file
(<literal>wpa_supplicant.conf</literal> under /nix/store).
This is achieved by storing the secrets in a secured
<link xlink:href="options.html#opt-networking.wireless.environmentFile">environmentFile</link>
and referring to them though environment variables that
are expanded inside the configuration.
</para>
</listitem>
<listitem>
<para>
With multiple interfaces declared, independent
wpa_supplicant daemons are started, one for each interface
(the services are named
<literal>wpa_supplicant-wlan0</literal>,
<literal>wpa_supplicant-wlan1</literal>, etc.).
</para>
</listitem>
<listitem>
<para>
The generated <literal>wpa_supplicant.conf</literal> file
is now formatted for easier reading.
</para>
</listitem>
<listitem>
<para>
A new
<link xlink:href="options.html#opt-networking.wireless.scanOnLowSignal">scanOnLowSignal</link>
option has been added to facilitate fast roaming between
access points (enabled by default).
</para>
</listitem>
<listitem>
<para>
A new
<link xlink:href="options.html#opt-networking.wireless.networks._name_.authProtocols">networks.&lt;name&gt;.authProtocols</link>
option has been added to change the authentication
protocols used when connecting to a network.
</para>
</listitem>
</itemizedlist>
</listitem>
<listitem>
<para>
The
@ -1218,6 +1635,23 @@ Superuser created successfully.
option.
</para>
</listitem>
<listitem>
<para>
The
<link xlink:href="options.html#opt-services.smokeping.host">services.smokeping.host</link>
option was added and defaulted to
<literal>localhost</literal>. Before,
<literal>smokeping</literal> listened to all interfaces by
default. NixOS defaults generally aim to provide
non-Internet-exposed defaults for databases and internal
monitoring tools, see e.g.
<link xlink:href="https://github.com/NixOS/nixpkgs/issues/100192">#100192</link>.
Further, the systemd service for <literal>smokeping</literal>
got reworked defaults for increased operational stability, see
<link xlink:href="https://github.com/NixOS/nixpkgs/pull/144127">PR
#144127</link> for details.
</para>
</listitem>
<listitem>
<para>
The
@ -1324,6 +1758,184 @@ Superuser created successfully.
<literal>/etc/xdg/mimeapps.list</literal>.
</para>
</listitem>
<listitem>
<para>
Kopia was upgraded from 0.8.x to 0.9.x. Please read the
<link xlink:href="https://github.com/kopia/kopia/releases/tag/v0.9.0">upstream
release notes</link> for changes and upgrade instructions.
</para>
</listitem>
<listitem>
<para>
The <literal>systemd.network</literal> module has gained
support for the FooOverUDP link type.
</para>
</listitem>
<listitem>
<para>
The <literal>networking</literal> module has a new
<literal>networking.fooOverUDP</literal> option to configure
Foo-over-UDP encapsulations.
</para>
</listitem>
<listitem>
<para>
<literal>networking.sits</literal> now supports Foo-over-UDP
encapsulation.
</para>
</listitem>
<listitem>
<para>
Changing systemd <literal>.socket</literal> units now restarts
them and stops the service that is activated by them.
Additionally, services with
<literal>stopOnChange = false</literal> don’t break anymore
when they are socket-activated.
</para>
</listitem>
<listitem>
<para>
The <literal>virtualisation.libvirtd</literal> module has been
refactored and updated with new options:
</para>
<itemizedlist spacing="compact">
<listitem>
<para>
<literal>virtualisation.libvirtd.qemu*</literal> options
(e.g.:
<literal>virtualisation.libvirtd.qemuRunAsRoot</literal>)
were moved to
<link xlink:href="options.html#opt-virtualisation.libvirtd.qemu"><literal>virtualisation.libvirtd.qemu</literal></link>
submodule,
</para>
</listitem>
<listitem>
<para>
software TPM1/TPM2 support (e.g.: Windows 11 guests)
(<link xlink:href="options.html#opt-virtualisation.libvirtd.qemu.swtpm"><literal>virtualisation.libvirtd.qemu.swtpm</literal></link>),
</para>
</listitem>
<listitem>
<para>
custom OVMF package (e.g.:
<literal>pkgs.OVMFFull</literal> with HTTP, CSM and Secure
Boot support)
(<link xlink:href="options.html#opt-virtualisation.libvirtd.qemu.ovmf.package"><literal>virtualisation.libvirtd.qemu.ovmf.package</literal></link>).
</para>
</listitem>
</itemizedlist>
</listitem>
<listitem>
<para>
The <literal>cawbird</literal> Twitter client now uses its own
API keys to count as different application than upstream
builds. This is done to evade application-level rate limiting.
While existing accounts continue to work, users may want to
remove and re-register their account in the client to enjoy a
better user experience and benefit from this change.
</para>
</listitem>
<listitem>
<para>
A new option
<literal>services.prometheus.enableReload</literal> has been
added which can be enabled to reload the prometheus service
when its config file changes instead of restarting.
</para>
</listitem>
<listitem>
<para>
The option
<literal>services.prometheus.environmentFile</literal> has
been removed since it was causing
<link xlink:href="https://github.com/NixOS/nixpkgs/issues/126083">issues</link>
and Prometheus now has native support for secret files, i.e.
<literal>basic_auth.password_file</literal> and
<literal>authorization.credentials_file</literal>.
</para>
</listitem>
<listitem>
<para>
Dokuwiki now supports caddy! However
</para>
<itemizedlist spacing="compact">
<listitem>
<para>
the nginx option has been removed, in the new
configuration, please use the
<literal>dokuwiki.webserver = &quot;nginx&quot;</literal>
instead.
</para>
</listitem>
<listitem>
<para>
The <quote>${hostname}</quote> option has been deprecated,
please use
<literal>dokuwiki.sites = [ &quot;${hostname}&quot; ]</literal>
instead
</para>
</listitem>
</itemizedlist>
</listitem>
<listitem>
<para>
The
<link xlink:href="options.html#opt-services.unifi.enable">services.unifi</link>
module has been reworked, solving a number of issues. This
leads to several user facing changes:
</para>
<itemizedlist spacing="compact">
<listitem>
<para>
The <literal>services.unifi.dataDir</literal> option is
removed and the data is now always located under
<literal>/var/lib/unifi/data</literal>. This is done to
make better use of systemd state direcotiry and thus
making the service restart more reliable.
</para>
</listitem>
<listitem>
<para>
The unifi logs can now be found under:
<literal>/var/log/unifi</literal> instead of
<literal>/var/lib/unifi/logs</literal>.
</para>
</listitem>
<listitem>
<para>
The unifi run directory can now be found under:
<literal>/run/unifi</literal> instead of
<literal>/var/lib/unifi/run</literal>.
</para>
</listitem>
</itemizedlist>
</listitem>
<listitem>
<para>
<literal>security.pam.services.&lt;name&gt;.makeHomeDir</literal>
now uses <literal>umask=0077</literal> instead of
<literal>umask=0022</literal> when creating the home
directory.
</para>
</listitem>
<listitem>
<para>
Loki has had another release. Some default values have been
changed for the configuration and some configuration options
have been renamed. For more details, please check
<link xlink:href="https://grafana.com/docs/loki/latest/upgrading/#240">the
upgrade guide</link>.
</para>
</listitem>
<listitem>
<para>
<literal>julia</literal> now refers to
<literal>julia-stable</literal> instead of
<literal>julia-lts</literal>. In practice this means it has
been upgraded from <literal>1.0.4</literal> to
<literal>1.5.4</literal>.
</para>
</listitem>
</itemizedlist>
</section>
</section>

5
nixos/doc/manual/installation/installing.chapter.md
View File

@ -15,7 +15,10 @@ finished booting, it should have detected most of your hardware.
The NixOS manual is available by running `nixos-help`.
You are logged-in automatically as `nixos`. The `nixos` user account has
an empty password so you can use `sudo` without a password.
an empty password so you can use `sudo` without a password:
```ShellSession
$ sudo -i
```
If you downloaded the graphical ISO image, you can run `systemctl
start display-manager` to start the desktop environment. If you want

6
nixos/doc/manual/md-to-db.sh
View File

@ -6,7 +6,7 @@
# into DocBook files in the from_md folder.
DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )"
pushd $DIR
pushd "$DIR"
# NOTE: Keep in sync with Nixpkgs manual (/doc/Makefile).
# TODO: Remove raw-attribute when we can get rid of DocBook altogether.
@ -29,7 +29,7 @@ mapfile -t MD_FILES < <(find . -type f -regex '.*\.md$')
for mf in ${MD_FILES[*]}; do
if [ "${mf: -11}" == ".section.md" ]; then
mkdir -p $(dirname "$OUT/$mf")
mkdir -p "$(dirname "$OUT/$mf")"
OUTFILE="$OUT/${mf%".section.md"}.section.xml"
pandoc "$mf" "${pandoc_flags[@]}" \
-o "$OUTFILE"
@ -37,7 +37,7 @@ for mf in ${MD_FILES[*]}; do
fi
if [ "${mf: -11}" == ".chapter.md" ]; then
mkdir -p $(dirname "$OUT/$mf")
mkdir -p "$(dirname "$OUT/$mf")"
OUTFILE="$OUT/${mf%".chapter.md"}.chapter.xml"
pandoc "$mf" "${pandoc_flags[@]}" \
--top-level-division=chapter \

257
nixos/doc/manual/release-notes/rl-2111.section.md
View File

@ -1,4 +1,4 @@
# Release 21.11 (“?”, 2021.11/??) {#sec-release-21.11}
# Release 21.11 (“Porcupine”, 2021.11/??) {#sec-release-21.11}
In addition to numerous new and upgraded packages, this release has the following highlights:
@ -6,6 +6,10 @@ In addition to numerous new and upgraded packages, this release has the followin
## Highlights {#sec-release-21.11-highlights}
- Nix has been updated to version 2.4, reference its [release notes](https://discourse.nixos.org/t/nix-2-4-released/15822) for more information on what has changed. The previous version of Nix, 2.3.16, remains available for the time being in the `nix_2_3` package.
- `iptables` now uses `nf_tables` backend.
- PHP now defaults to PHP 8.0, updated from 7.4.
- kOps now defaults to 1.21.1, which uses containerd as the default runtime.
@ -16,23 +20,54 @@ In addition to numerous new and upgraded packages, this release has the followin
- spark now defaults to spark 3, updated from 2. A [migration guide](https://spark.apache.org/docs/latest/core-migration-guide.html#upgrading-from-core-24-to-30) is available.
- Improvements have been made to the Hadoop module and package:
- HDFS and YARN now support production-ready highly available deployments with automatic failover.
- Hadoop now defaults to Hadoop 3, updated from 2.
- JournalNode, ZKFS and HTTPFS services have been added.
- Activation scripts can now opt int to be run when running `nixos-rebuild dry-activate` and detect the dry activation by reading `$NIXOS_ACTION`.
This allows activation scripts to output what they would change if the activation was really run.
The users/modules activation script supports this and outputs some of is actions.
- KDE Plasma now finally works on Wayland.
- bash now defaults to major version 5.
- Systemd was updated to version 249 (from 247).
- Pantheon desktop has been updated to version 6. Due to changes of screen locker, if locking doesn't work for you, please try `gsettings set org.gnome.desktop.lockdown disable-lock-screen false`.
- `kubernetes-helm` now defaults to 3.7.0, which introduced some breaking changes to the experimental OCI manifest format. See [HIP 6](https://github.com/helm/community/blob/main/hips/hip-0006.md) for more details.
`helmfile` also defaults to 0.141.0, which is the minimum compatible version.
- GNOME has been upgraded to 41. Please take a look at their [Release Notes](https://help.gnome.org/misc/release-notes/41.0/) for details.
- LXD support was greatly improved:
- building LXD images from configurations is now directly possible with just nixpkgs
- hydra is now building nixOS LXD images that can be used standalone with full nixos-rebuild support
- OpenSSH was updated to version 8.8p1
- This breaks connections to old SSH daemons as ssh-rsa host keys and ssh-rsa public keys that were signed with SHA-1 are disabled by default now
- These can be re-enabled, see the [OpenSSH changelog](https://www.openssh.com/txt/release-8.8) for details
## New Services {#sec-release-21.11-new-services}
- [btrbk](https://digint.ch/btrbk/index.html), a backup tool for btrfs subvolumes, taking advantage of btrfs specific capabilities to create atomic snapshots and transfer them incrementally to your backup locations. Available as [services.btrbk](options.html#opt-services.brtbk.instances).
- [clipcat](https://github.com/xrelkd/clipcat/), an X11 clipboard manager written in Rust. Available at [services.clipcat](options.html#o
pt-services.clipcat.enable).
- [clipcat](https://github.com/xrelkd/clipcat/), an X11 clipboard manager written in Rust. Available at [services.clipcat](options.html#opt-services.clipcat.enable).
- [dex](https://github.com/dexidp/dex), an OpenID Connect (OIDC) identity and OAuth 2.0 provider. Available at [services.dex](options.html#opt-services.dex.enable).
- [geoipupdate](https://github.com/maxmind/geoipupdate), a GeoIP database updater from MaxMind. Available as [services.geoipupdate](options.html#opt-services.geoipupdate.enable).
- [Jibri](https://github.com/jitsi/jibri), a service for recording or streaming a Jitsi Meet conference. Available as [services.jibri](options.html#opt-services.jibri.enable).
- [Kea](https://www.isc.org/kea/), ISCs 2nd generation DHCP and DDNS server suite. Available at [services.kea](options.html#opt-services.kea).
- [owncast](https://owncast.online/), self-hosted video live streaming solution. Available at [services.owncast](options.html#opt-services.owncast).
- [PeerTube](https://joinpeertube.org/), developed by Framasoft, is the free and decentralized alternative to video platforms. Available at [services.peertube](options.html#opt-services.peertube).
- [sourcehut](https://sr.ht), a collection of tools useful for software development. Available as [services.sourcehut](options.html#opt-services.sourcehut.enable).
- [ucarp](https://download.pureftpd.org/pub/ucarp/README), an userspace implementation of the Common Address Redundancy Protocol (CARP). Available as [networking.ucarp](options.html#opt-networking.ucarp.enable).
@ -65,7 +100,7 @@ pt-services.clipcat.enable).
Available as [isso](#opt-services.isso.enable)
- [navidrome](https://www.navidrome.org/), a personal music streaming server with
subsonic-compatible api. Available as [navidrome](#opt-services.navidrome.enable).
subsonic-compatible api. Available as [navidrome](#opt-services.navidrome.enable).
- [fluidd](https://docs.fluidd.xyz/), a Klipper web interface for managing 3d printers using moonraker. Available as [fluidd](#opt-services.fluidd.enable).
@ -73,6 +108,8 @@ subsonic-compatible api. Available as [navidrome](#opt-services.navidrome.enable
- [postfixadmin](https://postfixadmin.sourceforge.io/), a web based virtual user administration interface for Postfix mail servers. Available as [postfixadmin](#opt-services.postfixadmin.enable).
- [prowlarr](https://wiki.servarr.com/prowlarr), an indexer manager/proxy built on the popular arr .net/reactjs base stack [services.prowlarr](#opt-services.prowlarr.enable).
- [soju](https://sr.ht/~emersion/soju), a user-friendly IRC bouncer. Available as [services.soju](options.html#opt-services.soju.enable).
- [nats](https://nats.io/), a high performance cloud and edge messaging system. Available as [services.nats](#opt-services.nats.enable).
@ -90,59 +127,84 @@ subsonic-compatible api. Available as [navidrome](#opt-services.navidrome.enable
- [pantheon-tweaks](https://github.com/pantheon-tweaks/pantheon-tweaks), an unofficial system settings panel for Pantheon. Available as [programs.pantheon-tweaks](#opt-programs.pantheon-tweaks.enable).
- [joycond](https://github.com/DanielOgorchock/joycond), a service that uses `hid-nintendo` to provide nintendo joycond pairing and better nintendo switch pro controller support.
- [multipath](https://github.com/opensvc/multipath-tools), the device mapper multipath (DM-MP) daemon. Available as [services.multipath](#opt-services.multipath.enable).
- [seafile](https://www.seafile.com/en/home/), an open source file syncing & sharing software. Available as [services.seafile](options.html#opt-services.seafile.enable).
- [rasdaemon](https://github.com/mchehab/rasdaemon), a hardware error logging daemon. Available as [hardware.rasdaemon](#opt-hardware.rasdaemon.enable).
- `code-server`-module now available
- [xmrig](https://github.com/xmrig/xmrig), a high performance, open source, cross platform RandomX, KawPow, CryptoNight and AstroBWT unified CPU/GPU miner and RandomX benchmark.
- Auto nice daemons [ananicy](https://github.com/Nefelim4ag/Ananicy) and [ananicy-cpp](https://gitlab.com/ananicy-cpp/ananicy-cpp/). Available as [services.ananicy](#opt-services.ananicy.enable).
## Backward Incompatibilities {#sec-release-21.11-incompatibilities}
- The NixOS VM test framework, `pkgs.nixosTest`/`make-test-python.nix`, now requires detaching commands such as `succeed("foo &")` and `succeed("foo | xclip -i")` to close stdout.
This can be done with a redirect such as `succeed("foo >&2 &")`. This breaking change was necessitated by a race condition causing tests to fail or hang.
It applies to all methods that invoke commands on the nodes, including `execute`, `succeed`, `fail`, `wait_until_succeeds`, `wait_until_fails`.
- The `services.wakeonlan` option was removed, and replaced with `networking.interfaces.<name>.wakeOnLan`.
- The `security.wrappers` option now requires to always specify an owner, group and whether the setuid/setgid bit should be set.
This is motivated by the fact that before NixOS 21.11, specifying either setuid or setgid but not owner/group resulted in wrappers owned by nobody/nogroup, which is unsafe.
- Since `iptables` now uses `nf_tables` backend and `ipset` doesn't support it, some applications (ferm, shorewall, firehol) may have limited functionality.
- The `paperless` module and package have been removed. All users should migrate to the
successor `paperless-ng` instead. The Paperless project [has been
archived](https://github.com/the-paperless-project/paperless/commit/9b0063c9731f7c5f65b1852cb8caff97f5e40ba4)
and advises all users to use `paperless-ng` instead.
Users can use the `services.paperless-ng` module as a replacement while noting the following incompatibilities:
- `services.paperless.ocrLanguages` has no replacement. Users should migrate to [`services.paperless-ng.extraConfig`](options.html#opt-services.paperless-ng.extraConfig) instead:
```nix
{
services.paperless-ng.extraConfig = {
# Provide languages as ISO 639-2 codes
# separated by a plus (+) sign.
# https://en.wikipedia.org/wiki/List_of_ISO_639-2_codes
PAPERLESS_OCR_LANGUAGE = "deu+eng+jpn"; # German & English & Japanse
};
}
```
- If you previously specified `PAPERLESS_CONSUME_MAIL_*` settings in
`services.paperless.extraConfig` you should remove those options now. You
now *must* define those settings in the admin interface of paperless-ng.
- `services.paperless.ocrLanguages` has no replacement. Users should migrate to [`services.paperless-ng.extraConfig`](options.html#opt-services.paperless-ng.extraConfig) instead:
- Option `services.paperless.manage` no longer exists.
Use the script at `${services.paperless-ng.dataDir}/paperless-ng-manage` instead.
Note that this script only exists after the `paperless-ng` service has been
started at least once.
```nix
{
services.paperless-ng.extraConfig = {
# Provide languages as ISO 639-2 codes
# separated by a plus (+) sign.
# https://en.wikipedia.org/wiki/List_of_ISO_639-2_codes
PAPERLESS_OCR_LANGUAGE = "deu+eng+jpn"; # German & English & Japanse
};
}
```
- After switching to the new system configuration you should run the Django
management command to reindex your documents and optionally create a user,
if you don't have one already.
- If you previously specified `PAPERLESS_CONSUME_MAIL_*` settings in
`services.paperless.extraConfig` you should remove those options now. You
now _must_ define those settings in the admin interface of paperless-ng.
To do so, enter the data directory (the value of
`services.paperless-ng.dataDir`, `/var/lib/paperless` by default), switch
to the paperless user and execute the management command like below:
```
$ cd /var/lib/paperless
$ su paperless -s /bin/sh
$ ./paperless-ng-manage document_index reindex
# if not already done create a user account, paperless-ng requires a login
$ ./paperless-ng-manage createsuperuser
Username (leave blank to use 'paperless'): my-user-name
Email address: me@example.com
Password: **********
Password (again): **********
Superuser created successfully.
```
- Option `services.paperless.manage` no longer exists.
Use the script at `${services.paperless-ng.dataDir}/paperless-ng-manage` instead.
Note that this script only exists after the `paperless-ng` service has been
started at least once.
- The `staticjinja` package has been upgraded from 1.0.4 to 4.1.0
- After switching to the new system configuration you should run the Django
management command to reindex your documents and optionally create a user,
if you don't have one already.
To do so, enter the data directory (the value of
`services.paperless-ng.dataDir`, `/var/lib/paperless` by default), switch
to the paperless user and execute the management command like below:
```
$ cd /var/lib/paperless
$ su paperless -s /bin/sh
$ ./paperless-ng-manage document_index reindex
# if not already done create a user account, paperless-ng requires a login
$ ./paperless-ng-manage createsuperuser
Username (leave blank to use 'paperless'): my-user-name
Email address: me@example.com
Password: **********
Password (again): **********
Superuser created successfully.
```
- The `staticjinja` package has been upgraded from 1.0.4 to 4.1.1
- Firefox v91 does not support addons with invalid signature anymore. Firefox ESR needs to be used for nix addon support.
@ -171,6 +233,8 @@ subsonic-compatible api. Available as [navidrome](#opt-services.navidrome.enable
- `services.geoip-updater` was broken and has been replaced by [services.geoipupdate](options.html#opt-services.geoipupdate.enable).
- `ihatemoney` has been updated to version 5.1.1 ([release notes](https://github.com/spiral-project/ihatemoney/blob/5.1.1/CHANGELOG.rst)). If you serve ihatemoney by HTTP rather than HTTPS, you must set [services.ihatemoney.secureCookie](options.html#opt-services.ihatemoney.secureCookie) to `false`.
- PHP 7.3 is no longer supported due to upstream not supporting this version for the entire lifecycle of the 21.11 release.
- Those making use of `buildBazelPackage` will need to regenerate the fetch hashes (preferred), or set `fetchConfigured = false;`.
@ -237,28 +301,32 @@ subsonic-compatible api. Available as [navidrome](#opt-services.navidrome.enable
* The `bitwarden_rs` packages and modules were renamed to `vaultwarden`
[following upstream](https://github.com/dani-garcia/vaultwarden/discussions/1642). More specifically,
* `pkgs.bitwarden_rs`, `pkgs.bitwarden_rs-sqlite`, `pkgs.bitwarden_rs-mysql` and
- `pkgs.bitwarden_rs`, `pkgs.bitwarden_rs-sqlite`, `pkgs.bitwarden_rs-mysql` and
`pkgs.bitwarden_rs-postgresql` were renamed to `pkgs.vaultwarden`, `pkgs.vaultwarden-sqlite`,
`pkgs.vaultwarden-mysql` and `pkgs.vaultwarden-postgresql`, respectively.
* Old names are preserved as aliases for backwards compatibility, but may be removed in the future.
* The `bitwarden_rs` executable was also renamed to `vaultwarden` in all packages.
* `pkgs.bitwarden_rs-vault` was renamed to `pkgs.vaultwarden-vault`.
* `pkgs.bitwarden_rs-vault` is preserved as an alias for backwards compatibility, but may be removed in the future.
* The static files were moved from `/usr/share/bitwarden_rs` to `/usr/share/vaultwarden`.
- Old names are preserved as aliases for backwards compatibility, but may be removed in the future.
- The `bitwarden_rs` executable was also renamed to `vaultwarden` in all packages.
* The `services.bitwarden_rs` config module was renamed to `services.vaultwarden`.
* `services.bitwarden_rs` is preserved as an alias for backwards compatibility, but may be removed in the future.
- `pkgs.bitwarden_rs-vault` was renamed to `pkgs.vaultwarden-vault`.
* `systemd.services.bitwarden_rs`, `systemd.services.backup-bitwarden_rs` and `systemd.timers.backup-bitwarden_rs`
- `pkgs.bitwarden_rs-vault` is preserved as an alias for backwards compatibility, but may be removed in the future.
- The static files were moved from `/usr/share/bitwarden_rs` to `/usr/share/vaultwarden`.
- The `services.bitwarden_rs` config module was renamed to `services.vaultwarden`.
- `services.bitwarden_rs` is preserved as an alias for backwards compatibility, but may be removed in the future.
- `systemd.services.bitwarden_rs`, `systemd.services.backup-bitwarden_rs` and `systemd.timers.backup-bitwarden_rs`
were renamed to `systemd.services.vaultwarden`, `systemd.services.backup-vaultwarden` and
`systemd.timers.backup-vaultwarden`, respectively.
* Old names are preserved as aliases for backwards compatibility, but may be removed in the future.
* `users.users.bitwarden_rs` and `users.groups.bitwarden_rs` were renamed to `users.users.vaultwarden` and
- Old names are preserved as aliases for backwards compatibility, but may be removed in the future.
- `users.users.bitwarden_rs` and `users.groups.bitwarden_rs` were renamed to `users.users.vaultwarden` and
`users.groups.vaultwarden`, respectively.
* The data directory remains located at `/var/lib/bitwarden_rs`, for backwards compatibility.
- The data directory remains located at `/var/lib/bitwarden_rs`, for backwards compatibility.
- `yggdrasil` was upgraded to a new major release with breaking changes, see [upstream changelog](https://github.com/yggdrasil-network/yggdrasil-go/releases/tag/v0.4.0).
@ -271,6 +339,7 @@ subsonic-compatible api. Available as [navidrome](#opt-services.navidrome.enable
- `tt-rss` was upgraded to the commit on 2021-06-21, which has breaking changes. If you use `services.tt-rss.extraConfig` you should migrate to the `putenv`-style configuration. See [this Discourse post](https://community.tt-rss.org/t/rip-config-php-hello-classes-config-php/4337) in the tt-rss forums for more details.
- The following Visual Studio Code extensions were renamed to keep the naming convention uniform.
- `bbenoist.Nix` -> `bbenoist.nix`
- `CoenraadS.bracket-pair-colorizer` -> `coenraads.bracket-pair-colorizer`
- `golang.Go` -> `golang.go`
@ -290,12 +359,12 @@ subsonic-compatible api. Available as [navidrome](#opt-services.navidrome.enable
- The `yambar` package has been split into `yambar` and `yambar-wayland`, corresponding to the xorg and wayland backend respectively. Please switch to `yambar-wayland` if you are on wayland.
- The `services.minio` module gained an additional option `consoleAddress`, that
configures the address and port the web UI is listening, it defaults to `:9001`.
To be able to access the web UI this port needs to be opened in the firewall.
configures the address and port the web UI is listening, it defaults to `:9001`.
To be able to access the web UI this port needs to be opened in the firewall.
- The `varnish` package was upgraded from 6.3.x to 6.5.x. `varnish60` for the last LTS release is also still available.
- The `varnish` package was upgraded from 6.3.x to 7.x. `varnish60` for the last LTS release is also still available.
- The `kubernetes` package was upgraded to 1.22. The `kubernetes.apiserver.kubeletHttps` option was removed and HTTPS is always used.
- The `kubernetes` package was upgraded to 1.22. The `kubernetes.apiserver.kubeletHttps` option was removed and HTTPS is always used.
- The attribute `linuxPackages_latest_hardened` was dropped because the hardened patches
lag behind the upstream kernel which made version bumps harder. If you want to use
@ -314,11 +383,37 @@ To be able to access the web UI this port needs to be opened in the firewall.
respectively. As a result `services.datadog-agent` has had breaking changes to the
configuration file. For details, see the [upstream changelog](https://github.com/DataDog/datadog-agent/blob/main/CHANGELOG.rst).
- `opencv2` no longer includes the non-free libraries by default, and consequently `pfstools` no longer includes OpenCV support by default. Both packages now support an `enableUnfree` option to re-enable this functionality.
- `services.xserver.displayManager.defaultSession = "plasma5"` does not work anymore, instead use either `"plasma"` for the Plasma X11 session or `"plasmawayland"` for the Plasma Wayland sesison.
- `boot.kernelParams` now only accepts one command line parameter per string. This change is aimed to reduce common mistakes like "param = 12", which would be parsed as 3 parameters.
- The `coursier` package's binary was renamed from `coursier` to `cs`. Completions which haven't worked for a while should now work with the renamed binary. To keep using `coursier`, you can create a shell alias.
- The `services.mosquitto` module has been rewritten to support multiple listeners and per-listener configuration.
Module configurations from previous releases will no longer work and must be updated.
- Nextcloud 20 (`pkgs.nextcloud20`) has been dropped because it was EOLed by upstream in 2021-10.
- The `virtualisation.pathsInNixDB` option was renamed
[`virtualisation.additionalPaths`](options.html#opt-virtualisation.additionalPaths).
- The `services.ddclient.password` option was removed, and replaced with `services.ddclient.passwordFile`.
## Other Notable Changes {#sec-release-21.11-notable-changes}
- The linux kernel package infrastructure was moved out of `all-packages.nix`, and restructured. Linux related functions and attributes now live under the `pkgs.linuxKernel` attribute set.
In particular the versioned `linuxPackages_*` package sets (such as `linuxPackages_5_4`) and kernels from `pkgs` were moved there and now live under `pkgs.linuxKernel.packages.*`. The unversioned ones (such as `linuxPackages_latest`) remain untouched.
- In NixOS virtual machines (QEMU), the `virtualisation` module has been updated with new options:
- [`forwardPorts`](options.html#opt-virtualisation.forwardPorts) to configure IPv4 port forwarding,
- [`sharedDirectories`](options.html#opt-virtualisation.sharedDirectories) to set up shared host directories,
- [`resolution`](options.html#opt-virtualisation.resolution) to set the screen resolution,
- [`useNixStoreImage`](options.html#opt-virtualisation.useNixStoreImage) to use a disk image for the Nix store instead of 9P.
In addition, the default [`msize`](options.html#opt-virtualisation.msize) parameter in 9P filesystems (including /nix/store and all shared directories) has been increased to 16K for improved performance.
- The setting [`services.openssh.logLevel`](options.html#opt-services.openssh.logLevel) `"VERBOSE"` `"INFO"`. This brings NixOS in line with upstream and other Linux distributions, and reduces log spam on servers due to bruteforcing botnets.
However, if [`services.fail2ban.enable`](options.html#opt-services.fail2ban.enable) is `true`, the `fail2ban` will override the verbosity to `"VERBOSE"`, so that `fail2ban` can observe the failed login attempts from the SSH logs.
@ -365,8 +460,20 @@ To be able to access the web UI this port needs to be opened in the firewall.
`myhostname`, but before `dns` should use the default priority
- NSS modules which should come after `dns` should use mkAfter.
- The [networking.wireless](options.html#opt-networking.wireless.enable) module (based on wpa_supplicant) has been heavily reworked, solving a number of issues and adding useful features:
- The automatic discovery of wireless interfaces at boot has been made reliable again (issues [#101963](https://github.com/NixOS/nixpkgs/issues/101963), [#23196](https://github.com/NixOS/nixpkgs/issues/23196)).
- WPA3 and Fast BSS Transition (802.11r) are now enabled by default for all networks.
- Secrets like pre-shared keys and passwords can now be handled safely, meaning without including them in a world-readable file (`wpa_supplicant.conf` under /nix/store).
This is achieved by storing the secrets in a secured [environmentFile](options.html#opt-networking.wireless.environmentFile) and referring to them though environment variables that are expanded inside the configuration.
- With multiple interfaces declared, independent wpa_supplicant daemons are started, one for each interface (the services are named `wpa_supplicant-wlan0`, `wpa_supplicant-wlan1`, etc.).
- The generated `wpa_supplicant.conf` file is now formatted for easier reading.
- A new [scanOnLowSignal](options.html#opt-networking.wireless.scanOnLowSignal) option has been added to facilitate fast roaming between access points (enabled by default).
- A new [networks.&lt;name&gt;.authProtocols](options.html#opt-networking.wireless.networks._name_.authProtocols) option has been added to change the authentication protocols used when connecting to a network.
- The [networking.wireless.iwd](options.html#opt-networking.wireless.iwd.enable) module has a new [networking.wireless.iwd.settings](options.html#opt-networking.wireless.iwd.settings) option.
- The [services.smokeping.host](options.html#opt-services.smokeping.host) option was added and defaulted to `localhost`. Before, `smokeping` listened to all interfaces by default. NixOS defaults generally aim to provide non-Internet-exposed defaults for databases and internal monitoring tools, see e.g. [#100192](https://github.com/NixOS/nixpkgs/issues/100192). Further, the systemd service for `smokeping` got reworked defaults for increased operational stability, see [PR #144127](https://github.com/NixOS/nixpkgs/pull/144127) for details.
- The [services.syncoid.enable](options.html#opt-services.syncoid.enable) module now properly drops ZFS permissions after usage. Before it delegated permissions to whole pools instead of datasets and didn't clean up after execution. You can manually look this up for your pools by running `zfs allow your-pool-name` and use `zfs unallow syncoid your-pool-name` to clean this up.
- Zfs: `latestCompatibleLinuxPackages` is now exported on the zfs package. One can use `boot.kernelPackages = config.boot.zfs.package.latestCompatibleLinuxPackages;` to always track the latest compatible kernel with a given version of zfs.
@ -389,3 +496,39 @@ To be able to access the web UI this port needs to be opened in the firewall.
directories, thus increasing the purity of the build.
- Three new options, [xdg.mime.addedAssociations](#opt-xdg.mime.addedAssociations), [xdg.mime.defaultApplications](#opt-xdg.mime.defaultApplications), and [xdg.mime.removedAssociations](#opt-xdg.mime.removedAssociations) have been added to the [xdg.mime](#opt-xdg.mime.enable) module to allow the configuration of `/etc/xdg/mimeapps.list`.
- Kopia was upgraded from 0.8.x to 0.9.x. Please read the [upstream release notes](https://github.com/kopia/kopia/releases/tag/v0.9.0) for changes and upgrade instructions.
- The `systemd.network` module has gained support for the FooOverUDP link type.
- The `networking` module has a new `networking.fooOverUDP` option to configure Foo-over-UDP encapsulations.
- `networking.sits` now supports Foo-over-UDP encapsulation.
- Changing systemd `.socket` units now restarts them and stops the service that is activated by them. Additionally, services with `stopOnChange = false` don't break anymore when they are socket-activated.
- The `virtualisation.libvirtd` module has been refactored and updated with new options:
- `virtualisation.libvirtd.qemu*` options (e.g.: `virtualisation.libvirtd.qemuRunAsRoot`) were moved to [`virtualisation.libvirtd.qemu`](options.html#opt-virtualisation.libvirtd.qemu) submodule,
- software TPM1/TPM2 support (e.g.: Windows 11 guests) ([`virtualisation.libvirtd.qemu.swtpm`](options.html#opt-virtualisation.libvirtd.qemu.swtpm)),
- custom OVMF package (e.g.: `pkgs.OVMFFull` with HTTP, CSM and Secure Boot support) ([`virtualisation.libvirtd.qemu.ovmf.package`](options.html#opt-virtualisation.libvirtd.qemu.ovmf.package)).
- The `cawbird` Twitter client now uses its own API keys to count as different application than upstream builds. This is done to evade application-level rate limiting. While existing accounts continue to work, users may want to remove and re-register their account in the client to enjoy a better user experience and benefit from this change.
- A new option `services.prometheus.enableReload` has been added which can be enabled to reload the prometheus service when its config file changes instead of restarting.
- The option `services.prometheus.environmentFile` has been removed since it was causing [issues](https://github.com/NixOS/nixpkgs/issues/126083) and Prometheus now has native support for secret files, i.e. `basic_auth.password_file` and `authorization.credentials_file`.
- Dokuwiki now supports caddy! However
- the nginx option has been removed, in the new configuration, please use the `dokuwiki.webserver = "nginx"` instead.
- The "${hostname}" option has been deprecated, please use `dokuwiki.sites = [ "${hostname}" ]` instead
- The [services.unifi](options.html#opt-services.unifi.enable) module has been reworked, solving a number of issues. This leads to several user facing changes:
- The `services.unifi.dataDir` option is removed and the data is now always located under `/var/lib/unifi/data`. This is done to make better use of systemd state direcotiry and thus making the service restart more reliable.
- The unifi logs can now be found under: `/var/log/unifi` instead of `/var/lib/unifi/logs`.
- The unifi run directory can now be found under: `/run/unifi` instead of `/var/lib/unifi/run`.
- `security.pam.services.<name>.makeHomeDir` now uses `umask=0077` instead of `umask=0022` when creating the home directory.
- Loki has had another release. Some default values have been changed for the configuration and some configuration options have been renamed. For more details, please check [the upgrade guide](https://grafana.com/docs/loki/latest/upgrading/#240).
- `julia` now refers to `julia-stable` instead of `julia-lts`. In practice this means it has been upgraded from `1.0.4` to `1.5.4`.

25
nixos/lib/build-vms.nix
View File

@ -4,15 +4,14 @@
, # Ignored
config ? null
, # Nixpkgs, for qemu, lib and more
pkgs
pkgs, lib
, # !!! See comment about args in lib/modules.nix
specialArgs ? {}
, # NixOS configuration to add to the VMs
extraConfigurations ? []
}:
with pkgs.lib;
with import ../lib/qemu-flags.nix { inherit pkgs; };
with lib;
rec {
@ -69,9 +68,8 @@ rec {
prefixLength = 24;
} ];
});
in
{ key = "ip-address";
config =
networkConfig =
{ networking.hostName = mkDefault m.fst;
networking.interfaces = listToAttrs interfaces;
@ -93,10 +91,19 @@ rec {
"${config.networking.hostName}\n"));
virtualisation.qemu.options =
forEach interfacesNumbered
({ fst, snd }: qemuNICFlags snd fst m.snd);
let qemu-common = import ../lib/qemu-common.nix { inherit lib pkgs; };
in flip concatMap interfacesNumbered
({ fst, snd }: qemu-common.qemuNICFlags snd fst m.snd);
};
}
in
{ key = "ip-address";
config = networkConfig // {
# Expose the networkConfig items for tests like nixops
# that need to recreate the network config.
system.build.networkConfig = networkConfig;
};
}
)
(getAttr m.fst nodes)
] );

2
nixos/lib/eval-config.nix
View File

@ -61,7 +61,7 @@ in rec {
args = extraArgs;
specialArgs =
{ modulesPath = builtins.toString ../modules; } // specialArgs;
}) config options _module;
}) config options _module type;
# These are the extra arguments passed to every module. In
# particular, Nixpkgs is passed through the "pkgs" argument.

88
nixos/lib/make-disk-image.nix
View File

@ -44,11 +44,14 @@
# most likely fails as GRUB will probably refuse to install.
partitionTableType ? "legacy"
, # Whether to invoke `switch-to-configuration boot` during image creation
installBootLoader ? true
, # The root file system type.
fsType ? "ext4"
, # Filesystem label
label ? "nixos"
label ? if onlyNixStore then "nix-store" else "nixos"
, # The initial NixOS configuration file to be copied to
# /etc/nixos/configuration.nix.
@ -57,10 +60,24 @@
, # Shell code executed after the VM has finished.
postVM ? ""
, # Copy the contents of the Nix store to the root of the image and
# skip further setup. Incompatible with `contents`,
# `installBootLoader` and `configFile`.
onlyNixStore ? false
, name ? "nixos-disk-image"
, # Disk image format, one of qcow2, qcow2-compressed, vdi, vpc, raw.
format ? "raw"
, # Whether a nix channel based on the current source tree should be
# made available inside the image. Useful for interactive use of nix
# utils, but changes the hash of the image when the sources are
# updated.
copyChannel ? true
, # Additional store paths to copy to the image's store.
additionalPaths ? []
}:
assert partitionTableType == "legacy" || partitionTableType == "legacy+gpt" || partitionTableType == "efi" || partitionTableType == "hybrid" || partitionTableType == "none";
@ -71,6 +88,7 @@ assert lib.all
(attrs: ((attrs.user or null) == null)
== ((attrs.group or null) == null))
contents;
assert onlyNixStore -> contents == [] && configFile == null && !installBootLoader;
with lib;
@ -163,7 +181,14 @@ let format' = format; in let
users = map (x: x.user or "''") contents;
groups = map (x: x.group or "''") contents;
closureInfo = pkgs.closureInfo { rootPaths = [ config.system.build.toplevel channelSources ]; };
basePaths = [ config.system.build.toplevel ]
++ lib.optional copyChannel channelSources;
additionalPaths' = subtractLists basePaths additionalPaths;
closureInfo = pkgs.closureInfo {
rootPaths = basePaths ++ additionalPaths';
};
blockSize = toString (4 * 1024); # ext4fs block size (not block device sector size)
@ -251,7 +276,13 @@ let format' = format; in let
chmod 755 "$TMPDIR"
echo "running nixos-install..."
nixos-install --root $root --no-bootloader --no-root-passwd \
--system ${config.system.build.toplevel} --channel ${channelSources} --substituters ""
--system ${config.system.build.toplevel} \
${if copyChannel then "--channel ${channelSources}" else "--no-channel-copy"} \
--substituters ""
${optionalString (additionalPaths' != []) ''
nix copy --to $root --no-check-sigs ${concatStringsSep " " additionalPaths'}
''}
diskImage=nixos.raw
@ -320,25 +351,29 @@ let format' = format; in let
''}
echo "copying staging root to image..."
cptofs -p ${optionalString (partitionTableType != "none") "-P ${rootPartition}"} -t ${fsType} -i $diskImage $root/* / ||
cptofs -p ${optionalString (partitionTableType != "none") "-P ${rootPartition}"} \
-t ${fsType} \
-i $diskImage \
$root${optionalString onlyNixStore builtins.storeDir}/* / ||
(echo >&2 "ERROR: cptofs failed. diskSize might be too small for closure."; exit 1)
'';
in pkgs.vmTools.runInLinuxVM (
pkgs.runCommand name
{ preVM = prepareImage;
moveOrConvertImage = ''
${if format == "raw" then ''
mv $diskImage $out/${filename}
'' else ''
${pkgs.qemu}/bin/qemu-img convert -f raw -O ${format} ${compress} $diskImage $out/${filename}
''}
diskImage=$out/${filename}
'';
buildImage = pkgs.vmTools.runInLinuxVM (
pkgs.runCommand name {
preVM = prepareImage;
buildInputs = with pkgs; [ util-linux e2fsprogs dosfstools ];
postVM = ''
${if format == "raw" then ''
mv $diskImage $out/${filename}
'' else ''
${pkgs.qemu}/bin/qemu-img convert -f raw -O ${format} ${compress} $diskImage $out/${filename}
''}
diskImage=$out/${filename}
${postVM}
'';
postVM = moveOrConvertImage + postVM;
memSize = 1024;
}
''
} ''
export PATH=${binPath}:$PATH
rootDisk=${if partitionTableType != "none" then "/dev/vda${rootPartition}" else "/dev/vda"}
@ -368,11 +403,13 @@ in pkgs.vmTools.runInLinuxVM (
cp ${configFile} /mnt/etc/nixos/configuration.nix
''}
# Set up core system link, GRUB, etc.
NIXOS_INSTALL_BOOTLOADER=1 nixos-enter --root $mountPoint -- /nix/var/nix/profiles/system/bin/switch-to-configuration boot
${lib.optionalString installBootLoader ''
# Set up core system link, GRUB, etc.
NIXOS_INSTALL_BOOTLOADER=1 nixos-enter --root $mountPoint -- /nix/var/nix/profiles/system/bin/switch-to-configuration boot
# The above scripts will generate a random machine-id and we don't want to bake a single ID into all our images
rm -f $mountPoint/etc/machine-id
# The above scripts will generate a random machine-id and we don't want to bake a single ID into all our images
rm -f $mountPoint/etc/machine-id
''}
# Set the ownerships of the contents. The modes are set in preVM.
# No globbing on targets, so no need to set -f
@ -398,4 +435,9 @@ in pkgs.vmTools.runInLinuxVM (
tune2fs -T now -c 0 -i 0 $rootDisk
''}
''
)
);
in
if onlyNixStore then
pkgs.runCommand name {}
(prepareImage + moveOrConvertImage + postVM)
else buildImage

8
nixos/lib/make-options-doc/default.nix
View File

@ -83,10 +83,13 @@ let
optionsListVisible = lib.filter (opt: opt.visible && !opt.internal) (lib.optionAttrSetToDocList options);
# Customly sort option list for the man page.
# Always ensure that the sort order matches sortXML.py!
optionsList = lib.sort optionLess optionsListDesc;
# Convert the list of options into an XML file.
optionsXML = builtins.toFile "options.xml" (builtins.toXML optionsList);
# This file is *not* sorted sorted to save on eval time, since the docbook XML
# and the manpage depend on it and thus we evaluate this on every system rebuild.
optionsXML = builtins.toFile "options.xml" (builtins.toXML optionsListDesc);
optionsNix = builtins.listToAttrs (map (o: { name = o.name; value = removeAttrs o ["name" "visible" "internal"]; }) optionsList);
@ -185,9 +188,10 @@ in {
exit 1
fi
${pkgs.python3Minimal}/bin/python ${./sortXML.py} $optionsXML sorted.xml
${pkgs.libxslt.bin}/bin/xsltproc \
--stringparam revision '${revision}' \
-o intermediate.xml ${./options-to-docbook.xsl} $optionsXML
-o intermediate.xml ${./options-to-docbook.xsl} sorted.xml
${pkgs.libxslt.bin}/bin/xsltproc \
-o "$out" ${./postprocess-option-descriptions.xsl} intermediate.xml
'';

46
nixos/lib/make-options-doc/options-to-docbook.xsl
View File

@ -54,7 +54,7 @@
<para>
<emphasis>Default:</emphasis>
<xsl:text> </xsl:text>
<xsl:apply-templates select="attr[@name = 'default']" mode="top" />
<xsl:apply-templates select="attr[@name = 'default']/*" mode="top" />
</para>
</xsl:if>
@ -62,14 +62,7 @@
<para>
<emphasis>Example:</emphasis>
<xsl:text> </xsl:text>
<xsl:choose>
<xsl:when test="attr[@name = 'example']/attrs[attr[@name = '_type' and string[@value = 'literalExample']]]">
<programlisting><xsl:value-of select="attr[@name = 'example']/attrs/attr[@name = 'text']/string/@value" /></programlisting>
</xsl:when>
<xsl:otherwise>
<xsl:apply-templates select="attr[@name = 'example']" mode="top" />
</xsl:otherwise>
</xsl:choose>
<xsl:apply-templates select="attr[@name = 'example']/*" mode="top" />
</para>
</xsl:if>
@ -107,20 +100,37 @@
</xsl:template>
<xsl:template match="*" mode="top">
<xsl:template match="attrs[attr[@name = '_type' and string[@value = 'literalExpression']]]" mode = "top">
<xsl:choose>
<xsl:when test="string[contains(@value, '&#010;')]">
<programlisting>
<xsl:text>''
</xsl:text><xsl:value-of select='str:replace(string/@value, "${", "&apos;&apos;${")' /><xsl:text>''</xsl:text></programlisting>
<xsl:when test="contains(attr[@name = 'text']/string/@value, '&#010;')">
<programlisting><xsl:value-of select="attr[@name = 'text']/string/@value" /></programlisting>
</xsl:when>
<xsl:otherwise>
<literal><xsl:apply-templates /></literal>
<literal><xsl:value-of select="attr[@name = 'text']/string/@value" /></literal>
</xsl:otherwise>
</xsl:choose>
</xsl:template>
<xsl:template match="attrs[attr[@name = '_type' and string[@value = 'literalDocBook']]]" mode = "top">
<xsl:value-of disable-output-escaping="yes" select="attr[@name = 'text']/string/@value" />
</xsl:template>
<xsl:template match="string[contains(@value, '&#010;')]" mode="top">
<programlisting>
<xsl:text>''&#010;</xsl:text>
<xsl:value-of select='str:replace(str:replace(@value, "&apos;&apos;", "&apos;&apos;&apos;"), "${", "&apos;&apos;${")' />
<xsl:text>''</xsl:text>
</programlisting>
</xsl:template>
<xsl:template match="*" mode="top">
<literal><xsl:apply-templates select="." /></literal>
</xsl:template>
<xsl:template match="null">
<xsl:text>null</xsl:text>
</xsl:template>
@ -129,10 +139,10 @@
<xsl:template match="string">
<xsl:choose>
<xsl:when test="(contains(@value, '&quot;') or contains(@value, '\')) and not(contains(@value, '&#010;'))">
<xsl:text>''</xsl:text><xsl:value-of select='str:replace(@value, "${", "&apos;&apos;${")' /><xsl:text>''</xsl:text>
<xsl:text>''</xsl:text><xsl:value-of select='str:replace(str:replace(@value, "&apos;&apos;", "&apos;&apos;&apos;"), "${", "&apos;&apos;${")' /><xsl:text>''</xsl:text>
</xsl:when>
<xsl:otherwise>
<xsl:text>"</xsl:text><xsl:value-of select="str:replace(str:replace(str:replace(str:replace(@value, '\', '\\'), '&quot;', '\&quot;'), '&#010;', '\n'), '$', '\$')" /><xsl:text>"</xsl:text>
<xsl:text>"</xsl:text><xsl:value-of select="str:replace(str:replace(str:replace(str:replace(@value, '\', '\\'), '&quot;', '\&quot;'), '&#010;', '\n'), '${', '\${')" /><xsl:text>"</xsl:text>
</xsl:otherwise>
</xsl:choose>
</xsl:template>
@ -163,7 +173,7 @@
</xsl:template>
<xsl:template match="attrs[attr[@name = '_type' and string[@value = 'literalExample']]]">
<xsl:template match="attrs[attr[@name = '_type' and string[@value = 'literalExpression']]]">
<xsl:value-of select="attr[@name = 'text']/string/@value" />
</xsl:template>

28
nixos/lib/make-options-doc/sortXML.py Normal file
View File

@ -0,0 +1,28 @@
import xml.etree.ElementTree as ET
import sys
tree = ET.parse(sys.argv[1])
# the xml tree is of the form
# <expr><list> {all options, each an attrs} </list></expr>
options = list(tree.getroot().find('list'))
def sortKey(opt):
def order(s):
if s.startswith("enable"):
return 0
if s.startswith("package"):
return 1
return 2
return [
(order(p.attrib['value']), p.attrib['value'])
for p in opt.findall('attr[@name="loc"]/list/string')
]
# always ensure that the sort order matches the order used in the nix expression!
options.sort(key=sortKey)
doc = ET.Element("expr")
newOptions = ET.SubElement(doc, "list")
newOptions.extend(options)
ET.ElementTree(doc).write(sys.argv[2], encoding='utf-8')

10
nixos/lib/qemu-flags.nix → nixos/lib/qemu-common.nix
View File

@ -1,12 +1,12 @@
# QEMU flags shared between various Nix expressions.
{ pkgs }:
# QEMU-related utilities shared between various Nix expressions.
{ lib, pkgs }:
let
zeroPad = n:
pkgs.lib.optionalString (n < 16) "0" +
lib.optionalString (n < 16) "0" +
(if n > 255
then throw "Can't have more than 255 nets or nodes!"
else pkgs.lib.toHexString n);
else lib.toHexString n);
in
rec {
@ -14,7 +14,7 @@ rec {
qemuNICFlags = nic: net: machine:
[ "-device virtio-net-pci,netdev=vlan${toString nic},mac=${qemuNicMac net machine}"
"-netdev vde,id=vlan${toString nic},sock=$QEMU_VDE_SOCKET_${toString net}"
''-netdev vde,id=vlan${toString nic},sock="$QEMU_VDE_SOCKET_${toString net}"''
];
qemuSerialDevice = if pkgs.stdenv.isi686 || pkgs.stdenv.isx86_64 then "ttyS0"

856
nixos/lib/test-driver/test-driver.py
View File

File diff suppressed because it is too large Load Diff

44
nixos/lib/testing-python.nix
View File

@ -43,7 +43,8 @@ rec {
from pydoc import importfile
with open('driver-symbols', 'w') as fp:
t = importfile('${testDriverScript}')
test_symbols = t._test_symbols()
d = t.Driver([],[],"")
test_symbols = d.test_symbols()
fp.write(','.join(test_symbols.keys()))
EOF
'';
@ -188,14 +189,6 @@ rec {
--set startScripts "''${vmStartScripts[*]}" \
--set testScript "$out/test-script" \
--set vlans '${toString vlans}'
${lib.optionalString (testScript == "") ''
ln -s ${testDriver}/bin/nixos-test-driver $out/bin/nixos-run-vms
wrapProgram $out/bin/nixos-run-vms \
--set startScripts "''${vmStartScripts[*]}" \
--set testScript "${pkgs.writeText "start-all" "start_all(); join_all();"}" \
--set vlans '${toString vlans}'
''}
'');
# Make a full-blown test
@ -216,11 +209,41 @@ rec {
let
nodes = qemu_pkg:
let
testScript' =
# Call the test script with the computed nodes.
if lib.isFunction testScript
then testScript { nodes = nodes qemu_pkg; }
else testScript;
build-vms = import ./build-vms.nix {
inherit system pkgs minimal specialArgs;
inherit system lib pkgs minimal specialArgs;
extraConfigurations = extraConfigurations ++ [(
{ config, ... }:
{
virtualisation.qemu.package = qemu_pkg;
# Make sure all derivations referenced by the test
# script are available on the nodes. When the store is
# accessed through 9p, this isn't important, since
# everything in the store is available to the guest,
# but when building a root image it is, as all paths
# that should be available to the guest has to be
# copied to the image.
virtualisation.additionalPaths =
lib.optional
# A testScript may evaluate nodes, which has caused
# infinite recursions. The demand cycle involves:
# testScript -->
# nodes -->
# toplevel -->
# additionalPaths -->
# hasContext testScript' -->
# testScript (ad infinitum)
# If we don't need to build an image, we can break this
# cycle by short-circuiting when useNixStoreImage is false.
(config.virtualisation.useNixStoreImage && builtins.hasContext testScript')
(pkgs.writeStringReferencesToFile testScript');
# Ensure we do not use aliases. Ideally this is only set
# when the test framework is used by Nixpkgs NixOS tests.
nixpkgs.config.allowAliases = false;
@ -257,7 +280,6 @@ rec {
inherit test driver driverInteractive nodes;
};
abortForFunction = functionName: abort ''The ${functionName} function was
removed because it is not an essential part of the NixOS testing
infrastructure. It had no usage in NixOS or Nixpkgs and it had no designated

2
nixos/lib/utils.nix
View File

@ -10,7 +10,7 @@ rec {
# Check whenever fileSystem is needed for boot. NOTE: Make sure
# pathsNeededForBoot is closed under the parent relationship, i.e. if /a/b/c
# is in the list, put /a and /a/b in as well.
pathsNeededForBoot = [ "/" "/nix" "/nix/store" "/var" "/var/log" "/var/lib" "/etc" ];
pathsNeededForBoot = [ "/" "/nix" "/nix/store" "/var" "/var/log" "/var/lib" "/var/lib/nixos" "/etc" ];
fsNeededForBoot = fs: fs.neededForBoot || elem fs.mountPoint pathsNeededForBoot;
# Check whenever `b` depends on `a` as a fileSystem

2
nixos/maintainers/scripts/azure-new/examples/basic/image.nix
View File

@ -1,6 +1,6 @@
let
pkgs = (import ../../../../../../default.nix {});
machine = import "${pkgs.path}/nixos/lib/eval-config.nix" {
machine = import (pkgs.path + "/nixos/lib/eval-config.nix") {
system = "x86_64-linux";
modules = [
({config, ...}: { imports = [ ./system.nix ]; })

11
nixos/maintainers/scripts/ec2/amazon-image.nix
View File

@ -4,6 +4,7 @@ with lib;
let
cfg = config.amazonImage;
amiBootMode = if config.ec2.efi then "uefi" else "legacy-bios";
in {
@ -27,7 +28,7 @@ in {
};
contents = mkOption {
example = literalExample ''
example = literalExpression ''
[ { source = pkgs.memtest86 + "/memtest.bin";
target = "boot/memtest.bin";
}
@ -104,12 +105,14 @@ in {
${pkgs.jq}/bin/jq -n \
--arg system_label ${lib.escapeShellArg config.system.nixos.label} \
--arg system ${lib.escapeShellArg pkgs.stdenv.hostPlatform.system} \
--arg root_logical_bytes "$(${pkgs.qemu}/bin/qemu-img info --output json "$bootDisk" | ${pkgs.jq}/bin/jq '."virtual-size"')" \
--arg boot_logical_bytes "$(${pkgs.qemu}/bin/qemu-img info --output json "$rootDisk" | ${pkgs.jq}/bin/jq '."virtual-size"')" \
--arg root_logical_bytes "$(${pkgs.qemu}/bin/qemu-img info --output json "$rootDisk" | ${pkgs.jq}/bin/jq '."virtual-size"')" \
--arg boot_logical_bytes "$(${pkgs.qemu}/bin/qemu-img info --output json "$bootDisk" | ${pkgs.jq}/bin/jq '."virtual-size"')" \
--arg boot_mode "${amiBootMode}" \
--arg root "$rootDisk" \
--arg boot "$bootDisk" \
'{}
| .label = $system_label
| .boot_mode = $boot_mode
| .system = $system
| .disks.boot.logical_bytes = $boot_logical_bytes
| .disks.boot.file = $boot
@ -145,9 +148,11 @@ in {
--arg system_label ${lib.escapeShellArg config.system.nixos.label} \
--arg system ${lib.escapeShellArg pkgs.stdenv.hostPlatform.system} \
--arg logical_bytes "$(${pkgs.qemu}/bin/qemu-img info --output json "$diskImage" | ${pkgs.jq}/bin/jq '."virtual-size"')" \
--arg boot_mode "${amiBootMode}" \
--arg file "$diskImage" \
'{}
| .label = $system_label
| .boot_mode = $boot_mode
| .system = $system
| .logical_bytes = $logical_bytes
| .file = $file

126
nixos/maintainers/scripts/ec2/create-amis.sh
View File

@ -1,6 +1,9 @@
#!/usr/bin/env nix-shell
#!nix-shell -p awscli -p jq -p qemu -i bash
# shellcheck shell=bash
#
# Future Deprecation?
# This entire thing should probably be replaced with a generic terraform config
# Uploads and registers NixOS images built from the
# <nixos/release.nix> amazonImage attribute. Images are uploaded and
@ -15,18 +18,22 @@
# set -x
set -euo pipefail
# configuration
state_dir=$HOME/amis/ec2-images
home_region=eu-west-1
bucket=nixos-amis
service_role_name=vmimport
var () { true; }
regions=(eu-west-1 eu-west-2 eu-west-3 eu-central-1 eu-north-1
# configuration
var ${state_dir:=$HOME/amis/ec2-images}
var ${home_region:=eu-west-1}
var ${bucket:=nixos-amis}
var ${service_role_name:=vmimport}
var ${regions:=eu-west-1 eu-west-2 eu-west-3 eu-central-1 eu-north-1
us-east-1 us-east-2 us-west-1 us-west-2
ca-central-1
ap-southeast-1 ap-southeast-2 ap-northeast-1 ap-northeast-2
ap-south-1 ap-east-1
sa-east-1)
sa-east-1}
regions=($regions)
log() {
echo "$@" >&2
@ -60,10 +67,21 @@ read_image_info() {
# We handle a single image per invocation, store all attributes in
# globals for convenience.
image_label=$(read_image_info .label)
zfs_disks=$(read_image_info .disks)
is_zfs_image=
if jq -e .boot <<< "$zfs_disks"; then
is_zfs_image=1
zfs_boot=".disks.boot"
fi
image_label="$(read_image_info .label)${is_zfs_image:+-ZFS}"
image_system=$(read_image_info .system)
image_file=$(read_image_info .file)
image_logical_bytes=$(read_image_info .logical_bytes)
image_files=( $(read_image_info ".disks.root.file") )
image_logical_bytes=$(read_image_info "${zfs_boot:-.disks.root}.logical_bytes")
if [[ -n "$is_zfs_image" ]]; then
image_files+=( $(read_image_info .disks.boot.file) )
fi
# Derived attributes
@ -113,11 +131,11 @@ wait_for_import() {
local state snapshot_id
log "Waiting for import task $task_id to be completed"
while true; do
read -r state progress snapshot_id < <(
read -r state message snapshot_id < <(
aws ec2 describe-import-snapshot-tasks --region "$region" --import-task-ids "$task_id" | \
jq -r '.ImportSnapshotTasks[].SnapshotTaskDetail | "\(.Status) \(.Progress) \(.SnapshotId)"'
jq -r '.ImportSnapshotTasks[].SnapshotTaskDetail | "\(.Status) \(.StatusMessage) \(.SnapshotId)"'
)
log " ... state=$state progress=$progress snapshot_id=$snapshot_id"
log " ... state=$state message=$message snapshot_id=$snapshot_id"
case "$state" in
active)
sleep 10
@ -179,41 +197,48 @@ make_image_public() {
upload_image() {
local region=$1
local aws_path=${image_file#/}
for image_file in "${image_files[@]}"; do
local aws_path=${image_file#/}
local state_key="$region.$image_label.$image_system"
local task_id
task_id=$(read_state "$state_key" task_id)
local snapshot_id
snapshot_id=$(read_state "$state_key" snapshot_id)
local ami_id
ami_id=$(read_state "$state_key" ami_id)
if [ -z "$task_id" ]; then
log "Checking for image on S3"
if ! aws s3 ls --region "$region" "s3://${bucket}/${aws_path}" >&2; then
log "Image missing from aws, uploading"
aws s3 cp --region "$region" "$image_file" "s3://${bucket}/${aws_path}" >&2
if [[ -n "$is_zfs_image" ]]; then
local suffix=${image_file%.*}
suffix=${suffix##*.}
fi
log "Importing image from S3 path s3://$bucket/$aws_path"
local state_key="$region.$image_label${suffix:+.${suffix}}.$image_system"
local task_id
task_id=$(read_state "$state_key" task_id)
local snapshot_id
snapshot_id=$(read_state "$state_key" snapshot_id)
local ami_id
ami_id=$(read_state "$state_key" ami_id)
task_id=$(aws ec2 import-snapshot --role-name "$service_role_name" --disk-container "{
\"Description\": \"nixos-image-${image_label}-${image_system}\",
\"Format\": \"vhd\",
\"UserBucket\": {
\"S3Bucket\": \"$bucket\",
\"S3Key\": \"$aws_path\"
}
}" --region "$region" | jq -r '.ImportTaskId')
if [ -z "$task_id" ]; then
log "Checking for image on S3"
if ! aws s3 ls --region "$region" "s3://${bucket}/${aws_path}" >&2; then
log "Image missing from aws, uploading"
aws s3 cp --region "$region" "$image_file" "s3://${bucket}/${aws_path}" >&2
fi
write_state "$state_key" task_id "$task_id"
fi
log "Importing image from S3 path s3://$bucket/$aws_path"
if [ -z "$snapshot_id" ]; then
snapshot_id=$(wait_for_import "$region" "$task_id")
write_state "$state_key" snapshot_id "$snapshot_id"
fi
task_id=$(aws ec2 import-snapshot --role-name "$service_role_name" --disk-container "{
\"Description\": \"nixos-image-${image_label}-${image_system}\",
\"Format\": \"vhd\",
\"UserBucket\": {
\"S3Bucket\": \"$bucket\",
\"S3Key\": \"$aws_path\"
}
}" --region "$region" | jq -r '.ImportTaskId')
write_state "$state_key" task_id "$task_id"
fi
if [ -z "$snapshot_id" ]; then
snapshot_id=$(wait_for_import "$region" "$task_id")
write_state "$state_key" snapshot_id "$snapshot_id"
fi
done
if [ -z "$ami_id" ]; then
log "Registering snapshot $snapshot_id as AMI"
@ -222,6 +247,18 @@ upload_image() {
"DeviceName=/dev/xvda,Ebs={SnapshotId=$snapshot_id,VolumeSize=$image_logical_gigabytes,DeleteOnTermination=true,VolumeType=gp3}"
)
if [[ -n "$is_zfs_image" ]]; then
local root_snapshot_id=$(read_state "$region.$image_label.root.$image_system" snapshot_id)
local root_image_logical_bytes=$(read_image_info ".disks.root.logical_bytes")
local root_image_logical_gigabytes=$(((root_image_logical_bytes-1)/1024/1024/1024+1)) # Round to the next GB
block_device_mappings+=(
"DeviceName=/dev/xvdb,Ebs={SnapshotId=$root_snapshot_id,VolumeSize=$root_image_logical_gigabytes,DeleteOnTermination=true,VolumeType=gp3}"
)
fi
local extra_flags=(
--root-device-name /dev/xvda
--sriov-net-support simple
@ -241,6 +278,7 @@ upload_image() {
--region "$region" \
--architecture $amazon_arch \
--block-device-mappings "${block_device_mappings[@]}" \
--boot-mode $(read_image_info .boot_mode) \
"${extra_flags[@]}" \
| jq -r '.ImageId'
)
@ -248,7 +286,7 @@ upload_image() {
write_state "$state_key" ami_id "$ami_id"
fi
make_image_public "$region" "$ami_id"
[[ -v PRIVATE ]] || make_image_public "$region" "$ami_id"
echo "$ami_id"
}
@ -276,7 +314,7 @@ copy_to_region() {
write_state "$state_key" ami_id "$ami_id"
fi
make_image_public "$region" "$ami_id"
[[ -v PRIVATE ]] || make_image_public "$region" "$ami_id"
echo "$ami_id"
}

102
nixos/maintainers/scripts/lxd/lxd-image-inner.nix Normal file
View File

@ -0,0 +1,102 @@
# Edit this configuration file to define what should be installed on
# your system. Help is available in the configuration.nix(5) man page
# and in the NixOS manual (accessible by running ‘nixos-help’).
{ config, pkgs, lib, ... }:
with lib;
{
imports =
[ # Include the default lxd configuration.
../../../modules/virtualisation/lxc-container.nix
# Include the container-specific autogenerated configuration.
./lxd.nix
];
# networking.hostName = mkForce "nixos"; # Overwrite the hostname.
# networking.wireless.enable = true; # Enables wireless support via wpa_supplicant.
# Set your time zone.
# time.timeZone = "Europe/Amsterdam";
# The global useDHCP flag is deprecated, therefore explicitly set to false here.
# Per-interface useDHCP will be mandatory in the future, so this generated config
# replicates the default behaviour.
networking.useDHCP = false;
networking.interfaces.eth0.useDHCP = true;
# Configure network proxy if necessary
# networking.proxy.default = "http://user:password@proxy:port/";
# networking.proxy.noProxy = "127.0.0.1,localhost,internal.domain";
# Select internationalisation properties.
# i18n.defaultLocale = "en_US.UTF-8";
# console = {
# font = "Lat2-Terminus16";
# keyMap = "us";
# };
# Enable the X11 windowing system.
# services.xserver.enable = true;
# Configure keymap in X11
# services.xserver.layout = "us";
# services.xserver.xkbOptions = "eurosign:e";
# Enable CUPS to print documents.
# services.printing.enable = true;
# Enable sound.
# sound.enable = true;
# hardware.pulseaudio.enable = true;
# Enable touchpad support (enabled default in most desktopManager).
# services.xserver.libinput.enable = true;
# Define a user account. Don't forget to set a password with ‘passwd’.
# users.users.jane = {
# isNormalUser = true;
# extraGroups = [ "wheel" ]; # Enable ‘sudo’ for the user.
# };
# List packages installed in system profile. To search, run:
# $ nix search wget
# environment.systemPackages = with pkgs; [
# vim # Do not forget to add an editor to edit configuration.nix! The Nano editor is also installed by default.
# wget
# firefox
# ];
# Some programs need SUID wrappers, can be configured further or are
# started in user sessions.
# programs.mtr.enable = true;
# programs.gnupg.agent = {
# enable = true;
# enableSSHSupport = true;
# };
# List services that you want to enable:
# Enable the OpenSSH daemon.
# services.openssh.enable = true;
# Open ports in the firewall.
# networking.firewall.allowedTCPPorts = [ ... ];
# networking.firewall.allowedUDPPorts = [ ... ];
# Or disable the firewall altogether.
# networking.firewall.enable = false;
# This value determines the NixOS release from which the default
# settings for stateful data, like file locations and database versions
# on your system were taken. It‘s perfectly fine and recommended to leave
# this value at the release version of the first install of this system.
# Before changing this value read the documentation for this option
# (e.g. man configuration.nix or on https://nixos.org/nixos/options.html).
system.stateVersion = "21.05"; # Did you read the comment?
# As this is intended as a stadalone image, undo some of the minimal profile stuff
documentation.enable = true;
documentation.nixos.enable = true;
environment.noXlibs = false;
}

34
nixos/maintainers/scripts/lxd/lxd-image.nix Normal file
View File

@ -0,0 +1,34 @@
{ lib, config, pkgs, ... }:
with lib;
{
imports = [
../../../modules/virtualisation/lxc-container.nix
];
virtualisation.lxc.templates.nix = {
enable = true;
target = "/etc/nixos/lxd.nix";
template = ./nix.tpl;
when = [ "create" "copy" ];
};
# copy the config for nixos-rebuild
system.activationScripts.config = ''
if [ ! -e /etc/nixos/configuration.nix ]; then
mkdir -p /etc/nixos
cat ${./lxd-image-inner.nix} > /etc/nixos/configuration.nix
sed 's|../../../modules/virtualisation/lxc-container.nix|<nixpkgs/nixos/modules/virtualisation/lxc-container.nix>|g' -i /etc/nixos/configuration.nix
fi
'';
# Network
networking.useDHCP = false;
networking.interfaces.eth0.useDHCP = true;
# As this is intended as a stadalone image, undo some of the minimal profile stuff
documentation.enable = true;
documentation.nixos.enable = true;
environment.noXlibs = false;
}

9
nixos/maintainers/scripts/lxd/nix.tpl Normal file
View File

@ -0,0 +1,9 @@
{ lib, config, pkgs, ... }:
with lib;
# WARNING: THIS CONFIGURATION IS AUTOGENERATED AND WILL BE OVERWRITTEN AUTOMATICALLY
{
networking.hostName = "{{ container.name }}";
}

2
nixos/modules/config/console.nix
View File

@ -116,7 +116,7 @@ in
{ console.keyMap = with config.services.xserver;
mkIf cfg.useXkbConfig
(pkgs.runCommand "xkb-console-keymap" { preferLocalBuild = true; } ''
'${pkgs.ckbcomp}/bin/ckbcomp' \
'${pkgs.buildPackages.ckbcomp}/bin/ckbcomp' \
${optionalString (config.environment.sessionVariables ? XKB_CONFIG_ROOT)
"-I${config.environment.sessionVariables.XKB_CONFIG_ROOT}"
} \

3
nixos/modules/config/fonts/fontdir.nix
View File

@ -50,9 +50,8 @@ in
config = mkIf cfg.enable {
# This is enough to make a symlink because the xserver
# module already links all /share/X11 paths.
environment.systemPackages = [ x11Fonts ];
environment.pathsToLink = [ "/share/X11/fonts" ];
services.xserver.filesSection = ''
FontPath "${x11Fonts}/share/X11/fonts"

2
nixos/modules/config/fonts/fonts.nix
View File

@ -61,7 +61,7 @@ in
fonts = mkOption {
type = types.listOf types.path;
default = [];
example = literalExample "[ pkgs.dejavu_fonts ]";
example = literalExpression "[ pkgs.dejavu_fonts ]";
description = "List of primary font paths.";
};

2
nixos/modules/config/i18n.nix
View File

@ -14,7 +14,7 @@ with lib;
allLocales = any (x: x == "all") config.i18n.supportedLocales;
locales = config.i18n.supportedLocales;
};
example = literalExample "pkgs.glibcLocales";
example = literalExpression "pkgs.glibcLocales";
description = ''
Customized pkg.glibcLocales package.

16
nixos/modules/config/krb5/default.nix
View File

@ -83,8 +83,8 @@ in {
kerberos = mkOption {
type = types.package;
default = pkgs.krb5Full;
defaultText = "pkgs.krb5Full";
example = literalExample "pkgs.heimdalFull";
defaultText = literalExpression "pkgs.krb5Full";
example = literalExpression "pkgs.heimdal";
description = ''
The Kerberos implementation that will be present in
<literal>environment.systemPackages</literal> after enabling this
@ -96,7 +96,7 @@ in {
type = with types; either attrs lines;
default = {};
apply = attrs: filterEmbeddedMetadata attrs;
example = literalExample ''
example = literalExpression ''
{
default_realm = "ATHENA.MIT.EDU";
};
@ -109,7 +109,7 @@ in {
realms = mkOption {
type = with types; either attrs lines;
default = {};
example = literalExample ''
example = literalExpression ''
{
"ATHENA.MIT.EDU" = {
admin_server = "athena.mit.edu";
@ -127,7 +127,7 @@ in {
domain_realm = mkOption {
type = with types; either attrs lines;
default = {};
example = literalExample ''
example = literalExpression ''
{
"example.com" = "EXAMPLE.COM";
".example.com" = "EXAMPLE.COM";
@ -142,7 +142,7 @@ in {
capaths = mkOption {
type = with types; either attrs lines;
default = {};
example = literalExample ''
example = literalExpression ''
{
"ATHENA.MIT.EDU" = {
"EXAMPLE.COM" = ".";
@ -161,7 +161,7 @@ in {
appdefaults = mkOption {
type = with types; either attrs lines;
default = {};
example = literalExample ''
example = literalExpression ''
{
pam = {
debug = false;
@ -182,7 +182,7 @@ in {
plugins = mkOption {
type = with types; either attrs lines;
default = {};
example = literalExample ''
example = literalExpression ''
{
ccselect = {
disable = "k5identity";

14
nixos/modules/config/malloc.nix
View File

@ -30,6 +30,15 @@ let
vulnerabilities, while maintaining good performance.
'';
};
mimalloc = {
libPath = "${pkgs.mimalloc}/lib/libmimalloc.so";
description = ''
A compact and fast general purpose allocator, which may
optionally be built with mitigations against various heap
vulnerabilities.
'';
};
};
providerConf = providers.${cfg.provider};
@ -91,7 +100,10 @@ in
"abstractions/base" = ''
r /etc/ld-nix.so.preload,
r ${config.environment.etc."ld-nix.so.preload".source},
mr ${providerLibPath},
include "${pkgs.apparmorRulesFromClosure {
name = "mallocLib";
baseRules = ["mr $path/lib/**.so*"];
} [ mallocLib ] }"
'';
};
};

6
nixos/modules/config/networking.nix
View File

@ -21,7 +21,7 @@ in
networking.hosts = lib.mkOption {
type = types.attrsOf (types.listOf types.str);
example = literalExample ''
example = literalExpression ''
{
"127.0.0.1" = [ "foo.bar.baz" ];
"192.168.0.2" = [ "fileserver.local" "nameserver.local" ];
@ -34,8 +34,8 @@ in
networking.hostFiles = lib.mkOption {
type = types.listOf types.path;
defaultText = lib.literalExample "Hosts from `networking.hosts` and `networking.extraHosts`";
example = lib.literalExample ''[ "''${pkgs.my-blocklist-package}/share/my-blocklist/hosts" ]'';
defaultText = literalDocBook "Hosts from <option>networking.hosts</option> and <option>networking.extraHosts</option>";
example = literalExpression ''[ "''${pkgs.my-blocklist-package}/share/my-blocklist/hosts" ]'';
description = ''
Files that should be concatenated together to form <filename>/etc/hosts</filename>.
'';

4
nixos/modules/config/power-management.nix
View File

@ -35,7 +35,7 @@ in
powerUpCommands = mkOption {
type = types.lines;
default = "";
example = literalExample ''
example = literalExpression ''
"''${pkgs.hdparm}/sbin/hdparm -B 255 /dev/sda"
'';
description =
@ -49,7 +49,7 @@ in
powerDownCommands = mkOption {
type = types.lines;
default = "";
example = literalExample ''
example = literalExpression ''
"''${pkgs.hdparm}/sbin/hdparm -B 255 /dev/sda"
'';
description =

10
nixos/modules/config/pulseaudio.nix
View File

@ -149,8 +149,8 @@ in {
default = if config.services.jack.jackd.enable
then pkgs.pulseaudioFull
else pkgs.pulseaudio;
defaultText = "pkgs.pulseaudio";
example = literalExample "pkgs.pulseaudioFull";
defaultText = literalExpression "pkgs.pulseaudio";
example = literalExpression "pkgs.pulseaudioFull";
description = ''
The PulseAudio derivation to use. This can be used to enable
features (such as JACK support, Bluetooth) via the
@ -161,7 +161,7 @@ in {
extraModules = mkOption {
type = types.listOf types.package;
default = [];
example = literalExample "[ pkgs.pulseaudio-modules-bt ]";
example = literalExpression "[ pkgs.pulseaudio-modules-bt ]";
description = ''
Extra pulseaudio modules to use. This is intended for out-of-tree
pulseaudio modules like extra bluetooth codecs.
@ -184,7 +184,7 @@ in {
type = types.attrsOf types.unspecified;
default = {};
description = "Config of the pulse daemon. See <literal>man pulse-daemon.conf</literal>.";
example = literalExample ''{ realtime-scheduling = "yes"; }'';
example = literalExpression ''{ realtime-scheduling = "yes"; }'';
};
};
@ -204,7 +204,7 @@ in {
allowedIpRanges = mkOption {
type = types.listOf types.str;
default = [];
example = literalExample ''[ "127.0.0.1" "192.168.1.0/24" ]'';
example = literalExpression ''[ "127.0.0.1" "192.168.1.0/24" ]'';
description = ''
A list of IP subnets that are allowed to stream to the server.
'';

8
nixos/modules/config/shells-environment.nix
View File

@ -136,10 +136,8 @@ in
environment.binsh = mkOption {
default = "${config.system.build.binsh}/bin/sh";
defaultText = "\${config.system.build.binsh}/bin/sh";
example = literalExample ''
"''${pkgs.dash}/bin/dash"
'';
defaultText = literalExpression ''"''${config.system.build.binsh}/bin/sh"'';
example = literalExpression ''"''${pkgs.dash}/bin/dash"'';
type = types.path;
visible = false;
description = ''
@ -152,7 +150,7 @@ in
environment.shells = mkOption {
default = [];
example = literalExample "[ pkgs.bashInteractive pkgs.zsh ]";
example = literalExpression "[ pkgs.bashInteractive pkgs.zsh ]";
description = ''
A list of permissible login shells for user accounts.
No need to mention <literal>/bin/sh</literal>

12
nixos/modules/config/swap.nix
View File

@ -47,6 +47,15 @@ let
'';
};
allowDiscards = mkOption {
default = false;
type = types.bool;
description = ''
Whether to allow TRIM requests to the underlying device. This option
has security implications; please read the LUKS documentation before
activating it.
'';
};
};
};
@ -224,7 +233,8 @@ in
fi
''}
${optionalString sw.randomEncryption.enable ''
cryptsetup plainOpen -c ${sw.randomEncryption.cipher} -d ${sw.randomEncryption.source} ${optionalString (sw.discardPolicy != null) "--allow-discards"} ${sw.device} ${sw.deviceName}
cryptsetup plainOpen -c ${sw.randomEncryption.cipher} -d ${sw.randomEncryption.source} \
${optionalString sw.randomEncryption.allowDiscards "--allow-discards"} ${sw.device} ${sw.deviceName}
mkswap ${sw.realDevice}
''}
'';

2
nixos/modules/config/sysctl.nix
View File

@ -22,7 +22,7 @@ in
boot.kernel.sysctl = mkOption {
default = {};
example = literalExample ''
example = literalExpression ''
{ "net.ipv4.tcp_syncookies" = false; "vm.swappiness" = 60; }
'';
type = types.attrsOf sysctlOption;

Some files were not shown because too many files have changed in this diff Show More