dev/nixpkgs
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity

doc/cross-compilation: fixup

Browse Source 
More cleanups and stuff. May need to be split up.
This commit is contained in:
Matthew Bauer 2018-11-18 23:23:22 -06:00
parent ee58ab3cb9
commit 9d3108c3ae
2 changed files with 98 additions and 98 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

188
doc/cross-compilation.xml
View File

@ -6,17 +6,17 @@
<title>Introduction</title> <title>Introduction</title>
<para> <para>
"Cross-compilation" means compiling a program on one machine for another "Cross-compilation" means compiling a program on one machine for another type
type of machine. For example, a typical use of cross compilation is to of machine. For example, a typical use of cross-compilation is to compile
compile programs for embedded devices. These devices often don't have the programs for embedded devices. These devices often don't have the computing
computing power and memory to compile their own programs. One might think power and memory to compile their own programs. One might think that
that cross-compilation is a fairly niche concern, but there are advantages cross-compilation is a fairly niche concern. However, there are significant
to being rigorous about distinguishing build-time vs run-time environments advantages to rigorously distinguishing between build-time and run-time
even when one is developing and deploying on the same machine. Nixpkgs is environments! This applies even when one is developing and deploying on the
increasingly adopting the opinion that packages should be written with same machine. Nixpkgs is increasingly adopting the opinion that packages
cross-compilation in mind, and nixpkgs should evaluate in a similar way (by should be written with cross-compilation in mind, and nixpkgs should evaluate
minimizing cross-compilation-specific special cases) whether or not one is in a similar way (by minimizing cross-compilation-specific special cases)
cross-compiling. whether or not one is cross-compiling.
</para> </para>
<para> <para>
@ -34,15 +34,15 @@
<title>Platform parameters</title> <title>Platform parameters</title>
<para> <para>
Nixpkgs follows the Nixpkgs follows the <link
<link xlink:href="https://gcc.gnu.org/onlinedocs/gccint/Configure-Terms.html">common xlink:href="https://gcc.gnu.org/onlinedocs/gccint/Configure-Terms.html">conventions
historical convention of GNU autoconf</link> of distinguishing between 3 of GNU autoconf</link>. We distinguish between 3 types of platforms when
types of platform: <wordasword>build</wordasword>, building a derivation: <wordasword>build</wordasword>,
<wordasword>host</wordasword>, and <wordasword>target</wordasword>. In <wordasword>host</wordasword>, and <wordasword>target</wordasword>. In
summary, <wordasword>build</wordasword> is the platform on which a package summary, <wordasword>build</wordasword> is the platform on which a package
is being built, <wordasword>host</wordasword> is the platform on which it is being built, <wordasword>host</wordasword> is the platform on which it
is to run. The third attribute, <wordasword>target</wordasword>, is will run. The third attribute, <wordasword>target</wordasword>, is relevant
relevant only for certain specific compilers and build tools. only for certain specific compilers and build tools.
</para> </para>
<para> <para>
@ -64,7 +64,7 @@
<para> <para>
The "build platform" is the platform on which a package is built. Once The "build platform" is the platform on which a package is built. Once
someone has a built package, or pre-built binary package, the build someone has a built package, or pre-built binary package, the build
platform should not matter and be safe to ignore. platform should not matter and can be ignored.
</para> </para>
</listitem> </listitem>
</varlistentry> </varlistentry>
@ -94,11 +94,11 @@
<para> <para>
The build process of certain compilers is written in such a way that the The build process of certain compilers is written in such a way that the
compiler resulting from a single build can itself only produce binaries compiler resulting from a single build can itself only produce binaries
for a single platform. The task specifying this single "target platform" for a single platform. The task of specifying this single "target
is thus pushed to build time of the compiler. The root cause of this platform" is thus pushed to build time of the compiler. The root cause of
mistake is often that the compiler (which will be run on the host) and this that the compiler (which will be run on the host) and the standard
the the standard library/runtime (which will be run on the target) are library/runtime (which will be run on the target) are built by a single
built by a single build process. build process.
</para> </para>
<para> <para>
There is no fundamental need to think about a single target ahead of There is no fundamental need to think about a single target ahead of
@ -135,8 +135,10 @@
<para> <para>
This is a two-component shorthand for the platform. Examples of this This is a two-component shorthand for the platform. Examples of this
would be "x86_64-darwin" and "i686-linux"; see would be "x86_64-darwin" and "i686-linux"; see
<literal>lib.systems.doubles</literal> for more. This format isn't very <literal>lib.systems.doubles</literal> for more. The first component
standard, but has built-in support in Nix, such as the corresponds to the CPU architecture of the platform and the second to the
operating system of the platform (<literal>[cpu]-[os]</literal>). This
format has built-in support in Nix, such as the
<varname>builtins.currentSystem</varname> impure string. <varname>builtins.currentSystem</varname> impure string.
</para> </para>
</listitem> </listitem>
@ -147,12 +149,13 @@
</term> </term>
<listitem> <listitem>
<para> <para>
This is a 3- or 4- component shorthand for the platform. Examples of This is a 3- or 4- component shorthand for the platform. Examples of this
this would be "x86_64-unknown-linux-gnu" and "aarch64-apple-darwin14". would be <literal>x86_64-unknown-linux-gnu</literal> and
This is a standard format called the "LLVM target triple", as they are <literal>aarch64-apple-darwin14</literal>. This is a standard format
pioneered by LLVM and traditionally just used for the called the "LLVM target triple", as they are pioneered by LLVM. In the
<varname>targetPlatform</varname>. This format is strictly more 4-part form, this corresponds to
informative than the "Nix host double", as the previous format could <literal>[cpu]-[vendor]-[os]-[abi]</literal>. This format is strictly
more informative than the "Nix host double", as the previous format could
analogously be termed. This needs a better name than analogously be termed. This needs a better name than
<varname>config</varname>! <varname>config</varname>!
</para> </para>
@ -164,12 +167,11 @@
</term> </term>
<listitem> <listitem>
<para> <para>
This is a nix representation of a parsed LLVM target triple with This is a Nix representation of a parsed LLVM target triple
white-listed components. This can be specified directly, or actually with white-listed components. This can be specified directly,
parsed from the <varname>config</varname>. [Technically, only one need or actually parsed from the <varname>config</varname>. See
be specified and the others can be inferred, though the precision of <literal>lib.systems.parse</literal> for the exact
inference may not be very good.] See representation.
<literal>lib.systems.parse</literal> for the exact representation.
</para> </para>
</listitem> </listitem>
</varlistentry> </varlistentry>
@ -249,17 +251,17 @@
</para> </para>
<para> <para>
Some examples will probably make this clearer. If a package is being built Some examples will make this clearer. If a package is being built with a
with a <literal>(build, host, target)</literal> platform triple of <literal>(build, host, target)</literal> platform triple of <literal>(foo,
<literal>(foo, bar, bar)</literal>, then its build-time dependencies would bar, bar)</literal>, then its build-time dependencies would have a triple of
have a triple of <literal>(foo, foo, bar)</literal>, and <emphasis>those <literal>(foo, foo, bar)</literal>, and <emphasis>those packages'</emphasis>
packages'</emphasis> build-time dependencies would have triple of build-time dependencies would have a triple of <literal>(foo, foo,
<literal>(foo, foo, foo)</literal>. In other words, it should take two foo)</literal>. In other words, it should take two "rounds" of following
"rounds" of following build-time dependency edges before one reaches a build-time dependency edges before one reaches a fixed point where, by the
fixed point where, by the sliding window principle, the platform triple no sliding window principle, the platform triple no longer changes. Indeed,
longer changes. Indeed, this happens with cross compilation, where only this happens with cross-compilation, where only rounds of native
rounds of native dependencies starting with the second necessarily coincide dependencies starting with the second necessarily coincide with native
with native packages. packages.
</para> </para>
<note> <note>
@ -271,23 +273,23 @@
</note> </note>
<para> <para>
How does this work in practice? Nixpkgs is now structured so that How does this work in practice? Nixpkgs is now structured so that build-time
build-time dependencies are taken from <varname>buildPackages</varname>, dependencies are taken from <varname>buildPackages</varname>, whereas
whereas run-time dependencies are taken from the top level attribute set. run-time dependencies are taken from the top level attribute set. For
For example, <varname>buildPackages.gcc</varname> should be used at build example, <varname>buildPackages.gcc</varname> should be used at build-time,
time, while <varname>gcc</varname> should be used at run time. Now, for while <varname>gcc</varname> should be used at run-time. Now, for most of
most of Nixpkgs's history, there was no <varname>buildPackages</varname>, Nixpkgs's history, there was no <varname>buildPackages</varname>, and most
and most packages have not been refactored to use it explicitly. Instead, packages have not been refactored to use it explicitly. Instead, one can use
one can use the six (<emphasis>gasp</emphasis>) attributes used for the six (<emphasis>gasp</emphasis>) attributes used for specifying
specifying dependencies as documented in dependencies as documented in <xref linkend="ssec-stdenv-dependencies"/>. We
<xref linkend="ssec-stdenv-dependencies"/>. We "splice" together the "splice" together the run-time and build-time package sets with
run-time and build-time package sets with <varname>callPackage</varname>, <varname>callPackage</varname>, and then <varname>mkDerivation</varname> for
and then <varname>mkDerivation</varname> for each of four attributes pulls each of four attributes pulls the right derivation out. This splicing can be
the right derivation out. This splicing can be skipped when not cross skipped when not cross-compiling as the package sets are the same, but is a
compiling as the package sets are the same, but is a bit slow for cross bit slow for cross-compiling. Because of this, a best-of-both-worlds
compiling. Because of this, a best-of-both-worlds solution is in the works solution is in the works with no splicing or explicit access of
with no splicing or explicit access of <varname>buildPackages</varname> <varname>buildPackages</varname> needed. For now, feel free to use either
needed. For now, feel free to use either method. method.
</para> </para>
<note> <note>
@ -305,11 +307,11 @@
<title>Cross packaging cookbook</title> <title>Cross packaging cookbook</title>
<para> <para>
Some frequently problems when packaging for cross compilation are good to Some frequently encountered problems when packaging for cross-compilation
just spell and answer. Ideally the information above is exhaustive, so this should be answered here. Ideally, the information above is exhaustive, so
section cannot provide any new information, but its ludicrous and cruel to this section cannot provide any new information, but it is ludicrous and
expect everyone to spend effort working through the interaction of many cruel to expect everyone to spend effort working through the interaction of
features just to figure out the same answer to the same common problem. many features just to figure out the same answer to the same common problem.
Feel free to add to this list! Feel free to add to this list!
</para> </para>
@ -366,15 +368,14 @@
<note> <note>
<para> <para>
More information needs to moved from the old wiki, especially More information needs to be moved from the old wiki, especially <link
<link xlink:href="https://nixos.org/wiki/CrossCompiling" />, for this xlink:href="https://nixos.org/wiki/CrossCompiling" />, for this section.
section.
</para> </para>
</note> </note>
<para> <para>
Nixpkgs can be instantiated with <varname>localSystem</varname> alone, in Nixpkgs can be instantiated with <varname>localSystem</varname> alone, in
which case there is no cross compiling and everything is built by and for which case there is no cross-compiling and everything is built by and for
that system, or also with <varname>crossSystem</varname>, in which case that system, or also with <varname>crossSystem</varname>, in which case
packages run on the latter, but all building happens on the former. Both packages run on the latter, but all building happens on the former. Both
parameters take the same schema as the 3 (build, host, and target) platforms parameters take the same schema as the 3 (build, host, and target) platforms
@ -440,15 +441,14 @@ nix-build &lt;nixpkgs&gt; --arg crossSystem.config '&lt;arch&gt;-&lt;os&gt;-&lt;
build plan or package set. A simple "build vs deploy" dichotomy is adequate: build plan or package set. A simple "build vs deploy" dichotomy is adequate:
the sliding window principle described in the previous section shows how to the sliding window principle described in the previous section shows how to
interpolate between the these two "end points" to get the 3 platform triple interpolate between the these two "end points" to get the 3 platform triple
for each bootstrapping stage. That means for any package a given package for each bootstrapping stage. That means for any package a given package set,
set, even those not bound on the top level but only reachable via even those not bound on the top level but only reachable via dependencies or
dependencies or <varname>buildPackages</varname>, the three platforms will <varname>buildPackages</varname>, the three platforms will be defined as one
be defined as one of <varname>localSystem</varname> or of <varname>localSystem</varname> or <varname>crossSystem</varname>, with the
<varname>crossSystem</varname>, with the former replacing the latter as one former replacing the latter as one traverses build-time dependencies. A last
traverses build-time dependencies. A last simple difference then is simple difference is that <varname>crossSystem</varname> should be null when
<varname>crossSystem</varname> should be null when one doesn't want to one doesn't want to cross-compile, while the <varname>*Platform</varname>s
cross-compile, while the <varname>*Platform</varname>s are always non-null. are always non-null. <varname>localSystem</varname> is always non-null.
<varname>localSystem</varname> is always non-null.
</para> </para>
</section> </section>
<!--============================================================--> <!--============================================================-->
@ -461,14 +461,14 @@ nix-build &lt;nixpkgs&gt; --arg crossSystem.config '&lt;arch&gt;-&lt;os&gt;-&lt;
<note> <note>
<para> <para>
If one explores nixpkgs, they will see derivations with names like If one explores Nixpkgs, they will see derivations with names like
<literal>gccCross</literal>. Such <literal>*Cross</literal> derivations is <literal>gccCross</literal>. Such <literal>*Cross</literal> derivations is a
a holdover from before we properly distinguished between the host and holdover from before we properly distinguished between the host and target
target platforms —the derivation with "Cross" in the name covered the platforms—the derivation with "Cross" in the name covered the <literal>build
<literal>build = host != target</literal> case, while the other covered the = host != target</literal> case, while the other covered the <literal>host =
<literal>host = target</literal>, with build platform the same or not based target</literal>, with build platform the same or not based on whether one
on whether one was using its <literal>.nativeDrv</literal> or was using its <literal>.nativeDrv</literal> or <literal>.crossDrv</literal>.
<literal>.crossDrv</literal>. This ugliness will disappear soon. This ugliness will disappear soon.
</para> </para>
</note> </note>
</section> </section>

8
doc/stdenv.xml
View File

@ -258,15 +258,15 @@ genericBuild
</para> </para>
<para> <para>
It is important to note dependencies are not necessarily propagated as the It is important to note that dependencies are not necessarily propagated as
same sort of dependency that they were before, but rather as the the same sort of dependency that they were before, but rather as the
corresponding sort so that the platform rules still line up. The exact rules corresponding sort so that the platform rules still line up. The exact rules
for dependency propagation can be given by assigning to each dependency two for dependency propagation can be given by assigning to each dependency two
integers based one how its host and target platforms are offset from the integers based one how its host and target platforms are offset from the
depending derivation's platforms. Those offsets are given below in the depending derivation's platforms. Those offsets are given below in the
descriptions of each dependency list attribute. Algorithmically, we traverse descriptions of each dependency list attribute. Algorithmically, we traverse
propagated inputs, accumulating every propagated dependency's propagated propagated inputs, accumulating every propagated dependency's propagated
dependenciess and adjusting them to account for the "shift in perspective" dependencies and adjusting them to account for the "shift in perspective"
described by the current dependency's platform offsets. This results in sort described by the current dependency's platform offsets. This results in sort
a transitive closure of the dependency relation, with the offsets being a transitive closure of the dependency relation, with the offsets being
approximately summed when two dependency links are combined. We also prune approximately summed when two dependency links are combined. We also prune
@ -424,7 +424,7 @@ let f(h, h + 1, i) = i + h
target offset from the new derivation's platforms. These are programs used target offset from the new derivation's platforms. These are programs used
at build time that produce code to run with code produced by the depending at build time that produce code to run with code produced by the depending
package. Most commonly, these are tools used to build the runtime or package. Most commonly, these are tools used to build the runtime or
standard library taht the currently-being-built compiler will inject into standard library that the currently-being-built compiler will inject into
any code it compiles. In many cases, the currently-being-built-compiler is any code it compiles. In many cases, the currently-being-built-compiler is
itself employed for that task, but when that compiler won't run (i.e. its itself employed for that task, but when that compiler won't run (i.e. its
build and host platform differ) this is not possible. Other times, the build and host platform differ) this is not possible. Other times, the