From 59ead6227c97da6acf6abfced89e1ed27a1c4dd3 Mon Sep 17 00:00:00 2001 From: Juan RP Date: Sat, 22 Nov 2014 17:19:25 +0100 Subject: [PATCH] README: import documentation from void, with some twerks. --- README.md | 213 +++++++++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 211 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index 8b09e36a6ab..a3763d51095 100644 --- a/README.md +++ b/README.md @@ -3,14 +3,36 @@ This repository contains the XBPS source packages collection to build binary packages for the Void Linux distribution. -To start using it first you'll need some external dependencies: +The included `xbps-src` script will fetch and compile the sources, and install its +files into a `fake destdir` to generate XBPS binary packages that can be installed +through the `xbps-install(8)` utility. -- bash +The `xbps-src` utility uses `xbps-uchroot(8)` to build packages in lightweight linux +`containers` thru the use of `namespaces`, that means that processes and bind mounts +(among others) won't be visible to other processes in the system. + +### Requirements + +- GNU bash - xbps >= 0.41 Make sure your user is added to the `xbuilder` group to be able to use `xbps-uchroot(8)`, otherwise `xbps-src` won't work correctly. +### Quick setup in Void + +Add your user to the `xbuilder` group: + + # usermod -a -G xbuilder + +Clone the `void-packages` git repository, install the bootstrap packages: + +``` +$ git clone git://github.com/voidlinux/void-packages.git +$ cd void-packages +$ ./xbps-src binary-bootstrap +``` + Type: $ ./xbps-src -h @@ -18,6 +40,15 @@ Type: to see all available targets/options and start building any available package in the `srcpkgs` directory. +### Install the bootstrap packages + +The `bootstrap` packages are a set of packages required to build any available source package in a container. There are two methods to install the `bootstrap`: + + - `bootstrap`: all bootstrap packages will be built from scratch. + - `binary-bootstrap`: the bootstrap binary packages are downloaded via XBPS repositories. + +If you don't want to waste your time building everything from scratch probably it's better to use `binary-bootstrap`. + ### Configuration The `etc/defaults.conf` file contains the possible settings that can be overrided @@ -38,6 +69,184 @@ used as dependencies in the source packages tree. If you want to customize those replacements, copy `etc/defaults.virtual` to `etc/virtual` and edit it accordingly to your needs. +### Directory tree + +The following directory tree is used with a default configuration file: + + /home/foo + |- void-packages + |- common + |- etc + |- srcpkgs + | |- xbps + | |- template + | + |- hostdir + | |- binpkgs ... + | |- ccache- ... + | |- distcc- ... + | |- repocache ... + | |- sources ... + | + |- masterdir + | |- builddir -> ... + | |- destdir -> ... + | |- host -> bind mounted from + | |- void-packages -> bind mounted from + + +The description of these directories is as follows: + + - `masterdir`: master directory to be used as rootfs to build/install packages. + - `builddir`: to unpack package source tarballs and where packages are built. + - `destdir`: to install packages, aka **fake destdir**. + - `hostdir/ccache-`: to store ccache data if the `XBPS_CCACHE` option is enabled. + - `hostdir/distcc-`: to store distcc data if the `XBPS_DISTCC` option is enabled. + - `hostdir/repocache`: to store downloaded binary packages from remote repositories to resolve package dependencies. + - `hostdir/sources`: to store package source tarballs. + - `hostdir/binpkgs`: local repository to store generated binary packages. + +### Building packages + +The simplest form of building package is accomplished by running the `pkg` target in `xbps-src`: + +``` +$ cd void-packages +$ ./xbps-src pkg +``` + +When the package and its required dependencies are built, the binary packages will be in +default local repository at `hostdir/binpkgs`; the path to this local repository can be added to +the xbps configuration files or by explicitly appending it via cmdline, i.e `xbps-install --repository=/path/to/hostdir/binpkgs`. + +By default **xbps-src** will try to resolve package dependencies in this order: + + - If dependency exists in the local repository, use it (`hostdir/binpkgs`). + - If dependency exists in a remote repository, use it. + - If dependency exists in a source package, use it. + +It is possible to avoid using remote repositories completely by using the `-N` flag. + +### Rebuilding and overwriting existing local packages + +If for whatever reason a package has been built and it is available in your local repository +and you have to rebuild it without bumping its `version` or `revision` fields, it is possible +to accomplish this task easily with `xbps-src`: + + $ ./xbps-src -f pkg xbps + +Reinstalling this package in your target `rootdir` can be easily done too: + + $ xbps-install --repository=/path/to/local/repo -yff xbps-0.25_1 + +> Please note that the `package expression` must be properly defined to explicitly pick up +the package from the desired repository. + +### Enabling distcc for distributed compilation + +Setup the slaves (machines that will compile the code): + + # xbps-install -Sy distcc + +Enable and start the `distccd` service: + + # ln -s /etc/sv/distccd /var/service + +In the host (machine that executes xbps-src) enable the following settings in the `void-packages/etc/conf` file: + + XBPS_DISTCC=yes + XBPS_DISTCC_HOSTS="192.168.2.101 192.168.2.102" + +### Cross building packages for a target architecture + +Currently `xbps-src` can cross build packages for some target architectures with a cross compiler. The supported target list is the following: + +* i686 - for Linux i686 GNU. +* i686-musl - for Linux i686 Musl libc. +* armv6hf - for Linux ARMv6 EABI5 (LE Hard Float / GNU) +* armv6hf-musl - for Linux ARMv6 EABI5 (LE Hard Float / Musl libc) +* armv7hf - for Linux ARMv7 EABI5 (LE Hard Float / GNU) +* armv7hf-musl - for Linux ARMv7 EABI5 (LE Hard Float / Musl libc) +* mips - for Linux MIPS o32 (BE Soft Float / GNU) +* mipsel - for Linux MIPS o32 (LE Soft Float / GNU) +* x86_64-musl - for x86_64 Musl/Linux + +If a source package has been adapted to be **cross buildable** `xbps-src` will automatically build the binary package(s) with a simple command: + + $ ./xbps-src -a pkg + +If the build for whatever reason fails, might be a new build issue or simply because it hasn't been adapted to be **cross built**. Join #xbps on irc.freenode.net for more information. + +### Using xbps-src in a foreign linux distribution + +xbps-src can be used in any recent linux distribution matching the cpu architecture. At least a linux kernel 2.6.32 is required. + +To use xbps-src in your linux distribution use the following instructions. Let's start downloading the xbps static binaries: + + $ wget http://repo.voidlinux.eu/static/xbps-static-latest.-musl.tar.xz + $ mkdir ~/XBPS + $ tar xvf xbps-static-latest..tar.xz -C ~/XBPS + $ export PATH=~/XBPS/usr/sbin:$PATH + +A privileged group is required to be able to chroot with xbps-src, by default it's set to the `xbuilder` group, change this to your desired group: + + # chown root: ~/XBPS/usr/sbin/xbps-uchroot.static + # chmod 4750 ~/XBPS/usr/sbin/xbps-uchroot.static + +Clone the `void-packages` git repository: + + $ git clone git://github.com/voidlinux/void-packages + +and `xbps-src` should be fully functional; just start the `bootstrap` process, i.e: + + $ ./xbps-src binary-bootstrap + +The default masterdir is created in the current working directory, i.e `void-packages/masterdir`. + +### Remaking the masterdir + +If for some reason you must update xbps-src and the `bootstrap-update` target is not enough, it's possible to recreate a masterdir with two simple commands (please note that `zap` keeps your `ccache/distcc/host` directories intact): + + $ ./xbps-src zap + $ ./xbps-src binary-bootstrap + +### Keeping your masterdir uptodate + +Sometimes the bootstrap packages must be updated to the latest available version in repositories, this is accomplished with the `bootstrap-update` target: + + $ ./xbps-src bootstrap-update + +### Building i686/32bit packages on x86_64 + +A new x86 `masterdir` must be created to build 32bit packages: + + $ ./xbps-src -m masterdir-x86 binary-bootstrap i686 + +Packages that are multilib only (32bit) must be built on a 32bit masterdir. + + $ ./xbps-src -m masterdir-x86 ... + +#### Building packages natively for the musl C library + +A native build environment is required to be able to cross compile the bootstrap packages for the musl C library; this is accomplished by installing them via `binary-bootstrap`: + + $ ./xbps-src binary-bootstrap + +Now cross compile `base-chroot-musl` for your native architecture: + + $ ./xbps-src -a x86_64-musl pkg base-chroot-musl + +Wait until all packages are built and when ready, prepare a new masterdir with the musl packages: + + $ ./xbps-src -m masterdir-x86_64-musl binary-bootstrap x86_64-musl + +Your new masterdir is now ready to build natively packages for the musl C library. Try: + + $ ./xbps-src -m masterdir-x86_64-musl chroot + $ ldd + +To see if the musl C dynamic linker is working as expected. + ### Contributing See [Manual](https://github.com/voidlinux/xbps-packages/blob/master/Manual.md)