Commit 58968cf fixed symlinks for package products in $startdir in
light of the simplified chroot setup. However, a similar change needs
to be made for source-package products. This was an easy omission to
make because makechrootpkg does not produce source-pakcages by
default.
The bug isn't currently triggered, but I accidentally did trigger when I
was trying to modify the command a bit. I figure a "caution" sign would be
helpful to any future developers.
What this is really doing is fixing a conflict that I had incorrectly
resolved when rebasing what became 2fd5931 onto cda9cf4. Of course,
because of dynamic scoping, everything worked out, and everything worked as
intended.
Before cda9cf4, it was appropriate for download_sources to take src_owner
as an argument, but after cda9cf4, it is now appropriate to take
makepkg_user as an argument. However, it still takes src_owner as an
argument, but pays 0 attention to it; instead looking at makepkg_user which
it happily inherited because of dynamic scoping.
So change it to take makepkg_user as the argument.
This is inspired by the thought that went in to the delete_chroot
is_subvolume commit.
sync_chroot($chrootdir, $copydir) copies `$chrootdir/root` to `$copydir`.
That seems a little silly; why do we care about "$chrootdir"? Have it just
be sync_chroot(source, destination) like every other sync/copy command.
Where this becomes tricky is check to decide if we are going to use btrfs
subvolumes or not. We don't care if "$source/.." is on btrfs; the root
could be a directly-mounted subvolume, but and the destination could be
another subvolume of the same btrfs mounted somewhere else.
The things we do care about are:
- The source is a btrfs subvolume (so that we can snapshot it)
- The source is on the same filesystem as the directory that the copy will
be created in.
- If the destination exists:
* that it is not a mountpoint (so that we can delete and recreate it)
* that it is a btrfs subvolume (so that we can quickly delete it)
On the last point, it isn't necessary for creating the new snapshot, just
for quick deletion. That can be a separate check, where we use regular
`rm` for deleting the existing copy, but use subvolume snapshots for
creating the new one.
Also, shorten the "Synchronizing" message to only include the full path
to the copy if it was specified.
The capslocked variable names in the Usage comment were references to
things in Parabola's tools, that didn't make much sense here out of
context.
First of all, it ran `is_btrfs "$chrootdir"` to decide if it was on
btrfs, but $chrootdir wasn't defined locally; it just happens to work
because $chrootdir was defined in main(). (I noticed this because in
Parabola, it is called differently, so $chrootdir was empty).
So I was tempted to just change it to `is_btrfs "$copydir"`, but if
$copydir is just a regular directory on a btrfs filesystem, then it
It would leave much of $copydir intact. What we really care about is
if $copydir is a btrfs subvolume; which we can check by combining the
is_btrfs check with inspecting the inum of the directory.
I put this combined check in lib/archroot.sh:is_subvolume.
https://lists.archlinux.org/pipermail/arch-projects/2013-September/003901.html
This means wrapping variable initialization in init_variables(), and the
main program routine in main().
I did NOT put `shopt -s nullglob` in to a function.
It make make sense to move init_variables() down into the main()
function, instead of having it as a separate function up top (if this
done, then the `-g` flag passed to `declare` in init_variables() can
be dropped). However, in interest of keeping the `diff -w` small, and
merges/rebases simpler, this isn't done here.
In sync_chroot(), this makes the messages be a bit more precise with
exactly which thing they are syncing where. This is based on my users
expressing confusion at what is going on (especially when something is
taking a long time, and they have to blame something for blocking).
With these changes, I haven't gotten such confusion in a long time
(but maybe my users just got used to it).
In delete_chroot(), this changes "temporary copy" to "chroot copy",
since in Parabola's version of the tools, the function can get called
from other places, and it isn't necessarily operating on a temporary
copy.
Even though main() doesn't call `set -u`; this way the functions will
continue to work if copied into an environment with `set -u`, or so
that we are ready if we ever want to start using `set -u`.
Rather than them simply being named blocks of code with braces around
them.
That is: have them take things via arguments rather than global
variables.
Specific notes:
- create_chroot->sync_chroot:
I pulled out locking the destination chroot; getting that lock is
now the caller's responsibility. It still handles locking the
source chroot though.
I pulled the `if [[ ! -d $copydir ]] || $clean_first;` check out; it is
now the caller's responsibility to use that check when deciding if to
call sync_chroot.
However, when pulling that check out, I left it as `if true;`, to
keep an indentation level. This patch has had to be rebased/merged
many times, and changing the indentation is a sure way to make that
go less smoothly; I'm not going to re-indent this block until I see
the check removed in the git.archlinux.org/devtools.git repository.
- install_packages:
1. Receive the list of packages as arguments, rather than a global
variable.
2. Make the caller responsible for looking at PKGBUILD. From the
name and arguments, one would never expect it to look at PKGBUILD.
It was displaing the value of the `makepkg_args` variable, which may
have already been changed by the argument parsing by the time it gets
to `-h`. Now there is a separate `default_makepkg_args` variable.
This involves extending the signature of lib/common.sh's `stat_busy()`,
`lock()`, and `slock()`. The `mesg=$1; shift` in stat_busy even suggests
that this is what was originally intended from it.
In cases where there is no license specified, the file is tagged as
"License: Unspecified". Obviously, that is not ideal, but it
highlights the fact, and I hope that it encourages whoever has the
authority to specify the license to do so.
On that note, to anyone who may have the authority to specify the
license of files in devtools: the current licence of many files is
GPLv2 with no option for later versions; I impore you to re-license
them to have the "or any later version" option.
`lock_close FD` is easier to remember than 'exec FD>&-`; and is especially
easier if FD is a variable (though that isn't actually taken advantage of
here).
This uses Bash 4.1+ `exec {var}>&-`, rather than the clunkier
`eval exec "$var>&-"` that was necessary in older versions of Bash.
Thanks to Dave Reisner for pointing this new bit of syntax out to me
the last time I submitted this (back in 2014, 4.1 had just come out).
The systemd package creates a subvolume at /var/lib/machines (through
tmpfiles), if it can. We need to delete this subvolume before we can
delete the parent subvolume.
Look through the root for inodes with the number 256. These identify
subvolume roots.
The way in which makechrootpkg reads variables from makepkg.conf(5) is
different from makepkg, in that it reads a subset of defined
variables, and only if the were not set in the environment before.
Mention this in the usage text.
Fixes FS#44827
This removes the preservation of HOME being /build just for the pacman
sudo call. Former leads to unbuildable packages when an to be installed
dependency writes something into the HOME dir (f.e. .config). The
resulting directories won't be writable by the builduser as they are
owned by root:root and ultimately will fail to build anything that
requires so.
The gnustep-base package ships a profile.d script that adds
"$HOME/GNUstep/Tools" to the PATH, which breaks when the user changes
and causes meson to exit with a "permission denied" error.
Copy both UID and primary GID of the invoker to the builduser. Mount
srcdest and startdir read-write.
v2: Fixed GnuPG keyring owner and moved running namcap from a heredoc
to a function.
This way the HOME dir is writable and no ugly hacks are required
in the PKGBUILD if $HOME is accessed (f.e. maven, gradle and also
some python tests etc.)
This is needed in order to use GPG's auto-key-retrieve keyserver option,
otherwise the keyring will get copied to the chroot before the required
keys are retrieved during 'makepkg --verifysource'.
Chances are that pubring.kbx has been created by gpgsm but pubring.gpg
is still around with valid data. We do not know what file contains what
we need, so just copy both.
Signed-off-by: Christian Hesse <mail@eworm.de>
Signed-off-by: Pierre Schmitz <pierre@archlinux.de>
In collaborative builder machine, these scripts are often allowed to become root
via sudo. This patch avoid to prefix them by sudo each time or call su.
Signed-off-by: Pierre Schmitz <pierre@archlinux.de>
If getopts comes across an unknown argument, $arg it set to '?' and
$OPTARG is unset. Therefore the getopts line detecting unknown arguments
doesn't work. Arguments to pass to makepkg are already handled by
passing all the aguments after the end-of-options marker (--), but this
wasn't documented in the usage text.
Signed-off-by: Pierre Schmitz <pierre@archlinux.de>
We shouldn't be in the business of reparsing makepkg's arguments, but
since we have to treat the case of repackaging separately, do a better
job of trying to find signs of it happening. This change lets you pass
the longopt, --repackage, or multiple shortopts such as -RA, and still
get the intended effect.
Signed-off-by: Dave Reisner <dreisner@archlinux.org>
Signed-off-by: Pierre Schmitz <pierre@archlinux.de>
Piggyback on systemd-nspawn's --bind and --bind-ro flags to allow
arbitrary mount points to be added to the build container.
Signed-off-by: Dave Reisner <dreisner@archlinux.org>
Signed-off-by: Pierre Schmitz <pierre@archlinux.de>
Now that die() properly forwards arguments to error(), we can expect
that the first arg is a format string and not the entirety of the
output.
Signed-off-by: Dave Reisner <dreisner@archlinux.org>
Signed-off-by: Pierre Schmitz <pierre@archlinux.de>
- Ensure sources are available before entering chroot
- Bind STARTDIR and SRCDEST into the chroot read-only
- Refactor makechrootpkg and introduce meaningful functions
Avoids copying stuff from/to the chroot as much as possible. With
VCS sources these copies can get quite expensive.
Reduces code duplication.
With makechrootpkg not calling mkarchroot anymore,
the lock handover protocol is unneeded.
arch-nspawn does not do any locking, so add protection to archbuild.
Separates the two features of mkarchroot. Provides users of the new
arch-nspawn with the full feature set of systemd-nspawn.
For example, this can be used to bind custom directories into the chroot.
Add option -T to build in a temporary chroot. This apply to any kind of
filesytem and allow to easily parrallelize builds.
This patch also simplify how $default_copy and $copy are defined.
Signed-off-by: Sébastien Luttringer <seblu@seblu.net>
Signed-off-by: Pierre Schmitz <pierre@archlinux.de>
Move detection of chrootdir type after have check if the directory exists.
This avoid the following messages when -r is not given
stat: cannot read file system information for '': No such file or directory
Signed-off-by: Sébastien Luttringer <seblu@seblu.net>
Signed-off-by: Pierre Schmitz <pierre@archlinux.de>
Since TODO [1] which state:
It would be good for base-devel to install everything needed for a build chroot
we can remove base and sudo
[1] https://www.archlinux.org/todo/add-more-to-base-devel/
Signed-off-by: Sébastien Luttringer <seblu@seblu.net>
Signed-off-by: Pierre Schmitz <pierre@archlinux.de>
Enable btrfs features only if the underlying filesystem is btrfs and not rely
on the presence of the btrfs tools.
Signed-off-by: Sébastien Luttringer <seblu@seblu.net>
Signed-off-by: Pierre Schmitz <pierre@archlinux.de>
Since commit cb3a6ce, running makechroot 2 times to insert a package in a build
directory require to find a directory without PKGBUILD
cd /var/empty
makechrootpkg -cu -I virtualbox-host-dkms-*-i686.pkg.tar.xz -r <dir>
makechrootpkg -I virtualbox-host-dkms-*-i686.pkg.tar.xz -r <dir>
cd -
makechrootpkg -n -r <dir>
This patch allow makechrootpkg to handle more than one package to be installed
before the build is run and simplify the previous case in
makechrootpkg -ncu -I virtualbox-host-dkms-*-i686.pkg.tar.xz -I virtualbox-guest-dkms-*-i686.pkg.tar.xz -r <dir>
Signed-off-by: Sébastien Luttringer <seblu@seblu.net>
Signed-off-by: Pierre Schmitz <pierre@archlinux.de>
This allow to build in one shot a package depending of a missing package
Signed-off-by: Sébastien Luttringer <seblu@seblu.net>
Signed-off-by: Pierre Schmitz <pierre@archlinux.de>
The usage messages now begins with a "Usage:", i.e. capitalized and with a colon. Fixes FS#26956.
Signed-off-by: Eric Bélanger <snowmaniscool@gmail.com>
Signed-off-by: Pierre Schmitz <pierre@archlinux.de>
Stop trap_exit from forcing a 0 exit code. This fixes makechrootpkg,
which used to always return success, even if the build failed.
Signed-off-by: Pierre Schmitz <pierre@archlinux.de>
Previously files were always owned by nobody which means trying to write
to them directly would fail because only the owner has +w.
Signed-off-by: Florian Pritz <bluewind@xinu.at>
Signed-off-by: Pierre Schmitz <pierre@archlinux.de>
* bind /sys and /dev/pts from host
* drop support for devtmpfs as it is no longer needed
* add /run and /dev/rtc0
* clone own ipc, uts and mount namespaces for chroot
* set localtime, timezone and locale within chroot environment
* copy /etc/pacman.d/gnupg from host
Passing a directory that does not exist to makechrootpkg results in
an error message:
==> ERROR: No chroot dir defined, or invalid path ''
The path is not being printed as the readlink command blanks it if
the directory does not exist. Fix this.
Signed-off-by: Allan McRae <allan@archlinux.org>
Signed-off-by: Pierre Schmitz <pierre@archlinux.de>
This ensures the PATH used when building is the default path and
not the value set by the user calling makechrootpkg.
Signed-off-by: Allan McRae <allan@archlinux.org>
Signed-off-by: Pierre Schmitz <pierre@archlinux.de>
repack is defined as a boolean. set it true when -R is passed
/usr/sbin/makechrootpkg: line 295: 1: command not found
Signed-off-by: Ionut Biru <ibiru@archlinux.org>
Signed-off-by: Pierre Schmitz <pierre@archlinux.de>
Some of the output/error messages were capitalized, some were
not. This patch capitalize everything for consistency sake. Other
minor changes were done to the messages like removing the superfluous
"error:" from die messages and adding a final period to messages that
were complete sentences as appropriate.
Signed-off-by: Eric Bélanger <snowmaniscool@gmail.com>
Signed-off-by: Pierre Schmitz <pierre@archlinux.de>
It is very helpful to always log the build output in makechrootpkg
so pass "-L" as a standard option to makepkg.
Signed-off-by: Allan McRae <allan@archlinux.org>
Signed-off-by: Lukas Fleischer <archlinux@cryptocrack.de>
Installs namcap if needed, *after* building the package,
contrary to the former way of having to have namcap installed,
e.g. via makedepends.
Signed-off-by: Lukas Fleischer <archlinux@cryptocrack.de>
Martchus
Jan Alexander Steffens (heftig)
Luke Shumaker
Alad Wenter
Levente Polyak
NicoHood
Johannes Löthberg
Evangelos Foutras
Christian Hesse
Dave Reisner
Luke Shumaker
Sébastien Luttringer
Joel Teichroeb
Markus M. May
Maxime Gauduin
WorMzy Tykashi
Eric Bélanger
Pierre Schmitz
Florian Pritz
Allan McRae
Ionut Biru