find_cached_package was unnecessarily looping over all packages which
uses a lot of CPU and could be exceptionally slow when PKGDEST contains
a lot of packages.
Fix this by adding the target pkgname, pkgver and arch to the glob and
only process potential candidates.
Instead only enable it for whatever operation requires them.
Example sides effects:
commitpkg can accidently execute PKGBUILD functions when sourcing the
PKGBUILD that has function names like package_libsigc++()
Fixes#87
This implements our current debug package detection logic.
Mostly taken from our dbscripts project.
Signed-off-by: Morten Linderud <foxboron@archlinux.org>
Signed-off-by: Levente Polyak <anthraxx@archlinux.org>
Add the SPDX license identifier GPL-3.0-or-later to the header of all
scripts without a specific license and upgrading those that are stated
as GPL-2.0 to become GPL-3.0-or-later.
We need to modify the matching of valid package files to support formats
like zstd. Let's try to use an eager approach instead of a simple
whitelist in order to be functional for arbitrary formats that may be
introduced in the future without the need to adjust any code.
Allow any single fragment word as compression type but filter out known
non-package content like signature files.
Signed-off-by: Levente Polyak <anthraxx@archlinux.org>
Transform sogrep into an in-prog so we can benefit from the m4 macro
to specify valid repos in a single place of truth.
Signed-off-by: Levente Polyak <anthraxx@archlinux.org>
Programs can freely define the value of argv0 and thus it means nothing.
Instead, use the bash-specific variable explicitly designed to safely
and accurately reference the name of the currently sourced file.
This also fixes the case where simple debugging mechanisms like using
"bash -x foo" tried to treat "foo" as the unqualified $0 and therefore
broke horribly due to lack of pathnames.
Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Levente Polyak <anthraxx@archlinux.org>
This reverts commit 6d1992909c.
It has never worked. In commit c86823a2d4
it was noted that it compared the device numbers for [[ $1 = $1 ]] which
was a useless check and always returned true, for *any* btrfs
filesystem. Now that the function is corrected to compare [[ $1 = $2 ]]
the check is still useless, but this time because it always returns
false -- btrfs subvolumes on the same filesystem do *not* share device
numbers.
So let's go back to the original working implementation that only
matters in terms of makechrootpkg, and just checks if makechrootpkg's
root working directory is btrfs (in which case we know it will be a
subvolume because mkarchroot will create it that way).
This restores our special support for the btrfs filesystem.
Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Support for working with `set -u` was broken by 94160d6. Egg on my
face; I'm the one who wants `set -u` support, and I'm the author of
that commit!
libmakepkg does not work with `set -u`; but mostly because of the include
guards! So we just need to temporarily disable `set -u` (nounset) while
loading libmakepkg. Instead of introducing a new variable, just store the
initial nounset status in _INCLUDE_COMMON_SH; rather than a useless
fixed-string "true".
While we're at it, disable POSIX-mode (just in case we're running as "sh"
instead of "bash"), since libmakepkg uses bash-isms that won't parse in
POSIX mode.
Recent development versions of makepkg support reproducible builds
through the environment variable SOURCE_DATE_EPOCH. Pass this variable
through makechrootpkg to makepkg when available.
Also initialize SOURCE_DATE_EPOCH whenever running archbuild to enforce
reproducible builds for repository packages.
Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Levente Polyak <anthraxx@archlinux.org>
- Use `read -r` instead of other forms of read or looping
- Use arrays instead of strings with whitespaces.
- In one instance, use ${var%%.*} instead of $(echo $var|cut -f. -d1)
The `-xdev` flag to `find` makes it not recurse over subvolumes; so it only
supports recursion with depth=1. Fix this by having the function
recursively call itself.
This is inspired by the thought that went in to the delete_chroot
is_subvolume commit.
sync_chroot($chrootdir, $copydir) copies `$chrootdir/root` to `$copydir`.
That seems a little silly; why do we care about "$chrootdir"? Have it just
be sync_chroot(source, destination) like every other sync/copy command.
Where this becomes tricky is check to decide if we are going to use btrfs
subvolumes or not. We don't care if "$source/.." is on btrfs; the root
could be a directly-mounted subvolume, but and the destination could be
another subvolume of the same btrfs mounted somewhere else.
The things we do care about are:
- The source is a btrfs subvolume (so that we can snapshot it)
- The source is on the same filesystem as the directory that the copy will
be created in.
- If the destination exists:
* that it is not a mountpoint (so that we can delete and recreate it)
* that it is a btrfs subvolume (so that we can quickly delete it)
On the last point, it isn't necessary for creating the new snapshot, just
for quick deletion. That can be a separate check, where we use regular
`rm` for deleting the existing copy, but use subvolume snapshots for
creating the new one.
First of all, it ran `is_btrfs "$chrootdir"` to decide if it was on
btrfs, but $chrootdir wasn't defined locally; it just happens to work
because $chrootdir was defined in main(). (I noticed this because in
Parabola, it is called differently, so $chrootdir was empty).
So I was tempted to just change it to `is_btrfs "$copydir"`, but if
$copydir is just a regular directory on a btrfs filesystem, then it
It would leave much of $copydir intact. What we really care about is
if $copydir is a btrfs subvolume; which we can check by combining the
is_btrfs check with inspecting the inum of the directory.
I put this combined check in lib/archroot.sh:is_subvolume.
https://lists.archlinux.org/pipermail/arch-projects/2013-September/003901.html
This is similar to common C #ifdef guards.
I was tempted to wrap the entire thing in the if/fi, rather than use
'return' to bail early. However, that means it won't execute anything
until after it reaches 'fi'. And if `shopt -s extglob` isn't executed
before parsing, then it will syntax-error on the extended globs. One
solution would have been to move `shopt -s extglob` up above the
include-guard. But the committed solution is all-around simpler.
This involves extending the signature of lib/common.sh's `stat_busy()`,
`lock()`, and `slock()`. The `mesg=$1; shift` in stat_busy even suggests
that this is what was originally intended from it.
In cases where there is no license specified, the file is tagged as
"License: Unspecified". Obviously, that is not ideal, but it
highlights the fact, and I hope that it encourages whoever has the
authority to specify the license to do so.
On that note, to anyone who may have the authority to specify the
license of files in devtools: the current licence of many files is
GPLv2 with no option for later versions; I impore you to re-license
them to have the "or any later version" option.
Allow for locks to be inherited. Inheriting the lock is something that
mkarchroot could do previously, but has since lost the ability to do. This
allows for the programs to be more compos-able.
Do this by instead of unconditionally opening $file on $fd, first check if
$file is already open on $fd; and go ahead use it if it is.
The naive way of doing this would be to `$(readlink /dev/fd/$fd)` and
compare that to `$file`. However, if `$file` is itself a symlink; or there
is a symlink somewhere in the path to `$file`, then this could easily fail.
Instead, check `[[ "/dev/fd/$fd" -ef "$file" ]]`. Even though the Bash
documentation (`help test`) says that `-ef` checks for if the two files are
hard links to eachother, because it uses stat(3) (which resolves symlinks)
to do this check, it also works with the /dev/fd/ soft links.
`lock_close FD` is easier to remember than 'exec FD>&-`; and is especially
easier if FD is a variable (though that isn't actually taken advantage of
here).
This uses Bash 4.1+ `exec {var}>&-`, rather than the clunkier
`eval exec "$var>&-"` that was necessary in older versions of Bash.
Thanks to Dave Reisner for pointing this new bit of syntax out to me
the last time I submitted this (back in 2014, 4.1 had just come out).
The systemd package creates a subvolume at /var/lib/machines (through
tmpfiles), if it can. We need to delete this subvolume before we can
delete the parent subvolume.
Look through the root for inodes with the number 256. These identify
subvolume roots.
We run from a non-interactive shell, so the exec which is inevitably
called will replace the current process and 'die' will never run under
any circumstances.
This also fixes a bug with the su fallback which would cause multiple
arguments to be concatenated without any whitespace between them.
In collaborative builder machine, these scripts are often allowed to become root
via sudo. This patch avoid to prefix them by sudo each time or call su.
Signed-off-by: Pierre Schmitz <pierre@archlinux.de>
If PKGDEST is set when makepkg was run, the package will be present in
find_cached_package's search path by default, causing an error.
This also fixes a display bug which causes no output to be shown when
multiple packages are found.
Fixes FS#37626.
Signed-off-by: Dave Reisner <dreisner@archlinux.org>
Signed-off-by: Pierre Schmitz <pierre@archlinux.de>
This function (currently) searches through $PWD and $PKGDEST looking
for a tarball matching the requested package name, architecture, and
pkgver. If found, it writes the full path to the located package to
stdout and returns 0, else 1. If more than 1 match is found, it's
treated as an error and the user will need to figure out what to do.
Use this in checkpkg and commitpkg, which previously implemented their
own less complete logic, to locate the build artifacts they rely on.
Signed-off-by: Dave Reisner <dreisner@archlinux.org>
Signed-off-by: Pierre Schmitz <pierre@archlinux.de>
Also allow this function to be called without arguments, in which case,
don't call error at all. Some uses of this function wrongly assumed
that this was already allowed.
Signed-off-by: Dave Reisner <dreisner@archlinux.org>
Signed-off-by: Pierre Schmitz <pierre@archlinux.de>
Reduces code duplication.
With makechrootpkg not calling mkarchroot anymore,
the lock handover protocol is unneeded.
arch-nspawn does not do any locking, so add protection to archbuild.
Stop trap_exit from forcing a 0 exit code. This fixes makechrootpkg,
which used to always return success, even if the build failed.
Signed-off-by: Pierre Schmitz <pierre@archlinux.de>
Levente Polyak
Felix Yan
Morten Linderud
David Runge
Ivy Foster
Eli Schwartz
Eli Schwartz via arch-projects
Erich Eckner
Luke Shumaker
Bartłomiej Piotrowski
Luke Shumaker
Jan Alexander Steffens (heftig)
Jelle van der Waa
Dave Reisner
Sébastien Luttringer
BlackEagle