consolidate logic flows in the same area for parsing and building
arrays. Don't bother having a special function just to build the
mount_args array, since we now use the same handling for adding any
cachedir (including host mirrors) to the mount arguments, this becomes a
trivial for loop -- and it really did not need to be delayed until after
the sanity check, anyway.
Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Levente Polyak <anthraxx@archlinux.org>
In commit 27ff286ee7, we moved from
sourcing the primary cachedir via /etc/pacman.conf, to using the
pacman.conf in the workdir. One unanticipated side effect of this was
breaking the special host mirrors magic we used to turn a host mirror
into a cachedir. It was still processed as a server, but we relied on it
being in the host's cachedirs in order to be persisted, and this no
longer occurred.
Solve this by explicitly adding each host mirror root as a cachedir.
Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Levente Polyak <anthraxx@archlinux.org>
Transform sogrep into an in-prog so we can benefit from the m4 macro
to specify valid repos in a single place of truth.
Signed-off-by: Levente Polyak <anthraxx@archlinux.org>
Lets prefer the explicit variant of gpg --verify by providing both, the
signature and the data file as parameters.
For the unlikely case there is a matching signature file already present
that was created outside of the toolchain and has an embedded signature
with data, we at least could detect it early with this check.
Signed-off-by: Levente Polyak <anthraxx@archlinux.org>
Several cases showed that we release packages that were built with
different PKGBUILDs than the one commited to the source tree. This is
bad for obvious reasons plus sploils reproducible builds.
We, under no circumstances, want to allow using commitpkg to publish and
release a packages whose PKGBUILD doesn't match the one to be commited.
Signed-off-by: Levente Polyak <anthraxx@archlinux.org>
The unknown packager check didn't worked so far as the wrongly ordered
call to find_cached_package lead to the enclosing block never being
executed.
Signed-off-by: Levente Polyak <anthraxx@archlinux.org>
Sometimes its desired to be explicitly made aware of differences
reporter by checkpkg via printing a warning instead of a regular
message.
Automatically use --warn for makechrootpkg builds so packagers are made
visibly aware of a soname bump by simply looking out for colors
indicating non success messages.
Signed-off-by: Levente Polyak <anthraxx@archlinux.org>
In some cases, like default makechrootpkg execution, the temporary
directory used to assemble the differences is not required. Add an
option to checkpkg that allows to get rid of that directory after
run and call it automatically like that in makechrootpkg.
Signed-off-by: Levente Polyak <anthraxx@archlinux.org>
Cache previous versions required for checkpkg via pacman to avoid
multiple downloads when running multiple times.
In case we can't download the packages, like while building out of repo
packages, print a warning instead of running checkpkg
Signed-off-by: Levente Polyak <anthraxx@archlinux.org>
This reverts commit be44b9cde1.
This was a nice idea in theory, because it means that we can catch
conflicting files before releasing them into the repos. In practice,
there were unanticipated side effects: single-package installs which
conflict against their own makedepends cannot be installed either.
Examples include:
- kernel modules which makedepend on their dkms equivalent
- jack2, which makedepends/optdepends on portaudio, which requires
jack... but jack2 is a drop-in provides/conflicts jack.
We cannot reliably detect when makepkg --install will error out because
of dependency conflicts vs. packages which are simply broken. So, back
out this change for now.
Revisit this once pacutils has a new release, because it will add the
option --resolve-conflicts=all, allowing for much better scripted
responses to "foo conflicts with bar, remove bar? [y/N]" than simply
"--noconfirm and fail".
Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Levente Polyak <anthraxx@archlinux.org>
We previously whitelisted this return code because split packages can
frequently conflict each other, so makepkg -i is *expected* to fail in
such a case. However, there is no good reason to let this succeed if the
pkgbase only builds one pkgname -- that will always be a severe issue.
Add a check for how many split
Signed-off-by: Levente Polyak <anthraxx@archlinux.org>
The -U option was initially introduced in commit
cda9cf436b in order to enable running
makechrootpkg as root, delegating to another, manually selected, user to
perform various non-root tasks (given that makepkg was modified to throw
fatal errors when run as root without the option of --asroot to disable
that). However, it was only ever implemented for the --verifysource
option outside of the chroot, and the builduser inside the chroot is
created with the same uid as the makechrootpkg invoker. It needs to run
as the same uid, because it needs rw access to $startdir and $SRCDEST!
Additionally this lets the invoking user more easily inspect the build
directory in case of problems...
The correct solution for this is to properly implement the initial
intention of the -U option, and make it override the autodetection of
the "invoking user" which is normally done by inspecting $SUDO_USER.
This is then used as the single source of truth for "who am I pretending
to be".
Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Levente Polyak <anthraxx@archlinux.org>
When svn ls fails due to network timeouts, this currently results in
archrelease deleting all files, then committing this as the changeset.
This causes data loss...
With bash 4.4 and using wait $! we can get return the return code of the
last backgrounded command -- which process substitution qualifies as.
Key off of this to make sure that `svn ls` actually succeeded.
Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Levente Polyak <anthraxx@archlinux.org>
When mixing and matching different repos and architectures not present
in mainline archlinux, it is sometimes desirable to set up differing
presets with more granularity than devtools currently allows.
One example of this is when building for architectures that are only
supported by another project -- in order to coexist on a mainline
archlinux host, a different mirrorlist needs to be used.
Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Levente Polyak <anthraxx@archlinux.org>
Packages should never be getting downgraded... unless a package is
pulled from testing, e.g. for example if gcc9 totally breaks the linux
kernel. In such cases, the master repo says there is a downgrade, so
we'd better go with that. Basically, ensure that packages match the repo
they are being built against. Consistency at all costs!
Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Levente Polyak <anthraxx@archlinux.org>
noconfirm is wrong here, as we don't want to accept the default answer
-- we want to install the new package, even if it conflicts and provides
an existing one. After all, we explicitly asked for it.
Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Levente Polyak <anthraxx@archlinux.org>
gpg-agent is really annoying and leaves useless copies of itself around.
Using unshare ensures that all such processes are killed as soon as the
main gpg process dies.
Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Levente Polyak <anthraxx@archlinux.org>
If the find command cannot descend into a directory in order to search
for a PKGBUILD, it is likely a "$pkgdir" which makepkg sets as
unreadable. As far as finddeps is concerned, this error message is not
needed.
Also convert to using null-delimited paths on general principle to
prevent read from splitting on odd paths.
Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Levente Polyak <anthraxx@archlinux.org>
And pass them on to download_sources outside the chroot.
Fixes FS#35652
Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Levente Polyak <anthraxx@archlinux.org>
In commit bd826752c9, support for short
options was added to the heuristic for --noextract, but in the process,
we changed to loop over the set of user options plus the builtin
defaults for inside the chroot. This was wrong, as we only care about
the user options -- moreover, it prevents us from adding verifysource
support *outside* the chroot, for options that are also chroot options,
like --holdver.
Also remove uselessly duplicated line.
Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Levente Polyak <anthraxx@archlinux.org>
If a user umask is restrictive, a chroot may be created as root without the
ability for the user to read it, which then causes makepkg
--verifysource to fail.
Do not set this in lib/common.sh, where it would apply to all scripts,
as we do not want to override the user's policy for things like $SRCDEST
files, svn checkouts, etc.
Fixes FS#47625
Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Levente Polyak <anthraxx@archlinux.org>
When parsing paths to automatically make available to the container, the
":" is used internally by systemd-nspawn to signify destinations in the
container. Replace automatically with "\:" for the mounts that we set
up, in order to safely handle a working directory etc. that contains
this character.
For bind options exposed to the user, it is assumed the user takes care
of passing systemd-nspawn compatible paths themselves.
Fixes FS#60845
Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Levente Polyak <anthraxx@archlinux.org>
This allows comparing the currently built set of packages against
targets named by filename, url, or pkgname. One example use is to
compare a package against a different version that was never in the
repos; another example use is to compare a *-git package against the
non-git version.
Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Levente Polyak <anthraxx@archlinux.org>
Using the literal strings "true" and "false" is inaccurate and may
result in uncertainty of whether it is set when doing string comparison,
or simply rely on the shell implementation of treating the string as a
command builtin, then executing the value as a shell command. Emulate
makepkg, which makes heavy use of shell arithmetic for this purpose.
Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Levente Polyak <anthraxx@archlinux.org>
This reverts (the bulk of) commit 2fd5931a8c.
Reducing globals makes little sense in in a oneshot bash script, but
reduces code clarity and in fact resulted in bugs because even the
commit author couldn't keep track of the script state.
An exit was changed to a return, even though that made no sense outside
of a function, and has been duly returned to being an exit. This was
never tested and later papered over by wrapping the entire script in a
main() function and then calling the function for hysterical raisins.
The functiony nature of sync_chroot/delete_chroot is preserved, as those
functions demonstrate meaningfully standalone functionality -- who
knows? we may want to reuse this. Everything else is tightly bound to
the internal logic of makechrootpkg.
Completely separate functionality that was silently implemented in the
original commit is also preserved:
- declare a couple of variables as locals
- move the abort-on-no-PKGBUILD outside the install_packages function
Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Levente Polyak <anthraxx@archlinux.org>
This reverts commit 49088b0860.
The fundamental intention was flawed and broken, it caused annoying
issues and regressions, and the self-avowed sole purpose of the change
was so that a downstream project could *post-modify the script and
source it as a library*.
That is not okay. You don't wrap non-factorable code in a function
called main() and call it a library. The only possible use for this is
to treat makechrootpkg *internals* as a library, which is not supported.
Downstream projects that wish to use the functionality of makechrootpkg
should treat makepkg as a command with a public API in the form of
command line options. That is kind of how commands of all kinds work,
since forever. That is how all users of makechrootpkg *except for
parabola* use it.
Arguments that "it saves us the cost of fork+exec to bash" are simply
invalid.
Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Levente Polyak <anthraxx@archlinux.org>
Programs can freely define the value of argv0 and thus it means nothing.
Instead, use the bash-specific variable explicitly designed to safely
and accurately reference the name of the currently sourced file.
This also fixes the case where simple debugging mechanisms like using
"bash -x foo" tried to treat "foo" as the unqualified $0 and therefore
broke horribly due to lack of pathnames.
Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Levente Polyak <anthraxx@archlinux.org>
This reverts commit 578a62f1e0.
mkarchroot is run as root (via check_root if needed) so the environment
should already be clean. If not, the user has broken their root
environment, and we cannot support this. It's unclear what environment
settings may or may not be messing with anything, ever, but the original
bug report happened on Parabola who perform extensive patching to
"libretools" such that the code no longer resembles devtools at all.
It's therefore likely any such bug is parabola specific, but we will
never know since the original commit message states that they don't know
why they do it either.
Parsing the user's entire exported environment via both sed and grep is
overkill for a non-bug, especially when it doesn't work for variables
declared -rx and doesn't work for things like:
export fooled_you=$'wow such hax\ndeclare -x http_proxy=lol'
Also if done properly this would rely on compgen -e to print all
exported shell variables. Or even better, loop through /proc/$$/environ
which is both null-delimited and easily parsed with the read builtin and
[[ ]]
Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Levente Polyak <anthraxx@archlinux.org>
Previously, arch-nspawn was using the hosts' pacman cache in
the chroot even when the chroot was set up with a different
cache by mkarchroot, unless specified with the -c flag.
Problem is that makechrootpkg passes no -C, -M nor -c flags
to arch-nspawn, so all values must be obtained from the
working directory.
This change take the cache directories from the pacman.conf
specified with the -C option unless the -c option was given
(as is the case when the chroot is set up with mkarchroot),
and, when neither -C nor -c is given (as is the case when
invoked by makechrootpkg), the cache directory is taken
from the pacman.conf in the working directory.
This wasn't such an issue when i686 was mainline, however,
which building packages in a chroot against archlinux32 on
an x86_64 platform, the cache of the host should _never_ be
used.
Rebased by eschwartz on top of cachedir reworking.
Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Levente Polyak <anthraxx@archlinux.org>
It's important to ensure the guest has up to date data because updating
a chroot after quite some time can potentially rely on updated
archlinux-keyring, something which the host machine either kept up to
date on or manually fixed, but it kills automation to mess around with
chroot configs like that. Alternatively, signed packages added with -I
need to work, and we assume the host is configured to accept these.
That is *not* a good reason to completely nuke whatever is in the guest,
though. A guest might have been manually configured to accept keys which
aren't accepted by the host; one example of this happening in practice,
is archlinux32 when building 32-bit packages from an archlinux host.
The right solution is to append to, rather thna overwrite, the chrooted
guest's pacman keyring.
To do this, we will use gpg's native facility to dump the keyring from
one GNUPGHOME and import it into another. We'd use pacman-key's --import
option directly, but this doesn't support passing custom options like
--import-options import-local-sigs
Finally use pacman-key's native facility to import the trust status from
the host.
While we are at it, fix a bug where we didn't respect the host's
pacman.conf settings for the GpgDir. While it isn't wildly likely a user
will choose to customize this, it is a valid and supported use case and
we must think about this ourselves.
Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Levente Polyak <anthraxx@archlinux.org>
In commit d9b7350448, a line was deleted
that had a shellcheck ignore marker, but the marker itself was left in
(and had nothing to do). So, remove it.
Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Levente Polyak <anthraxx@archlinux.org>
It's incorrect to make pacman completely useless inside the chroot by
starting off with no pacman keyring. Assuming that the only consumers of
a new chroot will be arch-nspawn (which copies over the hostconf) is
bad design, and furthermore makes it impossible to fix other issues in
arch-nspawn itself.
Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Levente Polyak <anthraxx@archlinux.org>
Since makepkg.conf is a bash-compatible configuration file, it must be
sourced.
Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Levente Polyak <anthraxx@archlinux.org>
In commit 46c4def073, we added support for
nonstandard PREFIX installations, but DESTDIR was and is never supposed
to be a part of that. While DESTDIR isn't terribly likely to be used
during `make all` invocations, that's no reason to break horribly if it
is used for some reason.
Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Levente Polyak <anthraxx@archlinux.org>
And while we're at it, make this more consistent. Currently we
unnecessarily support only one -c /path/to/cachedir option.
This requires slightly more thorough handling in mkarchroot to ensure
all custom cachedirs are passed on to arch-nspawn. Rework
to simply forward all arguments to arch-nspawn (minus final arguments
used for pacman -Sy packagelist).
Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Levente Polyak <anthraxx@archlinux.org>
This tool is useful for running makechrootpkg on a remote build server,
and is by default hooked up to send a PKGBUILD and initiate a build on
our shiny new build server "dragon".
Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
This reverts commit 6d1992909c.
It has never worked. In commit c86823a2d4
it was noted that it compared the device numbers for [[ $1 = $1 ]] which
was a useless check and always returned true, for *any* btrfs
filesystem. Now that the function is corrected to compare [[ $1 = $2 ]]
the check is still useless, but this time because it always returns
false -- btrfs subvolumes on the same filesystem do *not* share device
numbers.
So let's go back to the original working implementation that only
matters in terms of makechrootpkg, and just checks if makechrootpkg's
root working directory is btrfs (in which case we know it will be a
subvolume because mkarchroot will create it that way).
This restores our special support for the btrfs filesystem.
Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Eli Schwartz
Daniel M. Capella
Levente Polyak
jelle van der Waa
Brian Bidulock
Eli Schwartz via arch-projects