This attempts to recreate a package that was probably created using
makechrootpkg, and see if it conforms to the
https://reproducible-builds.org/ specification.
Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
In commit 40a90e2cab we tried to protect
against system umasks resulting in unreadable chroots. However, we tried
to do this in a targeted manner due to not wanting to fiddle with
permissions for user-owned files. Unfortuantely, mkdir -p -m755 does not
actually work that way -- the parent directory is created with broken
permissions. We need umask.
Run umask and mkdir in a subshell to prevent leakage.
Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
In commit 75d23eec94 we moved to include
commitpkg arguments as the first line of the svn commit message, but we
simply dumped the result after the version number without separating the
two, increasing the cognitive burden of parsing the rationale. Since the
whole point of the change was to make it easier to see what happened
when using git log --oneline (reducing the cognitive burden of parsing
'pretty' output with author/date info), it makes sense to also delineate
the reason correctly.
Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Instead of comparing exact mirror urls to see if they are in
host_mirrors in order to "skip" the official mirrors
Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Right now there is a bug in makepkg that leaves back an empty src
directory if SRCDEST is set. This is purely cosmetic, but lets just
politely try to rmdir it and fail silently if its empty or non-existent.
Signed-off-by: Levente Polyak <anthraxx@archlinux.org>
It may be not enough to just listen on EXIT depending on the shell used
so lets make sure we clean up SRCPKGDEST by listening to more sigs.
Signed-off-by: Levente Polyak <anthraxx@archlinux.org>
- drop homebrew function in makechrootpkg
- use better mock to find invoking user's $HOME
- make offload-build respect makepkg.conf to determine where to sync
files, matching the behavior of makechrootpkg
Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Levente Polyak <anthraxx@archlinux.org>
consolidate logic flows in the same area for parsing and building
arrays. Don't bother having a special function just to build the
mount_args array, since we now use the same handling for adding any
cachedir (including host mirrors) to the mount arguments, this becomes a
trivial for loop -- and it really did not need to be delayed until after
the sanity check, anyway.
Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Levente Polyak <anthraxx@archlinux.org>
In commit 27ff286ee7, we moved from
sourcing the primary cachedir via /etc/pacman.conf, to using the
pacman.conf in the workdir. One unanticipated side effect of this was
breaking the special host mirrors magic we used to turn a host mirror
into a cachedir. It was still processed as a server, but we relied on it
being in the host's cachedirs in order to be persisted, and this no
longer occurred.
Solve this by explicitly adding each host mirror root as a cachedir.
Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Levente Polyak <anthraxx@archlinux.org>
Transform sogrep into an in-prog so we can benefit from the m4 macro
to specify valid repos in a single place of truth.
Signed-off-by: Levente Polyak <anthraxx@archlinux.org>
Lets prefer the explicit variant of gpg --verify by providing both, the
signature and the data file as parameters.
For the unlikely case there is a matching signature file already present
that was created outside of the toolchain and has an embedded signature
with data, we at least could detect it early with this check.
Signed-off-by: Levente Polyak <anthraxx@archlinux.org>
Several cases showed that we release packages that were built with
different PKGBUILDs than the one commited to the source tree. This is
bad for obvious reasons plus sploils reproducible builds.
We, under no circumstances, want to allow using commitpkg to publish and
release a packages whose PKGBUILD doesn't match the one to be commited.
Signed-off-by: Levente Polyak <anthraxx@archlinux.org>
The unknown packager check didn't worked so far as the wrongly ordered
call to find_cached_package lead to the enclosing block never being
executed.
Signed-off-by: Levente Polyak <anthraxx@archlinux.org>
Sometimes its desired to be explicitly made aware of differences
reporter by checkpkg via printing a warning instead of a regular
message.
Automatically use --warn for makechrootpkg builds so packagers are made
visibly aware of a soname bump by simply looking out for colors
indicating non success messages.
Signed-off-by: Levente Polyak <anthraxx@archlinux.org>
In some cases, like default makechrootpkg execution, the temporary
directory used to assemble the differences is not required. Add an
option to checkpkg that allows to get rid of that directory after
run and call it automatically like that in makechrootpkg.
Signed-off-by: Levente Polyak <anthraxx@archlinux.org>
Cache previous versions required for checkpkg via pacman to avoid
multiple downloads when running multiple times.
In case we can't download the packages, like while building out of repo
packages, print a warning instead of running checkpkg
Signed-off-by: Levente Polyak <anthraxx@archlinux.org>
This reverts commit be44b9cde1.
This was a nice idea in theory, because it means that we can catch
conflicting files before releasing them into the repos. In practice,
there were unanticipated side effects: single-package installs which
conflict against their own makedepends cannot be installed either.
Examples include:
- kernel modules which makedepend on their dkms equivalent
- jack2, which makedepends/optdepends on portaudio, which requires
jack... but jack2 is a drop-in provides/conflicts jack.
We cannot reliably detect when makepkg --install will error out because
of dependency conflicts vs. packages which are simply broken. So, back
out this change for now.
Revisit this once pacutils has a new release, because it will add the
option --resolve-conflicts=all, allowing for much better scripted
responses to "foo conflicts with bar, remove bar? [y/N]" than simply
"--noconfirm and fail".
Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Levente Polyak <anthraxx@archlinux.org>
We previously whitelisted this return code because split packages can
frequently conflict each other, so makepkg -i is *expected* to fail in
such a case. However, there is no good reason to let this succeed if the
pkgbase only builds one pkgname -- that will always be a severe issue.
Add a check for how many split
Signed-off-by: Levente Polyak <anthraxx@archlinux.org>
The -U option was initially introduced in commit
cda9cf436b in order to enable running
makechrootpkg as root, delegating to another, manually selected, user to
perform various non-root tasks (given that makepkg was modified to throw
fatal errors when run as root without the option of --asroot to disable
that). However, it was only ever implemented for the --verifysource
option outside of the chroot, and the builduser inside the chroot is
created with the same uid as the makechrootpkg invoker. It needs to run
as the same uid, because it needs rw access to $startdir and $SRCDEST!
Additionally this lets the invoking user more easily inspect the build
directory in case of problems...
The correct solution for this is to properly implement the initial
intention of the -U option, and make it override the autodetection of
the "invoking user" which is normally done by inspecting $SUDO_USER.
This is then used as the single source of truth for "who am I pretending
to be".
Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Levente Polyak <anthraxx@archlinux.org>
When svn ls fails due to network timeouts, this currently results in
archrelease deleting all files, then committing this as the changeset.
This causes data loss...
With bash 4.4 and using wait $! we can get return the return code of the
last backgrounded command -- which process substitution qualifies as.
Key off of this to make sure that `svn ls` actually succeeded.
Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Levente Polyak <anthraxx@archlinux.org>
When mixing and matching different repos and architectures not present
in mainline archlinux, it is sometimes desirable to set up differing
presets with more granularity than devtools currently allows.
One example of this is when building for architectures that are only
supported by another project -- in order to coexist on a mainline
archlinux host, a different mirrorlist needs to be used.
Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Levente Polyak <anthraxx@archlinux.org>
Packages should never be getting downgraded... unless a package is
pulled from testing, e.g. for example if gcc9 totally breaks the linux
kernel. In such cases, the master repo says there is a downgrade, so
we'd better go with that. Basically, ensure that packages match the repo
they are being built against. Consistency at all costs!
Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Levente Polyak <anthraxx@archlinux.org>
noconfirm is wrong here, as we don't want to accept the default answer
-- we want to install the new package, even if it conflicts and provides
an existing one. After all, we explicitly asked for it.
Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Levente Polyak <anthraxx@archlinux.org>
gpg-agent is really annoying and leaves useless copies of itself around.
Using unshare ensures that all such processes are killed as soon as the
main gpg process dies.
Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Levente Polyak <anthraxx@archlinux.org>
If the find command cannot descend into a directory in order to search
for a PKGBUILD, it is likely a "$pkgdir" which makepkg sets as
unreadable. As far as finddeps is concerned, this error message is not
needed.
Also convert to using null-delimited paths on general principle to
prevent read from splitting on odd paths.
Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Levente Polyak <anthraxx@archlinux.org>
And pass them on to download_sources outside the chroot.
Fixes FS#35652
Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Levente Polyak <anthraxx@archlinux.org>
In commit bd826752c9, support for short
options was added to the heuristic for --noextract, but in the process,
we changed to loop over the set of user options plus the builtin
defaults for inside the chroot. This was wrong, as we only care about
the user options -- moreover, it prevents us from adding verifysource
support *outside* the chroot, for options that are also chroot options,
like --holdver.
Also remove uselessly duplicated line.
Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Levente Polyak <anthraxx@archlinux.org>
If a user umask is restrictive, a chroot may be created as root without the
ability for the user to read it, which then causes makepkg
--verifysource to fail.
Do not set this in lib/common.sh, where it would apply to all scripts,
as we do not want to override the user's policy for things like $SRCDEST
files, svn checkouts, etc.
Fixes FS#47625
Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Levente Polyak <anthraxx@archlinux.org>
When parsing paths to automatically make available to the container, the
":" is used internally by systemd-nspawn to signify destinations in the
container. Replace automatically with "\:" for the mounts that we set
up, in order to safely handle a working directory etc. that contains
this character.
For bind options exposed to the user, it is assumed the user takes care
of passing systemd-nspawn compatible paths themselves.
Fixes FS#60845
Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Levente Polyak <anthraxx@archlinux.org>
This allows comparing the currently built set of packages against
targets named by filename, url, or pkgname. One example use is to
compare a package against a different version that was never in the
repos; another example use is to compare a *-git package against the
non-git version.
Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Levente Polyak <anthraxx@archlinux.org>
Using the literal strings "true" and "false" is inaccurate and may
result in uncertainty of whether it is set when doing string comparison,
or simply rely on the shell implementation of treating the string as a
command builtin, then executing the value as a shell command. Emulate
makepkg, which makes heavy use of shell arithmetic for this purpose.
Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Levente Polyak <anthraxx@archlinux.org>
This reverts (the bulk of) commit 2fd5931a8c.
Reducing globals makes little sense in in a oneshot bash script, but
reduces code clarity and in fact resulted in bugs because even the
commit author couldn't keep track of the script state.
An exit was changed to a return, even though that made no sense outside
of a function, and has been duly returned to being an exit. This was
never tested and later papered over by wrapping the entire script in a
main() function and then calling the function for hysterical raisins.
The functiony nature of sync_chroot/delete_chroot is preserved, as those
functions demonstrate meaningfully standalone functionality -- who
knows? we may want to reuse this. Everything else is tightly bound to
the internal logic of makechrootpkg.
Completely separate functionality that was silently implemented in the
original commit is also preserved:
- declare a couple of variables as locals
- move the abort-on-no-PKGBUILD outside the install_packages function
Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Levente Polyak <anthraxx@archlinux.org>
This reverts commit 49088b0860.
The fundamental intention was flawed and broken, it caused annoying
issues and regressions, and the self-avowed sole purpose of the change
was so that a downstream project could *post-modify the script and
source it as a library*.
That is not okay. You don't wrap non-factorable code in a function
called main() and call it a library. The only possible use for this is
to treat makechrootpkg *internals* as a library, which is not supported.
Downstream projects that wish to use the functionality of makechrootpkg
should treat makepkg as a command with a public API in the form of
command line options. That is kind of how commands of all kinds work,
since forever. That is how all users of makechrootpkg *except for
parabola* use it.
Arguments that "it saves us the cost of fork+exec to bash" are simply
invalid.
Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Levente Polyak <anthraxx@archlinux.org>
Programs can freely define the value of argv0 and thus it means nothing.
Instead, use the bash-specific variable explicitly designed to safely
and accurately reference the name of the currently sourced file.
This also fixes the case where simple debugging mechanisms like using
"bash -x foo" tried to treat "foo" as the unqualified $0 and therefore
broke horribly due to lack of pathnames.
Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Levente Polyak <anthraxx@archlinux.org>
This reverts commit 578a62f1e0.
mkarchroot is run as root (via check_root if needed) so the environment
should already be clean. If not, the user has broken their root
environment, and we cannot support this. It's unclear what environment
settings may or may not be messing with anything, ever, but the original
bug report happened on Parabola who perform extensive patching to
"libretools" such that the code no longer resembles devtools at all.
It's therefore likely any such bug is parabola specific, but we will
never know since the original commit message states that they don't know
why they do it either.
Parsing the user's entire exported environment via both sed and grep is
overkill for a non-bug, especially when it doesn't work for variables
declared -rx and doesn't work for things like:
export fooled_you=$'wow such hax\ndeclare -x http_proxy=lol'
Also if done properly this would rely on compgen -e to print all
exported shell variables. Or even better, loop through /proc/$$/environ
which is both null-delimited and easily parsed with the read builtin and
[[ ]]
Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Levente Polyak <anthraxx@archlinux.org>
Ivy Foster
Eli Schwartz
Levente Polyak
Daniel M. Capella
jelle van der Waa