Martchus
c2d5e2f749
Use DB path when checking timestamps in repo cleanup
...
Otherwise it can break if files DBs are used.
2022-05-02 22:23:35 +02:00
Martchus
26caa78956
Reduce memory usage when loading packages
...
* Avoid creation of map with all archive contents; instead parse packages
while walking though the archive
* Avoid instantiation of std::string in come cases (using std::string_view)
* Reuse libarchive's archive entry when walking though archive
* Use visitor-based database parser in all places to avoid intermediate big
array with all package objects
2022-04-20 22:47:10 +02:00
Martchus
1f3f0b0df1
Avoid high memory usage when reloading database
...
Especially when enabling files DBs it is quite problematic to store all
package objects of a database within one big array. This change avoids the
array and instead adds the packages directly. The disadvantage is that
clearing the database isn't as simple anymore.
2022-04-19 00:23:46 +02:00
Martchus
08421fa719
Don't rely on local pkg dir when reloading library deps
...
Since 11f1849fde
the cache dir is also set as local pkg dir
so we cannot rely on packages really existing there.
2022-04-07 21:35:05 +02:00
Martchus
d5e2b5489d
Provide filename for artefact downloads
2022-03-15 23:22:34 +01:00
Martchus
69a81f7583
Remove unused function declaration
2022-03-15 22:27:50 +01:00
Martchus
83c7426fce
Allow specifying output path for license info
2022-03-15 22:23:34 +01:00
Martchus
13cccd3bc9
Improve creating license info
2022-03-15 01:10:15 +01:00
Martchus
f87829ba85
Save state after reloading configuration
2022-03-15 01:07:51 +01:00
Martchus
59cd593a3b
Display template name
2022-03-12 22:46:42 +01:00
Martchus
70a98ffedc
Make name of build action to execute custom command shorter
2022-03-12 21:37:59 +01:00
Martchus
55f081153e
Add command to delete invalid build actions from DB
2022-03-07 23:36:48 +01:00
Martchus
7ade757c8d
Avoid locking whole config when updating DBs
...
* Only lock the config for writing the reloading the config file
* Make sure all write operations to the database acquire an "update mutex"
to ensure only one write operation happens at a time
* Do *not* acquire any additional locks when reading from a database as it
should be safe to do so even when a write operation happens because
* LMDB read and write transactions can happen at the same time
* The package cache has its own mutex anyways
* Write ops to the package cache try to lock the "update mutex" to
prevent writing "old" data to the cache during updates
* Make "lastUpdate" atomic to avoid locking the config when accessing it
2022-03-05 18:55:52 +01:00
Martchus
9b6be51dd3
Allow reloading library dependencies of specific packages
2022-03-04 21:51:25 +01:00
Martchus
4293eb92bf
Ignore libs for cross-compilation when checking any packages for binaries
...
Not really generic yet (e.g. doesn't cover `aarch64-linux-gnu-glibc`) but
good enough for now.
2022-03-04 21:50:56 +01:00
Martchus
c633eea607
Add any-packages containing binaries as problem when checking for errors
2022-03-03 23:08:14 +01:00
Martchus
c2e7f40767
Fix typos
2022-03-01 00:43:06 +01:00
Martchus
2071327a71
Fix building packages using signed git repositories as source
2022-02-25 23:09:16 +01:00
Martchus
ff7f039519
Ensure problems of repos with different archs but the same name aren't mixed up
2022-02-22 22:50:51 +01:00
Martchus
382169ab6e
Fix varoius build action related problems
...
* Fix crashes in some situations
* Fix aborting reloading library dependencies
2022-02-22 22:49:40 +01:00
Martchus
5c62014b3b
Fix variable name in fallback code when BOOST_ASIO_HAS_FILE
2022-02-22 20:22:00 +01:00
Martchus
839a4b0831
Fix wrong variable name in `buildactionlivestreaming.cpp`
2022-02-22 00:05:35 +01:00
Martchus
8ea2f5fd62
Reintroduce cache file as it is still required for "lastUpdate" for DBs
...
Maybe it makes more sense to use LMDB for this as well. On the other hand
it is an easy way of keeping the remaining state persistent.
2022-02-20 20:17:42 +01:00
Martchus
9c4f0277bd
Unlock output lock early
...
The following things are read-only or guarded by other mutex.
2022-02-20 19:51:46 +01:00
Martchus
bdbfd42e5e
Avoid package object being reused after 3d7310af3
2022-02-20 19:50:44 +01:00
Martchus
334bb1faa9
Use io_uring for async file I/O
2022-02-20 18:44:07 +01:00
Martchus
8115721913
Fix handling error computing db order when reloading dependencies
2022-02-20 00:18:49 +01:00
Martchus
93afb3883d
Write build action "output" log to a logfile like for sub-processes
2022-02-20 00:11:23 +01:00
Martchus
b3b396df5f
Allow ignoring copying errors due to existing files when moving packages
2022-02-16 19:47:25 +01:00
Martchus
557fd1a738
lmdb: Use lmdb to store build actions
2022-02-13 21:04:54 +01:00
Martchus
e582d5f406
Remove useless blank line
2022-02-03 22:07:22 +01:00
Martchus
66f59fecb9
Make storing lib deps more efficient and allow for intermediate reads
2022-01-31 21:32:07 +01:00
Martchus
218dfecf56
Avoid loading databases when modification date is <= last update
...
* Do HTTP head request first when loading database from mirror to avoid
downloading the full database all the time
* Use the last modification date of the local database file because with
the persistent storage even local database reloads became a bit expensive
2022-01-25 00:13:10 +01:00
Martchus
f088e54c61
lmdb: Use package cache when querying packages for dependencies
...
* Use package cache when querying packages for dependencies or library
names
* Avoid manual instantiations of `std::shared_ptr<Package>`
2022-01-21 20:35:43 +01:00
Martchus
e44cbf94e5
lmdb: Enable versioning for objects we'd like to store in the DB
2022-01-18 22:48:34 +01:00
Martchus
2ffa6629c8
lmdb: Use lmdb to store packages and dependency indices
2022-01-18 21:50:53 +01:00
Martchus
0a2c41ee0c
Fix shadowing warning
2021-12-07 22:48:22 +01:00
Martchus
567c3aaf56
Support "trunk" subdirectory for using official svntogit repos as source
2021-11-16 15:00:19 +01:00
Martchus
0f7718c934
Locate PKGBUILDs locally also without variant dir
...
This makes it possible to checkout
https://github.com/archlinux/svntogit-packages.git and
https://github.com/archlinux/svntogit-community.git locally to build
official packages.
2021-11-14 23:37:33 +01:00
Martchus
6f652922cd
Add lookup for repo-specific config file path
...
This could be useful to create another repository with packages that are
built using different compile flags, e.g. optimized for certain processors.
2021-11-14 22:59:46 +01:00
Martchus
2adc80e5fc
Override existing symlinks when adding signatures to repo
2021-11-03 16:32:38 +01:00
Martchus
893a8473ad
Override existing symlinks when adding packages to repo
2021-11-03 16:06:16 +01:00
Martchus
755e1958d3
Allow overriding existing files in repository
...
When `repo-add` fails but files could be copied before refusing to override
is actually very annoying because one has to delete the files before
restarting the build task.
2021-10-07 18:12:44 +02:00
Martchus
672f3b4e97
Fix staging check for split packages
...
* Consider all split packages when skipping an affected package
2021-09-30 22:23:54 +02:00
Martchus
a93924bb3a
Fix cleanup of signatures
...
* Record full path for the deletion of orphaned signatures
* Test for the existence of the signature symlink itself and not the target
because the symlink target (in the any directory) might not exist anymore
but the symlink should be removed in any case
2021-08-29 00:32:19 +02:00
Martchus
1cc8a3d6db
Add warnings about missing DLLs to the build action's result
2021-08-28 15:06:24 +02:00
Martchus
2ad4a39aea
Handle case when a split package overrides the archs from the base
2021-08-28 00:25:09 +02:00
Martchus
075f539480
Avoid copying strings when splitting lock names for custom command
2021-07-25 00:41:46 +02:00
Martchus
bbc7c3a533
Ignore empty packages within the cache dir
...
Those are most likely leftovers from previous failed download attempts and
can be ignored. (Existing files are overridden.)
2021-07-17 23:58:37 +02:00
Martchus
e5f4057c61
Use BufferSearch from c++utilities
2021-07-16 16:01:23 +02:00