Martchus
/
arch-repo-manager
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
181 Commits 5 Branches 4 Tags 27 MiB
Commit Graph

107 Commits

Author SHA1 Message Date
Martchus bdbfd42e5e Avoid package object being reused after 3d7310af3 2022-02-20 19:50:44 +01:00
Martchus 334bb1faa9 Use io_uring for async file I/O 2022-02-20 18:44:07 +01:00
Martchus 8115721913 Fix handling error computing db order when reloading dependencies 2022-02-20 00:18:49 +01:00
Martchus 93afb3883d Write build action "output" log to a logfile like for sub-processes 2022-02-20 00:11:23 +01:00
Martchus a7de520549 Use lazy-deserialization and package cache when searching in package name 2022-02-19 00:11:59 +01:00
Martchus b3b396df5f Allow ignoring copying errors due to existing files when moving packages 2022-02-16 19:47:25 +01:00
Martchus 8949b5f3c1 Fix resetting running build actions on server startup 2022-02-16 19:17:29 +01:00
Martchus 557fd1a738 lmdb: Use lmdb to store build actions 2022-02-13 21:04:54 +01:00
Martchus 4fbe3df095 Allow enabling Boost.Asio's support for using `io_uring` 
It won't likely do much regarding file handling as code is currently still
using `boost::asio::posix::stream_descriptor` or Boost.Beast's file
handling (which uses blocking I/O). I suppose one needs to use the new file
API described here under "Asio 1.22.0 / Boost 1.78":
https://www.boost.org/doc/libs/1_78_0/doc/html/boost_asio/history.html
 2022-02-05 20:31:11 +01:00
Martchus e582d5f406 Remove useless blank line 2022-02-03 22:07:22 +01:00
Martchus 4ed645be94 Fix handling of head requests 
* Fix condition for skipping
* Open destination file only if the request is not skipped (when doing a
  head request first)
 2022-02-02 21:18:07 +01:00
Martchus 66f59fecb9 Make storing lib deps more efficient and allow for intermediate reads 2022-01-31 21:32:07 +01:00
Martchus 45922b47ec lmdb: Allow configuring cache limit 2022-01-26 00:41:53 +01:00
Martchus d4d187463a Fix problems with global locks 
* Aquire the mutex before setting `m_exclusivelyOwned = false`; otherwise
  it might be set after `lock()` has checked it but before `lock()` has
  entered `wait()` leading to a deadlock
* Check state again after `wait()` because it may also be unblocked
  "spuriously" so it isn't guaranteed that the state will have actually
  changed after unblocking
 2022-01-25 22:49:20 +01:00
Martchus 218dfecf56 Avoid loading databases when modification date is <= last update 
* Do HTTP head request first when loading database from mirror to avoid
  downloading the full database all the time
* Use the last modification date of the local database file because with
  the persistent storage even local database reloads became a bit expensive
 2022-01-25 00:13:10 +01:00
Martchus 300f74fb10 lmdb: Open LMDB file after restoring cache so DBs added from cache have the storage initialized 2022-01-25 00:01:59 +01:00
Martchus 1c820fc2f6 lmdb: Fix storage initialization when config file cannot be loaded 2022-01-23 19:47:50 +01:00
Martchus f088e54c61 lmdb: Use package cache when querying packages for dependencies 
* Use package cache when querying packages for dependencies or library
  names
* Avoid manual instantiations of `std::shared_ptr<Package>`
 2022-01-21 20:35:43 +01:00
Martchus e44cbf94e5 lmdb: Enable versioning for objects we'd like to store in the DB 2022-01-18 22:48:34 +01:00
Martchus 2ffa6629c8 lmdb: Use lmdb to store packages and dependency indices 2022-01-18 21:50:53 +01:00
Martchus 0a2c41ee0c Fix shadowing warning 2021-12-07 22:48:22 +01:00
Martchus 567c3aaf56 Support "trunk" subdirectory for using official svntogit repos as source 2021-11-16 15:00:19 +01:00
Martchus 0f7718c934 Locate PKGBUILDs locally also without variant dir 
This makes it possible to checkout
https://github.com/archlinux/svntogit-packages.git and
https://github.com/archlinux/svntogit-community.git locally to build
official packages.
 2021-11-14 23:37:33 +01:00
Martchus 6f652922cd Add lookup for repo-specific config file path 
This could be useful to create another repository with packages that are
built using different compile flags, e.g. optimized for certain processors.
 2021-11-14 22:59:46 +01:00
Martchus 2adc80e5fc Override existing symlinks when adding signatures to repo 2021-11-03 16:32:38 +01:00
Martchus 893a8473ad Override existing symlinks when adding packages to repo 2021-11-03 16:06:16 +01:00
Martchus 755e1958d3 Allow overriding existing files in repository 
When `repo-add` fails but files could be copied before refusing to override
is actually very annoying because one has to delete the files before
restarting the build task.
 2021-10-07 18:12:44 +02:00
Martchus 672f3b4e97 Fix staging check for split packages 
* Consider all split packages when skipping an affected package
 2021-09-30 22:23:54 +02:00
Martchus ace478be22 Fix tests when test working directory is on another device 
* Fallback to copying if hard link creation fails due to cross device link
 2021-09-30 22:22:22 +02:00
Martchus a93924bb3a Fix cleanup of signatures 
* Record full path for the deletion of orphaned signatures
* Test for the existence of the signature symlink itself and not the target
  because the symlink target (in the any directory) might not exist anymore
  but the symlink should be removed in any case
 2021-08-29 00:32:19 +02:00
Martchus c68202bdc1 Add test for repository cleanup 2021-08-29 00:29:26 +02:00
Martchus 1cc8a3d6db Add warnings about missing DLLs to the build action's result 2021-08-28 15:06:24 +02:00
Martchus 2ad4a39aea Handle case when a split package overrides the archs from the base 2021-08-28 00:25:09 +02:00
Martchus 95ae827c74 Fix applying relative working directory multiple times 2021-07-31 14:55:49 +02:00
Martchus 1ce736d66f Fix passing the body limit to nested `cachePackages()` calls 2021-07-25 01:25:30 +02:00
Martchus 075f539480 Avoid copying strings when splitting lock names for custom command 2021-07-25 00:41:46 +02:00
Martchus f3ec908bcc Hold lock for named lock table not while acquiring named lock 
Otherwise all other attempts to acquire named locks are blocked. It should
be ok because references are not invalidated when inserting/accessing
elements of an `std::unique_map`. When cleaning locks up elements are
erased, though. Hence an additional cleanup lock is required to prevent
acquiring a named lock which has already been cleaned up.
 2021-07-25 00:40:39 +02:00
Martchus bbc7c3a533 Ignore empty packages within the cache dir 
Those are most likely leftovers from  previous failed download attempts and
can be ignored. (Existing files are overridden.)
 2021-07-17 23:58:37 +02:00
Martchus e5f4057c61 Use BufferSearch from c++utilities 2021-07-16 16:01:23 +02:00
Martchus bc8a0549b2 Log package download limit 2021-07-14 15:27:49 +02:00
Martchus 909f7bfab0 Use default package download size limit if it is set to zero 2021-07-14 15:23:18 +02:00
Martchus e0a913b16c Apply clang-format 2021-07-13 00:41:49 +02:00
Martchus c05a581322 Increase the max. download size for packages and allow configuring it 2021-07-12 15:00:43 +02:00
Martchus a2fbba9713 Allow constructing BadRequest from std::string 2021-07-07 19:46:43 +02:00
Martchus b5ca815e4c Allow excluding packages via regex when reloading library dependencies 
This recently failed because the cuda package exceeded the max. body size.
I suppose it makes generally sense to be able to exclude packages as I
don't need that package anyways.
 2021-07-07 19:14:03 +02:00
Martchus d006c9ce4f Fix error handling when instantiating `std::regex` 2021-07-07 19:09:32 +02:00
Martchus 17c5f50440 Do not exit process if config values are invalid 
This is not a good idea when re-reading the config after startup.
 2021-07-07 19:08:47 +02:00
Martchus 49aebdd8a5 Fix typos found via `codespell --skip .git -w` 2021-07-03 19:59:57 +02:00
Martchus 8b70cc5f8e Clear added/removed provides from previous builds 2021-06-22 00:02:41 +02:00
Martchus f685fddd20 Add `.pid`-equivalent for Boost.Asio's Windows-executor 2021-06-22 00:01:55 +02:00