Commit Graph

83 Commits

Author SHA1 Message Date
Martchus fe80e5b970 Remove pragma to ignore warning
This seems no longer required.
2024-03-08 19:19:17 +01:00
Martchus c3ea4f42f0 Add stress test for package updater
This test passes, supposedly the package updater is *not* what's causing
sporatic problems with indexes being messed up.
2024-02-27 01:55:59 +01:00
Martchus 5d5b673b3c Improve updating package database when loading library dependencies
* Make use of `PackageUpdater` more efficient by avoiding multiple calls of
  `addDepsAndProvidesFromOtherPackage`
* Fix leaving old provides in the database by splitting the update via new
  `beginUpdate()` and `endUpdate()` functions
2023-12-21 20:51:38 +01:00
Martchus da19885d8b Ensure case of empty package name is handled in find/remove functions 2023-12-19 21:41:38 +01:00
Martchus 8deeb0a82b Remove non-forcing package updates
This was only used in tests but was otherwise not very useful as we
normally need to ensure the persistent database is updated as well.
2023-12-19 21:34:18 +01:00
Martchus f0761ed70e Apply clang-format 2023-12-19 21:32:05 +01:00
Martchus 303d0bd17b Clear entries from package cache correctly 2023-12-17 01:41:13 +01:00
Martchus 40e2fb82de Avoid unqualified use of `std::move` 2023-12-16 23:07:00 +01:00
Martchus 141d8fa3f5 Allow dumping package databases 2023-12-16 22:13:44 +01:00
Martchus ec493b3454 Remove unused includes 2023-12-09 23:28:02 +01:00
Martchus e25866c904 Apply clang-format 2023-06-12 22:18:47 +02:00
Martchus 524b16815a Do basic sanity check of a package's most important fields when checking for problems 2023-06-03 18:49:02 +02:00
Martchus 77000ac7d3 Improve logging status when rebuilding DB
Override previous "Processing package …" line to avoid huge output.
2023-06-03 17:48:36 +02:00
Martchus 5f25e6e8b7 Avoid unqualified calls to `std::move` 2023-01-22 22:30:07 +01:00
Martchus 407e3159e0 Update/remove packages from database in a whole transaction 2022-11-01 15:46:10 +01:00
Martchus 84ef92aaa2 Use one transaction when updating packages via `PackageUpdater`
This should likely be done in the other updating functions of the
`Database` class as well to avoid indexes and actual data appearing
inconsistent during the write operations.
2022-11-01 15:11:49 +01:00
Martchus 9f5474e128 Fix passing architecture when creating database
When adding a new database when reloading the config at runtime the
architecture must be passed when creating the database. Otherwise we end up
with the new database sharing the internal storage with the x86_64 database
which leads to very confusing behavior.
2022-06-19 23:06:25 +02:00
Martchus 6a633252a8 Fix search by libraries broken by recent optimizations 2022-06-09 18:51:03 +02:00
Martchus c155176098 Fix missing build date in package search
* Avoid using buildDate of PackageInfo in favor of the same field in
  PackageBase
    * PackageInfo::buildDate should be removed on the next breaking change
* Do not really use the deprecated attribute because it is still used by
  (de)serialization code until it is removed for good
2022-06-08 22:44:58 +02:00
Martchus 0095424696 Support static-compat variant when computing regular package name 2022-05-31 20:13:21 +02:00
Martchus f2902f1d08 Apply clang-format 2022-05-31 20:13:21 +02:00
Martchus ecebf9fbde Fix showing arch of packages after optimization
* Move the binary package arch truly into the base package
2022-05-31 20:13:15 +02:00
Martchus b74392c34d Speed up deserialization when serving providing packages as well 2022-05-31 19:40:12 +02:00
Martchus 68d67f543f Refactor to optimize deserialization of base-data
* Break backwards compatibility
* Allow to deserialize only base-data of packages and build
  actions to potentially speed up showing tables
* Speed up package search in many cases by only deserializing
  base-data (unless details are actually wanted)
2022-05-31 19:04:57 +02:00
Martchus ae44624989 Rebuild package DBs as well when fixing DBs 2022-05-15 00:32:46 +02:00
Martchus 142f6466ae Show resource usage in UI 2022-04-23 19:56:22 +02:00
Martchus 6965d9b40b Pass package pointer by reference when parsing database file 2022-04-20 23:14:49 +02:00
Martchus 26caa78956 Reduce memory usage when loading packages
* Avoid creation of map with all archive contents; instead parse packages
  while walking though the archive
* Avoid instantiation of std::string in come cases (using std::string_view)
* Reuse libarchive's archive entry when walking though archive
* Use visitor-based database parser in all places to avoid intermediate big
  array with all package objects
2022-04-20 22:47:10 +02:00
Martchus 1f3f0b0df1 Avoid high memory usage when reloading database
Especially when enabling files DBs it is quite problematic to store all
package objects of a database within one big array. This change avoids the
array and instead adds the packages directly. The disadvantage is that
clearing the database isn't as simple anymore.
2022-04-19 00:23:46 +02:00
Martchus f07ef9f147 Remove disabled code 2022-04-18 19:23:18 +02:00
Martchus edb014011c Make package search more efficient
* Avoid keeping list of all packages in memory by pushing each package
  directly to the JSON array
* Avoid check for database before pushing each package; do it only before
  reaching the next database
* Remove no longer required functions
2022-04-17 23:39:25 +02:00
Martchus 4fc3bf0928 Fix version comparison when epoch is only present in one version 2022-03-29 22:08:20 +02:00
Martchus f5394db6b1 Avoid crash when searching for empty dependency/library 2022-03-20 17:21:41 +01:00
Martchus 11f1849fde Keep local pkg paths deduced from pacman config
It makes the setup easier and shouldn't hurt otherwise.
2022-03-15 01:07:36 +01:00
Martchus 6699bfa49e lmdb: Avoid storing empty keys possibly leading to `MDB_BAD_VALSIZE`
This might prevent errors like:

```
Unable to parse retrieved database file for "community-staging@x86_64": Getting data: MDB_BAD_VALSIZE: Unsupported size of key/DB name/data, or wrong DUPFIXED size
```
2022-03-10 22:45:11 +01:00
Martchus 7ade757c8d Avoid locking whole config when updating DBs
* Only lock the config for writing the reloading the config file
* Make sure all write operations to the database acquire an "update mutex"
  to ensure only one write operation happens at a time
* Do *not* acquire any additional locks when reading from a database as it
  should be safe to do so even when a write operation happens because
    * LMDB read and write transactions can happen at the same time
    * The package cache has its own mutex anyways
    * Write ops to the package cache try to lock the "update mutex" to
      prevent writing "old" data to the cache during updates
* Make "lastUpdate" atomic to avoid locking the config when accessing it
2022-03-05 18:55:52 +01:00
Martchus 1f3abd5596 Distinguish Android libs from GNU/Linux libs 2022-03-04 21:48:28 +01:00
Martchus cbfdf2236e Skip DLLs provided by Windows when checking for unresolvable DLLs 2022-03-03 22:40:48 +01:00
Martchus c2e7f40767 Fix typos 2022-03-01 00:43:06 +01:00
Martchus 90ff9678fe Limit results returned by the API for better scalability 2022-02-25 00:29:43 +01:00
Martchus 9e15129d9d Ignore null-dereference warning
It is about
```
return result_type{ storageEntry.id, storageEntry.ref.relatedStorage };
```
but it isn't clear why the compiler thinks there's a problem.
2022-02-22 22:46:53 +01:00
Martchus bdbfd42e5e Avoid package object being reused after 3d7310af3 2022-02-20 19:50:44 +01:00
Martchus d11ba43bf0 lmdb: Avoid getting packages with empty name, can lead to exception 2022-02-20 19:28:44 +01:00
Martchus a7de520549 Use lazy-deserialization and package cache when searching in package name 2022-02-19 00:11:59 +01:00
Martchus 3d7310af30 lmdb: Get `std::shared_ptr` directly from iterator 2022-02-18 19:23:00 +01:00
Martchus 68c11be6bb Call `addDepsAndProvidesFromOtherPackage` *before* clearing existing packages 2022-02-16 19:54:21 +01:00
Martchus 557fd1a738 lmdb: Use lmdb to store build actions 2022-02-13 21:04:54 +01:00
Martchus ff8b644221 Avoid redundant code in PackageUpdaterPrivate::submit() applying previous commit in all versions 2022-02-04 17:57:11 +01:00
Martchus e5a3b5f311 Fix accidentally removing dependency from index when already provided by existing package 2022-02-03 22:10:22 +01:00
Martchus d1d65dac80 Use `CacheRef(storage, entry)` c'tor as it is generally preferred
Passing any string here is more likely a mistake, see previous commit
2022-02-01 22:59:09 +01:00