Commit Graph

30 Commits

Author SHA1 Message Date
Martchus 68d67f543f Refactor to optimize deserialization of base-data
* Break backwards compatibility
* Allow to deserialize only base-data of packages and build
  actions to potentially speed up showing tables
* Speed up package search in many cases by only deserializing
  base-data (unless details are actually wanted)
2022-05-31 19:04:57 +02:00
Martchus ae44624989 Rebuild package DBs as well when fixing DBs 2022-05-15 00:32:46 +02:00
Martchus 26caa78956 Reduce memory usage when loading packages
* Avoid creation of map with all archive contents; instead parse packages
  while walking though the archive
* Avoid instantiation of std::string in come cases (using std::string_view)
* Reuse libarchive's archive entry when walking though archive
* Use visitor-based database parser in all places to avoid intermediate big
  array with all package objects
2022-04-20 22:47:10 +02:00
Martchus 1f3f0b0df1 Avoid high memory usage when reloading database
Especially when enabling files DBs it is quite problematic to store all
package objects of a database within one big array. This change avoids the
array and instead adds the packages directly. The disadvantage is that
clearing the database isn't as simple anymore.
2022-04-19 00:23:46 +02:00
Martchus f5394db6b1 Avoid crash when searching for empty dependency/library 2022-03-20 17:21:41 +01:00
Martchus 11f1849fde Keep local pkg paths deduced from pacman config
It makes the setup easier and shouldn't hurt otherwise.
2022-03-15 01:07:36 +01:00
Martchus 6699bfa49e lmdb: Avoid storing empty keys possibly leading to `MDB_BAD_VALSIZE`
This might prevent errors like:

```
Unable to parse retrieved database file for "community-staging@x86_64": Getting data: MDB_BAD_VALSIZE: Unsupported size of key/DB name/data, or wrong DUPFIXED size
```
2022-03-10 22:45:11 +01:00
Martchus 7ade757c8d Avoid locking whole config when updating DBs
* Only lock the config for writing the reloading the config file
* Make sure all write operations to the database acquire an "update mutex"
  to ensure only one write operation happens at a time
* Do *not* acquire any additional locks when reading from a database as it
  should be safe to do so even when a write operation happens because
    * LMDB read and write transactions can happen at the same time
    * The package cache has its own mutex anyways
    * Write ops to the package cache try to lock the "update mutex" to
      prevent writing "old" data to the cache during updates
* Make "lastUpdate" atomic to avoid locking the config when accessing it
2022-03-05 18:55:52 +01:00
Martchus cbfdf2236e Skip DLLs provided by Windows when checking for unresolvable DLLs 2022-03-03 22:40:48 +01:00
Martchus c2e7f40767 Fix typos 2022-03-01 00:43:06 +01:00
Martchus bdbfd42e5e Avoid package object being reused after 3d7310af3 2022-02-20 19:50:44 +01:00
Martchus a7de520549 Use lazy-deserialization and package cache when searching in package name 2022-02-19 00:11:59 +01:00
Martchus 3d7310af30 lmdb: Get `std::shared_ptr` directly from iterator 2022-02-18 19:23:00 +01:00
Martchus 68c11be6bb Call `addDepsAndProvidesFromOtherPackage` *before* clearing existing packages 2022-02-16 19:54:21 +01:00
Martchus ff8b644221 Avoid redundant code in PackageUpdaterPrivate::submit() applying previous commit in all versions 2022-02-04 17:57:11 +01:00
Martchus e5a3b5f311 Fix accidentally removing dependency from index when already provided by existing package 2022-02-03 22:10:22 +01:00
Martchus 66f59fecb9 Make storing lib deps more efficient and allow for intermediate reads 2022-01-31 21:32:07 +01:00
Martchus 92f83fadb6 Fix/simplify move c'tor/assignment of Database 2022-01-31 20:48:02 +01:00
Martchus 0e61a726e1 Set `lastUpdate` to zero when clearing packages
Since reloading packages is now avoided clearing packages should not itself
count as update as it would prevent loading packages again.
2022-01-26 00:46:36 +01:00
Martchus f088e54c61 lmdb: Use package cache when querying packages for dependencies
* Use package cache when querying packages for dependencies or library
  names
* Avoid manual instantiations of `std::shared_ptr<Package>`
2022-01-21 20:35:43 +01:00
Martchus f73753792c lmdb: Use caching when finding package via ID 2022-01-20 23:33:02 +01:00
Martchus 231571f2d1 lmdb: Generalize caching to be able to use it for other types than packages 2022-01-19 23:33:07 +01:00
Martchus 2ffa6629c8 lmdb: Use lmdb to store packages and dependency indices 2022-01-18 21:50:53 +01:00
Martchus 2ad4a39aea Handle case when a split package overrides the archs from the base 2021-08-28 00:25:09 +02:00
Martchus c10df26871 Adapt to versioning changes in reflective rapidjson 2021-07-13 00:41:25 +02:00
Martchus 4ac9e729d0 Exclude current db from order computation when detecting unresolved packages 2021-02-09 11:59:23 +01:00
Martchus 7a8601010e Fix dependency lookup in Database::replacePackages() 2021-02-08 23:54:03 +01:00
Martchus 5127ca351f Allow ignoring certain dependencies/libraries when checking for problems 2021-02-08 23:54:03 +01:00
Martchus b06a1dfcd2 Allow considering the regular package when checking for updates 2021-02-08 23:54:03 +01:00
Martchus c1554f4c87 Initial import 2021-02-08 23:53:56 +01:00