Commit Graph

277 Commits

Author SHA1 Message Date
André Colomb 7931af1078 lib/db: Fix comparison of pending folder timestamps (fixes #7532) (#7535) 2021-04-06 10:37:11 +02:00
Jakob Borg 3ac858b150
all: Remove miscellaneous vestigial code (#7495) 2021-03-17 22:23:12 +01:00
Jakob Borg 6e5514419d
lib/db: Fix some omitted error checks, unused variable (#7489) 2021-03-17 21:41:07 +01:00
Simon Frei 273ee09925
lib/db, lib/model: Allow needing invalid files (fixes #7474) (#7476) 2021-03-15 07:58:01 +01:00
Jakob Borg 4d979a1ce9
all: Truncate some timestamps (fixes #7457) (#7459)
This truncates times meant for API consumption to second precision,
where fractions won't typically matter or add any value. Exception to
this is timestamps on logs and events, and of course I'm not touching
things like file metadata.

I'm not 100% certain this is an exhaustive change, but it's the things I
found by grepping and following the breadcrumbs from lib/api...

I also considered general-but-ugly solutions, like having the API
serializer itself do reflection magic or even regexps on returned
objects, but decided against it because aurgh...
2021-03-12 10:35:10 +01:00
Simon Frei 310fba4c12
lib: Return error from db.FileSet.Snapshot (fixes #7419, ref #5907) (#7424) 2021-03-07 13:43:22 +01:00
Jakob Borg 55d5e03639
lib/db: Remove Badger experiment (#7413) 2021-03-01 09:16:08 +01:00
Simon Frei 80fc238bec
all: Automatic/disabled folder-config when receive-encrypted (#7327) 2021-02-12 22:51:29 +01:00
Simon Frei c0f353c0e8
lib: Do not set ModifiedBy on meta only changes (#7345) 2021-02-08 15:30:39 +01:00
Simon Frei 46bbc78e82
lib/db: Fix and improve removing entries from global (ref #6501) (#7336) 2021-02-08 08:38:41 +01:00
Simon Frei 27a34609a1
all: Failure reporting fixes (#7331) 2021-02-05 11:21:14 +01:00
Simon Frei ffc14a77c6
all: Add configurable defaults (fixes #4224, fixes #6086) (#7131) 2021-02-04 21:10:41 +01:00
Simon Frei 070bf3b776
lib/db: Report number of repaired items from checkGlobal (#7329) 2021-02-04 14:42:46 +01:00
André Colomb e32a516b5f
lib/model: Forget pending folders no longer announced in ClusterConfig (fixes #5187) (#7205)
* lib/db: Add ExpirePendingFolders().

Use-case is to drop any no-longer-pending folders for a specific
device when parsing its ClusterConfig message where previously offered
folders are not mentioned any more.

The timestamp in ObservedFolder is stored with only second precision,
so round to seconds here as well.  This allows calling the function
within the same second of adding or updating entries.

* lib/model: Weed out pending folders when receiving ClusterConfig.

Filter the entries by timestamp, which must be newer than or equal to
the reception time of the ClusterConfig.  For just mentioned ones,
this assumption will hold as AddOrUpdatePendingFolder() updates the
timestamp.

* lib/model, gui: Notify when one or more pending folders expired.

Introduce new event type FolderOfferCancelled and use it to trigger a
complete refreshCluster() cycle.  Listing individual entries would be
much more code and probably just as much work to answer the API
request.

* lib/model: Add comment and rename ExpirePendingFolders().

* lib/events: Rename FolderOfferCancelled to ClusterPendingChanged.

* lib/model: Reuse ClusterPendingChanged event for cleanPending()

Changing the config does not necessarily mean that the
/resut/cluster/pending endpoints need to be refreshed, but only if
something was actually removed.  Detect this and indicate it through
the ClusterPendingChanged event, which is already hooked up to requery
respective endpoints within the GUI.

No more need for a separate refreshCluster() in reaction to
ConfigSaved event or calling refreshConfig().

* lib/model: Gofmt.

* lib/db: Warn instead of info log for failed removal.

* gui: Fix pending notifications not loading on GUI start.

* lib/db: Use short device ID in log message.

* lib/db: Return list of expired folder IDs after deleting them.

* lib/model: Refactor Pending...Changed events.

* lib/model: Adjust format of removed pending folders enumeration.

Use an array of objects with device / folder ID properties, matching
the other places where it's used.

* lib/db: Drop invalid entries in RemovePendingFoldersBeforeTime().

* lib/model: Gofmt.

My local gofmt did not complain here, strangely...

* gui: Handle PendingDevicesChanged event.

Even though it currently only holds one device at a time, wrap the
contents in an array under the "added" property name.

* lib/model: Fix null values in PendingFoldersChanged removed member.

* gui: Handle PendingFoldersChanged event.

* lib/model: Simplify construction of expiredPendingList.

* lib/model: Reduce code duplication in cleanPending().

Use goto and a label for the common parts of calling the DB removal
function and building the event data part.

* lib/events, gui: Mark ...Rejected events deprecated.

Extend comments explaining the conditions when the replacement event
types are emitted.

* lib/model: Wrap removed devices in array of objects as well.

* lib/db: Use iter.Value() instead of needless db.Get(iter.Key())

* lib/db: Add comment explaining RemovePendingFoldersBeforeTime().

* lib/model: Rename fields folderID and deviceID in event data.

* lib/db: Only list actually expired IDs as removed.

Skip entries where Delete() failed as well as invalid entries that got
removed automatically.

* lib/model: Gofmt
2021-01-25 10:58:10 +00:00
Jakob Borg 0b193b76c2
lib/config, lib/connections: Add optional connection limits (fixes #7176) (#7223)
This adds two new configuration options:

    // The number of connections at which we stop trying to connect to more
    // devices, zero meaning no limit. Does not affect incoming connections.
    ConnectionLimitEnough int

    // The maximum number of connections which we will allow in total, zero
    // meaning no limit. Affects incoming connections and prevents
    // attempting outgoing connections.
    ConnectionLimitMax int

These can be used to limit the number of concurrent connections in
various ways.
2021-01-11 15:14:44 +01:00
Simon Frei a20a5f61f0
lib/ur: Send unreported failures on shutdown (#7164) 2020-12-22 20:17:14 +01:00
Simon Frei a744dee94c
lib/fs: Correct wrapping order for meaningful log-caller (#7209) 2020-12-21 13:01:34 +01:00
Simon Frei 78bd0341a8
all: Handle errors opening db/creating file-set (ref #5907) (#7150) 2020-12-21 12:59:22 +01:00
Simon Frei 4a787986cd
lib/db: Prevent IndexID creation race (#7211) 2020-12-21 11:32:59 +01:00
Simon Frei bd0c9913cf
lib/db: Remove index ids when dropping folder (#7200) 2020-12-21 11:10:59 +01:00
Simon Frei fa40ccece1
lib: Consistently set suture logging (#7202) 2020-12-18 19:44:00 +01:00
André Colomb 7502997e7e
all: Store pending devices and folders in database (fixes #7178) (#6443) 2020-12-17 19:54:31 +01:00
Simon Frei a9764fc16c
lib: Skip deleted, locally changed on recv-enc folders (fixes #7153) (#7154) 2020-11-27 11:26:36 +01:00
Simon Frei 3169212046
lib/db: Do not reset index-id when dropping device (ref #7135) (#7156)
This reverts commit 641b7aee38.
2020-11-25 22:54:05 +01:00
Simon Frei 8ebd893349
lib/db: Store versions for last successful db migration (#7140) 2020-11-23 18:31:32 +01:00
Simon Frei 641b7aee38
lib/db: Reset index-id when dropping device (#7135) 2020-11-20 14:17:09 +01:00
Simon Frei 9524b51708
all: Implement suture v4-api (#6947) 2020-11-17 13:19:04 +01:00
Simon Frei 1d3b9876f6
lib/api, lib/db: Add file debug endpoint (#7095) 2020-11-10 09:24:45 +01:00
Simon Frei 31559e908b
all: Add untrusted folders behind feature flag (ref #62) (#7055) 2020-11-09 15:33:32 +01:00
Jakob Borg 7892547873
lib: Remove USE_BADGER experiment (#7089)
This removes the switch for using a Badger database, because it has bugs
that it seems there is no interest in fixing, and no actual bug tracker
to track them in.

It retains the actual implementation for the sole purpose of being able
to do the conversion back to LevelDB if anyone is actually running with
USE_BADGER. At some point in a couple of versions we can remove the
implementation as well.
2020-11-03 09:10:35 +01:00
Simon Frei 2ba3be5e4d
lib/db: Add mechanism to repair db without schema update (ref #7044) (#7047) 2020-10-21 14:21:09 +02:00
Simon Frei 27c91c57d5
lib/db: Remove need for the right dev removing globals (fixes #7036) (#7044) 2020-10-21 08:26:10 +02:00
Simon Frei 01a7ef3b0f
lib/db: Undo adding user info to panic msgs (ref #7029) (#7040) 2020-10-19 08:40:37 +02:00
Simon Frei 23c935b05a
lib/db: Ignore not found on delete in recalcGlobal (ref #7026) (#7041) 2020-10-19 08:28:53 +02:00
Simon Frei d828adb648
cmd/stcrashreceiver, lib/db: Improve panic message handling (#7029) 2020-10-08 17:37:45 +02:00
Simon Frei 14ae330eff
lib/db: Improve error handling checking db (ref #7026) (#7027) 2020-10-06 20:14:09 +02:00
Audrius Butkevicius e027175446
all: Move remaining protos to use the vanity plugin (#7009) 2020-10-02 08:07:05 +02:00
Simon Frei 08bebbe59b
lib/db, lib/syncthing: Don't repair DB on upgrade, but on error (fixes #6917) (#6971) 2020-09-10 10:54:41 +02:00
Simon Frei ecc24428ac
lib/model: Prevent duplicate need count on recalculating metadata (#6964) 2020-09-07 20:19:03 +02:00
Jakob Borg 674fca3868
lib/db, lib/protocol: Never need empty-version entries (fixes #6961) (#6962)
Avoid havoc when discovering locally-deleted-upgraded files during repair / need calculation...

Co-authored-by: Simon Frei <freisim93@gmail.com>
2020-09-07 20:18:25 +02:00
Jakob Borg 415adfbae6
lib/db: Don't hang on double iterator release with Badger (#6960)
Since iterators must be released before committing or discarding a
transaction we have the pattern of both deferring a release plus doing
it manually. But we can't release twice because we track this with a
WaitGroup that will panic when there are more Done()s than Add()s. This
just adds a boolean to let an iterator keep track.
2020-09-07 10:31:33 +02:00
Simon Frei 52d1681fe6
lib/db: Copy returned, old FileVersion (#6952) 2020-09-04 14:13:38 +02:00
Simon Frei 7b821d2550
lib/db: Add local need check&repair (#6950) 2020-09-04 14:01:46 +02:00
Simon Frei 0e3e0a7c9e
cmd/stindex, lib/db: Use same condition to check for needed files (#6954) 2020-09-04 13:59:04 +02:00
Jakob Borg 71bfad0bc6 cmd/stindex: Add dumping folder meta 2020-09-04 11:39:21 +02:00
Jakob Borg b7986801cd cmd/stindex: Print more hashes and versions 2020-09-04 09:10:33 +02:00
Audrius Butkevicius d507d932b8
all: Use protobuf to generate config structs (fixes #6734) (#6900) 2020-08-25 08:11:14 +02:00
Simon Frei cc1f6e4d4a
lib/db, lib/model: Cover exec-paths with debug logging (#6918) 2020-08-20 16:11:20 +02:00
Audrius Butkevicius cbbc262161
lib/db: Dont recurse on flush (fixes #6905) (#6906)
Or else we crash.

Co-authored-by: Jakob Borg <jakob@kastelo.net>
Co-authored-by: Simon Frei <freisim93@gmail.com>
2020-08-19 13:13:44 +02:00
Simon Frei 8f5215878b
lib/db: Don't put truncated files (ref #6855, ref #6501) (#6888) 2020-08-18 09:20:12 +02:00