Commit Graph

716 Commits

Author SHA1 Message Date
Simon Frei 3d91f7c975
lib: Use counterfeiter to mock interfaces in tests (#7375) 2021-03-03 08:53:50 +01:00
Simon Frei 80fc238bec
all: Automatic/disabled folder-config when receive-encrypted (#7327) 2021-02-12 22:51:29 +01:00
Simon Frei 7e4e2f3720
lib/versioner: Improve error messages (fixes #7354) (#7357) 2021-02-12 20:30:51 +01:00
Jakob Borg 55afa625fc
cmd/syncthing: Add decrypt subcommand (#7332)
This adds the `syncthing decrypt` subcommand that is used to
(offline-)decrypt or just verify the contents of an encrypted folder.
2021-02-12 08:38:43 +01:00
Simon Frei d842197931
lib/model: Disable tests involving scrypt with -short (fixes #7344) (#7346) 2021-02-08 17:13:28 +01:00
Simon Frei c0f353c0e8
lib: Do not set ModifiedBy on meta only changes (#7345) 2021-02-08 15:30:39 +01:00
Simon Frei 11e9d575c8
lib/model: Refactor folder.scanSubdirs into smaller parts (#7321) 2021-02-08 08:40:57 +01:00
Jakob Borg 6db8dc33f2
lib/model: Correctly verify short read blocks (fixes #7333) (#7334)
An untrusted device will receive padded info for small blocks, and hence
sometimes request a larger block than actually exists on disk.
Previously we let this pass because we didn't have a hash to compare to
in that case and we ignored the EOF error based on that.

Now the untrusted device does pass an encrypted hash that we decrypt and
verify. This means we can't check for len(hash)==0 any more, but on the
other hand we do have a valid hash we can apply to the data we actually
read. If it matches then we don't need to worry about the read
supposedly being a bit short.
2021-02-05 16:07:21 +01:00
Simon Frei ffc14a77c6
all: Add configurable defaults (fixes #4224, fixes #6086) (#7131) 2021-02-04 21:10:41 +01:00
Audrius Butkevicius a7d9268e4d
lib/model: Make /browse endpoint return sane objects (#7306) 2021-02-01 09:27:34 +01:00
Simon Frei 052dc13487
lib/model: Correct pull progress for small files (fixes #7263) (#7316) 2021-01-31 23:40:15 +01:00
Simon Frei 249bcb3a01
lib/model: Optimize rename detection in scanner (#7315) 2021-01-31 21:02:42 +01:00
André Colomb e32a516b5f
lib/model: Forget pending folders no longer announced in ClusterConfig (fixes #5187) (#7205)
* lib/db: Add ExpirePendingFolders().

Use-case is to drop any no-longer-pending folders for a specific
device when parsing its ClusterConfig message where previously offered
folders are not mentioned any more.

The timestamp in ObservedFolder is stored with only second precision,
so round to seconds here as well.  This allows calling the function
within the same second of adding or updating entries.

* lib/model: Weed out pending folders when receiving ClusterConfig.

Filter the entries by timestamp, which must be newer than or equal to
the reception time of the ClusterConfig.  For just mentioned ones,
this assumption will hold as AddOrUpdatePendingFolder() updates the
timestamp.

* lib/model, gui: Notify when one or more pending folders expired.

Introduce new event type FolderOfferCancelled and use it to trigger a
complete refreshCluster() cycle.  Listing individual entries would be
much more code and probably just as much work to answer the API
request.

* lib/model: Add comment and rename ExpirePendingFolders().

* lib/events: Rename FolderOfferCancelled to ClusterPendingChanged.

* lib/model: Reuse ClusterPendingChanged event for cleanPending()

Changing the config does not necessarily mean that the
/resut/cluster/pending endpoints need to be refreshed, but only if
something was actually removed.  Detect this and indicate it through
the ClusterPendingChanged event, which is already hooked up to requery
respective endpoints within the GUI.

No more need for a separate refreshCluster() in reaction to
ConfigSaved event or calling refreshConfig().

* lib/model: Gofmt.

* lib/db: Warn instead of info log for failed removal.

* gui: Fix pending notifications not loading on GUI start.

* lib/db: Use short device ID in log message.

* lib/db: Return list of expired folder IDs after deleting them.

* lib/model: Refactor Pending...Changed events.

* lib/model: Adjust format of removed pending folders enumeration.

Use an array of objects with device / folder ID properties, matching
the other places where it's used.

* lib/db: Drop invalid entries in RemovePendingFoldersBeforeTime().

* lib/model: Gofmt.

My local gofmt did not complain here, strangely...

* gui: Handle PendingDevicesChanged event.

Even though it currently only holds one device at a time, wrap the
contents in an array under the "added" property name.

* lib/model: Fix null values in PendingFoldersChanged removed member.

* gui: Handle PendingFoldersChanged event.

* lib/model: Simplify construction of expiredPendingList.

* lib/model: Reduce code duplication in cleanPending().

Use goto and a label for the common parts of calling the DB removal
function and building the event data part.

* lib/events, gui: Mark ...Rejected events deprecated.

Extend comments explaining the conditions when the replacement event
types are emitted.

* lib/model: Wrap removed devices in array of objects as well.

* lib/db: Use iter.Value() instead of needless db.Get(iter.Key())

* lib/db: Add comment explaining RemovePendingFoldersBeforeTime().

* lib/model: Rename fields folderID and deviceID in event data.

* lib/db: Only list actually expired IDs as removed.

Skip entries where Delete() failed as well as invalid entries that got
removed automatically.

* lib/model: Gofmt
2021-01-25 10:58:10 +00:00
Simon Frei f63cdbfcfa
lib: Apply config changes sequentially (ref #5298) (#7188) 2021-01-15 15:43:34 +01:00
Simon Frei b2d82da20d
lib/model: Pull when folder leaves error state (fixes #7280) (#7281) 2021-01-14 13:29:01 +01:00
Jakob Borg 253049a4cc
lib/api, lib/model: Avoid contention on filesystem for DB status call (ref #7270) (#7271)
This splits the ignore getting to two methods, one that loads from disk
(the old one) and one that just returns whatever is already loaded (the
new one). The folder summary service which is just interested in stats
now uses the latter method. This means that it, and API calls that call
it, does not get blocked by folder I/O.
2021-01-12 16:25:21 +01:00
Jakob Borg 8f199e12b3
lib/fs, lib/model: Reduce lock contention on NewCaseFilesystem (fixes #7268) (#7269) 2021-01-12 16:22:21 +01:00
Jakob Borg 0b193b76c2
lib/config, lib/connections: Add optional connection limits (fixes #7176) (#7223)
This adds two new configuration options:

    // The number of connections at which we stop trying to connect to more
    // devices, zero meaning no limit. Does not affect incoming connections.
    ConnectionLimitEnough int

    // The maximum number of connections which we will allow in total, zero
    // meaning no limit. Affects incoming connections and prevents
    // attempting outgoing connections.
    ConnectionLimitMax int

These can be used to limit the number of concurrent connections in
various ways.
2021-01-11 15:14:44 +01:00
Jakob Borg b13b15758d
lib/connections, lib/model: Track last connection duration (ref #7223) (#7242)
This adds a statistic to track the last connection duration per device.
It isn't used for much in this PR, but it's available for #7223 to use
in deciding how to order device connection attempts (deprioritizing
devices that just dropped our connection the last time).
2021-01-05 17:45:07 +01:00
Simon Frei c48eb4241a
lib/model: Fix child-check when deleting dirs in pull (#7236) 2021-01-02 21:40:37 +01:00
Simon Frei 0f8290485e
lib/model: Handle index sender terminating due to error (fixes #7231) (#7232) 2020-12-30 09:59:11 +01:00
Simon Frei a05dc6cc47
lib/model: Cleanup redundant filesystem variables in folders (#7237) 2020-12-27 22:26:25 +01:00
Simon Frei f13e6ca631
lib/model: Remove obsolete return val from ccHandleFolders (ref #6443) (#7229) 2020-12-23 13:10:08 +01:00
Simon Frei a20a5f61f0
lib/ur: Send unreported failures on shutdown (#7164) 2020-12-22 20:17:14 +01:00
Simon Frei 78bd0341a8
all: Handle errors opening db/creating file-set (ref #5907) (#7150) 2020-12-21 12:59:22 +01:00
Simon Frei c845e245a1
lib: Close underlying conn in protocol (fixes #7165) (#7212) 2020-12-21 11:40:51 +01:00
Simon Frei d904dfa191
lib/model: Fix flaky test and add some scanning debug (#7214) 2020-12-20 18:13:35 +01:00
Simon Frei fa40ccece1
lib: Consistently set suture logging (#7202) 2020-12-18 19:44:00 +01:00
Simon Frei 7919310dc6
lib/model: Unflake TestIgnoreDeleteUnignore (#7208) 2020-12-18 18:42:09 +01:00
André Colomb 7502997e7e
all: Store pending devices and folders in database (fixes #7178) (#6443) 2020-12-17 19:54:31 +01:00
Jakob Borg 4142a431b5 model: Actually print folder description in "Overriding" log message 2020-12-12 12:32:24 +01:00
Simon Frei 240ae0c14f
lib/model: Unflake TestRequestReceiveEncryptedLocalNoSend (#7167) 2020-11-27 20:53:03 +01:00
Simon Frei a9764fc16c
lib: Skip deleted, locally changed on recv-enc folders (fixes #7153) (#7154) 2020-11-27 11:26:36 +01:00
Simon Frei bf7d03d029
lib/model: Fix enc file size when pulling (fixes #7152) (#7155) 2020-11-25 22:57:25 +01:00
Simon Frei 54b50e3d52
lib/fs, lib/model: Cover more windowsyness sanitizing paths (fixes #7075) (#7158) 2020-11-25 22:52:46 +01:00
André Colomb e6595c1ab9
lib/model: Simplify access to Folder and Device configuration. (#7151)
Use the accessor methods with a device argument instead of looking up
in a temporary map of the same data.
2020-11-24 22:20:50 +01:00
Simon Frei 100ef10d84
lib/model: Don't send locally changed on recv-enc (fixes #7137) (#7147) 2020-11-24 21:49:45 +01:00
Simon Frei db1f20603a
lib/model: Retain index info for new folders/devs (ref #7100) (#7133) 2020-11-20 15:53:13 +01:00
Simon Frei 24af89c8e2
all: Refactor preparing configuration (#7127) 2020-11-20 14:21:54 +01:00
Simon Frei b43eccf2fe
lib/model: Never send unpaused folder without index info (#7134) 2020-11-20 14:13:50 +01:00
Simon Frei e11b309379
lib/model, lib/util: Replace IsComplete with NoRestartErr (ref #6947) (#7126) 2020-11-18 13:43:57 +01:00
Simon Frei 2d3a535ced
lib/model: Handle cluster-config before folder start (fixes #7122) (#7123) 2020-11-17 15:30:21 +01:00
Simon Frei 9524b51708
all: Implement suture v4-api (#6947) 2020-11-17 13:19:04 +01:00
Simon Frei b8051fb37e
lib/model: Prevent test deadlock (#7110) 2020-11-10 12:32:07 +01:00
Simon Frei 31559e908b
all: Add untrusted folders behind feature flag (ref #62) (#7055) 2020-11-09 15:33:32 +01:00
Simon Frei 54643e86b5
lib/model: Fix locking when resending cluster-configs (#7107) 2020-11-09 14:05:21 +01:00
Simon Frei 1f1729ba43
lib/model: Add done chan to track folder-lifetime (fixes #6664) (#7094) 2020-11-09 09:05:48 +01:00
Simon Frei d4ce0dfd84
lib/model: Send indexes for newly shared folder (fixes #7098) (#7100) 2020-11-09 08:58:46 +01:00
Simon Frei cc9ea9db89
lib/folder: Clear pull errors when nothing is needed anymore (#7093) 2020-11-06 14:22:20 +01:00
Simon Frei 1c2be84e4e
lib/model: Pass device infos as struct (fixes #7051) (#7052) 2020-10-22 13:05:31 +02:00