Commit Graph

1861 Commits

Author SHA1 Message Date
Simon Frei 58592e3ef1
lib/db: Add logging for GC (#7707) 2021-05-22 21:36:43 +02:00
Simon Frei 0126188ba7
lib/config: Set DisableTempIndexes to true on receive-encrypted (#7701) 2021-05-20 22:33:23 +02:00
Simon Frei 5bdb6798a9
all: Regenerate proto (#7696) 2021-05-19 13:30:20 +02:00
greatroar 0e52ce830a
lib/fs: Fix UnicodeLowercaseNormalized on lowercase NFD (#7692)
Co-authored-by: greatroar <@>
2021-05-17 20:43:07 +02:00
Jakob Borg 97437cad64
lib/fs: Ignore normalization differences in case insensitive lookup (fixes #7677) (#7678) 2021-05-17 12:35:03 +02:00
Simon Frei 5b90a98650
lib/model: Fix addFakeConn and other test improvements (#7684) 2021-05-16 17:23:27 +02:00
Simon Frei 93a02c677e
lib/scanner: Do not drop all not-exist-errors and debug logging (#7675) 2021-05-15 11:51:35 +02:00
Simon Frei 0d054f9b64
lib/model: Don't use empty folder cfg for index sender (fixes #7649) (#7671) 2021-05-15 11:13:39 +02:00
Audrius Butkevicius 1107f6eb5f
lib/connections: Reduce default quic redial interval (fixes #7471) (#7672)
* lib/connections: Reduce default quic redial interval (fixes #7471)

* Update quic_dial.go
2021-05-14 14:26:02 +01:00
Simon Frei c494ced21f
lib/connections: Actually remove listenerSupervisor (ref #7644) (#7663) 2021-05-11 14:35:13 +02:00
Audrius Butkevicius aca1b45e93
lib/connections: Update pfilter to pick up bugfix/oob stuff, support OOB connections (fixes #7636) (#7654) 2021-05-11 07:59:56 +02:00
Simon Frei 5cb2a10138
lib/model: Improve encryption cluster-config errors (#7658) 2021-05-11 07:55:44 +02:00
Audrius Butkevicius 411796606c
lib/connections: Correct service termination order (#7657) 2021-05-10 22:29:27 +02:00
Simon Frei 1a9b54c9fa
lib/connections: Separate listener supervisors and lower backoff time (#7644) 2021-05-10 22:26:51 +02:00
Simon Frei c7f4f15272
lib/relay, lib/svcutil: Improve service logging (fixes #7580) (#7647) 2021-05-10 22:26:25 +02:00
Simon Frei 713527facf
all: Refactor relay invitations (#7646) 2021-05-10 22:25:43 +02:00
Simon Frei 6e662dc9fc
lib/suture: Use ServeBackground to start main supervisor (#7626) 2021-05-10 16:50:45 +02:00
Audrius Butkevicius eb178caf3a
lib/connections: Add connection benchmarks, allow binding to port zero addresses (#7648)
* Add connbench

* Refactor port fixup

* More cleanup

* touch for build

Co-authored-by: Jakob Borg <jakob@kastelo.net>
2021-05-10 15:44:47 +01:00
Simon Frei 6157c766de
lib/connections: Correct comments on quic wrapper type (#7652) 2021-05-09 19:15:10 +01:00
Audrius Butkevicius 745cd4744a
lib/connections: Revert add more methods to the quic conn wrapper (#7651)
This reverts commit faf15b4567.
2021-05-09 19:43:16 +02:00
Simon Frei faf15b4567
lib/connections: Add more methods to the quic conn wrapper (#7643) 2021-05-09 12:45:08 +01:00
greatroar 3746c899b7
build: List go:generate tools in tools.go (#7599) 2021-05-08 12:52:06 +02:00
Audrius Butkevicius 87a0eecc31
lib/fs, lib/api, lib/model: Expose mtime remappings as part of /db/file (#7624)
* lib/fs, lib/api, lib/model: Expose mtime remappings as part of /db/file

* Fix wrong error returned by CLI

* Gofmt

* Better names

* Review comments

* Review comments
2021-05-03 11:28:25 +01:00
Simon Frei 75b58eb480
lib/fs: Watch attrib. changes on inotify for mod. time (#7623)
* lib/fs: Watch attrib. changes on inotify for mod. time

* fix bsds (real) and darwin (test only)
2021-05-02 16:45:44 +02:00
Simon Frei dd39556759
lib: Revert needing invalid files (fixes #7608, ref #7476) (#7609) 2021-04-29 22:01:46 +02:00
Simon Frei 1f30383866
lib/model: Remove path from enc errors and report only once (#7610) 2021-04-29 19:21:07 +02:00
Gahl Saraf 66662cd678
Trigger connection loop on config device addition (fixes #7600) (#7604)
* Trigger connection loop on config device addition (fixes #7600)

* Also check for device address equality

* Move EqualStrings from api_test to utils, and use in connections/service.go

* Make sure CommitConfiguration cannot block due on the deviceAddressesChanged channel

* Update lib/connections/service.go

Co-authored-by: Jakob Borg <jakob@kastelo.net>
2021-04-26 21:13:59 +01:00
Simon Frei 8734fa65fc
lib/model: Handle invalid needed items on send-only (ref #7476) (#7596) 2021-04-26 15:36:51 +02:00
Jakob Borg 74823e81e9
all: Deprecate TLS 1.2 on sync connections (fixes #7594) (#7598)
This makes us use TLS 1.3+ on sync connections by default. A new option
`insecureAllowOldTLSVersions` exists to allow communication with TLS
1.2-only clients (roughly Syncthing 1.2.2 and older). Even with that
option set you get a slightly simplified setup, with the cipher suite
order fixed instead of auto detected.
2021-04-26 10:04:35 +02:00
Jakob Borg e4c995a321
lib/model: Don't remove non-empty .stfolder (#7578) 2021-04-15 18:13:35 +02:00
André Colomb 130d14cec9
api: Log API authorization failures. (#7575) 2021-04-15 07:33:02 +02:00
Simon Frei 15d2dc3a4f
lib/connections: Add SyscallConn() to quic conn (fixes #7551) (#7570) 2021-04-13 12:59:58 +01:00
André Colomb f6df1a760d
lib/api: Log the remote address on login attempts (#7560)
This enables usage of the audit log to e.g. automatically block remote
addresses from connecting after repeated login failures.
2021-04-13 10:14:44 +02:00
Simon Frei f71fcd440a
all: Remove untrusted feature flag (fixes #109) (#7567)
No longer hide the web UI controls for the new untrusted/encrypted
device feature. Testing hasn't been very widespread, but there has been
some and quite a few bugs have been caught and fixed. I believe its time
to not hide it anymore, and cautiously recommend usage. E.g. mention
that the feature hasn't been widely used yet and anyone using it is an
early adopter, but drop the bit about not using it with production data.
We can maybe stress the need for backups in general and especially
using this.
2021-04-13 10:12:56 +02:00
Simon Frei 1a00ea7c6e
lib: Prevent using protocol method with native path (fixes #7557) (#7563) 2021-04-11 15:29:43 +02:00
Simon Frei ec0a66c75b
lib/db, lib/model: Refactor removing expired pending folders (#7537) 2021-04-11 15:24:08 +02:00
Jakob Borg 8e38ecdeb2
Merge pull request from GHSA-x462-89pf-6r5h 2021-04-06 08:00:00 +02:00
Simon Frei f30f9c50f8
lib/db: Handle indirection error repairing sequences (fixes #7026) (#7525) 2021-04-05 10:24:16 +02:00
Simon Frei 0dcd9794d4
lib/protocol: Deterministic encrypted version (fixes #7533) (#7538) 2021-03-31 08:59:15 +02:00
André Colomb fb2d85b9d5
lib/db: Fix comparison of pending folder timestamps (fixes #7532) (#7535) 2021-03-30 21:24:53 +02:00
Simon Frei 7f0d4f6ba8
lib/db: Don't panic debugging an inexistent file (#7534) 2021-03-30 20:06:01 +02:00
Simon Frei 0d7a77ba85
lib/model: Reset conn when enc token is missing (fixes #7198) (#7518) 2021-03-23 10:38:40 +01:00
Simon Frei 924b96856f
lib: Handle adding enc folders on an existing conn (fixes #7509) (#7510) 2021-03-22 21:50:19 +01:00
Simon Frei 6b25eb2e79
lib/ur: Prevent panic when blocksResult is nil (ref #7495) (#7515) 2021-03-22 15:07:41 +01:00
Simon Frei bc08a951f1
lib/model: Encrypted fileinfo trailer needs to be in wire format (#7505) 2021-03-21 10:34:08 +01:00
Lars Lehtonen a87c5515bd
lib/model: Error check in test (#7508) 2021-03-21 10:32:17 +01:00
Jakob Borg ebcd22b02b
lib/osutil: Fix raising max FDs on macOS (#7507)
There was a logic mistake, so the limit in question wasn't used. On my
macOS this doesn't seem to matter, the hard limit returned is 2^63-1 and
setting the soft limit to that works. However I'm assuming that's not
the case for older macOSes since it was so nicely documented, so we
should still have this working. (10240 FDs should be enough for
anybody.)
2021-03-20 16:32:36 +01:00
Audrius Butkevicius 4b02b7e6f1
lib/connections: Silence "connected to myself" messages. (#7500) 2021-03-17 23:53:20 +01:00
Jakob Borg fdd823d2cb lib/osutil: Remove unused code 2021-03-17 23:18:07 +01:00
Jakob Borg 8ef504f745
all: Simplify some method calls (#7499)
strings.Replace(a, b, c, -1) -> strings.ReplaceAll(a, b, c)

(Go 1.12) and who knows what was up with that dialQueue.Sort() thing.
2021-03-17 23:12:26 +01:00
deepsource-autofix[bot] 6c573a5762
Remove unnecessary guard around `delete` (#7496)
Co-authored-by: deepsource-autofix[bot] <62050782+deepsource-autofix[bot]@users.noreply.github.com>
2021-03-17 22:23:36 +01:00
Jakob Borg 3ac858b150
all: Remove miscellaneous vestigial code (#7495) 2021-03-17 22:23:12 +01:00
Jakob Borg f4372710bf
all: Remove crypto/md5 (#7493)
This is a mostly pointless change to make security scanners and static
analysis tools happy, as they all hate seeing md5. None of our md5 uses
were security relevant, but still. Only visible effect of this change is
that our temp file names for very long file names become slightly longer
than they were previously...
2021-03-17 22:22:49 +01:00
Jakob Borg f39477bbd5
lib/api: Missing error handling in API delete-device (#7494) 2021-03-17 22:08:44 +01:00
Jakob Borg 6e5514419d
lib/db: Fix some omitted error checks, unused variable (#7489) 2021-03-17 21:41:07 +01:00
Jakob Borg 81484699db
lib/model: Actually break puller loop on context cancel (#7492)
Current break does nothing (breaks the select).
2021-03-17 21:34:52 +01:00
deepsource-autofix[bot] 0930bccf88
cmd/ursrv, lib/scanner: Remove unnecessary slicing of slices (#7491)
Co-authored-by: deepsource-autofix[bot] <62050782+deepsource-autofix[bot]@users.noreply.github.com>
2021-03-17 21:04:36 +01:00
Jakob Borg e321bd3941
lib/*/auto: Add noassets files (#7490)
This adds a couple of dummy asset files protected by the "noassets"
build tag. The purpose is that it should be possible for, for example,
CI tools and static analysis things to compile and analyze the source
tree without our custom asset generation step. Also makes `go test -tags
noassets ./...` work without building assets first.
2021-03-17 21:03:35 +01:00
Simon Frei 4b02937862
lib/model: Add missing lock on availability func (#7487) 2021-03-17 20:46:13 +01:00
Simon Frei 3f2b584c4e
lib/model: Don't use ignore patterns for recv-enc folders (fixes #7469) (#7472) 2021-03-16 15:04:11 +01:00
Simon Frei d2d4fcc1df
lib/protocol: Improve messages when an error occurs receiving (ref #7466) (#7470) 2021-03-15 19:14:09 +01:00
Simon Frei 273ee09925
lib/db, lib/model: Allow needing invalid files (fixes #7474) (#7476) 2021-03-15 07:58:01 +01:00
Simon Frei f80ee472c2
lib/protocol: Set invalid flag on encrypted file infos (fixes #7466) (#7467) 2021-03-13 16:57:36 +01:00
Simon Frei 97a8777d03
lib/fs: Check both old and new path when renaming (fixes #7426) (#7463) 2021-03-12 21:15:50 +01:00
Simon Frei 8a4c00d82e
lib/model: Send failure report on CC encryption check error (#7460) 2021-03-12 12:21:54 +01:00
Simon Frei 31f859e909
lib/model: Return correct error in puller-iteration (ref #7424) (#7461) 2021-03-12 12:21:28 +01:00
Jakob Borg 4d979a1ce9
all: Truncate some timestamps (fixes #7457) (#7459)
This truncates times meant for API consumption to second precision,
where fractions won't typically matter or add any value. Exception to
this is timestamps on logs and events, and of course I'm not touching
things like file metadata.

I'm not 100% certain this is an exhaustive change, but it's the things I
found by grepping and following the breadcrumbs from lib/api...

I also considered general-but-ugly solutions, like having the API
serializer itself do reflection magic or even regexps on returned
objects, but decided against it because aurgh...
2021-03-12 10:35:10 +01:00
Simon Frei 4465cdf8bc
lib/api: Fix body of renamed config/restart-required endpoint (ref #7402) (#7453) 2021-03-11 15:54:05 +01:00
Simon Frei 3938b61c3f
lib/fs: Expose fs option on interface (fixes #7385, ref #7381) (#7389) 2021-03-11 15:23:56 +01:00
Jakob Borg cdef503db6
all: Make config.Wrapper an actual suture.Service (fixes #7451) (#7452) 2021-03-11 14:51:00 +01:00
Jakob Borg df08984a58
lib/api: Sanitize names used in certificates (fixes #7434) (#7435) 2021-03-11 13:15:03 +01:00
Simon Frei 9a001051d6
cmd/ursrv, lib/ur: Collect and present encryption usage (#7448) 2021-03-10 22:26:56 +00:00
greatroar c00520281b
lib/protocol: Optimize FileKey (#7440) 2021-03-07 18:44:21 +01:00
Simon Frei 310fba4c12
lib: Return error from db.FileSet.Snapshot (fixes #7419, ref #5907) (#7424) 2021-03-07 13:43:22 +01:00
Simon Frei 767e1c6f58
lib/connections: Expose SetReadBuffer on conn passed to quic (ref #7417) (#7432) 2021-03-05 08:06:37 +01:00
Simon Frei 3d91f7c975
lib: Use counterfeiter to mock interfaces in tests (#7375) 2021-03-03 08:53:50 +01:00
greatroar 56b5352f64
all: Use crypt/rand through its buffered version, but not in benchmarks (#7420) 2021-03-02 19:17:20 +01:00
Jakob Borg 55d5e03639
lib/db: Remove Badger experiment (#7413) 2021-03-01 09:16:08 +01:00
greatroar ffcaffa32f
lib/protocol: Optimize encrypted filename handling + make it more strict (#7408) 2021-02-27 08:57:12 +01:00
Simon Frei 0ffd80f380
lib/protocol: Alwasy return buffers to the pool (#7409) 2021-02-27 08:55:51 +01:00
wouter bolsterlee 25151b14e7
lib/api: Treat *.localhost as valid localhost addresses (#7412) (ref #4815)
This loosens the ‘is this localhost?’ check to include *.localhost host
names.

This allows for clearer (hence better) names to be used in browsers,
e.g. when accessing a remote syncthing instance ‘foo’ using a ssh port
forward, one can use foo.localhost to remind oneself which one is which.
💡 Without these changes, Syncthing shows a ‘Host check error’ when
pointing a browser at http://foo.localhost/, and with these changes, the
interface loads as usual.

The .localhost top level domain is a reserved top-level domain (RFC 2606):

> The ".localhost" TLD has traditionally been statically defined in
> host DNS implementations as having an A record pointing to the
> loop back IP address and is reserved for such use.  Any other use
> would conflict with widely deployed code which assumes this use.
> – https://tools.ietf.org/html/rfc2606

As Wikipedia puts it:

> This allows the use of these names for either documentation purposes
or in local testing scenarios. – https://en.wikipedia.org/wiki/.localhost

On Linux systems, systemd-resolved resolves *.localhost, on purpose:
https://www.freedesktop.org/software/systemd/man/systemd-resolved.service.html

See also #4815, #4816.
2021-02-27 08:52:49 +01:00
Simon Frei fff8805ff6
all: Fix versioning path handling (#7407) 2021-02-26 12:04:05 +01:00
Simon Frei 0bf9645f2f
lib/api: Rename config insync endpoint to restart-required (#7402) 2021-02-25 09:29:44 +01:00
tomasz1986 1ed0116147
all: Fix Microsoft documentation links in code comments (#7387) 2021-02-20 14:56:45 +01:00
Simon Frei aa1c274231
lib/fs: Consider options in case-fs caching (fixes #7371) (#7381) 2021-02-19 11:06:25 +01:00
Simon Frei 78c2844e3f
lib/scanner: Pass on errors while hashing (#7380) 2021-02-19 08:51:39 +01:00
Jakob Borg 327604719a
lib/connections: Allow QUIC with Go 1.16 (#7372) 2021-02-17 11:09:16 +01:00
Audrius Butkevicius fb078068b4
cmd/syncthing: Add cli as a subcommand (fixes #6566, fixes #4719) (#7364)
* cmd/syncthing: Add cli as a subcommand (fixes #6566, fixes #4719)

* Hijack help

* Add comment

* Revert go.mod/go.sum
2021-02-15 18:50:53 +01:00
Simon Frei 80fc238bec
all: Automatic/disabled folder-config when receive-encrypted (#7327) 2021-02-12 22:51:29 +01:00
Simon Frei 7e4e2f3720
lib/versioner: Improve error messages (fixes #7354) (#7357) 2021-02-12 20:30:51 +01:00
Jakob Borg 55afa625fc
cmd/syncthing: Add decrypt subcommand (#7332)
This adds the `syncthing decrypt` subcommand that is used to
(offline-)decrypt or just verify the contents of an encrypted folder.
2021-02-12 08:38:43 +01:00
Jakob Borg 4f20c900d0
cmd/syncthing: Refactor command line parsing (#7330) 2021-02-10 20:35:37 +01:00
Simon Frei d842197931
lib/model: Disable tests involving scrypt with -short (fixes #7344) (#7346) 2021-02-08 17:13:28 +01:00
Simon Frei c0f353c0e8
lib: Do not set ModifiedBy on meta only changes (#7345) 2021-02-08 15:30:39 +01:00
Simon Frei 11e9d575c8
lib/model: Refactor folder.scanSubdirs into smaller parts (#7321) 2021-02-08 08:40:57 +01:00
Simon Frei 46bbc78e82
lib/db: Fix and improve removing entries from global (ref #6501) (#7336) 2021-02-08 08:38:41 +01:00
Jakob Borg 6db8dc33f2
lib/model: Correctly verify short read blocks (fixes #7333) (#7334)
An untrusted device will receive padded info for small blocks, and hence
sometimes request a larger block than actually exists on disk.
Previously we let this pass because we didn't have a hash to compare to
in that case and we ignored the EOF error based on that.

Now the untrusted device does pass an encrypted hash that we decrypt and
verify. This means we can't check for len(hash)==0 any more, but on the
other hand we do have a valid hash we can apply to the data we actually
read. If it matches then we don't need to worry about the read
supposedly being a bit short.
2021-02-05 16:07:21 +01:00
Jakob Borg 194501c958
lib/api: Give the config changes some more time? (#7335) 2021-02-05 15:33:37 +01:00
Simon Frei 27a34609a1
all: Failure reporting fixes (#7331) 2021-02-05 11:21:14 +01:00
Simon Frei ffc14a77c6
all: Add configurable defaults (fixes #4224, fixes #6086) (#7131) 2021-02-04 21:10:41 +01:00