This makes the new default $XDG_STATE_HOME/syncthing or
~/.local/state/syncthing, while still looking in legacy locations first
for existing installs.
Note that this does not *move* existing installs, and nor should we.
Existing paths will continue to be used as-is, but the user can move the
dir into the new place if they want to use it (as they could prior to
this change as well, for that matter).
### Documentation
Needs update to the config docs about our default locations.
lib/fs: Fix conflicts on Android due to fluctuating inode change time
[1] added inode change time to file info in order to support syncing
extended attributes. However, in the case of Android, this inode change
time fluctuates, leading to unexpected conflicts even when the user has
not even touched the files on the Android device itself. Thus, in order
to prevent those conflicts from happening, do not write inode change
time on Android.
[1] 6cac308bcd
Signed-off-by: Tomasz Wilczyński <twilczynski@naver.com>
### Purpose
Deduplicated files are apparently considered 'irregular' under the hood,
this causes them to simply be ignored by Syncthing. This change is more
of a workaround than a proper fix, as the fix should probably happen in
the underlying libraries? - which may take some time. In the meanwhile,
this change should make deduplicated files be treated as regular files
and be indexed and synced as they should.
### Testing
Create some volume where deduplication is turned on (see the relevant
issue for details, including a proper description of how to reproduce
it). Prior to this change, the deduplicated files were simply ignored
(even by the indexer). After this change, the deduplicated files are
being index and synced properly.
This is motivated by the Android app:
https://github.com/syncthing/syncthing-android/pull/1982#issuecomment-1752042554
The planned fix in response to basic auth behaviour changing in #8757
was to add the `Authorization` header when opening the WebView, but it
turns out the function used only applies the header to the initial page
load, not any subsequent script loads or AJAX calls. The
`basicAuthAndSessionMiddleware` checks for no-auth exceptions before
checking the `Authorization` header, so the header has no effect on the
initial page load since the `/` path is a no-auth exception. Thus the
Android app fails to log in when opening the WebView.
This changes the order of checks in `basicAuthAndSessionMiddleware` so
that the `Authorization` header is always checked if present, and a
session cookie is set if it is valid. Only after that does the
middleware fall back to checking for no-auth exceptions.
`api_test.go` has been expanded with additional checks:
- Check that a session cookie is set whenever correct basic auth is
provided.
- Check that a session cookie is not set when basic auth is incorrect.
- Check that a session cookie is not set when authenticating with an API
token (either via `X-Api-Key` or `Authorization: Bearer`).
And an additional test case:
- Check that requests to `/` always succeed, but receive a session
cookie when correct basic auth is provided.
I have manually verified that
- The new assertions fail if the `createSession` call is removed in
`basicAuthAndSessionMiddleware`.
- The new test cases in e6e4df4d7034302b729ada6d91cff6e2b29678da fail
before the change in 0e47d37e738d4c15736c496e01cd949afb372e71 is
applied.
improve parsing of gui-address overrides
make checks for whether the gui-address is overridden consistent by
checking whether the environment variable is set and not an empty
string. the `Network()` function however checked for the inclusion of
a slash instead of the presence of any characters. If the config file's
gui address was set to a unix socket and the gui override to a tcp
address, then the function would have wrongly returned "unix".
the `URL()` function always returned the config file's gui address if a
unix socket was configured, even if an override was specified.
the `URL()` function wrongly formatted unix addresses. the http(s)
protocol was used as the sheme and the path was percent escaped. because
of the previous bug, this could only be triggered if the config file's
gui address was tcp and an unix socket override was given.
simplify the `useTLS()` function's codepath for overrides.
Co-authored-by: digital <didev@dinid.net>
I don't really understand under what circumstances, but sometimes these
calls panic with a "panic: counter cannot decrease in value" because the
value passed to Add() was negative.
In principle a connection can close while it's in progress with
starting, and then it's undefined if we wait for goroutines to exit etc.
With this change, we will wait for start to complete before starting to
stop everything.
This adds the ability to have multiple concurrent connections to a single device. This is primarily useful when the network has multiple physical links for aggregated bandwidth. A single connection will never see a higher rate than a single link can give, but multiple connections are load-balanced over multiple links.
It is also incidentally useful for older multi-core CPUs, where bandwidth could be limited by the TLS performance of a single CPU core -- using multiple connections achieves concurrency in the required crypto calculations...
Co-authored-by: Simon Frei <freisim93@gmail.com>
Co-authored-by: tomasz1986 <twilczynski@naver.com>
Co-authored-by: bt90 <btom1990@googlemail.com>
Instead of separately tracking the token.
Also changes serviceMap to have a channel version of RemoveAndWait, so
that it's possible to do the removal under a lock but wait outside of
the lock. And changed where we do that in connection close, reversing
the change that happened when I added the serviceMap in 40b3b9ad1.
By creating the http.Transport and tls.Configuration ourselves we
override some default behavior and end up with a client that speaks only
HTTP/1.1.
This adds a call to http.ConfigureTransport to do the relevant magic to
enable HTTP/2.
Also tweaks the keepalive settings to be a little kinder to the
server(s).
The allowed IPv4 ranges are the same as before. But we now also accept IPv6 addresses in the ULA range FC00::/7. These addresses don't require an interface identifier and are roughly equivalent to the IPv4 private ranges.
Typical usecases:
VPN interface IPs: Wireguard, OpenVPN, Tailscale, ...
fixed IPv6 LAN addressing while the provider assigns a dynamic prefix. e.g used by pihole
https://cs.opensource.google/go/go/+/refs/tags/go1.21.0:src/net/ip.go;l=146
* lib/versioner: Factor out DefaultPath constant.
Replace several instances where .stversions is named literally to all
use the same definition in the versioner package. Exceptions are the
packages where a cyclic dependency on versioner is impossible, or some
tests which combine the versions base path with other components.
* lib/versioner: Fix comment about trash can in simple versioner.
* lib/versioner: Fix wrong versioning type string in error message.
The error message shows the folder type instead of the versioning
type, although the correct field is used in the comparison.
Safety check added in v1.23.6 introduced bug. Bug unshares folders with untrusted devices if folder does not have an encryption password set, regardless of whether the folder is shared with the untrusted device as encrypted or not. Prevents sharing with untrusted devices in some cases where sharing would be encrypted.
Patch preserves safety check but permits sharing folders with untrusted devices if they are shared as encrypted.
Signed-off-by: kewiha <keithh@protonmail.com>
Currently, historically, we look for the `X-API-Key` header to
authenticate with an API key. There's nothing wrong with this, but in
some scenarios it's easier to produce an `Authorization` header with a
`Bearer $token` content, which is nowadays more common. This change adds
support for both, so that we will accept an API key either in our custom
header or as a bearer token.
Currently, because of devices with unset RTC clock, the 100% percentile
for Uptime on [1] is calculated since the Unix epoch which is useless as
far as usage statistics are concerned. Thus, if the Syncthing start time
is set to a past date, assume that the clock is wrong and do not even
try to report the uptime.
[1] https://data.syncthing.net
Signed-off-by: Tomasz Wilczyński <twilczynski@naver.com>
Co-authored-by: Jakob Borg <jakob@kastelo.net>
refactor: replace empty slice literal with `var`
An empty slice can be represented by `nil` or an empty slice literal. They are
functionally equivalent — their `len` and `cap` are both zero — but the `nil`
slice is the preferred style. For more information about empty slices,
see [Declaring Empty Slices](https://github.com/golang/go/wiki/CodeReviewComments#declaring-empty-slices).
Co-authored-by: deepsource-autofix[bot] <62050782+deepsource-autofix[bot]@users.noreply.github.com>
refactor: fix unused method receiver
Methods with unused receivers can be a symptom of unfinished refactoring or a bug. To keep
the same method signature, omit the receiver name or '_' as it is unused.
Co-authored-by: deepsource-autofix[bot] <62050782+deepsource-autofix[bot]@users.noreply.github.com>
refactor: unused parameter should be replaced by underscore
Unused parameters in functions or methods should be replaced with `_`
(underscore) or removed.
Co-authored-by: deepsource-autofix[bot] <62050782+deepsource-autofix[bot]@users.noreply.github.com>
This adds an environment variable STVERSIONEXTRA that, when set, gets
added to the version information in the API and GUI.
The purpose of all this is to be able to communicate something about the
bundling or packaging, through the log & GUI and the end user, to the
potential person supporting it -- i.e., us. :) A wrapper can set this
variable to indicate that Syncthing is being run via `SyncTrayzor`,
`Syncthing-macOS`, etc., and thus indicate to the end user that the GUI
they are looking at is perhaps not the only source of truth and
management for this instance.
With this change, error messages include the offending characters or
name parts. Examples:
nul.txt: name is invalid, contains Windows reserved name: "nul"
foo>bar.txt: name is invalid, contains Windows reserved character: ">"
foo \bar.txt: name is invalid, must not end in space or period on Windows
This prevents combining untrusted with introducer and auto-accept, and
also verifies that folders shared with untrusted devices have passwords
at config loading time.
Co-authored-by: Simon Frei <freisim93@gmail.com>
We usually want to ensure that our own device is present. However if the
given device ID is the empty ID, we shouldn't do that. This is a
legimate (though way too non-obvious) use-case when opening the config
without knowing/caring about the device ID.
* Platform data (ownership, xattrs, etc.) is now set correctly for newly-received folders, even if the received folder has the NoPermissions flag.
* Call setPlatformData on receivers that have ignorePerms set to true.
This fixes various test issues with Go 1.20.
- Most tests rewritten to use fakefs where possible
- Some tests that were already skipped, or dubious (invasive,
unmaintainable, unclear what they even tested) have been removed
- Some actual code rewritten to better support testing in fakefs
Co-authored-by: Eric P <eric@kastelo.net>
In the sequence of loading ignores, the error File Does Not Exist is not being considered a fatal error, since the .stignore file is allowed to not exist. However, included ignore files also tossed that same error in case those do not exist while in those cases it's considered an error and it should lead to the folder stopping. Changing the error when opening an included ignore file to something other than the regular does fix this issue, as in it now works again as described in the Documentation.
This makes the various protocol priorities configurable among the other
options. With this, it's possible to prefer QUIC over TCP for WAN
connections, for example. Both sides need to be similarly configured for
this to work properly.
The default priority order remains the same as previously (TCP, QUIC,
Relay, with LAN better than WAN).
To make this happen I made each dialer & listener more priority aware,
and moved the check for whether a connection is LAN or not into the
dialer / listener -- this is the new "lanChecker" type that's passed
around.
In the original fix in #8563 I simply forgot this. Which meant #8556
wasn't actually fixed, as the trialer size would have been 0 (default),
and thus we would have still sent the inflated size to encrypted peers.
lib/model: Fix file size inconsisency due to enc. trailer
Fixes a regression due to PR #8563, while arguable the bug was actually
introduced in a much older PR #7155, but didn't have any bad effects so
far:
We account for the encryption trailer in the db updater routine,
calculating the file-info size there. However there's no guarantee that
the file-info at this point is still the exact same as when it was
written. It was before, but isn't anymore since introducing the new
EncryptedTrailerSize field.
Fix: Adjust the size in the info at the same place where the trailer is
written, i.e. we definitely have the actual size on disk.
The layout of the request differs based on whether it comes from an
untrusted device or a trusted device with encrypted enabled. Handle
both.
Closes#8819.
Allow the watcher delay to take fractional values, effectively allowing
for much shorter delays. The minimum value is limited at 0.01, which
effectively translates to 10ms. This is required in order to guarantee
that there is still enough time to aggregate multiple single change
events.
Signed-off-by: Tomasz Wilczyński <twilczynski@naver.com>
This adds a cache to the expensive key generation operations. It's fixes
size LRU/MRU stuff to keep memory usage bounded under absurd conditions.
Also closes#8600.
This adds the BlocksHash field from the FileInfo to our API output. It
can be useful for debugging, or for external tools. I'm intentionally
leaving it as an opaque base64 string because no meaning should be
derived from it: it's just a string.
This makes sure the service manager doesn't interpret timeout errors, or any other error, as a signal to stop the service instead of restarting it.
I added it directly to our service utility function, as it may help catch other instances of the same problem... We would typically want timeouts etc to be a retryable error, unless it is the top level context that has timed out and we check for that specifically.
This adds a word to the version string when running containerized. The
purpose is mostly to facilitate troubleshooting via screenshot by
"leaking" this rather important aspect of the setup. Additionally, the
version row gets "no-overflow-ellipsis" treatment so that the whole
thing is actually visible in the GUI and the (now useless) tooltip is
removed. In production releases this won't make a difference as the
whole thing will typically fit, but in odd setups it provides more info
up front.
* lib/connections: Cache isLAN decision for later external access.
The check whether a remote device's address is on a local network
currently happens when handling the Hello message, to configure the
limiters. Save the result to the ConnectionInfo and pass it out as
part of the model's ConnectionInfo struct in ConnectionStats().
* gui: Use provided connection attribute to distinguish LAN / WAN.
Replace the dumb IP address check which didn't catch common cases and
actually could contradict what the backend decided. That could have
been confusing if the GUI says WAN, but the limiter is not actually
applied because the backend thinks it's a LAN.
Add strings for QUIC and relay connections to also differentiate
between LAN and WAN.
* gui: Redefine reception level icons for all connection types.
Move the mapping to the JS code, as it is much easier to handle
multiple switch cases by fall-through there.
QUIC is regarded no less than TCP anymore. LAN and WAN make the
difference between levels 4 / 3 and 2 / 1:
{TCP,QUIC} LAN --> {TCP,QUIC} WAN --> Relay LAN --> Relay WAN -->
Disconnected.
Previous debug input didn't really give enough info to show what was
happening, while it also printed full block lists which are enormously
verbose. Now it consistently prints 1. what it sees on disk, 2. what it
got from CurrentFile (without blocks), 3. the action taken on that file.
Jakob Borg
tomasz1986
Emil Lundberg
Eric P
bt90
Maximilian
André Colomb
Keith Harrison
Chih-Hsuan Yen
Christian Kujau
deepsource-autofix[bot]
guangwu
Simon Frei
Anthony Goeckner
Eng Zer Jun
Will Rouesnel
Ross Smith II
Alexander Seiler
Dimitri Papadopoulos Orfanos
entity0xfe
Andreas Sommer
greatroar
Aleksey Vasenev