Commit Graph

7318 Commits

Author SHA1 Message Date
Martchus 36acf3e66b Add basic C bindings to start and stop Syncthing and invoke its CLI 2023-12-11 10:48:31 +01:00
Jakob Borg 3850a08252 lib/model: Remove spurious "replacing service" failure event (ref #9271)
This is no longer a notable condition, as we do this pretty much all the
time.
2023-12-11 07:44:04 +01:00
Simon Frei a28de73031
lib/model: Remove runner during folder cleanup (fixes #9269) (#9271)
Before introducing the service map and using it for folder runners, the
entries in folderCfgs and folderRunners for the same key/folder were
removed under a single lock. Stopping the folder happens separately
before that with just the read lock. Now with the service map stopping
the folder and removing it from the map is a single operation. And that
still happens with just a read-lock. However even with a full lock it’s
still problematic: After the folder stopped, the runner isn’t present
anymore while the folder-config still is and sais the folder isn't
paused.

The index handler in turn looks at the folder config that is not paused,
thus assumes the runner has to be present -> nil deref on the runner.

A better solution might be to push most of these fmut maps into the
folder - they anyway are needed in there. Then there's just a single
map/source of info that's necessarily consistent. That's quite a bit of
work though, and probably/likely there will be corner cases there too.
2023-12-08 07:13:09 +01:00
Jakob Borg 75310b58a0
build: Update dependencies (#9265) 2023-12-06 10:53:45 +01:00
Jakob Borg 8064957270
build: Revert specifics for Go 1.21.4, build using Go 1.21.5 (#9264)
This reverts commit e477777f49.

In principle, we could have stayed with `~1.21.1`, but `check-latest:
true` apparently checks some cache/manifest/something that is only
periodically refreshed and isn't aware of 1.21.5 yet. So update the
constraints to force an upgrade.

Also the infrastructure images weren't actually using the constraint
since there was no `setup-go` action...
2023-12-06 09:02:12 +01:00
Jakob Borg c1ec9a8826
lib/fs: Reduce memory usage in xattrs handling (#9251)
This reduces allocations, in number and in size, while getting extended
attributes. This is mostly noticable when there is a large number of new
files to scan and we're running with the default scanProgressInterval --
then a queue of files is built in-memory, and this queue includes
extended attributes as part of file metadata. (Arguable it shouldn't,
but that's a more difficult and involved change.)

With 1M files to scan, each with one extended attribute, current peak
memory usage looks like this:

	Showing nodes accounting for 1425.30MB, 98.19% of 1451.64MB total
	Dropped 1435 nodes (cum <= 7.26MB)
	Showing top 10 nodes out of 54
	      flat  flat%   sum%        cum   cum%
976.56MB 67.27% 67.27% 976.56MB 67.27%
github.com/syncthing/syncthing/lib/fs.getXattr
305.44MB 21.04% 88.31% 305.44MB 21.04%
github.com/syncthing/syncthing/lib/scanner.(*walker).walk.func1
45.78MB 3.15% 91.47% 1045.23MB 72.00%
github.com/syncthing/syncthing/lib/fs.(*BasicFilesystem).GetXattr
22.89MB 1.58% 93.04% 22.89MB 1.58%
github.com/syncthing/syncthing/lib/fs.listXattr
22.89MB 1.58% 94.62% 22.89MB 1.58%
github.com/syncthing/syncthing/lib/protocol.(*PlatformData).SetXattrs
16MB 1.10% 95.72% 16.01MB 1.10%
github.com/syndtr/goleveldb/leveldb/memdb.New

After the change, it's this:

	Showing nodes accounting for 502.32MB, 95.70% of 524.88MB total
	Dropped 1400 nodes (cum <= 2.62MB)
	Showing top 10 nodes out of 91
	      flat  flat%   sum%        cum   cum%
305.43MB 58.19% 58.19% 305.43MB 58.19%
github.com/syncthing/syncthing/lib/scanner.(*walker).walk.func1
45.79MB 8.72% 66.91% 68.68MB 13.09%
github.com/syncthing/syncthing/lib/fs.(*BasicFilesystem).GetXattr
32MB 6.10% 73.01% 32.01MB 6.10%
github.com/syndtr/goleveldb/leveldb/memdb.New
22.89MB 4.36% 77.37% 22.89MB 4.36%
github.com/syncthing/syncthing/lib/fs.listXattr
22.89MB 4.36% 81.73% 22.89MB 4.36%
github.com/syncthing/syncthing/lib/protocol.(*PlatformData).SetXattrs
15.35MB 2.92% 84.66% 15.36MB 2.93%
github.com/syndtr/goleveldb/leveldb/util.(*BufferPool).Get
	   15.28MB  2.91% 87.57%    15.28MB  2.91%  strings.(*Builder).grow

(The usage for xattrs is reduced from 976 MB to 68 MB)
2023-12-04 12:48:17 +01:00
Jakob Borg 1625b44892
lib/model: Improve LastSeen handling (#9256)
LastSeen for a device was only updated when they connected. This now
updates it when they disconnect, so that we remember the last time we
actually saw them. When asking the API for current stats, currently
connected devices get a last seen value of the current time.
2023-12-04 09:24:10 +01:00
Jakob Borg d51760f410
lib/scanner: Record inode change time for directories and symlinks (#9250) 2023-12-04 07:48:24 +01:00
Jakob Borg 7b1932d64e
lib/api: Improve ignore loading error handling (fixes #9253) (#9254) 2023-12-04 07:11:35 +01:00
Syncthing Release Automation 5bfc540c88 gui, man, authors: Update docs, translations, and contributors 2023-12-04 03:45:22 +00:00
Jakob Borg 4cba99fcd4 lib/fs: Better equality comparison in mtimefs 2023-12-03 16:01:46 +01:00
Jakob Borg 2ae15aa454 cmd/stcrashreceiver: Add metrics for diskstore inventory 2023-11-27 08:24:59 +01:00
Jakob Borg 47bcf4f8f4 cmd/stcrashreceiver: Minor cleanup, stricter file permissions 2023-11-27 08:24:59 +01:00
Jakob Borg a8b9096353 cmd/stcrashreceiver: Add metrics for incoming reports 2023-11-27 08:24:59 +01:00
Jakob Borg 5328380691 cmd/ursrv: Add metrics for incoming reports 2023-11-27 08:24:59 +01:00
Syncthing Release Automation 6069cf39e5 gui, man, authors: Update docs, translations, and contributors 2023-11-27 03:45:21 +00:00
Emil Lundberg 1f7d236742
gui: Specialize a special-purpose checkbox style (#9236)
### Purpose

Discovered while working on the WebAuthn credentials table in #9175:
there's a style on `td input[type="checkbox"]` that modifies margins for
all checkboxes in `<table>`s. It looks like this style is specially
tailored to the particular table that added it (PR #8734), so it should
have a correspondingly special-purpose class to not accidentally apply
it to other tables.

As best as I could tell there are only 2 instances of `<input
type="checkbox">` in `<td>`s, shown in the screenshots below.

### Testing

- Open "Actions > Logging > Debugging Facilities" and observe the
vertical spacing of the checkboxes.
- Open "Edit Folder > Advanced", check "Sync Extended Attributes" or
"Send Extended Attributes", click "Add filter entry" and observe the
vertical spacing of the checkbox that appears.

### Screenshots

#### Before

![Logs > Debugging
Facilities](https://github.com/syncthing/syncthing/assets/1367758/998fc66d-a0ad-41d9-a476-7a2b3da622d1)
![Add filter
entry](https://github.com/syncthing/syncthing/assets/1367758/647cb565-fcd0-4a81-a6ca-1f75137039b0)

#### After

Logs > Debugging Facilities now more compact:
![Logs > Debugging Facilities now
](https://github.com/syncthing/syncthing/assets/1367758/7cf8fc77-610e-4b4a-be21-c50d30be7bb9)

Add filter entry unchanged:
![Add filter
entry](https://github.com/syncthing/syncthing/assets/1367758/0ba710d6-cee1-49b4-92bc-acfc0c22c2bd)
2023-11-21 10:00:18 +01:00
Syncthing Release Automation 8e9ee3fbe8 gui, man, authors: Update docs, translations, and contributors 2023-11-20 08:49:38 +00:00
Jakob Borg 35b0afc131 build: Support new nested namespaces in Weblate downloads 2023-11-20 09:47:08 +01:00
Simon Frei 958ff67ccc
lib/model: Acquire fmut lock in ensureIndexHandler (fixes #9234) (#9235)
Towards the end of the function f.folderCfgs and f.folderRunners are
read without the lock.
2023-11-20 08:12:25 +01:00
tomasz1986 13d9317a38
gui: Allow to translate "unknown device" (#9229)
The string is currently hard-coded in English, so allow to translate it
into other languages. It appears mainly in the browser title bar before
logging in into the GUI or when the GUI is still loading.

Signed-off-by: Tomasz Wilczyński <twilczynski@naver.com>
2023-11-16 22:07:15 +01:00
Anatoli Babenia b184d46d8a
cmd/ursrv: Add link to source code (#9224)
To see that https://data.syncthing.net is open source, study the code
and change it.
2023-11-15 08:51:23 +00:00
Jakob Borg 3f32c5cb4b cmd/ursrv: Anchor distribution expressions to avoid mismatches (ref #9141) 2023-11-15 09:32:46 +01:00
Jakob Borg 5c65a1bc83 build: Ursrv image for infrastructure 2023-11-15 08:48:00 +01:00
Jakob Borg d0a6dc5b13 cmd/ursv: Report on copy range method 2023-11-15 08:48:00 +01:00
Jakob Borg 439c6c5b7c
lib/api: Add cache busting for basic auth (ref #9208) (#9215)
This adds our short device ID to the basic auth realm. This has at least
two consequences:

- It is different from what's presented by another device on the same
address (e.g., if I use SSH forwards to different dives on the same
local address), preventing credentials for one from being sent to
another.

- It is different from what we did previously, meaning we avoid cached
credentials from old versions interfering with the new login flow.

I don't *think* there should be things that depend on our precise realm
string, so this shouldn't break any existing setups...

Sneakily this also changes the session cookie and CSRF name, because I
think `id.Short().String()` is nicer than `id.String()[:5]` and the
short ID is two characters longer. That's also not a problem...
2023-11-14 11:57:39 +01:00
Jakob Borg aaee0c126b cmd/stdiscorv: Expose build info in metrics 2023-11-14 09:31:53 +01:00
André Colomb f3bd4d71de
gui: Fix Weblate merge conflict (#9222)
The manual translation updates in the recently merged PR #9220 caused
conflicts with the existing (test) entries on Weblate. This will fix it.
2023-11-14 08:21:40 +01:00
Jakob Borg 876d056705
build: Fixup Docker changes from previous (#9223) 2023-11-14 08:17:34 +01:00
DerRockWolf e988978fa1
Add `org.opencontainers.image.source` to Dockerfiles (#9211)
### Purpose

The OCI image spec specifies well-defined
[annotations](https://github.com/opencontainers/image-spec/blob/main/annotations.md)
that can be added to images.
Theses annotations can then be used by other tools to gather more
information of an image.

This PR adds the `org.opencontainers.image.source` to allow tools such
as [renovate](https://github.com/renovatebot/renovate) to find the
release notes of a give version.

~~I've only done this change for `Dockerfile`. Should I also add the
label to the other dockerfiles?~~
I've now added the source annotations to all `Dockerfile`s & action
workflows.

### Testing

None, change was done by following the [renovate
documentation](https://docs.renovatebot.com/modules/datasource/docker/).
2023-11-14 07:46:14 +01:00
Jakob Borg d5deede7a1 build: Update dependencies 2023-11-14 07:41:53 +01:00
André Colomb 4f70f5c280
gui: Use nested namespace for theme name translation keys (#9220)
Following up on #9192, this makes use of the new mechanism for the theme
names.

The dummy string added for testing is removed again here. All
translations are updated to the new nested syntax, except Chinese
(zh-HK) where the string weren't actually translated.
2023-11-14 07:22:52 +01:00
Emil Lundberg a1ad020b63
Support explicit translation ID and dotted namespaces in translation extraction (#9192)
Some translations, especially single words or other short
labels for buttons and the like, may not be transferable between
contexts even if they happen to be equal in English. In these cases,
setting an explicit translation ID is important for context separation.
Angular Translate also supports nested JSON in translation tables,
addressed using `.` as namespace separator; this enhancement makes use
of this when extracting translation with an explicit translation ID.
2023-11-13 21:04:24 +01:00
Jakob Borg 8f1b0df74b
lib/api: Improve cookie handling (fixes #9208) (#9214) 2023-11-13 20:37:29 +01:00
Jakob Borg 0f8dc6c1d3 test: Update testing configs
Somewhere along the way I snuck in a change to the test configs that is
quite annoying. This reverts that back to the more usual setup it was
before.
2023-11-13 13:51:17 +01:00
Jakob Borg 8ae9db3b2d
build: Use actual Go version as cache key (#9216)
We use `env.GO_VERSION` as cache key for the build cache, but this is
nowadays typically something like `~1.21.1` which doesn't change when
1.21.2, 1.21.3 etc are released, making the cache fairly useless as
everything gets rebuilt. This re-sets the `GO_VERSION` variable after
installing Go so that it contains the actual installed version.
2023-11-13 12:20:40 +01:00
Jakob Borg e477777f49
build: Version constraint to avoid Go 1.21.4 on Windows (ref #9207) (#9213) 2023-11-13 09:52:59 +00:00
Syncthing Release Automation 7a132bdf24 gui, man, authors: Update docs, translations, and contributors 2023-11-13 03:45:25 +00:00
Jakob Borg 5e2b7825dc cmd/stdiscosrv: Metric for returned retry-after 2023-11-08 12:18:59 +01:00
Jakob Borg 6d30c109e4 build: Push to Docker :edge tag for infrastructure builds 2023-11-08 12:18:59 +01:00
Jakob Borg 58bd931d90 cmd/stdiscosrv: Account IPv4 & IPv6 2023-11-08 12:18:59 +01:00
vapatel2 854499382e
cmd/stdiscosrv: Prevent nil IPs from X-Forwarded-For (fixes #9189) (#9190)
### Purpose

Treat X-Forwarded-For as a comma-separated string to prevent nil IP being returned by the Discovery Server

### Testing

Unit Tests implemented

Testing with a Discovery Client can be done as follows:
```
A simple example to replicate this entails running Discovery with HTTP, use Nginx as a reverse proxy and hardcode (as an example) a list of IPs in the X-Forwarded-For header.
1. Send an Announcement with tcp://0.0.0.0:<some-port>
2. Query the DeviceID
3. Observe the returned IP Address is no longer nil; i.e.  `tcp://<nil>:<some-port>`
```
2023-11-08 11:10:23 +00:00
Jakob Borg cb4c1f9ad2
build: Update dependencies (#9202) 2023-11-06 16:43:11 -08:00
Catfriend1 b452fb3ad2
gui: Add id attribute to login button, allows form filling tools to be used (fixes #9200) (#9201)
Add an id attribute to the submit button shown on the login form. This
allows my password manager's form filling function to interact with the
button after filling in username and password (which already have the id
attribute in place).
2023-11-06 16:30:19 -08:00
Syncthing Release Automation c17a1fea77 gui, man, authors: Update docs, translations, and contributors 2023-11-06 03:45:25 +00:00
Syncthing Release Automation d50511c5c6 gui, man, authors: Update docs, translations, and contributors 2023-10-30 03:45:44 +00:00
Jakob Borg bae6d5f375
lib/location: Fix regression of timestamp handling (ref #9180) (#9185) 2023-10-26 07:41:02 +02:00
Jakob Borg b5082f6af8
lib/locations: Change default config/data location to new XDG recommendation (fixes #9178, fixes #9179) (#9180)
This makes the new default $XDG_STATE_HOME/syncthing or
~/.local/state/syncthing, while still looking in legacy locations first
for existing installs.

Note that this does not *move* existing installs, and nor should we.
Existing paths will continue to be used as-is, but the user can move the
dir into the new place if they want to use it (as they could prior to
this change as well, for that matter).

### Documentation

Needs update to the config docs about our default locations.
2023-10-25 11:16:24 +02:00
dependabot[bot] 9666e9701b
build(deps): bump github.com/quic-go/quic-go from 0.39.0 to 0.39.1 (#9181) 2023-10-24 07:23:38 +02:00
Syncthing Release Automation 86e1c5ff18 gui, man, authors: Update docs, translations, and contributors 2023-10-23 03:45:41 +00:00