Compare commits

...

325 Commits

Author SHA1 Message Date
Martchus 388f287fcb Add basic C bindings to start and stop Syncthing and invoke its CLI 2024-05-08 10:52:52 +02:00
Jakob Borg 9562cfd5c2 build: Use Go 1.22.3 at minimum 2024-05-08 08:02:01 +02:00
Syncthing Release Automation ff9b24f388 gui, man, authors: Update docs, translations, and contributors 2024-04-22 03:45:18 +00:00
Syncthing Release Automation 01b820dc78 gui, man, authors: Update docs, translations, and contributors 2024-04-15 04:10:05 +00:00
Jakob Borg 79ae24df76
lib/nat: Don't crash on empty address list (fixes #9503) (#9504) 2024-04-11 13:23:29 +02:00
Jakob Borg 61b94b9ea5
lib/db: Drop indexes for outgoing data to force refresh (ref #9496) (#9502)
### Purpose

Resend our indexes since we fixed that index-sending issue.

I made a new thing to only drop the non-local-device index IDs, i.e.,
those for other devices. This means we will see a mismatch and resend
all indexes, but they will not. This is somewhat cleaner as it avoids
resending everything twice when two devices are upgraded, and in any
case, we have no reason to force a resend of incoming indexes here.

### Testing

It happens on my computer...
2024-04-08 11:14:27 +02:00
tomasz1986 6fb3c5ccf2
gui: Fix missing link to device editor for names with superscript (ref #9472) (#9494)
The commit 7e4e65ebf5 added links to
devices listed in the Shared With list in the folder info. However, it
only added them to those that had no superscript next to them.

With this change, the links are added to all devices regardless of
whether they have the superscript next to their names or not. The commit
also simplifies the code by using anchors directly instead of wrapping
them in spans.

Signed-off-by: Tomasz Wilczyński <twilczynski@naver.com>
2024-04-07 21:37:31 +02:00
Jakob Borg 2e7c03420f
lib/db: Hold update lock while taking snapshot (#9496) 2024-04-05 21:32:43 +02:00
Jakob Borg faa56b4bb7
build: Update dependencies (#9497) 2024-04-05 16:13:20 +02:00
Syncthing Release Automation d7ba5316b8 gui, man, authors: Update docs, translations, and contributors 2024-04-01 03:45:41 +00:00
Syncthing Release Automation bdfd0f0548 gui, man, authors: Update docs, translations, and contributors 2024-03-25 03:45:14 +00:00
Tim Nordenfur 5d27185083
Removed no longer relevant Bountysource link (#9480)
### Purpose

Bountysource no longer exists and the readme link 404s. This removes the
Bountysource link and corresponding readme section.

Perhaps the section should instead be replaced by other instructions for
voting on features.
2024-03-23 00:23:02 +01:00
Jakob Borg 4dfb9d7c83 lib/api: Missing return after HTTP error 2024-03-21 08:57:43 -04:00
Emil Lundberg 2f15670094
lib/api: Extract session store (#9425)
This is an extract from PR #9175, which can be reviewed in isolation to
reduce the volume of changes to review all at once in #9175. There are
about to be several services and API handlers that read and set cookies
and session state, so this abstraction will prove helpful.

In particular a motivating cause for this is that with the current
architecture in PR #9175, in `api.go` the [`webauthnService` needs to
access the
session](https://github.com/syncthing/syncthing/pull/9175/files#diff-e2e14f22d818b8e635572ef0ee7718dee875c365e07225d760a6faae8be7772dR309-R310)
for authentication purposes but needs to be instantiated before the
`configMuxBuilder` for config purposes, because the WebAuthn additions
to config management need to perform WebAuthn registration ceremonies,
but currently the session management is embedded in the
`basicAuthAndSessionMiddleware` which is [instantiated much
later](https://github.com/syncthing/syncthing/pull/9175/files#diff-e2e14f22d818b8e635572ef0ee7718dee875c365e07225d760a6faae8be7772dL371-R380)
and only if authentication is enabled in `guiCfg`. This refactorization
extracts the session management out from `basicAuthAndSessionMiddleware`
so that `basicAuthAndSessionMiddleware` and `webauthnService` can both
use the same shared session management service to perform session
management logic.

### Testing

This is a refactorization intended to not change any externally
observable behaviour, so existing tests (e.g., `api_auth_test.go`)
should cover this where appropriate. I have manually verified that:

- Appending `+ "foo"` to the cookie name in `createSession` causes
`TestHtmlFormLogin/invalid_URL_returns_403_before_auth_and_404_after_auth`
and `TestHtmlFormLogin/UTF-8_auth_works` to fail
- Inverting the return value of `hasValidSession` cases a whole bunch of
tests in `TestHTTPLogin` and `TestHtmlFormLogin` to fail
- (Fixed) Changing the cookie to `MaxAge: 1000` in `destroySession` does
NOT cause any tests to fail!
- Added tests `TestHtmlFormLogin/Logout_removes_the_session_cookie`,
`TestHTTPLogin/*/Logout_removes_the_session_cookie`,
`TestHtmlFormLogin/Session_cookie_is_invalid_after_logout` and
`TestHTTPLogin/200_path#01/Session_cookie_is_invalid_after_logout` to
cover this.
- Manually verified that these tests pass both before and after the
changes in this PR, and that changing the cookie to `MaxAge: 1000` or
not calling `m.tokens.Delete(cookie.Value)` in `destroySession` makes
the respective pair of tests fail.
2024-03-21 08:09:47 -04:00
Syncthing Release Automation b49137ce36 gui, man, authors: Update docs, translations, and contributors 2024-03-18 03:45:22 +00:00
Jaspitta 7e4e65ebf5
gui: Open devices on click in Shared With list in folder info (fixes #8972) (#9472) 2024-03-17 20:13:09 +01:00
Simon Frei 8c8167a4ab
lib/model: Don't bump seq on error in index handler (#9459) 2024-03-11 07:30:21 +01:00
Simon Frei 73cc5553b6
lib/model: Prevent infinite index sending loop (fixes #9407) (#9458)
Explanation of what/why in a code comment.

Fixes https://github.com/syncthing/syncthing/issues/9407
2024-03-10 22:28:40 +01:00
Simon Frei 2ab2488274
lib/scanner: Fix ticker leak in scanner (fixes #9417) (#9451)
Move the ticker closer to where it's used and defer stop it to avoid
missing a branch.

Fixes regression introduced in
2f3eacdb6c

Fixes https://github.com/syncthing/syncthing/issues/9417
2024-03-05 19:04:26 +01:00
Jakob Borg eb9cd363d0
build: Update dependencies (#9448) 2024-03-04 20:39:43 +01:00
Syncthing Release Automation 7fe3906534 gui, man, authors: Update docs, translations, and contributors 2024-03-04 03:54:27 +00:00
André Colomb 5fdab1bf11
gui: Show encryption status for devices sharing folder (ref #8808) (#9355)
This re-implements the stalled enhancement from #8808. Thanks @Craeckie
for the idea and first implementation draft!

If a folder is shared to a device with encryption, add a lock icon in
front of the device name under "Shared With" in the folder details
panel. Be careful not to add whitespace caused by line wraps in HTML
source code, which would defeat the purpose of keeping the icon glued to
the name by a non-breaking space.

Apply the same lock icon for the list of folders shared with a device.
2024-03-03 21:09:57 +01:00
André Colomb 13a6d43f0b
gui: Fix wrapping in "Shared With" / "Folders" lists. (#9438)
This change was split off from #9355 as an independent clean-up / fix.
See that PR for review discussion, testing, and screenshots.

Improve the wrapping of folder labels / device names by going back to
word-wrapping, but making sure other spans, such as the trailing comma,
do not get separated from the label span.

* Avoid adding whitespace caused by line wraps in HTML source code.

The different cases within the ng-switch block are separated by
newlines for readability, but that gets parsed as whitespace.  For
wrapping purposes, this should not happen, because then there is no
way to keep other HTML parts glued to the name / label in each list
entry.

* Simplify redundant conditional comma code.

The separating comma after a device name or folder label (all but the
last) should always stick to it.  Use the HTML comment trick to avoid
whitespace and therefore a wrapping opportunity caused by the code
formatting newline.  Thus the conditional comma only needs to be
defined once, not in each ng-switch case.

* Wrap at word boundaries and only break up words if necessary.

Use the overflow-wrap: break-word; style instead of word-break:
break-all;.  While the latter is suitable for longish paths, breaking
device names or folder labels arbitrarily within words is ugly.

This also makes the the <sup> numbers actually stay glued to their
respective neighboring words.

Include legacy CSS alias "word-wrap" in the class definition.

* Fix indentation (unrelated).
2024-03-03 20:55:09 +01:00
Jakob Borg ac942e2481 github: Convert issue templates into forms (fixes #9442) 2024-03-02 16:27:57 +01:00
Luke Hamburg bbd2a7fbc5
lib/model: Ignore difference in extended attributes & ownership when deleting (fixes #9371) (#9430)
Adds a bool flag to `scanIfItemChanged()` to indicate when the scan was initiated from a delete function, and if so, tell `IsEquivalentOptional()` to ignore Xattrs and Ownership regardless of the global setting.

I tested this with my sledgehammer and it seems to pass.
2024-03-02 14:55:18 +00:00
Jakob Borg 07a9fa2dbd
all: Use own automaxprocs package that doesn't log (ref #9436) (#9437)
### Purpose

🤫
2024-02-27 13:05:19 +01:00
Thomas aa559bf496
all: Use Linux container CPU quota (fixes #9357, fixes #9435) (#9436)
Go is not cgroup aware and by default will set GOMAXPROCS to the number
of available threads, regardless of whether it is within the allocated
quota. This behaviour causes high amount of CPU throttling and degraded
application performance.
2024-02-26 12:23:14 +00:00
Jakob Borg 2d968d46b7 cmd/syncthing: Remove legacy GOMAXPROCS handling (ref #9436) 2024-02-26 13:12:57 +01:00
Syncthing Release Automation 86c4cafc96 gui, man, authors: Update docs, translations, and contributors 2024-02-26 03:45:28 +00:00
Beat Reichenbach c4dfb66d84
docker: Add support for setting umask (#9429)
Add support for setting umask value in the Docker `entrypoint.sh`
script. This is useful when
not syncing permissions and working with groups, and needing umask
values like `002` instead of `022`.
2024-02-22 08:47:43 +00:00
Syncthing Release Automation f4d160684b gui, man, authors: Update docs, translations, and contributors 2024-02-19 03:45:23 +00:00
Syncthing Release Automation b76e6ce70d gui, man, authors: Update docs, translations, and contributors 2024-02-12 03:45:25 +00:00
Jakob Borg 6b4028eede
build: Use correct Go version also for script runs (#9414)
The changes to go.mod in latest Go 1.21/1.22 are not fully understood by
older Go that might be pre-installed on builds, so make sure we always
have a modern one in place even for running small release scripts etc.
2024-02-11 09:24:00 +01:00
Jakob Borg ad81ac8da7
lib/api: Deflake TestAPIServiceRequests (#9413)
Somewhere along the way, the non-parallel test became parallel, and at
that point, timeouts occurred. Parallel is better, so increase the
timeout on the offending call a bit...
2024-02-11 09:20:29 +01:00
Jakob Borg 7ebeaefe77
lib/model: Deflake new IndexHandlerTest (#9412)
The new test has a flakiness factor on slow platforms, where the close
on the sending connection races with the last index message, potentially
messing up the count. This adds a wait to ensure that all sent messages
are received, or the test will eventually fail with a timeout.
2024-02-11 09:03:12 +01:00
Jakob Borg e1dd36561d
all: Use some Go 1.21 features (#9409) 2024-02-10 21:02:42 +01:00
Jakob Borg 96c30f8387
lib/model, lib/protocol: Remove FileInfoBatch reuse behavior (#9399) 2024-02-10 19:16:27 +01:00
Jakob Borg fc8b353011
build: Use Go 1.22, minimum is Go 1.21 (#9408)
Also updated dependencies, and an adjustment to build tags for how those
are handled and how irrelevant go1.15 is nowadays...
2024-02-09 16:35:29 +01:00
Jakob Borg 416b9e8924
lib/logger: Reduce API surface (#9404)
There is no need to expose the IsTraced() thing; it's just used in
initialisation, and thereafter ShouldDebug() is the corresponding
correct call.
2024-02-09 11:17:44 +01:00
gudvinr 9f6d732587
lib/logger: Split STTRACE into list of strings (#9402)
Currently `IsTraced("xyz")` will return true for
any inclusion of "xyz" in string.

This change splits `STTRACE` using `','`, `' '` and `';'`
as delimiters. That makes facilities separation
more clear.
2024-02-06 14:07:59 +01:00
Syncthing Release Automation f2f5786b33 gui, man, authors: Update docs, translations, and contributors 2024-02-05 03:45:33 +00:00
Jakob Borg eb617865d2 lib/model: Typo in method name (fixes #9389) 2024-02-01 15:13:33 +01:00
Jakob Borg a49e318d25 lib/model: Typo in debug print (fixes #9386) 2024-02-01 15:11:09 +01:00
Jakob Borg e74674a019
build: Update dependencies (#9379) 2024-01-31 09:11:04 +01:00
dependabot[bot] d98fa474ae
build(deps): bump actions/cache from 3 to 4 (#9363)
Bumps [actions/cache](https://github.com/actions/cache) from 3 to 4.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/actions/cache/releases">actions/cache's
releases</a>.</em></p>
<blockquote>
<h2>v4.0.0</h2>
<h2>What's Changed</h2>
<ul>
<li>Update action to node20 by <a
href="https://github.com/takost"><code>@​takost</code></a> in <a
href="https://redirect.github.com/actions/cache/pull/1284">actions/cache#1284</a></li>
<li>feat: save-always flag by <a
href="https://github.com/to-s"><code>@​to-s</code></a> in <a
href="https://redirect.github.com/actions/cache/pull/1242">actions/cache#1242</a></li>
</ul>
<h2>New Contributors</h2>
<ul>
<li><a href="https://github.com/takost"><code>@​takost</code></a> made
their first contribution in <a
href="https://redirect.github.com/actions/cache/pull/1284">actions/cache#1284</a></li>
<li><a href="https://github.com/to-s"><code>@​to-s</code></a> made their
first contribution in <a
href="https://redirect.github.com/actions/cache/pull/1242">actions/cache#1242</a></li>
</ul>
<p><strong>Full Changelog</strong>: <a
href="https://github.com/actions/cache/compare/v3...v4.0.0">https://github.com/actions/cache/compare/v3...v4.0.0</a></p>
<h2>v3.3.3</h2>
<h2>What's Changed</h2>
<ul>
<li>Cache v3.3.3 by <a
href="https://github.com/robherley"><code>@​robherley</code></a> in <a
href="https://redirect.github.com/actions/cache/pull/1302">actions/cache#1302</a></li>
</ul>
<h2>New Contributors</h2>
<ul>
<li><a href="https://github.com/robherley"><code>@​robherley</code></a>
made their first contribution in <a
href="https://redirect.github.com/actions/cache/pull/1302">actions/cache#1302</a></li>
</ul>
<p><strong>Full Changelog</strong>: <a
href="https://github.com/actions/cache/compare/v3...v3.3.3">https://github.com/actions/cache/compare/v3...v3.3.3</a></p>
<h2>v3.3.2</h2>
<h2>What's Changed</h2>
<ul>
<li>Fixed readme with new segment timeout values by <a
href="https://github.com/kotewar"><code>@​kotewar</code></a> in <a
href="https://redirect.github.com/actions/cache/pull/1133">actions/cache#1133</a></li>
<li>Readme fixes by <a
href="https://github.com/kotewar"><code>@​kotewar</code></a> in <a
href="https://redirect.github.com/actions/cache/pull/1134">actions/cache#1134</a></li>
<li>Updated description of the lookup-only input for main action by <a
href="https://github.com/kotewar"><code>@​kotewar</code></a> in <a
href="https://redirect.github.com/actions/cache/pull/1130">actions/cache#1130</a></li>
<li>Change two new actions mention as quoted text by <a
href="https://github.com/bishal-pdMSFT"><code>@​bishal-pdMSFT</code></a>
in <a
href="https://redirect.github.com/actions/cache/pull/1131">actions/cache#1131</a></li>
<li>Update Cross-OS Caching tips by <a
href="https://github.com/pdotl"><code>@​pdotl</code></a> in <a
href="https://redirect.github.com/actions/cache/pull/1122">actions/cache#1122</a></li>
<li>Bazel example (Take <a
href="https://redirect.github.com/actions/cache/issues/2">#2</a>️⃣) by
<a href="https://github.com/vorburger"><code>@​vorburger</code></a> in
<a
href="https://redirect.github.com/actions/cache/pull/1132">actions/cache#1132</a></li>
<li>Remove actions to add new PRs and issues to a project board by <a
href="https://github.com/jorendorff"><code>@​jorendorff</code></a> in <a
href="https://redirect.github.com/actions/cache/pull/1187">actions/cache#1187</a></li>
<li>Consume latest toolkit and fix dangling promise bug by <a
href="https://github.com/chkimes"><code>@​chkimes</code></a> in <a
href="https://redirect.github.com/actions/cache/pull/1217">actions/cache#1217</a></li>
<li>Bump action version to 3.3.2 by <a
href="https://github.com/bethanyj28"><code>@​bethanyj28</code></a> in <a
href="https://redirect.github.com/actions/cache/pull/1236">actions/cache#1236</a></li>
</ul>
<h2>New Contributors</h2>
<ul>
<li><a href="https://github.com/vorburger"><code>@​vorburger</code></a>
made their first contribution in <a
href="https://redirect.github.com/actions/cache/pull/1132">actions/cache#1132</a></li>
<li><a
href="https://github.com/jorendorff"><code>@​jorendorff</code></a> made
their first contribution in <a
href="https://redirect.github.com/actions/cache/pull/1187">actions/cache#1187</a></li>
<li><a href="https://github.com/chkimes"><code>@​chkimes</code></a> made
their first contribution in <a
href="https://redirect.github.com/actions/cache/pull/1217">actions/cache#1217</a></li>
<li><a
href="https://github.com/bethanyj28"><code>@​bethanyj28</code></a> made
their first contribution in <a
href="https://redirect.github.com/actions/cache/pull/1236">actions/cache#1236</a></li>
</ul>
<p><strong>Full Changelog</strong>: <a
href="https://github.com/actions/cache/compare/v3...v3.3.2">https://github.com/actions/cache/compare/v3...v3.3.2</a></p>
<h2>v3.3.1</h2>
<h2>What's Changed</h2>
<ul>
<li>Reduced download segment size to 128 MB and timeout to 10 minutes by
<a href="https://github.com/kotewar"><code>@​kotewar</code></a> in <a
href="https://redirect.github.com/actions/cache/pull/1129">actions/cache#1129</a></li>
</ul>
<p><strong>Full Changelog</strong>: <a
href="https://github.com/actions/cache/compare/v3...v3.3.1">https://github.com/actions/cache/compare/v3...v3.3.1</a></p>
<h2>v3.3.0</h2>
<h2>What's Changed</h2>
<ul>
<li>Bug: Permission is missing in cache delete example by <a
href="https://github.com/kotokaze"><code>@​kotokaze</code></a> in <a
href="https://redirect.github.com/actions/cache/pull/1123">actions/cache#1123</a></li>
</ul>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/actions/cache/blob/main/RELEASES.md">actions/cache's
changelog</a>.</em></p>
<blockquote>
<h1>Releases</h1>
<h3>3.0.0</h3>
<ul>
<li>Updated minimum runner version support from node 12 -&gt; node
16</li>
</ul>
<h3>3.0.1</h3>
<ul>
<li>Added support for caching from GHES 3.5.</li>
<li>Fixed download issue for files &gt; 2GB during restore.</li>
</ul>
<h3>3.0.2</h3>
<ul>
<li>Added support for dynamic cache size cap on GHES.</li>
</ul>
<h3>3.0.3</h3>
<ul>
<li>Fixed avoiding empty cache save when no files are available for
caching. (<a
href="https://redirect.github.com/actions/cache/issues/624">issue</a>)</li>
</ul>
<h3>3.0.4</h3>
<ul>
<li>Fixed tar creation error while trying to create tar with path as
<code>~/</code> home folder on <code>ubuntu-latest</code>. (<a
href="https://redirect.github.com/actions/cache/issues/689">issue</a>)</li>
</ul>
<h3>3.0.5</h3>
<ul>
<li>Removed error handling by consuming actions/cache 3.0 toolkit, Now
cache server error handling will be done by toolkit. (<a
href="https://redirect.github.com/actions/cache/pull/834">PR</a>)</li>
</ul>
<h3>3.0.6</h3>
<ul>
<li>Fixed <a
href="https://redirect.github.com/actions/cache/issues/809">#809</a> -
zstd -d: no such file or directory error</li>
<li>Fixed <a
href="https://redirect.github.com/actions/cache/issues/833">#833</a> -
cache doesn't work with github workspace directory</li>
</ul>
<h3>3.0.7</h3>
<ul>
<li>Fixed <a
href="https://redirect.github.com/actions/cache/issues/810">#810</a> -
download stuck issue. A new timeout is introduced in the download
process to abort the download if it gets stuck and doesn't finish within
an hour.</li>
</ul>
<h3>3.0.8</h3>
<ul>
<li>Fix zstd not working for windows on gnu tar in issues <a
href="https://redirect.github.com/actions/cache/issues/888">#888</a> and
<a
href="https://redirect.github.com/actions/cache/issues/891">#891</a>.</li>
<li>Allowing users to provide a custom timeout as input for aborting
download of a cache segment using an environment variable
<code>SEGMENT_DOWNLOAD_TIMEOUT_MINS</code>. Default is 60 minutes.</li>
</ul>
<h3>3.0.9</h3>
<ul>
<li>Enhanced the warning message for cache unavailablity in case of
GHES.</li>
</ul>
<h3>3.0.10</h3>
<ul>
<li>Fix a bug with sorting inputs.</li>
<li>Update definition for restore-keys in README.md</li>
</ul>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="13aacd865c"><code>13aacd8</code></a>
Merge pull request <a
href="https://redirect.github.com/actions/cache/issues/1242">#1242</a>
from to-s/main</li>
<li><a
href="53b35c5439"><code>53b35c5</code></a>
Merge branch 'main' into main</li>
<li><a
href="65b8989fab"><code>65b8989</code></a>
Merge pull request <a
href="https://redirect.github.com/actions/cache/issues/1284">#1284</a>
from takost/update-to-node-20</li>
<li><a
href="d0be34d544"><code>d0be34d</code></a>
Fix dist</li>
<li><a
href="66cf064d47"><code>66cf064</code></a>
Merge branch 'main' into update-to-node-20</li>
<li><a
href="1326563738"><code>1326563</code></a>
Merge branch 'main' into main</li>
<li><a
href="e71876755e"><code>e718767</code></a>
Fix format</li>
<li><a
href="01229828ff"><code>0122982</code></a>
Apply workaround for earlyExit</li>
<li><a
href="3185ecfd61"><code>3185ecf</code></a>
Update &quot;only-&quot; actions to node20</li>
<li><a
href="25618a0a67"><code>25618a0</code></a>
Bump version</li>
<li>Additional commits viewable in <a
href="https://github.com/actions/cache/compare/v3...v4">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=actions/cache&package-manager=github_actions&previous-version=3&new-version=4)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)


</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-01-31 08:33:02 +01:00
Jakob Borg f16817632f
lib/api: Improve folder summary event, verbose service (#9370)
This makes a couple of small improvements to the folder summary
mechanism:

- The folder summary includes the local and remote sequence numbers in
clear text, rather than some odd sum that I'm not sure what it was
intended to represent.
- The folder summary event is generated when appropriate, regardless of
whether there is an event listener. We did this before because
generating it was expensive, and we wanted to avoid doing it
unnecessarily. Nowadays, however, it's mostly just reading out
pre-calculated metadata, and anyway, it's nice if it shows up reliably
when running with -verbose.

The point of all this is to make it easier to use these events to judge
when devices are, in fact, in sync. As-is, if I'm looking at two
devices, it's very difficult to reliably determine if they are in sync
or not. The reason is that while we can ask device A if it thinks it's
in sync, we can't see if the answer is "yes" because it has processed
all changes from B, or if it just doesn't know about the changes from B
yet. With proper sequence numbers in the event we can compare the two
and determine the truth. This makes testing a lot easier.
2024-01-31 08:24:39 +01:00
Jakob Borg bda4016109
lib/protocol: Refactor interface (#9375)
This is a refactor of the protocol/model interface to take the actual
message as the parameter, instead of the broken-out fields:

```diff
type Model interface {
        // An index was received from the peer device
-       Index(conn Connection, folder string, files []FileInfo) error
+       Index(conn Connection, idx *Index) error
        // An index update was received from the peer device
-       IndexUpdate(conn Connection, folder string, files []FileInfo) error
+       IndexUpdate(conn Connection, idxUp *IndexUpdate) error
        // A request was made by the peer device
-       Request(conn Connection, folder, name string, blockNo, size int32, offset int64, hash []byte, weakHash uint32, fromTemporary bool) (RequestResponse, error)
+       Request(conn Connection, req *Request) (RequestResponse, error)
        // A cluster configuration message was received
-       ClusterConfig(conn Connection, config ClusterConfig) error
+       ClusterConfig(conn Connection, config *ClusterConfig) error
        // The peer device closed the connection or an error occurred
        Closed(conn Connection, err error)
        // The peer device sent progress updates for the files it is currently downloading
-       DownloadProgress(conn Connection, folder string, updates []FileDownloadProgressUpdate) error
+       DownloadProgress(conn Connection, p *DownloadProgress) error
 }
```

(and changing the `ClusterConfig` to `*ClusterConfig` for symmetry;
we'll be forced to use all pointers everywhere at some point anyway...)

The reason for this is that I have another thing cooking which is a
small troubleshooting change to check index consistency during transfer.
This required adding a field or two to the index/indexupdate messages,
and plumbing the extra parameters in umpteen changes is almost as big a
diff as this is. I figured let's do it once and avoid having to do that
in the future again...

The rest of the diff falls out of the change above, much of it being in
test code where we run these methods manually...
2024-01-31 08:18:27 +01:00
Syncthing Release Automation 8f5d07bd09 gui, man, authors: Update docs, translations, and contributors 2024-01-29 03:45:37 +00:00
kylosus 302b352d78
lib/fs: Add invalid UTF-8 guards to watcher (fixes #9369) (#9372)
Add invalid UTF-8 guards to fix #9369. Probably not a permanent fix, but
putting it up here in case someone else encounters the same panic.
2024-01-28 19:50:26 +01:00
Jakob Borg 45beb28fa5 lib/api: Remove remnants of CSRF tokens file mentions (ref #9284) 2024-01-23 12:07:58 +01:00
Syncthing Release Automation ee9b20e47a gui, man, authors: Update docs, translations, and contributors 2024-01-22 03:45:30 +00:00
tomasz1986 0f55d5fc3e
gui: Remove non-functional HTML from External Versioning tooltip (ref #8923) (#9358)
gui: Remove non-functional HTML from External Versioning tooltip (ref
#8923)

Since [1], it is no longer possible to use HTML in tooltips. This was
addressed in [2], however the commit missed one instance of HTML that
was used to change the font type of the External versioning command
tooltip. This remaining HTML is removed in this commit.

[1] f5e5af391a
[2] 73c52eafb6

Signed-off-by: Tomasz Wilczyński <twilczynski@naver.com>

### Screenshots


![image](https://github.com/syncthing/syncthing/assets/5626656/d5f6c553-35cb-48c2-b654-809d8bbe93b8)

Signed-off-by: Tomasz Wilczyński <twilczynski@naver.com>
2024-01-20 22:11:42 +01:00
bt90 35e153625c
cmd/ursrv: Add FreeBSD detection (#9351)
### Purpose

Classify `ports@freebsd` as `FreeBSD (3rd party)`
2024-01-16 17:14:12 +01:00
bt90 d5e1b99e6c
cmd/ursrv: Fix Arch detection (#9350)
### Purpose

Classify `syncthing@archlinux` as `Arch (3rd party)`
2024-01-16 17:13:34 +01:00
Jakob Borg 3297624037
lib/ignore: Optimise ignoring directories for filesystem watcher (fixes #9339) (#9340)
This improves the ignore handling so that directories can be fully
ignored (skipped in the watcher) in more cases. Specifically, where the
previous rule was that any complex `!`-pattern would disable skipping
directories, the new rule is that only matches on patterns *after* such
a `!`-pattern disable skipping. That is, the following now does the
intuitive thing:

```
/foo
/bar
!whatever
*
```

- `/foo/**` and `/bar/**` are completely skipped, since there is no
chance anything underneath them could ever be not-ignored
- `!whatever` toggles the "can't skip directories any more" flag
- Anything that matches `*` can't skip directories, because it's
possible we can have `whatever` match something deeper.

To enable this, some refactoring was necessary:

- The "can skip dirs" flag is now a property of the match result, not of
the pattern set as a whole.
- That meant returning a boolean is not good enough, we need to actually
return the entire `Result` (or, like, two booleans but that seemed
uglier and more annoying to use)
- `ShouldIgnore(string) boolean` went away with
`Match(string).IsIgnored()` being the obvious replacement (API
simplification!)
- The watcher then needed to import the `ignore` package (for the
`Result` type), but `fs` imports the watcher and `ignore` imports `fs`.
That's a cycle, so I broke out `Result` into a package of its own so
that it can be safely imported everywhere in things like `type Matcher
interface { Match(string) result.Result }`. There's a fair amount of
stuttering in `result.Result` and maybe we should go with something like
`ignoreresult.R` or so, leaving this open for discussion.

Tests refactored to suit, I think this change is in fact quite well
covered by the existing ones...

Also some noise because a few of the changed files were quite old and
got the `gofumpt` treatment by my editor. Sorry not sorry.

---------

Co-authored-by: Simon Frei <freisim93@gmail.com>
2024-01-15 10:13:22 +00:00
Syncthing Release Automation 445e8cc532 gui, man, authors: Update docs, translations, and contributors 2024-01-15 03:45:19 +00:00
Jakob Borg e041877488
lib/ignore: Refactor out result type (#9343) 2024-01-13 18:58:23 +01:00
Jakob Borg 36e08f8eee build: Testing infra images for infra-* branches 2024-01-13 11:24:59 +01:00
nf 8b321387c0
lib/versioner: Expand tildes in version directory (fixes #9241) (#9327)
### Purpose

Fix #9241 by expanding tildes in version paths.

When creating the versioner file system, first try to expand any leading
tildes to the user's home directory before handling relative paths. This
makes a version path `"~/p"` expand to `"$HOME/p"` instead of
`"/folder/~/p"`.

### Testing

Added a test to lib/versioner that exercises this code path. Also
manually tested with local syncthing instances.
2024-01-12 10:46:18 +01:00
Julian Lehrhuber 8edd67a569
lib/scanner: Prevent sync-conflict for receive-only local modifications (#9323)
### Purpose

This PR changes behaviour of syncthing related to `receive-only`
folders, which I believe to be a bug since I wouldn't expect the current
behaviour. With the current syncthing codebase, a file of a
`receive-only` folder that is only modified locally can cause the
creation of a `.sync-conflict` file.

### Testing

Consider this szenario: Setup two paired clients that sync a folder with
a given file (e.g. `Test.txt`). One of the clients configures the folder
to be `receive-only`. Now, change the contents of the file for the
receive-only client **_twice_**.

With the current syncthing codebase, this leads to the creation of a
`.sync-conflict` file that contains the modified contents, while the
regular `Test.txt` file is reset to the cluster's provided contents.
This is due to a `protocol.FileInfo#ShouldConflict` check, that is
succeeding on the locally modified file.

This PR changes this behaviour to not reset the file and not cause the
creation of a `.sync-conflict`. Instead, the second content update is
treated the same as the first content update.

This PR also contains a test that fails on the current codebase and
succeeds with the changes introduced in this PR.

### Screenshots

This is not a GUI change

### Documentation

This is not a user visible change.

## Authorship

Your name and email will be added automatically to the AUTHORS file
based on the commit metadata.

#### Thanks to all the syncthing folks for this awesome piece of
software!
2024-01-08 10:29:20 +01:00
Syncthing Release Automation e829a63295 gui, man, authors: Update docs, translations, and contributors 2024-01-08 03:45:23 +00:00
diemade 6881a6897a
Fix website security link in README.md (#9325)
https://syncthing.net/security/
html file does not exist

### Purpose
Fixing link that is supposed to link to https://syncthing.net/security/
2024-01-06 23:58:56 +01:00
Daniel Padrta 9387e107b3
cmd/syncthing: Add CLI completion functionality (fixes #8616) (#9226)
### Purpose

This implements CLI completion using the Kongplete module. As a side
effect a CLI structure for syncthing/cli was created for kongplete to be
able to parse and implement CLI completion.

### Testing

I've tested the autocompletion manually, and it had worked, but I hadn't
added any tests so as to test it automatically. Additionally, I ran `go
run build.go test` with all tests passing.
2024-01-04 10:20:53 +00:00
Jakob Borg aa901790b9
lib/api: Save session & CSRF tokens to database, add option to stay logged in (fixes #9151) (#9284)
This adds a "token manager" which handles storing and checking expired
tokens, used for both sessions and CSRF tokens. It removes the old,
corresponding functionality for CSRFs which saved things in a file. The
result is less crap in the state directory, and active login sessions
now survive a Syncthing restart (this really annoyed me).

It also adds a boolean on login to create a longer-lived session cookie,
which is now possible and useful. Thus we can remain logged in over
browser restarts, which was also annoying... :)

<img width="1001" alt="Screenshot 2023-12-12 at 09 56 34"
src="https://github.com/syncthing/syncthing/assets/125426/55cb20c8-78fc-453e-825d-655b94c8623b">

Best viewed with whitespace-insensitive diff, as a bunch of the auth
functions became methods instead of closures which changed indentation.
2024-01-04 10:07:12 +00:00
Jakob Borg 17df4b8634
Update dependencies (#9321)
```
% export GOTOOLCHAIN=go1.20.7
% go list -m all | cut -d ' ' -f 1 | xargs go get -u
% go mod tidy
```

Except:

- github.com/jackpal/gateway now requires Go 1.21
- github.com/shirou/gopsutil breaks linux-mips in latest version
2024-01-04 10:56:11 +01:00
tomasz1986 34ef30dd5c
gui: Always inform about loading data in Restore Versions modal (#9317)
Currently, with a large number of versioned files, there is a delay
between the Restore Versions modal showing up on the screen and
initialisation of the actual versions tree. This leads to a situation,
where the modal is initially empty, which confuses the user, making
them think that something is not working correctly.

To avoid the above, always show the loading data information. The string
is displayed using static HTML first, and then replaced with the exact
same content once the tree has been initialised. Both elements use the
same style and position, so there is no visual shift between the two.

Signed-off-by: Tomasz Wilczyński <twilczynski@naver.com>
2024-01-03 12:37:51 +01:00
Peter Badida fc1c7a3c49
lib/build: Allow semver build in version regex (fixes #9267) (#9316) 2024-01-02 20:43:22 +01:00
Peter Badida 2abfefc18c
gui: Keep short deviceID length consistent + xrefs (fixes #9313) (#9314)
Making short deviceID length consistent and referencing to protocol file
for future-proof edits. Closes #9313.
2024-01-02 17:31:57 +01:00
dependabot[bot] 86a08eb87d
build(deps): bump actions/download-artifact from 3 to 4 (#9294)
Bumps
[actions/download-artifact](https://github.com/actions/download-artifact)
from 3 to 4.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/actions/download-artifact/releases">actions/download-artifact's
releases</a>.</em></p>
<blockquote>
<h2>v4.0.0</h2>
<h2>What's Changed</h2>
<p>The release of upload-artifact@v4 and download-artifact@v4 are major
changes to the backend architecture of Artifacts. They have numerous
performance and behavioral improvements.</p>
<p>For more information, see the <a
href="https://github.com/actions/toolkit/tree/main/packages/artifact"><code>@​actions/artifact</code></a>
documentation.</p>
<h2>New Contributors</h2>
<ul>
<li><a href="https://github.com/bflad"><code>@​bflad</code></a> made
their first contribution in <a
href="https://redirect.github.com/actions/download-artifact/pull/194">actions/download-artifact#194</a></li>
</ul>
<p><strong>Full Changelog</strong>: <a
href="https://github.com/actions/download-artifact/compare/v3...v4.0.0">https://github.com/actions/download-artifact/compare/v3...v4.0.0</a></p>
<h2>v3.0.2</h2>
<ul>
<li>Bump <code>@actions/artifact</code> to v1.1.1 - <a
href="https://redirect.github.com/actions/download-artifact/pull/195">actions/download-artifact#195</a></li>
<li>Fixed a bug in Node16 where if an HTTP download finished too quickly
(&lt;1ms, e.g. when it's mocked) we attempt to delete a temp file that
has not been created yet <a
href="hhttps://redirect.github.com/actions/toolkit/pull/1278">actions/toolkit#1278</a></li>
</ul>
<h2>v3.0.1</h2>
<ul>
<li><a
href="https://redirect.github.com/actions/download-artifact/pull/178">Bump
<code>@​actions/core</code> to 1.10.0</a></li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="7a1cd3216c"><code>7a1cd32</code></a>
Merge pull request <a
href="https://redirect.github.com/actions/download-artifact/issues/246">#246</a>
from actions/v4-beta</li>
<li><a
href="8f32874a49"><code>8f32874</code></a>
licensed cache</li>
<li><a
href="b5ff8444b1"><code>b5ff844</code></a>
Merge pull request <a
href="https://redirect.github.com/actions/download-artifact/issues/245">#245</a>
from actions/robherley/v4-documentation</li>
<li><a
href="f07a0f73f5"><code>f07a0f7</code></a>
Update README.md</li>
<li><a
href="7226129829"><code>7226129</code></a>
update test workflow to use different artifact names for matrix</li>
<li><a
href="ada9446619"><code>ada9446</code></a>
update docs and bump <code>@​actions/artifact</code></li>
<li><a
href="7eafc8b729"><code>7eafc8b</code></a>
Merge pull request <a
href="https://redirect.github.com/actions/download-artifact/issues/244">#244</a>
from actions/robherley/bump-toolkit</li>
<li><a
href="3132d12662"><code>3132d12</code></a>
consume latest toolkit</li>
<li><a
href="5be1d38671"><code>5be1d38</code></a>
Merge pull request <a
href="https://redirect.github.com/actions/download-artifact/issues/243">#243</a>
from actions/robherley/v4-beta-updates</li>
<li><a
href="465b526e63"><code>465b526</code></a>
consume latest <code>@​actions/toolkit</code></li>
<li>Additional commits viewable in <a
href="https://github.com/actions/download-artifact/compare/v3...v4">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=actions/download-artifact&package-manager=github_actions&previous-version=3&new-version=4)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)


</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-01-01 11:09:23 +01:00
dependabot[bot] d330d65859
build(deps): bump actions/upload-artifact from 3 to 4 (#9293)
Bumps
[actions/upload-artifact](https://github.com/actions/upload-artifact)
from 3 to 4.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/actions/upload-artifact/releases">actions/upload-artifact's
releases</a>.</em></p>
<blockquote>
<h2>v4.0.0</h2>
<h2>What's Changed</h2>
<p>The release of upload-artifact@v4 and download-artifact@v4 are major
changes to the backend architecture of Artifacts. They have numerous
performance and behavioral improvements.</p>
<p>For more information, see the <a
href="https://github.com/actions/toolkit/tree/main/packages/artifact"><code>@​actions/artifact</code></a>
documentation.</p>
<h2>New Contributors</h2>
<ul>
<li><a href="https://github.com/vmjoseph"><code>@​vmjoseph</code></a>
made their first contribution in <a
href="https://redirect.github.com/actions/upload-artifact/pull/464">actions/upload-artifact#464</a></li>
</ul>
<p><strong>Full Changelog</strong>: <a
href="https://github.com/actions/upload-artifact/compare/v3...v4.0.0">https://github.com/actions/upload-artifact/compare/v3...v4.0.0</a></p>
<h2>v3.1.3</h2>
<h2>What's Changed</h2>
<ul>
<li>chore(github): remove trailing whitespaces by <a
href="https://github.com/ljmf00"><code>@​ljmf00</code></a> in <a
href="https://redirect.github.com/actions/upload-artifact/pull/313">actions/upload-artifact#313</a></li>
<li>Bump <code>@​actions/artifact</code> version to v1.1.2 by <a
href="https://github.com/bethanyj28"><code>@​bethanyj28</code></a> in <a
href="https://redirect.github.com/actions/upload-artifact/pull/436">actions/upload-artifact#436</a></li>
</ul>
<p><strong>Full Changelog</strong>: <a
href="https://github.com/actions/upload-artifact/compare/v3...v3.1.3">https://github.com/actions/upload-artifact/compare/v3...v3.1.3</a></p>
<h2>v3.1.2</h2>
<ul>
<li>Update all <code>@actions/*</code> NPM packages to their latest
versions- <a
href="https://redirect.github.com/actions/upload-artifact/issues/374">#374</a></li>
<li>Update all dev dependencies to their most recent versions - <a
href="https://redirect.github.com/actions/upload-artifact/issues/375">#375</a></li>
</ul>
<h2>v3.1.1</h2>
<ul>
<li>Update actions/core package to latest version to remove
<code>set-output</code> deprecation warning <a
href="https://redirect.github.com/actions/upload-artifact/issues/351">#351</a></li>
</ul>
<h2>v3.1.0</h2>
<h2>What's Changed</h2>
<ul>
<li>Bump <code>@​actions/artifact</code> to v1.1.0 (<a
href="https://redirect.github.com/actions/upload-artifact/pull/327">actions/upload-artifact#327</a>)
<ul>
<li>Adds checksum headers on artifact upload (<a
href="https://redirect.github.com/actions/toolkit/pull/1095">actions/toolkit#1095</a>)
(<a
href="https://redirect.github.com/actions/toolkit/pull/1063">actions/toolkit#1063</a>)</li>
</ul>
</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="c7d193f32e"><code>c7d193f</code></a>
Merge pull request <a
href="https://redirect.github.com/actions/upload-artifact/issues/466">#466</a>
from actions/v4-beta</li>
<li><a
href="13131bb095"><code>13131bb</code></a>
licensed cache</li>
<li><a
href="4a6c273b98"><code>4a6c273</code></a>
Merge branch 'main' into v4-beta</li>
<li><a
href="f391bb91a3"><code>f391bb9</code></a>
Merge pull request <a
href="https://redirect.github.com/actions/upload-artifact/issues/465">#465</a>
from actions/robherley/v4-documentation</li>
<li><a
href="9653d03c4b"><code>9653d03</code></a>
Apply suggestions from code review</li>
<li><a
href="875b630764"><code>875b630</code></a>
add limitations section</li>
<li><a
href="ecb21463e9"><code>ecb2146</code></a>
add compression example</li>
<li><a
href="5e7604f84a"><code>5e7604f</code></a>
trim some repeated info</li>
<li><a
href="d6437d0758"><code>d6437d0</code></a>
naming</li>
<li><a
href="1b56155703"><code>1b56155</code></a>
s/v4-beta/v4/g</li>
<li>Additional commits viewable in <a
href="https://github.com/actions/upload-artifact/compare/v3...v4">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=actions/upload-artifact&package-manager=github_actions&previous-version=3&new-version=4)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)


</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-01-01 11:08:55 +01:00
Syncthing Release Automation 4e1831fa3a gui, man, authors: Update docs, translations, and contributors 2024-01-01 03:45:30 +00:00
Simon Frei 2f3eacdb6c
gui, lib/scanner: Improve scan progress indication (ref #8331) (#9308) 2023-12-31 23:01:16 +01:00
Sven Bachmann 1ce2af1238
lib/protocol: handle empty names in unixOwnershipEqual (fixes #9039) (#9306)
If syncOwnership is enabled and the remote uses for example a dockerized
Syncthing it can't fetch the ownername and groupname of the local
instance. Without this patch this led to an endless cycle of detected
changes on the remote and failing re-sync attempts.

This patch skips comparing the ownername and groupname if they zare empty
on one side.

See https://github.com/syncthing/syncthing/issues/9039 for details.

### Testing

Proposed by @calmh in
https://github.com/syncthing/syncthing/issues/9039#issuecomment-1870584783
and tested locally in my setup,

Setup PC 1:
- Syncthing is run in Docker as user `root` and has none of the users
configured that synchronize their files

Setup PC 2:
  - this PC has all users locally setup
- Syncthing runs as `systemd` service as user `syncthing` and has
multiple capabilities set to set the correct owner and permissions

Setup PC 3:
  - same as PC 2

Handling:
- `PC 1` is send & receive and uses just the `UID` and `GID` identifiers
to store the files
- `PC 2` and `PC 3` synchronize their files over `PC 1` but not directly
to each other

Outcome:
- `PC 2` and `PC 3` should send and receive their files with the correct
ownership and groups from `PC 1`
2023-12-29 09:16:33 +01:00
Syncthing Release Automation 683b48182c gui, man, authors: Update docs, translations, and contributors 2023-12-25 03:45:16 +00:00
Sertonix 795aed306c
etc/linux-desktop: use double dash for long options (#9301)
Use a style for options that is consistent with the documentation and
other uses.
2023-12-23 11:16:47 +01:00
greatroar cdefa535ed
lib/connections: Skip allocation in check for missing port (#9297)
Micro-optimization. Already has unit tests.
2023-12-20 11:59:11 +01:00
gudvinr 91084b83b4
lib/upgrade: Extract signing key to embedded file (fixes #9247) (#9296)
### Purpose

Instead of hardcoding `SigningKey` as text use `go:embed`. Fixes #9247.

### Testing

* Building syncthing
* Trying to upgrade (signature verification)
2023-12-18 19:47:57 +00:00
Syncthing Release Automation 5360e7153b gui, man, authors: Update docs, translations, and contributors 2023-12-18 03:45:20 +00:00
Jakob Borg 5d0ca19350 build: Update quic-go (fixes #9287) 2023-12-13 12:35:59 +01:00
Eric P e8d3529fed
lib/model: Only handle relevant folder summaries (kqueue) (fixes #9183) (#9288)
On kqueue-systems, folders listen for folder summaries to (be able to)
warn for potential high resource usage. However, it listened for any
folder summary and not for the summary which matches the folder it's
about. This could cause that an unwatched folder causes a folder summary
containing more files than the threshold (10k), and the listening folder
(with the watcher enabled) triggers the warning.

This makes sure that only the folder summaries which are relevant to the
specific folder are being handled.

### Testing

- Fire up some kqueue-system (freebsd, I used).
- add folder A, disable the watcher, add 10001 files
- add folder B with the watcher enabled, no files are needed here

Before the change:
- add an item to folder A, trigger a rescan to speed up the process
- wait some seconds...warning triggered by folder B's
summarySubscription

After the change:
- Only a warning is triggered if the received folder summary matches the
folder which listens for the summaries
2023-12-13 12:34:24 +01:00
Jakob Borg 935a28c961
lib/model: Use a single lock (phase two: cleanup) (#9276)
Cleanup after #9275.

This renames `fmut` -> `mut`, removes the deadlock detector and
associated plumbing, renames some things from `...PRLocked` to
`...RLocked` and similar, and updates comments.

Apart from the removal of the deadlock detection machinery, no
functional code changes... i.e. almost 100% diff noise, have fun
reviewing.
2023-12-11 22:06:45 +01:00
dependabot[bot] d21a2de055
build(deps): bump actions/setup-go from 4 to 5 (#9279)
Bumps [actions/setup-go](https://github.com/actions/setup-go) from 4 to
5.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/actions/setup-go/releases">actions/setup-go's
releases</a>.</em></p>
<blockquote>
<h2>v5.0.0</h2>
<h2>What's Changed</h2>
<p>In scope of this release, we change Nodejs runtime from node16 to
node20 (<a
href="https://redirect.github.com/actions/setup-go/pull/421">actions/setup-go#421</a>).
Moreover, we update some dependencies to the latest versions (<a
href="https://redirect.github.com/actions/setup-go/pull/445">actions/setup-go#445</a>).</p>
<p>Besides, this release contains such changes as:</p>
<ul>
<li>Fix hosted tool cache usage on windows by <a
href="https://github.com/galargh"><code>@​galargh</code></a> in <a
href="https://redirect.github.com/actions/setup-go/pull/411">actions/setup-go#411</a></li>
<li>Improve documentation regarding dependencies caching by <a
href="https://github.com/artemgavrilov"><code>@​artemgavrilov</code></a>
in <a
href="https://redirect.github.com/actions/setup-go/pull/417">actions/setup-go#417</a></li>
</ul>
<h2>New Contributors</h2>
<ul>
<li><a href="https://github.com/galargh"><code>@​galargh</code></a> made
their first contribution in <a
href="https://redirect.github.com/actions/setup-go/pull/411">actions/setup-go#411</a></li>
<li><a
href="https://github.com/artemgavrilov"><code>@​artemgavrilov</code></a>
made their first contribution in <a
href="https://redirect.github.com/actions/setup-go/pull/417">actions/setup-go#417</a></li>
<li><a
href="https://github.com/chenrui333"><code>@​chenrui333</code></a> made
their first contribution in <a
href="https://redirect.github.com/actions/setup-go/pull/421">actions/setup-go#421</a></li>
</ul>
<p><strong>Full Changelog</strong>: <a
href="https://github.com/actions/setup-go/compare/v4...v5.0.0">https://github.com/actions/setup-go/compare/v4...v5.0.0</a></p>
<h2>v4.1.0</h2>
<h2>What's Changed</h2>
<p>In scope of this release, slow installation on Windows was fixed by
<a href="https://github.com/dsame"><code>@​dsame</code></a> in <a
href="https://redirect.github.com/actions/setup-go/pull/393">actions/setup-go#393</a>
and OS version was added to <code>primaryKey</code> for Ubuntu runners
to avoid conflicts (<a
href="https://redirect.github.com/actions/setup-go/pull/383">actions/setup-go#383</a>)</p>
<p>This release also includes the following changes:</p>
<ul>
<li>Remove implicit dependencies by <a
href="https://github.com/nikolai-laevskii"><code>@​nikolai-laevskii</code></a>
in <a
href="https://redirect.github.com/actions/setup-go/pull/378">actions/setup-go#378</a></li>
<li>Update action.yml by <a
href="https://github.com/mkelly"><code>@​mkelly</code></a> in <a
href="https://redirect.github.com/actions/setup-go/pull/379">actions/setup-go#379</a></li>
<li>Added a description that go-version should be specified as a string
type by <a href="https://github.com/n3xem"><code>@​n3xem</code></a> in
<a
href="https://redirect.github.com/actions/setup-go/pull/367">actions/setup-go#367</a></li>
<li>Add note about YAML parsing versions by <a
href="https://github.com/dmitry-shibanov"><code>@​dmitry-shibanov</code></a>
in <a
href="https://redirect.github.com/actions/setup-go/pull/382">actions/setup-go#382</a></li>
<li>Automatic update of configuration files from 05/23/2023 by <a
href="https://github.com/github-actions"><code>@​github-actions</code></a>
in <a
href="https://redirect.github.com/actions/setup-go/pull/377">actions/setup-go#377</a></li>
<li>Bump tough-cookie and <code>@​azure/ms-rest-js</code> by <a
href="https://github.com/dependabot"><code>@​dependabot</code></a> in <a
href="https://redirect.github.com/actions/setup-go/pull/392">actions/setup-go#392</a></li>
<li>Bump word-wrap from 1.2.3 to 1.2.4 by <a
href="https://github.com/dependabot"><code>@​dependabot</code></a> in <a
href="https://redirect.github.com/actions/setup-go/pull/397">actions/setup-go#397</a></li>
<li>Bump semver from 6.3.0 to 6.3.1 by <a
href="https://github.com/dependabot"><code>@​dependabot</code></a> in <a
href="https://redirect.github.com/actions/setup-go/pull/396">actions/setup-go#396</a></li>
</ul>
<h2>New Contributors</h2>
<ul>
<li><a href="https://github.com/mkelly"><code>@​mkelly</code></a> made
their first contribution in <a
href="https://redirect.github.com/actions/setup-go/pull/379">actions/setup-go#379</a></li>
<li><a href="https://github.com/n3xem"><code>@​n3xem</code></a> made
their first contribution in <a
href="https://redirect.github.com/actions/setup-go/pull/367">actions/setup-go#367</a></li>
</ul>
<p><strong>Full Changelog</strong>: <a
href="https://github.com/actions/setup-go/compare/v4...v4.1.0">https://github.com/actions/setup-go/compare/v4...v4.1.0</a></p>
<h2>v4.0.1</h2>
<h2>What's Changed</h2>
<ul>
<li>Update documentation for <code>v4</code> by <a
href="https://github.com/dsame"><code>@​dsame</code></a> in <a
href="https://redirect.github.com/actions/setup-go/pull/354">actions/setup-go#354</a></li>
<li>Fix glob bug in the package.json scripts section by <a
href="https://github.com/IvanZosimov"><code>@​IvanZosimov</code></a> in
<a
href="https://redirect.github.com/actions/setup-go/pull/359">actions/setup-go#359</a></li>
<li>Bump <code>xml2js</code> dependency by <a
href="https://github.com/dmitry-shibanov"><code>@​dmitry-shibanov</code></a>
in <a
href="https://redirect.github.com/actions/setup-go/pull/370">actions/setup-go#370</a></li>
<li>Bump <code>@actions/cache</code> dependency to v3.2.1 by <a
href="https://github.com/nikolai-laevskii"><code>@​nikolai-laevskii</code></a>
in <a
href="https://redirect.github.com/actions/setup-go/pull/374">actions/setup-go#374</a></li>
</ul>
<h2>New Contributors</h2>
<ul>
<li><a
href="https://github.com/nikolai-laevskii"><code>@​nikolai-laevskii</code></a>
made their first contribution in <a
href="https://redirect.github.com/actions/setup-go/pull/374">actions/setup-go#374</a></li>
</ul>
<p><strong>Full Changelog</strong>: <a
href="https://github.com/actions/setup-go/compare/v4...v4.0.1">https://github.com/actions/setup-go/compare/v4...v4.0.1</a></p>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="0c52d547c9"><code>0c52d54</code></a>
Update dependencies for node20 (<a
href="https://redirect.github.com/actions/setup-go/issues/445">#445</a>)</li>
<li><a
href="bfd2fb341f"><code>bfd2fb3</code></a>
Merge pull request <a
href="https://redirect.github.com/actions/setup-go/issues/421">#421</a>
from chenrui333/node20-runtime</li>
<li><a
href="3d65fa57fc"><code>3d65fa5</code></a>
feat: bump to use actions/checkout@v4</li>
<li><a
href="8a505c9cf2"><code>8a505c9</code></a>
feat: bump to use node20 runtime</li>
<li><a
href="883490dfd0"><code>883490d</code></a>
Merge pull request <a
href="https://redirect.github.com/actions/setup-go/issues/417">#417</a>
from artemgavrilov/main</li>
<li><a
href="d45ebba0ce"><code>d45ebba</code></a>
Rephrase sentence</li>
<li><a
href="317c6617fa"><code>317c661</code></a>
Replace <code>wildcards</code> term with <code>globs</code>.</li>
<li><a
href="f90673ad64"><code>f90673a</code></a>
Merge pull request <a
href="https://redirect.github.com/actions/setup-go/issues/1">#1</a> from
artemgavrilov/caching-docs-improvement</li>
<li><a
href="8018234347"><code>8018234</code></a>
Improve documentation regarding dependencies cachin</li>
<li><a
href="d085b4fe57"><code>d085b4f</code></a>
Merge pull request <a
href="https://redirect.github.com/actions/setup-go/issues/411">#411</a>
from galargh/fix/windows-hostedtoolcache</li>
<li>Additional commits viewable in <a
href="https://github.com/actions/setup-go/compare/v4...v5">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=actions/setup-go&package-manager=github_actions&previous-version=4&new-version=5)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)


</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-12-11 21:38:46 +01:00
Jakob Borg 6f1023665c
lib/model: Use a single lock (#9275)
I'm tired of the fmut/pmut shenanigans. This consolidates both under one
lock; I'm not convinced there are any significant performance
differences with this approach since we're literally just protecting map
juggling...

- The locking goes away when we were already under an appropriate fmut
lock.
- Where we had fmut.RLock()+pmut.Lock() it gets upgraded to an
fmut.Lock().
- Otherwise s/pmut/fmut/.

In order to avoid diff noise for an important change I did not do the
following cleanups, which will be filed in a PR after this one, if
accepted:

- Renaming fmut to just mut
- Renaming methods that refer to being "PRLocked" and stuff like that
- Removing the no longer relevant deadlock detector
- Comments referring to pmut and locking sequences...
2023-12-11 21:26:23 +01:00
Jakob Borg c53a1f210c
cmd/syncthing: Better cli stdin handling (ref #9166) (#9281)
Seems to work for me, @AudriusButkevicius.
2023-12-11 21:15:52 +01:00
cjc7373 b71a930bfc
cmd/syncthing: Mostly replace urfave/cli command line parser with alecthomas/kong (#9166)
`syncthing cli` subcommand was using urfave/cli as the command parser.
This PR replace it with kong, which the main command uses.

Some help texts and error message format are changed. Other than that,
all the command usage and logic remains unchanged.

There's only one place which still uses urfave/cli, which is `syncthing
cli config`, because it uses some magic to dynamically build commands
from struct reflects. I used kong's `passthrough:""` tag to pass any
argument following `syncthing cli config` to urfave/cli parser.

This PR also fixes #9041

---------

Co-authored-by: Jakob Borg <jakob@kastelo.net>
2023-12-11 11:35:57 +01:00
Jakob Borg a2cbc62521 lib/nat: Fix test build failure (ref #9010) 2023-12-11 10:13:14 +01:00
Jakob Borg 768fd6bff8 Merge branch 'release'
* release:
  lib/model: Add pmut locking for DeviceStatistics (fixes #9274)
  lib/model: Remove spurious "replacing service" failure event (ref #9271)
2023-12-11 08:13:28 +01:00
Jakob Borg 48883e0e32 lib/model: Add pmut locking for DeviceStatistics (fixes #9274)
Looking at deviceConnIDs requires this. Added in #9256.
2023-12-11 08:06:03 +01:00
Jakob Borg 30fe2cf514 lib/model: Add pmut locking for DeviceStatistics (fixes #9274)
Looking at deviceConnIDs requires this. Added in #9256.
2023-12-11 08:04:47 +01:00
Jakob Borg 3850a08252 lib/model: Remove spurious "replacing service" failure event (ref #9271)
This is no longer a notable condition, as we do this pretty much all the
time.
2023-12-11 07:44:04 +01:00
Jakob Borg d0e407f3c3 lib/model: Remove spurious "replacing service" failure event (ref #9271)
This is no longer a notable condition, as we do this pretty much all the
time.
2023-12-11 07:43:40 +01:00
Maximilian 16db6fcf3d
lib/nat, lib/upnp: IPv6 UPnP support (#9010)
This pull request allows syncthing to request an IPv6
[pinhole](https://en.wikipedia.org/wiki/Firewall_pinhole), addressing
issue #7406. This helps users who prefer to use IPv6 for hosting their
services or are forced to do so because of
[CGNAT](https://en.wikipedia.org/wiki/Carrier-grade_NAT). Otherwise,
such users would have to configure their firewall manually to allow
syncthing traffic to pass through while IPv4 users can use UPnP to take
care of network configuration already.

### Testing

I have tested this in a virtual machine setup with miniupnpd running on
the virtualized router. It successfully added an IPv6 pinhole when used
with IPv6 only, an IPv4 port mapping when used with IPv4 only and both
when dual-stack (IPv4 and IPv6) is used.

Automated tests could be added for SOAP responses from the router but
automatically testing this with a real network is likely infeasible.

### Documentation

https://docs.syncthing.net/users/firewall.html could be updated to
mention the fact that UPnP now works with IPv6, although this change is
more "behind the scenes".

---------

Co-authored-by: Simon Frei <freisim93@gmail.com>
Co-authored-by: bt90 <btom1990@googlemail.com>
Co-authored-by: André Colomb <github.com@andre.colomb.de>
2023-12-11 07:36:18 +01:00
Syncthing Release Automation 4c5528bd0e gui, man, authors: Update docs, translations, and contributors 2023-12-11 03:45:22 +00:00
tomasz1986 d42fff1016
gui: Show folder/device status on small screens (#8643)
gui: Show folder/device status on small screens

On larger screens, folder and device status is shown in a textual form
directly next to folder and device titles. However, on small screens,
only icons are currently shown, which may be ambiguous to new users, who
cannot possibly know what a specific icon means (see [1]). Thus, on
small screens only, display a new entry in folder/device info that
contains the same textual information that is shown in the title on
larger screens.

Signed-off-by: Tomasz Wilczyński <twilczynski@naver.com>
Co-authored-by: André Colomb <src@andre.colomb.de>
2023-12-10 15:14:10 +01:00
Simon Frei a28de73031
lib/model: Remove runner during folder cleanup (fixes #9269) (#9271)
Before introducing the service map and using it for folder runners, the
entries in folderCfgs and folderRunners for the same key/folder were
removed under a single lock. Stopping the folder happens separately
before that with just the read lock. Now with the service map stopping
the folder and removing it from the map is a single operation. And that
still happens with just a read-lock. However even with a full lock it’s
still problematic: After the folder stopped, the runner isn’t present
anymore while the folder-config still is and sais the folder isn't
paused.

The index handler in turn looks at the folder config that is not paused,
thus assumes the runner has to be present -> nil deref on the runner.

A better solution might be to push most of these fmut maps into the
folder - they anyway are needed in there. Then there's just a single
map/source of info that's necessarily consistent. That's quite a bit of
work though, and probably/likely there will be corner cases there too.
2023-12-08 07:13:09 +01:00
Jakob Borg 75310b58a0
build: Update dependencies (#9265) 2023-12-06 10:53:45 +01:00
Jakob Borg 8064957270
build: Revert specifics for Go 1.21.4, build using Go 1.21.5 (#9264)
This reverts commit e477777f49.

In principle, we could have stayed with `~1.21.1`, but `check-latest:
true` apparently checks some cache/manifest/something that is only
periodically refreshed and isn't aware of 1.21.5 yet. So update the
constraints to force an upgrade.

Also the infrastructure images weren't actually using the constraint
since there was no `setup-go` action...
2023-12-06 09:02:12 +01:00
Jakob Borg c1ec9a8826
lib/fs: Reduce memory usage in xattrs handling (#9251)
This reduces allocations, in number and in size, while getting extended
attributes. This is mostly noticable when there is a large number of new
files to scan and we're running with the default scanProgressInterval --
then a queue of files is built in-memory, and this queue includes
extended attributes as part of file metadata. (Arguable it shouldn't,
but that's a more difficult and involved change.)

With 1M files to scan, each with one extended attribute, current peak
memory usage looks like this:

	Showing nodes accounting for 1425.30MB, 98.19% of 1451.64MB total
	Dropped 1435 nodes (cum <= 7.26MB)
	Showing top 10 nodes out of 54
	      flat  flat%   sum%        cum   cum%
976.56MB 67.27% 67.27% 976.56MB 67.27%
github.com/syncthing/syncthing/lib/fs.getXattr
305.44MB 21.04% 88.31% 305.44MB 21.04%
github.com/syncthing/syncthing/lib/scanner.(*walker).walk.func1
45.78MB 3.15% 91.47% 1045.23MB 72.00%
github.com/syncthing/syncthing/lib/fs.(*BasicFilesystem).GetXattr
22.89MB 1.58% 93.04% 22.89MB 1.58%
github.com/syncthing/syncthing/lib/fs.listXattr
22.89MB 1.58% 94.62% 22.89MB 1.58%
github.com/syncthing/syncthing/lib/protocol.(*PlatformData).SetXattrs
16MB 1.10% 95.72% 16.01MB 1.10%
github.com/syndtr/goleveldb/leveldb/memdb.New

After the change, it's this:

	Showing nodes accounting for 502.32MB, 95.70% of 524.88MB total
	Dropped 1400 nodes (cum <= 2.62MB)
	Showing top 10 nodes out of 91
	      flat  flat%   sum%        cum   cum%
305.43MB 58.19% 58.19% 305.43MB 58.19%
github.com/syncthing/syncthing/lib/scanner.(*walker).walk.func1
45.79MB 8.72% 66.91% 68.68MB 13.09%
github.com/syncthing/syncthing/lib/fs.(*BasicFilesystem).GetXattr
32MB 6.10% 73.01% 32.01MB 6.10%
github.com/syndtr/goleveldb/leveldb/memdb.New
22.89MB 4.36% 77.37% 22.89MB 4.36%
github.com/syncthing/syncthing/lib/fs.listXattr
22.89MB 4.36% 81.73% 22.89MB 4.36%
github.com/syncthing/syncthing/lib/protocol.(*PlatformData).SetXattrs
15.35MB 2.92% 84.66% 15.36MB 2.93%
github.com/syndtr/goleveldb/leveldb/util.(*BufferPool).Get
	   15.28MB  2.91% 87.57%    15.28MB  2.91%  strings.(*Builder).grow

(The usage for xattrs is reduced from 976 MB to 68 MB)
2023-12-04 12:48:17 +01:00
Jakob Borg 1625b44892
lib/model: Improve LastSeen handling (#9256)
LastSeen for a device was only updated when they connected. This now
updates it when they disconnect, so that we remember the last time we
actually saw them. When asking the API for current stats, currently
connected devices get a last seen value of the current time.
2023-12-04 09:24:10 +01:00
Jakob Borg d51760f410
lib/scanner: Record inode change time for directories and symlinks (#9250) 2023-12-04 07:48:24 +01:00
Jakob Borg 7b1932d64e
lib/api: Improve ignore loading error handling (fixes #9253) (#9254) 2023-12-04 07:11:35 +01:00
Syncthing Release Automation 5bfc540c88 gui, man, authors: Update docs, translations, and contributors 2023-12-04 03:45:22 +00:00
Jakob Borg 4cba99fcd4 lib/fs: Better equality comparison in mtimefs 2023-12-03 16:01:46 +01:00
Jakob Borg 2ae15aa454 cmd/stcrashreceiver: Add metrics for diskstore inventory 2023-11-27 08:24:59 +01:00
Jakob Borg 47bcf4f8f4 cmd/stcrashreceiver: Minor cleanup, stricter file permissions 2023-11-27 08:24:59 +01:00
Jakob Borg a8b9096353 cmd/stcrashreceiver: Add metrics for incoming reports 2023-11-27 08:24:59 +01:00
Jakob Borg 5328380691 cmd/ursrv: Add metrics for incoming reports 2023-11-27 08:24:59 +01:00
Syncthing Release Automation 6069cf39e5 gui, man, authors: Update docs, translations, and contributors 2023-11-27 03:45:21 +00:00
Emil Lundberg 1f7d236742
gui: Specialize a special-purpose checkbox style (#9236)
### Purpose

Discovered while working on the WebAuthn credentials table in #9175:
there's a style on `td input[type="checkbox"]` that modifies margins for
all checkboxes in `<table>`s. It looks like this style is specially
tailored to the particular table that added it (PR #8734), so it should
have a correspondingly special-purpose class to not accidentally apply
it to other tables.

As best as I could tell there are only 2 instances of `<input
type="checkbox">` in `<td>`s, shown in the screenshots below.

### Testing

- Open "Actions > Logging > Debugging Facilities" and observe the
vertical spacing of the checkboxes.
- Open "Edit Folder > Advanced", check "Sync Extended Attributes" or
"Send Extended Attributes", click "Add filter entry" and observe the
vertical spacing of the checkbox that appears.

### Screenshots

#### Before

![Logs > Debugging
Facilities](https://github.com/syncthing/syncthing/assets/1367758/998fc66d-a0ad-41d9-a476-7a2b3da622d1)
![Add filter
entry](https://github.com/syncthing/syncthing/assets/1367758/647cb565-fcd0-4a81-a6ca-1f75137039b0)

#### After

Logs > Debugging Facilities now more compact:
![Logs > Debugging Facilities now
](https://github.com/syncthing/syncthing/assets/1367758/7cf8fc77-610e-4b4a-be21-c50d30be7bb9)

Add filter entry unchanged:
![Add filter
entry](https://github.com/syncthing/syncthing/assets/1367758/0ba710d6-cee1-49b4-92bc-acfc0c22c2bd)
2023-11-21 10:00:18 +01:00
Syncthing Release Automation 8e9ee3fbe8 gui, man, authors: Update docs, translations, and contributors 2023-11-20 08:49:38 +00:00
Jakob Borg 35b0afc131 build: Support new nested namespaces in Weblate downloads 2023-11-20 09:47:08 +01:00
Simon Frei 958ff67ccc
lib/model: Acquire fmut lock in ensureIndexHandler (fixes #9234) (#9235)
Towards the end of the function f.folderCfgs and f.folderRunners are
read without the lock.
2023-11-20 08:12:25 +01:00
tomasz1986 13d9317a38
gui: Allow to translate "unknown device" (#9229)
The string is currently hard-coded in English, so allow to translate it
into other languages. It appears mainly in the browser title bar before
logging in into the GUI or when the GUI is still loading.

Signed-off-by: Tomasz Wilczyński <twilczynski@naver.com>
2023-11-16 22:07:15 +01:00
Anatoli Babenia b184d46d8a
cmd/ursrv: Add link to source code (#9224)
To see that https://data.syncthing.net is open source, study the code
and change it.
2023-11-15 08:51:23 +00:00
Jakob Borg 3f32c5cb4b cmd/ursrv: Anchor distribution expressions to avoid mismatches (ref #9141) 2023-11-15 09:32:46 +01:00
Jakob Borg 5c65a1bc83 build: Ursrv image for infrastructure 2023-11-15 08:48:00 +01:00
Jakob Borg d0a6dc5b13 cmd/ursv: Report on copy range method 2023-11-15 08:48:00 +01:00
Jakob Borg 439c6c5b7c
lib/api: Add cache busting for basic auth (ref #9208) (#9215)
This adds our short device ID to the basic auth realm. This has at least
two consequences:

- It is different from what's presented by another device on the same
address (e.g., if I use SSH forwards to different dives on the same
local address), preventing credentials for one from being sent to
another.

- It is different from what we did previously, meaning we avoid cached
credentials from old versions interfering with the new login flow.

I don't *think* there should be things that depend on our precise realm
string, so this shouldn't break any existing setups...

Sneakily this also changes the session cookie and CSRF name, because I
think `id.Short().String()` is nicer than `id.String()[:5]` and the
short ID is two characters longer. That's also not a problem...
2023-11-14 11:57:39 +01:00
Jakob Borg aaee0c126b cmd/stdiscorv: Expose build info in metrics 2023-11-14 09:31:53 +01:00
André Colomb f3bd4d71de
gui: Fix Weblate merge conflict (#9222)
The manual translation updates in the recently merged PR #9220 caused
conflicts with the existing (test) entries on Weblate. This will fix it.
2023-11-14 08:21:40 +01:00
Jakob Borg 876d056705
build: Fixup Docker changes from previous (#9223) 2023-11-14 08:17:34 +01:00
DerRockWolf e988978fa1
Add `org.opencontainers.image.source` to Dockerfiles (#9211)
### Purpose

The OCI image spec specifies well-defined
[annotations](https://github.com/opencontainers/image-spec/blob/main/annotations.md)
that can be added to images.
Theses annotations can then be used by other tools to gather more
information of an image.

This PR adds the `org.opencontainers.image.source` to allow tools such
as [renovate](https://github.com/renovatebot/renovate) to find the
release notes of a give version.

~~I've only done this change for `Dockerfile`. Should I also add the
label to the other dockerfiles?~~
I've now added the source annotations to all `Dockerfile`s & action
workflows.

### Testing

None, change was done by following the [renovate
documentation](https://docs.renovatebot.com/modules/datasource/docker/).
2023-11-14 07:46:14 +01:00
Jakob Borg d5deede7a1 build: Update dependencies 2023-11-14 07:41:53 +01:00
André Colomb 4f70f5c280
gui: Use nested namespace for theme name translation keys (#9220)
Following up on #9192, this makes use of the new mechanism for the theme
names.

The dummy string added for testing is removed again here. All
translations are updated to the new nested syntax, except Chinese
(zh-HK) where the string weren't actually translated.
2023-11-14 07:22:52 +01:00
Emil Lundberg a1ad020b63
Support explicit translation ID and dotted namespaces in translation extraction (#9192)
Some translations, especially single words or other short
labels for buttons and the like, may not be transferable between
contexts even if they happen to be equal in English. In these cases,
setting an explicit translation ID is important for context separation.
Angular Translate also supports nested JSON in translation tables,
addressed using `.` as namespace separator; this enhancement makes use
of this when extracting translation with an explicit translation ID.
2023-11-13 21:04:24 +01:00
Jakob Borg 8f1b0df74b
lib/api: Improve cookie handling (fixes #9208) (#9214) 2023-11-13 20:37:29 +01:00
Jakob Borg 0f8dc6c1d3 test: Update testing configs
Somewhere along the way I snuck in a change to the test configs that is
quite annoying. This reverts that back to the more usual setup it was
before.
2023-11-13 13:51:17 +01:00
Jakob Borg 8ae9db3b2d
build: Use actual Go version as cache key (#9216)
We use `env.GO_VERSION` as cache key for the build cache, but this is
nowadays typically something like `~1.21.1` which doesn't change when
1.21.2, 1.21.3 etc are released, making the cache fairly useless as
everything gets rebuilt. This re-sets the `GO_VERSION` variable after
installing Go so that it contains the actual installed version.
2023-11-13 12:20:40 +01:00
Jakob Borg e477777f49
build: Version constraint to avoid Go 1.21.4 on Windows (ref #9207) (#9213) 2023-11-13 09:52:59 +00:00
Syncthing Release Automation 7a132bdf24 gui, man, authors: Update docs, translations, and contributors 2023-11-13 03:45:25 +00:00
Jakob Borg 5e2b7825dc cmd/stdiscosrv: Metric for returned retry-after 2023-11-08 12:18:59 +01:00
Jakob Borg 6d30c109e4 build: Push to Docker :edge tag for infrastructure builds 2023-11-08 12:18:59 +01:00
Jakob Borg 58bd931d90 cmd/stdiscosrv: Account IPv4 & IPv6 2023-11-08 12:18:59 +01:00
vapatel2 854499382e
cmd/stdiscosrv: Prevent nil IPs from X-Forwarded-For (fixes #9189) (#9190)
### Purpose

Treat X-Forwarded-For as a comma-separated string to prevent nil IP being returned by the Discovery Server

### Testing

Unit Tests implemented

Testing with a Discovery Client can be done as follows:
```
A simple example to replicate this entails running Discovery with HTTP, use Nginx as a reverse proxy and hardcode (as an example) a list of IPs in the X-Forwarded-For header.
1. Send an Announcement with tcp://0.0.0.0:<some-port>
2. Query the DeviceID
3. Observe the returned IP Address is no longer nil; i.e.  `tcp://<nil>:<some-port>`
```
2023-11-08 11:10:23 +00:00
Jakob Borg cb4c1f9ad2
build: Update dependencies (#9202) 2023-11-06 16:43:11 -08:00
Catfriend1 b452fb3ad2
gui: Add id attribute to login button, allows form filling tools to be used (fixes #9200) (#9201)
Add an id attribute to the submit button shown on the login form. This
allows my password manager's form filling function to interact with the
button after filling in username and password (which already have the id
attribute in place).
2023-11-06 16:30:19 -08:00
Syncthing Release Automation c17a1fea77 gui, man, authors: Update docs, translations, and contributors 2023-11-06 03:45:25 +00:00
Syncthing Release Automation d50511c5c6 gui, man, authors: Update docs, translations, and contributors 2023-10-30 03:45:44 +00:00
Jakob Borg bae6d5f375
lib/location: Fix regression of timestamp handling (ref #9180) (#9185) 2023-10-26 07:41:02 +02:00
Jakob Borg b5082f6af8
lib/locations: Change default config/data location to new XDG recommendation (fixes #9178, fixes #9179) (#9180)
This makes the new default $XDG_STATE_HOME/syncthing or
~/.local/state/syncthing, while still looking in legacy locations first
for existing installs.

Note that this does not *move* existing installs, and nor should we.
Existing paths will continue to be used as-is, but the user can move the
dir into the new place if they want to use it (as they could prior to
this change as well, for that matter).

### Documentation

Needs update to the config docs about our default locations.
2023-10-25 11:16:24 +02:00
dependabot[bot] 9666e9701b
build(deps): bump github.com/quic-go/quic-go from 0.39.0 to 0.39.1 (#9181) 2023-10-24 07:23:38 +02:00
Syncthing Release Automation 86e1c5ff18 gui, man, authors: Update docs, translations, and contributors 2023-10-23 03:45:41 +00:00
tomasz1986 16ae1fbe5e
lib/fs: Ignore inode change time on Android (#9177)
lib/fs: Fix conflicts on Android due to fluctuating inode change time

[1] added inode change time to file info in order to support syncing
extended attributes. However, in the case of Android, this inode change
time fluctuates, leading to unexpected conflicts even when the user has
not even touched the files on the Android device itself. Thus, in order
to prevent those conflicts from happening, do not write inode change
time on Android.

[1] 6cac308bcd

Signed-off-by: Tomasz Wilczyński <twilczynski@naver.com>
2023-10-21 08:24:29 +02:00
Jakob Borg 11f508d9be build: Post build logs to Syncthing Loki 2023-10-16 11:07:40 +02:00
Jakob Borg 9ce6a73f42 Revert "cmd/stcrashreceiver: Aggregate slice out of bounds errors"
This reverts commit dc6a10dff4.
2023-10-16 08:08:23 +02:00
Syncthing Release Automation c5a991cf0a gui, man, authors: Update docs, translations, and contributors 2023-10-16 03:45:30 +00:00
Emil Lundberg 14569f12d3
Hide log out button when auth is not enabled (#9158)
This was an oversight in #8757: the new "Log out" button is always shown
in the "Actions" menu, even when authentication is not enabled.
2023-10-15 14:10:41 +02:00
Jakob Borg a405c21ebb cmd/stdiscosrv: Only attempt unescaping when there are %-encodings in the header (fixes #9143) 2023-10-14 12:30:29 +02:00
Jakob Borg dc6a10dff4 cmd/stcrashreceiver: Aggregate slice out of bounds errors 2023-10-14 12:19:55 +02:00
Jakob Borg d4c2acf6f6 cmd/stcrashreceiver: Propagate synthetic user ID for crashes 2023-10-14 12:19:55 +02:00
Jakob Borg 483ecada80 build: Update dependencies 2023-10-14 12:18:36 +02:00
Eric P 9553365d31
lib/fs: Properly handle Windows deduplicated files (fixes #9120) (#9168)
### Purpose

Deduplicated files are apparently considered 'irregular' under the hood,
this causes them to simply be ignored by Syncthing. This change is more
of a workaround than a proper fix, as the fix should probably happen in
the underlying libraries? - which may take some time. In the meanwhile,
this change should make deduplicated files be treated as regular files
and be indexed and synced as they should.

### Testing

Create some volume where deduplication is turned on (see the relevant
issue for details, including a proper description of how to reproduce
it). Prior to this change, the deduplicated files were simply ignored
(even by the indexer). After this change, the deduplicated files are
being index and synced properly.
2023-10-11 14:40:55 +02:00
orangekame3 5eb20580b1
cmd/ursrv: Replace "2006-01-02" with time.DateOnly (#9157)
This commit replaces "2006-01-02" to time.DateOnly. time.DateOnly is
introduced since Go1.20
2023-10-11 10:32:19 +00:00
Emil Lundberg ea1ea366d2 lib/api: Check basic auth (and set session cookie) before noauth exceptions (#9159)
This is motivated by the Android app:
https://github.com/syncthing/syncthing-android/pull/1982#issuecomment-1752042554

The planned fix in response to basic auth behaviour changing in #8757
was to add the `Authorization` header when opening the WebView, but it
turns out the function used only applies the header to the initial page
load, not any subsequent script loads or AJAX calls. The
`basicAuthAndSessionMiddleware` checks for no-auth exceptions before
checking the `Authorization` header, so the header has no effect on the
initial page load since the `/` path is a no-auth exception. Thus the
Android app fails to log in when opening the WebView.

This changes the order of checks in `basicAuthAndSessionMiddleware` so
that the `Authorization` header is always checked if present, and a
session cookie is set if it is valid. Only after that does the
middleware fall back to checking for no-auth exceptions.

`api_test.go` has been expanded with additional checks:
- Check that a session cookie is set whenever correct basic auth is
provided.
- Check that a session cookie is not set when basic auth is incorrect.
- Check that a session cookie is not set when authenticating with an API
token (either via `X-Api-Key` or `Authorization: Bearer`).

And an additional test case:
- Check that requests to `/` always succeed, but receive a session
cookie when correct basic auth is provided.

I have manually verified that
- The new assertions fail if the `createSession` call is removed in
`basicAuthAndSessionMiddleware`.
- The new test cases in e6e4df4d7034302b729ada6d91cff6e2b29678da fail
before the change in 0e47d37e738d4c15736c496e01cd949afb372e71 is
applied.
2023-10-10 07:48:55 +02:00
Syncthing Release Automation 6e4574a9f7 gui, man, authors: Update docs, translations, and contributors 2023-10-09 03:45:35 +00:00
Jakob Borg 3d0da5ac60
lib/api: Better handle %s templates in LDAP strings (fixes #9072) (#9155)
Also add some escaping for good measure.
2023-10-07 02:29:53 +00:00
Jakob Borg 9f8e6966d8
docker: Allow start even if chown fails (fixes #9133) (#9152) 2023-10-07 02:12:07 +00:00
Jakob Borg a64ae36bcc
lib/model: Verify versioning on configuration reload (fixes #9106) (#9154) 2023-10-07 04:09:51 +02:00
Jakob Borg 690b55360f
cmd/stdiscosrv: Handle unescaped cert header from Traefik (fixes #9143) (#9153) 2023-10-07 04:09:07 +02:00
DeflateAwning 2f6187dc0e
Add oxford comma (#9137)
Co-authored-by: André Colomb <src@andre.colomb.de>
2023-10-06 17:25:28 +02:00
Emil Lundberg 8294870ffc
Add HTML login form (fixes #4137) (#8757) 2023-10-06 13:00:58 +02:00
bt90 ac2e444a97
gui: Fix favicon status (fixes #9149) (#9150) 2023-10-06 12:27:13 +02:00
Jakob Borg 4f6b86a1c0 cmd/stdiscosrv: Slightly tweak replication settings 2023-10-04 14:15:00 +02:00
Jakob Borg 516c057d43 build: Update deps 2023-10-03 10:00:16 +02:00
Jakob Borg d644dce4e7 build: Run release steps for workflow_dispatch as well 2023-10-03 09:33:52 +02:00
bt90 7c579880eb
cmd/ursrv: Add linuxserver.io detection (#9145)
Detect linuxserver
2023-10-02 12:48:04 +02:00
Jakob Borg 296db314f5
lib/config: Improve parsing of gui-address overrides (#9144)
improve parsing of gui-address overrides

make checks for whether the gui-address is overridden consistent by
checking whether the environment variable is set and not an empty
string. the `Network()` function however checked for the inclusion of
a slash instead of the presence of any characters. If the config file's
gui address was set to a unix socket and the gui override to a tcp
address, then the function would have wrongly returned "unix".

the `URL()` function always returned the config file's gui address if a
unix socket was configured, even if an override was specified.

the `URL()` function wrongly formatted unix addresses. the http(s)
protocol was used as the sheme and the path was percent escaped. because
of the previous bug, this could only be triggered if the config file's
gui address was tcp and an unix socket override was given.

simplify the `useTLS()` function's codepath for overrides.

Co-authored-by: digital <didev@dinid.net>
2023-10-02 08:40:03 +02:00
Syncthing Release Automation a8486b0468 gui, man, authors: Update docs, translations, and contributors 2023-10-02 03:45:41 +00:00
bt90 f8a7a034a7
cmd/ursrv: Fix f-droid detection (#9142)
Fix f-droid detection
2023-09-29 17:42:44 +02:00
bt90 ceae56a860
cmd/ursrv: Support new android build user (#9141)
Support new android build user
2023-09-29 16:34:28 +02:00
DeflateAwning dcafd6ec72
readme: Style fixes, add security note (#9136) 2023-09-28 11:55:48 +02:00
Jakob Borg 8619a03f01 build: Update Actions 2023-09-25 21:50:17 +02:00
Jakob Borg b91d7711aa
Update dependencies (#9129)
And some QUIC API changes, of course.
2023-09-25 21:45:57 +02:00
d-volution 9940c91ebf
gui: Scroll to bottom by clicking message in log viewer (#9128) 2023-09-25 19:42:27 +00:00
tomasz1986 80a577b025
gui: Show if device is untrusted in the main GUI (#9116)
Add a new entry to the unfolded device info to inform the user that the
device has been marked as "untrusted" and all folders shared with it
have to be password-protected or already Receive Encrypted.

Signed-off-by: Tomasz Wilczyński <twilczynski@naver.com>
2023-09-25 21:34:19 +02:00
tomasz1986 d672175ce4
gui: Show if device has Auto Accept enabled in the main GUI (#9118)
Add a new entry to the unfolded device info to inform the user that the
device has Auto Accept enabled.

Signed-off-by: Tomasz Wilczyński <twilczynski@naver.com>
2023-09-25 21:33:16 +02:00
tomasz1986 a44b31d173
gui: Fix body padding infinitely increasing due to overlapping modals (ref #9063) (#9078)
Opening and hiding multiple modals at the same time as well as opening a
modal before fully hiding the previous one can lead to the body padding
infinitely increasing by the scrollbar width each time, with the only
way to fix it being refreshing the GUI.

Therefore, always try to ensure to open and hide multiple modals one by
one, and also that the previous modal has fully been hidden before
proceeding to open the next one. The most common case when this problem
happens is when saving config changes which displays a GUI blocking
modal that overlaps, e.g. with folder or device modals that have not
been hidden yet.

Ref: https://github.com/twbs/bootstrap/issues/3902#issuecomment-1547187799

Signed-off-by: Tomasz Wilczyński <twilczynski@naver.com>
2023-09-25 21:17:57 +02:00
Martin Polehla 70065e6b13
gitignore: All exe files, no editor configs (#9126) 2023-09-25 14:17:01 +00:00
Syncthing Release Automation adbb3ed2e9 gui, man, authors: Update docs, translations, and contributors 2023-09-25 03:45:38 +00:00
Jakob Borg 6ed9c0c34c
lib/config: Accept pre-hashed password (fixes #9123) (#9124) 2023-09-24 19:23:49 +02:00
tomasz1986 19bbf4f6bf
gui: Add missing $scope in editDeviceUntrustedChanged function (#9117)
Because $scope is missing, there are JavaScript errors when ticking and
unticking the "Untrusted" checkbox in the Advanced tab of the Edit
Device modal.

Signed-off-by: Tomasz Wilczyński <twilczynski@naver.com>
2023-09-22 07:39:16 +02:00
bt90 cf46bf0297
lib/connections: Fix transport type detection for QUIC (fixes #8274) (#9114)
Check remote address
2023-09-20 11:23:48 +02:00
Jakob Borg 051cbdc713
lib/fs, lib/model: Be careful about potentially negative durations (fixes #9112) (#9113)
I don't really understand under what circumstances, but sometimes these
calls panic with a "panic: counter cannot decrease in value" because the
value passed to Add() was negative.
2023-09-20 09:04:47 +02:00
Syncthing Release Automation 58d1f3a471 gui, man, authors: Update docs, translations, and contributors 2023-09-18 03:45:31 +00:00
tomasz1986 c9dfd75d8e
gui: Block GUI when saving changes only if necessary (ref #9063) (#9079)
Currently, the UI is always blocked from modifications when changes are
being saved, even if the save process takes very little time. This leads
to a situation where showing and closing the blocking modal can take
more time than is actually required to perform the whole operation. The
modal opening and closing very quickly can also cause the screen to
flash for a brief moment, leading to visual discomfort.

Because of this, wait for at least 200 ms and only show the blocking
modal if the changes have not been saved until then yet. The value of
200 ms is loosely based on [1] which states that 'a delay of 0.2–1.0
seconds does mean that users notice the delay and thus feel the computer
is "working" on the command, as opposed to having the command be a
direct effect of the users' actions.' Additionally, the delay must not
be too long, because the main purpose of the blocking modal is to
prevent the user from making further changes, and a longer delay would
possibly allow to do so in that brief amount of time as long as the user
is quick enough with their input.

[1] https://nngroup.com/articles/response-times-3-important-limits

Signed-off-by: Tomasz Wilczyński <twilczynski@naver.com>
2023-09-12 15:02:37 +02:00
Jakob Borg f47de83914
lib/protocol: Ensure starting & closing a connection are exclusive (fixes #9102) (#9103)
In principle a connection can close while it's in progress with
starting, and then it's undefined if we wait for goroutines to exit etc.
With this change, we will wait for start to complete before starting to
stop everything.
2023-09-12 14:48:15 +02:00
tomasz1986 caedb19307
gui: Remove unused hard-coded styles from Recent Changes modal (#9101)
gui: Remove unused hard-coded styles from globalChangesModalView modal

Currently, the globalChangesModalView modal has hardcoded th and td
styles. However, they are not even used in the modal itself, because
Bootstrap overrides them with its own styles for these elements in the
same modal. Yet, when hard-coded like that, these styles can conflict
with other table elements in the GUI. Thus, remove them completely.

Signed-off-by: Tomasz Wilczyński <twilczynski@naver.com>
2023-09-12 14:47:31 +02:00
bt90 e860d3b974
lib/connections: Make assumptions about isLAN when interface address listing fails (#9093) 2023-09-12 12:34:30 +00:00
bt90 ed66fba42b
lib/beacon, lib/discover: Send IPv4 limited broadcast when address listing fails (fixes #1628) (#9087) 2023-09-12 14:28:17 +02:00
Jakob Borg 415f320005 build: Update dependencies 2023-09-12 14:08:59 +02:00
Jakob Borg 4812600098
lib/versioner: Don't complain when folder is stopping (#9097) 2023-09-11 23:10:18 +02:00
Jakob Borg 5ff11ce142
gui: Add help link for numConnections (#9082) 2023-09-11 14:59:48 +02:00
tomasz1986 541572781b
gui: Add missing translation related to Number of Connections (ref #8918) (#9095)
Signed-off-by: Tomasz Wilczyński <twilczynski@naver.com>
2023-09-11 05:50:23 +02:00
Syncthing Release Automation e38679d9bf gui, man, authors: Update docs, translations, and contributors 2023-09-11 03:45:45 +00:00
Jakob Borg f25a169c4c build: Go 1.21.1 or higher 2023-09-06 21:11:19 +02:00
bt90 06ac10ee37
cmd/stdiscosrv: Deduplicate addresses (fixes #8482) (#9080) 2023-09-06 14:36:00 +02:00
Jakob Borg 7c0223bd06 lib/build: Next version is the Gold Grasshopper 2023-09-06 13:13:39 +02:00
Jakob Borg c6334e61aa
all: Support multiple device connections (fixes #141) (#8918)
This adds the ability to have multiple concurrent connections to a single device. This is primarily useful when the network has multiple physical links for aggregated bandwidth. A single connection will never see a higher rate than a single link can give, but multiple connections are load-balanced over multiple links.

It is also incidentally useful for older multi-core CPUs, where bandwidth could be limited by the TLS performance of a single CPU core -- using multiple connections achieves concurrency in the required crypto calculations...

Co-authored-by: Simon Frei <freisim93@gmail.com>
Co-authored-by: tomasz1986 <twilczynski@naver.com>
Co-authored-by: bt90 <btom1990@googlemail.com>
2023-09-06 12:52:01 +02:00
Jakob Borg 38bbdebffa build: Use actions/checkout@v4 2023-09-05 09:52:19 +02:00
Jakob Borg e80d04845e build: Minor dependency update 2023-09-05 09:47:51 +02:00
Syncthing Release Automation 4138e22898 gui, man, authors: Update docs, translations, and contributors 2023-09-04 03:45:39 +00:00
Maximilian c42c0e7ceb
lib/connections: Fix WANAddresses returning only unspecified IPs (ref #9010) (#9073)
Avoids taking the address of the same variable twice.
2023-09-03 15:03:27 +00:00
Jakob Borg 5118538179
lib/model: Refactor folderRunners to use a serviceMap (#9071)
Instead of separately tracking the token.

Also changes serviceMap to have a channel version of RemoveAndWait, so
that it's possible to do the removal under a lock but wait outside of
the lock. And changed where we do that in connection close, reversing
the change that happened when I added the serviceMap in 40b3b9ad1.
2023-09-02 16:42:46 +02:00
tomasz1986 4d93648f75
gui: Don't hide default values for folders and devices (#8987)
Currently, some of the information for folders and devices displayed in
the GUI relies on arbitrary values that come pre-set as defaults on a
fresh Syncthing installation, i.e. if the value matches the default, it
is hidden, and if does not, then it is displayed.

With this change, the GUI always displays all information regardless
of their value, making the overall experience more consistent and
predictable.

Signed-off-by: Tomasz Wilczyński <twilczynski@naver.com>
2023-09-02 12:19:18 +02:00
tomasz1986 29f100c162
gui: Fix File Versioning icon to match in all places (#9070)
Currently, different icons are used for File Versioning when displayed
in the unfolded folder info in the main part of the GUI, and the icon
used in the Edit Folder modal. This changes the main GUI icon to match
the icon used in the modal.

Signed-off-by: Tomasz Wilczyński <twilczynski@naver.com>
2023-09-01 11:15:20 +02:00
tomasz1986 cd98a43b80
gui: Fix Logs modal icon to match header icon (ref #9067) (#9069)
The Logs icon was changed in [1] in the header, however the icon used in
the modal was left out. This changes it, so that the header and the
modal icons match.

[1] 2abba1dfb0

Signed-off-by: Tomasz Wilczyński <twilczynski@naver.com>
2023-09-01 11:14:28 +02:00
Jakob Borg 4bf982376e build: Be more subtle about cross compilation errors
Summarize platforms that fail to build, without overloading the build
log with errors that we anyway ignore. (Currently freebsd/riscv64 fails
to build.)
2023-09-01 09:16:14 +02:00
Jakob Borg 29056d5873
build: Update dependencies (#9068) 2023-09-01 08:39:15 +02:00
tomasz1986 2abba1dfb0
gui: Remove footer and move links to header (fixes #5607) (#9067)
* gui: Remove footer and move links to header (fixes #5607)

Currently, the footer is always present and takes space at the bottom of
the GUI. However, the links listed there are not part of everyday user
interaction, and as such, they unnecessarily clutter the page, reducing
the usable screen space. Thus, transform the current Help link in the
header into a Help dropdown menu, and move the links from the footer
into it.

Also apply the following tweaks:

1. Move the About dialog from Actions to Help.
2. Add an Introduction (to the GUI) link to Help.
3. Change the Support icon from a question mark to a group of people.
4. Change the Changelog and About icons to a filled version to match the
   other icons better.
5. Use a source code icon for Source Code instead of a wrench icon, and
   move the wrench icon to Logs. This is done to prevent Changelog and
   Logs from using the same icon.
6. Update all dropdown icons' Fork Awesome styles to "fa fa-fw <icon>".

Signed-off-by: Tomasz Wilczyński <twilczynski@naver.com>

* a few more Fork Awesome style updates

---------

Signed-off-by: Tomasz Wilczyński <twilczynski@naver.com>
2023-09-01 08:18:30 +02:00
tomasz1986 325b3b114f
gui: Fix lastSeenDays error due to undefined deviceStats when adding new devices (ref #8730) (#9066) 2023-09-01 07:22:04 +02:00
tomasz1986 03590e5ac7
gui: Automatically select device ID on click (ref #8544) (#9065)
The CSS method to select device IDs on click was added in [1]. However,
it was later mistakenly overwritten by [2]. This commit fixes the
regression and also applies the same behaviour to the Edit Device modal
which was omitted in the original commit.

[1] 5baf5fedb5
[2] 5e384c9185

Signed-off-by: Tomasz Wilczyński <twilczynski@naver.com>
2023-08-31 22:16:59 +02:00
tomasz1986 95b3c26da7
gui: Prevent modifications when saving changes (fixes #9019) (#9063) 2023-08-31 17:11:03 +02:00
tomasz1986 3e5f0b1d0e
gui: Show in GUI if limitBandwidthInLan is enabled (#9062) 2023-08-31 07:22:24 +02:00
Jakob Borg 3130af3773
lib/upgrade: Enable HTTP/2 for upgrade checks (#9060) 2023-08-30 21:58:34 +02:00
Jakob Borg abd89f15f7
lib/discover: Enable HTTP/2 for global discovery requests (#9059)
By creating the http.Transport and tls.Configuration ourselves we
override some default behavior and end up with a client that speaks only
HTTP/1.1.

This adds a call to http.ConfigureTransport to do the relevant magic to
enable HTTP/2.

Also tweaks the keepalive settings to be a little kinder to the
server(s).
2023-08-30 21:58:05 +02:00
Jakob Borg a80e6be353 cmd/stdiscosrv: Streamline context handling 2023-08-30 09:36:27 +02:00
Jakob Borg acc532fc60 cmd/stdiscosrv: Explicitly enable HTTP/2
The server supports it, but it's not negotiated unless explicitly
allowed in the TLS config NextProtos.
2023-08-30 09:09:52 +02:00
Syncthing Release Automation 3cc3fb7504 gui, man, authors: Update docs, translations, and contributors 2023-08-28 03:45:57 +00:00
Jakob Borg a04cc95005 cmd/stdiscosrv: Separate HTTPS and replication certificates 2023-08-23 13:43:54 +02:00
Jakob Borg 480fa4b915 cmd/stdiscosrv: Use larger database settings 2023-08-23 13:43:14 +02:00
Jakob Borg 92a4931850 cmd/stdiscosrv: Modernise TLS settings, remove excessive HTTP logging 2023-08-23 13:39:52 +02:00
Jakob Borg bdfef9010f cmd/stdiscosrv: Serve compressed responses 2023-08-23 13:39:14 +02:00
bt90 467522d04d
lib/connections: Allow IPv6 ULA in discovery announcements (fixes #7456) (#9048)
The allowed IPv4 ranges are the same as before. But we now also accept IPv6 addresses in the ULA range FC00::/7. These addresses don't require an interface identifier and are roughly equivalent to the IPv4 private ranges.

Typical usecases:

VPN interface IPs: Wireguard, OpenVPN, Tailscale, ...
fixed IPv6 LAN addressing while the provider assigns a dynamic prefix. e.g used by pihole
https://cs.opensource.google/go/go/+/refs/tags/go1.21.0:src/net/ip.go;l=146
2023-08-23 12:28:48 +02:00
bt90 3147285c60
lib/beacon: Check FlagRunning (#9051) 2023-08-22 11:27:43 +02:00
Jakob Borg acd767b30b
all: Remove lib/util package (#9049)
Grab-bag packages are nasty, this cleans it up a little by splitting it
into topical packages sempahore, netutil, stringutil, structutil.
2023-08-21 19:44:33 +02:00
Jakob Borg 40b3b9ad15
lib/model: Clean up index handler life cycle (fixes #9021) (#9038)
Co-authored-by: Simon Frei <freisim93@gmail.com>
2023-08-21 18:39:13 +02:00
bt90 c2c6133aa5
lib/osutil, lib/upnp: Check FlagRunning (fixes #8767) (#9047) 2023-08-21 14:49:28 +00:00
Jakob Borg ccec8a4cdb
build: Update dependencies (#9046) 2023-08-21 15:56:02 +02:00
Jakob Borg cbf0e31f69
all: Use Go 1.21, new QUIC API (#9040) 2023-08-21 15:25:52 +02:00
Syncthing Release Automation c40dae315b gui, man, authors: Update docs, translations, and contributors 2023-08-21 03:45:38 +00:00
Jakob Borg ac0ce1c38f script: Remove find-metrics which belongs in docs 2023-08-17 12:27:56 +02:00
Jakob Borg 72c683aaca gui: Fix inadvertently always-false comparison (ref #7726) 2023-08-16 11:51:45 +02:00
Syncthing Release Automation 8042bd1a54 gui, man, authors: Update docs, translations, and contributors 2023-08-14 03:45:48 +00:00
Jakob Borg 462389934b cmd/stupgrades: Serve friendlier URLs for upgrade assets (fixes #9033) 2023-08-09 21:01:15 +02:00
Jakob Borg b347c14bd1 build: Use correct range specification for Go version
The old `^1.20.7` means `1.x.y, >= 1.20.7` which allows 1.21.0, which
was not intended. The new `~1.20.7` means `1.20.x, >= 1.20.7`, which is
safer.
2023-08-09 16:05:11 +02:00
Jakob Borg 8dfec6983b build: WASM is not a thing we need to try to compile for 2023-08-09 11:02:43 +02:00
Jakob Borg 9ebf2dae7b build: Ability to manually trigger Actions builds 2023-08-09 10:50:07 +02:00
André Colomb a8cacdca94
lib/versioner: Minor fixes in comments and error message (#9031)
* lib/versioner: Factor out DefaultPath constant.

Replace several instances where .stversions is named literally to all
use the same definition in the versioner package.  Exceptions are the
packages where a cyclic dependency on versioner is impossible, or some
tests which combine the versions base path with other components.

* lib/versioner: Fix comment about trash can in simple versioner.

* lib/versioner: Fix wrong versioning type string in error message.

The error message shows the folder type instead of the versioning
type, although the correct field is used in the comparison.
2023-08-09 07:10:06 +00:00
Jakob Borg 8b87cd5229
lib/model: Reinstate setting folder idle state (#9029) 2023-08-08 07:24:02 +02:00
Syncthing Release Automation e09146ee03 gui, man, authors: Update docs, translations, and contributors 2023-08-07 03:45:35 +00:00
Jakob Borg b9c08d3814
all: Add Prometheus-style metrics to expose some internal performance counters (fixes #5175) (#9003) 2023-08-04 19:57:30 +02:00
Jakob Borg 58042b3129 build: Increase Go version to 1.20.7 2023-08-03 08:11:16 +02:00
Keith Harrison eed12f3ec5
lib/config: Allow sharing already encrypted folder with untrusted devices (fixes #8965) (#9012)
Safety check added in v1.23.6 introduced bug. Bug unshares folders with untrusted devices if folder does not have an encryption password set, regardless of whether the folder is shared with the untrusted device as encrypted or not. Prevents sharing with untrusted devices in some cases where sharing would be encrypted.

Patch preserves safety check but permits sharing folders with untrusted devices if they are shared as encrypted.

Signed-off-by: kewiha <keithh@protonmail.com>
2023-08-02 07:14:53 +00:00
tomasz1986 5323928159
gui: Use case-insensive and backslash-agnostic versions filter (fixes #7973) (#8995)
Currently, the versions filter is case-sensitive regardless of the
underlying OS. With this change, the filter becomes case-insensitive
everywhere, which is more user-friendly and makes it easier to search
for files whose exact case the user may not remember.

In addition, forward and backslashes are no longer distinguished,
whether used as path separators or as part of a file / directory
name (which is unlikely but possible on some platforms).

Signed-off-by: Tomasz Wilczyński <twilczynski@naver.com>
2023-08-01 14:20:01 +02:00
Syncthing Release Automation 97625ccc26 gui, man, authors: Update docs, translations, and contributors 2023-07-31 03:45:37 +00:00
Jakob Borg 4fe746d9aa build: Run govulncheck (fixes #8983) 2023-07-30 14:38:36 +02:00
Jakob Borg 4f8cdd41ee build: Run build & tests on main branch nightly 2023-07-30 14:24:17 +02:00
Jakob Borg 406e3646e5 build: Send test logs to Grafana Loki for statistics 2023-07-30 13:40:26 +02:00
Jakob Borg 9d21b91124
all: Refactor the protocol/model interface a bit (ref #8981) (#9007) 2023-07-29 10:24:44 +02:00
Chih-Hsuan Yen b806026990
lib/connections: Fix building with `-tags noquic` (#9009) 2023-07-28 10:08:50 +00:00
tomasz1986 341b79814e
gui: Fix tooltips on buttons inside button groups (ref #7984) (#9008)
As per Bootstrap recommendation, buttons with tooltips inside button
groups require to have container: 'body' set. This prevents tooltips
from causing the buttons to jump on hover and also allows the tooltips
to be wider instead of wrapping on every space.

Ref: https://getbootstrap.com/docs/3.3/components/#btn-groups

Signed-off-by: Tomasz Wilczyński <twilczynski@naver.com>
2023-07-27 14:38:48 +02:00
Jakob Borg 319916124b
cmd/strelaysrv: Handle accept error with debug set (fixes #9001) (#9004) 2023-07-26 23:55:48 +01:00
Emil Lundberg b08b99e284
lib/api: Fix data race in TestCSRFRequired (#9006) 2023-07-26 21:33:45 +00:00
Jakob Borg f565df628c
gui: Show full error for failed items (#9005)
Also closes #8992.
2023-07-26 23:20:17 +02:00
Jakob Borg 855c6dc67b
lib/api: Allow `Bearer` authentication style with API key (#9002)
Currently, historically, we look for the `X-API-Key` header to
authenticate with an API key. There's nothing wrong with this, but in
some scenarios it's easier to produce an `Authorization` header with a
`Bearer $token` content, which is nowadays more common. This change adds
support for both, so that we will accept an API key either in our custom
header or as a bearer token.
2023-07-26 13:13:06 +02:00
tomasz1986 dc5e10fa2c
gui: Remove Twitter link from footer (#9000) 2023-07-25 00:53:57 +02:00
Syncthing Release Automation b857e57a35 gui, man, authors: Update docs, translations, and contributors 2023-07-24 03:45:42 +00:00
tomasz1986 f42f041f53
lib/ur: Don't report uptime if start time is in the past (fixes #7698) (#8996)
Currently, because of devices with unset RTC clock, the 100% percentile
for Uptime on [1] is calculated since the Unix epoch which is useless as
far as usage statistics are concerned. Thus, if the Syncthing start time
is set to a past date, assume that the clock is wrong and do not even
try to report the uptime.

[1] https://data.syncthing.net

Signed-off-by: Tomasz Wilczyński <twilczynski@naver.com>
Co-authored-by: Jakob Borg <jakob@kastelo.net>
2023-07-22 21:25:03 +00:00
Christian Kujau 6b6b2c6194
lib/model: use WARN for "Unexpected folder" messages (#8998) 2023-07-22 21:17:32 +00:00
tomasz1986 d70eb569f2
lib/osutil: Skip setLowPriority in Windows if already lower (fixes #6597) (#8993) 2023-07-21 04:38:15 +00:00
deepsource-autofix[bot] 21c074cc2c
all: replace empty slice literal with `var` (#8990)
refactor: replace empty slice literal with `var`

An empty slice can be represented by `nil` or an empty slice literal. They are
functionally equivalent — their `len` and `cap` are both zero — but the `nil`
slice is the preferred style. For more information about empty slices,
see [Declaring Empty Slices](https://github.com/golang/go/wiki/CodeReviewComments#declaring-empty-slices).

Co-authored-by: deepsource-autofix[bot] <62050782+deepsource-autofix[bot]@users.noreply.github.com>
2023-07-18 14:44:37 +00:00
deepsource-autofix[bot] f23c41221b
all: fix unused method receiver (#8988)
refactor: fix unused method receiver

Methods with unused receivers can be a symptom of unfinished refactoring or a bug. To keep 
the same method signature, omit the receiver name or '_' as it is unused.

Co-authored-by: deepsource-autofix[bot] <62050782+deepsource-autofix[bot]@users.noreply.github.com>
2023-07-18 14:34:50 +00:00
deepsource-autofix[bot] 24e230d455
all: unused parameter should be replaced by underscore (#8989)
refactor: unused parameter should be replaced by underscore

Unused parameters in functions or methods should be replaced with `_`
(underscore) or removed.

Co-authored-by: deepsource-autofix[bot] <62050782+deepsource-autofix[bot]@users.noreply.github.com>
2023-07-18 14:33:13 +00:00
Syncthing Release Automation 31daa20367 gui, man, authors: Update docs, translations, and contributors 2023-07-17 03:46:46 +00:00
Jakob Borg e4d0f9dd6c cmd/syncthing: Mention STVERSIONEXTRA in --help output (ref #8980) 2023-07-16 17:48:24 +02:00
Jakob Borg df2ac7aaeb
gui, lib/api: Add possibility to feed through extra version information (#8980)
This adds an environment variable STVERSIONEXTRA that, when set, gets
added to the version information in the API and GUI.

The purpose of all this is to be able to communicate something about the
bundling or packaging, through the log & GUI and the end user, to the
potential person supporting it -- i.e., us. :) A wrapper can set this
variable to indicate that Syncthing is being run via `SyncTrayzor`,
`Syncthing-macOS`, etc., and thus indicate to the end user that the GUI
they are looking at is perhaps not the only source of truth and
management for this instance.
2023-07-16 17:43:10 +02:00
Jakob Borg b96b23957b cmd/ursrv: Update map tile URL 2023-07-16 17:36:05 +02:00
bt90 265ce139c5
cmd/strelaypoolsrv: Update map tile URL (#8985) 2023-07-16 17:20:40 +02:00
Jakob Borg 48c95eb41d cmd/stcrashreceiver: Correct parsing of current version string 2023-07-12 09:27:34 +02:00
Jakob Borg 937895be69 build: Update dependencies 2023-07-11 09:14:21 +02:00
Jakob Borg a3886f778d cmd/ursrv: Remove old, unused user movement code 2023-07-10 09:21:40 +02:00
Jakob Borg 6aecc2622c cmd/ursrv: Merge ursrv and uraggregate as subcommands 2023-07-10 09:00:57 +02:00
Jakob Borg c55b205a0b cmd/ursrv: Remove useless static TLS cert handling 2023-07-10 08:39:30 +02:00
Jakob Borg 2fcf7006e6 cmd/ursrv: Embed static assets 2023-07-10 08:33:09 +02:00
Jakob Borg bf61e485a6 cmd/ursrv: Refactor to use CLI options, fewer global vars 2023-07-10 08:27:16 +02:00
Jakob Borg b2886f11b1
gui: Show xattr filter editor when send xattrs checked (fixes #8958) (#8959) 2023-07-10 08:01:08 +02:00
Syncthing Release Automation 11ece5d89e gui, man, authors: Update docs, translations, and contributors 2023-07-10 03:46:49 +00:00
Jakob Borg cf68dfac43 build: Build Docker image for plain 32 bit arm (fixes #8973) 2023-07-08 10:55:09 +02:00
Jakob Borg c44de2cd58
lib/fs: Clarify errors for Windows filenames (fixes #8968) (#8969)
With this change, error messages include the offending characters or
name parts. Examples:

    nul.txt: name is invalid, contains Windows reserved name: "nul"
    foo>bar.txt: name is invalid, contains Windows reserved character: ">"
    foo \bar.txt: name is invalid, must not end in space or period on Windows
2023-07-07 11:00:40 +00:00
Jakob Borg 6ff5ed6d23
gui: Avoid spurious comma in shared-with device list (fixes #8967) (#8970) 2023-07-07 07:25:24 +02:00
Jakob Borg 25ec2b63ab cmd/ursrv: Summarize tiny fraction items into Other 2023-07-05 08:22:10 +02:00
Jakob Borg c5ab71d7a5 cmd/ursrv: Update distributions list 2023-07-05 08:08:29 +02:00
Jakob Borg 1fc4c9d9c5 build: Only push releases to cloud storage, and also use `latest` 2023-07-04 10:27:26 +02:00
Jakob Borg bebf2c259c readme: Remove outdated build badges/links 2023-07-03 13:03:09 +02:00
Jakob Borg b7d526903e build: Push to correct Docker images 2023-07-03 12:08:53 +02:00
Jakob Borg 92917c9f62 Merge branch 'release'
* release:
  build: Build and publish Docker images for relay and discovery server (fixes #8691, fixes #8960)
2023-07-03 11:52:11 +02:00
Jakob Borg fdde05cf12 build: Build and publish Docker images for relay and discovery server (fixes #8691, fixes #8960)
This adds builds for the discovery and relay servers in the same manner
as Syncthing, so that Docker images are kept up to date with releases.

Inspired by and closes #8834.

Co-authored-by: Migelo <miha@filetki.si>
2023-07-03 11:51:12 +02:00
Syncthing Release Automation a7d7325e9b gui, man, authors: Update docs, translations, and contributors 2023-07-03 03:45:49 +00:00
Jakob Borg 465823237f build: Build and publish Docker images for relay and discovery server (fixes #8691, fixes #8960)
This adds builds for the discovery and relay servers in the same manner
as Syncthing, so that Docker images are kept up to date with releases.

Inspired by and closes #8834.

Co-authored-by: Migelo <miha@filetki.si>
2023-07-01 08:48:12 +02:00
Jakob Borg 5a1f996e56 build: Build infrastructure images 2023-07-01 08:23:55 +02:00
Jakob Borg 229b6a292c cmd/stcrashreceiver: Add /ping endpoint 2023-07-01 07:53:50 +02:00
Jakob Borg c84e47a7b2 build: Update dependencies 2023-06-28 14:00:30 +02:00
otbutz daf7baaeff
etc: Update sysctl configs (#8955) 2023-06-28 07:05:12 +02:00
Jakob Borg a4a1231e92
gui: Remove tilde auto-expansions (fixes #8890) (#8954)
This removes the tilde expansion we had in the GUI. I'm not sure why we
had it, but there are valid reasons to have a folder path with tilde in
it in the config, and it was previously impossible to enter one of
those.
2023-06-28 07:03:53 +02:00
Jakob Borg b99dee3ac3
cmd/syncthing: Add environment variables for --home, --conf, and --data (fixes #8957) (#8952)
This allows environment overrides for our directories. This is
advantageous because, apart from the obvious, it means we can set it in
the Docker file and not add command line options there. Having the
command line option as we did meant that it was impossible to use the
Docker image for other commands than `serve` (because that is implied
when we see other options on the command line).
2023-06-28 07:03:36 +02:00
Syncthing Release Automation 89fc69249b gui, man, authors: Update docs, translations, and contributors 2023-06-26 03:47:58 +00:00
Jakob Borg d421d66a3f build: Docker images should have auto upgrade disabled... 2023-06-22 14:38:34 +02:00
guangwu 27aba3567b
all: Minor staticcheck fixes (#8939) 2023-06-19 06:50:53 +00:00
Syncthing Release Automation 5532532db9 gui, man, authors: Update docs, translations, and contributors 2023-06-19 03:46:01 +00:00
Felix c369f8abb2
gui: Have static link to favicon, for bookmarks etc (fixes #7638) (#8850) 2023-06-14 09:59:56 +02:00
Jakob Borg 9e52f6cf2f build: Update some dependencies 2023-06-14 09:43:45 +02:00
Jakob Borg d22a38d947 build: Make sure we get the latest matching Go version
Also, disable caching in setup-go when we do manual cache setup with a
better cache key. It became default-true in the latest action version.
2023-06-14 09:37:59 +02:00
Jakob Borg 439fa6c848 build: Multi arch Docker images with GitHub actions (ref #8834)
This builds and publishes our Docker images from GitHub.
2023-06-14 09:28:04 +02:00
Jakob Borg 6b475bdb78
lib/config, gui: Disallow some options in combination with "untrusted" (fixes #8920) (#8921)
This prevents combining untrusted with introducer and auto-accept, and
also verifies that folders shared with untrusted devices have passwords
at config loading time.

Co-authored-by: Simon Frei <freisim93@gmail.com>
2023-06-14 09:24:31 +02:00
Syncthing Release Automation 7d56fba321 gui, man, authors: Update docs, translations, and contributors 2023-06-12 03:45:39 +00:00
Simon Frei bf6ffbbd67
Don't add empty device to config on init (#8933)
We usually want to ensure that our own device is present. However if the
given device ID is the empty ID, we shouldn't do that. This is a
legimate (though way too non-obvious) use-case when opening the config
without knowing/caring about the device ID.
2023-06-10 20:33:39 +00:00
Jakob Borg a972811f54 build: Push release files to cloud storage 2023-06-06 17:53:05 +02:00
Jakob Borg 88da67d7c3 build: Generate .asc files for release packages (fixes #8897) 2023-06-06 13:58:44 +02:00
Jakob Borg 1f07e05470 build: Properly build all Debian archs (fixes #8898) 2023-06-06 12:52:30 +02:00
Jakob Borg 5cab08a36a Merge branch 'release'
* release:
  gui: Avoid code generating HTML (#8923)
  gui: Remove HTML support in tooltips
2023-06-06 09:09:30 +02:00
Jakob Borg 4e2bb58e2d
gui: Avoid code generating HTML (#8923) 2023-06-05 13:16:02 +02:00
Jakob Borg ae176ea9cd
build: Tests should run with Go 1.20 on Windows (#8924)
Tests should run with Go 1.20 on Windows
2023-06-05 10:19:47 +02:00
Syncthing Release Automation 2b17db8aa3 gui, man, authors: Update docs, translations, and contributors 2023-06-05 03:45:37 +00:00
Jakob Borg 81a4b22d43
lib/model: Improve test for unignored parent directories (#8926) 2023-06-04 15:32:03 +02:00
Jakob Borg f7da96fb82
build: Update dependencies (#8925)
The easy ones, there is also a quic-go update but it requires API
refactoring.
2023-06-04 15:31:20 +02:00
Jakob Borg f5e5af391a gui: Remove HTML support in tooltips 2023-06-03 09:58:08 +02:00
Jakob Borg 5a3ac86c3f
cmd/syncthing: Use correct binary when restarting monitor (#8919) 2023-06-02 07:54:31 +02:00
dependabot[bot] 3d78ff9f68
build(deps): bump github.com/minio/sha256-simd from 1.0.0 to 1.0.1 (#8916)
Bumps [github.com/minio/sha256-simd](https://github.com/minio/sha256-simd) from 1.0.0 to 1.0.1.
- [Release notes](https://github.com/minio/sha256-simd/releases)
- [Commits](https://github.com/minio/sha256-simd/compare/v1.0.0...v1.0.1)

---
updated-dependencies:
- dependency-name: github.com/minio/sha256-simd
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-06-01 12:16:46 +02:00
Jakob Borg f3127a66ee build: Increase concurrency, add "basic checks" gatekeeper 2023-06-01 12:15:31 +02:00
Syncthing Release Automation 90b4711ad2 gui, man, authors: Update docs, translations, and contributors 2023-05-29 03:45:27 +00:00
Syncthing Release Automation 716b42103a gui, man, authors: Update docs, translations, and contributors 2023-05-22 03:45:33 +00:00
Anthony Goeckner 405cdedcd3
lib/model: Set platform data for new folders w/ ignorePerms (ref #8883) (#8907)
* Platform data (ownership, xattrs, etc.) is now set correctly for newly-received folders, even if the received folder has the NoPermissions flag.

* Call setPlatformData on receivers that have ignorePerms set to true.
2023-05-17 09:06:50 +02:00
Syncthing Release Automation 0b3a101ccd gui, man, authors: Update docs, translations, and contributors 2023-05-15 03:45:55 +00:00
Eng Zer Jun 089320aadc
lib: replace `Readdir(-1)` with `os.ReadDir` (#8901) 2023-05-11 15:35:52 +00:00
Will Rouesnel b2fb2ef276
lib/api: Allow BindDN to exclude any username formatting (fixes #8899) (#8900)
This allows a syncthing instance to be locked to exactly 1 user without
needing search capability on the LDAP instance.
2023-05-10 07:52:02 +02:00
370 changed files with 15132 additions and 8229 deletions

View File

@ -1,13 +0,0 @@
---
name: Feature request
about: If you're just not sure how to do something, see "ask a question".
labels: enhancement, needs-triage
---
### Include required information
Please be sure to include at least:
- what problem your new feature would solve
- how or why you think it is generally useful (i.e., not just for you)
- what alternatives or workarounds you considered

28
.github/ISSUE_TEMPLATE/01-feature.yml vendored Normal file
View File

@ -0,0 +1,28 @@
name: Feature request
description: File a new feature request
labels: ["enhancement", "needs-triage"]
body:
- type: textarea
id: feature
attributes:
label: Feature description
description: Please describe the behavior you'd like to see.
validations:
required: true
- type: textarea
id: problem-usecase
attributes:
label: Problem or use case
description: Please explain which problem this would solve, or what the use case is for the feature. Keep in mind that it's more likely to be implemented if it's generally useful for a larger number of users.
validations:
required: true
- type: textarea
id: alternatives
attributes:
label: Alternatives or workarounds
description: Please describe any alternatives or workarounds you have considered and, possibly, rejected.
validations:
required: true

View File

@ -1,23 +0,0 @@
---
name: Bug report
about: If you're actually looking for support, see "ask a question".
labels: bug, needs-triage
---
### Does your log mention database corruption?
If your Syncthing log reports panics because of database corruption it is
most likely a fault with your system's storage or memory. Affected log
entries will contain lines starting with `panic: leveldb`. You will need to
delete the index database to clear this, by running `syncthing
-reset-database`.
### Include required information
Please be sure to include at least:
- which version of Syncthing and what operating system you are using
- browser and version, if applicable
- what happened,
- what you expected to happen instead, and
- any steps to reproduce the problem.

51
.github/ISSUE_TEMPLATE/02-bug.yml vendored Normal file
View File

@ -0,0 +1,51 @@
name: Bug report
description: If you're actually looking for support instead, see "I need help / I have a question".
labels: ["bug", "needs-triage"]
body:
- type: markdown
attributes:
value: |
:no_entry_sign: If you want to report a security issue, please see [our Security Policy](https://syncthing.net/security/) and do not report the issue here.
:interrobang: If you are not sure if there is a bug, but something isn't working right and you need help, please [use the forum](https://forum.syncthing.net/).
- type: textarea
id: what-happened
attributes:
label: What happened?
description: Also tell us, what did you expect to happen, and any steps we might use to reproduce the problem.
placeholder: Tell us what you see!
validations:
required: true
- type: input
id: version
attributes:
label: Syncthing version
description: What version of Syncthing are you running?
placeholder: v1.27.4
validations:
required: true
- type: input
id: platform
attributes:
label: Platform & operating system
description: On what platform(s) are you seeing the problem?
placeholder: Linux arm64
validations:
required: true
- type: input
id: browser
attributes:
label: Browser version
description: If the problem is related to the GUI, describe your browser and version.
placeholder: Safari 17.3.1
- type: textarea
id: logs
attributes:
label: Relevant log output
description: Please copy and paste any relevant log output or crash backtrace. This will be automatically formatted into code, so no need for backticks.
render: shell

View File

@ -0,0 +1,74 @@
name: Build Infrastructure Images
on:
push:
branches:
- infrastructure
- infra-*
env:
GO_VERSION: "~1.22.3"
CGO_ENABLED: "0"
BUILD_USER: docker
BUILD_HOST: github.syncthing.net
jobs:
docker-syncthing:
name: Build and push Docker images
if: github.repository == 'syncthing/syncthing'
runs-on: ubuntu-latest
environment: docker
strategy:
matrix:
pkg:
- stcrashreceiver
- strelaypoolsrv
- stupgrades
- ursrv
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- uses: actions/setup-go@v5
with:
go-version: ${{ env.GO_VERSION }}
check-latest: true
- name: Login to Docker Hub
uses: docker/login-action@v3
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
- name: Build binaries
run: |
for arch in arm64 amd64; do
go run build.go -goos linux -goarch "$arch" build ${{ matrix.pkg }}
mv ${{ matrix.pkg }} ${{ matrix.pkg }}-linux-"$arch"
done
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
- name: Set Docker tags (all branches)
run: |
tags=syncthing/${{ matrix.pkg }}:${{ github.sha }}
echo "TAGS=$tags" >> $GITHUB_ENV
- name: Set Docker tags (latest)
if: github.ref == 'refs/heads/infrastructure'
run: |
tags=syncthing/${{ matrix.pkg }}:latest,${{ env.TAGS }}
echo "TAGS=$tags" >> $GITHUB_ENV
- name: Build and push
uses: docker/build-push-action@v5
with:
context: .
file: ./Dockerfile.${{ matrix.pkg }}
platforms: linux/amd64,linux/arm64
push: true
tags: ${{ env.TAGS }}
labels: |
org.opencontainers.image.revision=${{ github.sha }}

View File

@ -3,10 +3,16 @@ name: Build Syncthing
on:
pull_request:
push:
schedule:
# Run nightly build at 05:00 UTC
- cron: '00 05 * * *'
workflow_dispatch:
env:
# The go version to use for builds.
GO_VERSION: "^1.20.3"
# The go version to use for builds. We set check-latest to true when
# installing, so we get the latest patch version that matches the
# expression.
GO_VERSION: "~1.22.3"
# Optimize compatibility on the slow archictures.
GO386: softfloat
@ -42,7 +48,7 @@ jobs:
runner: ["windows-latest", "ubuntu-latest", "macos-latest"]
# The oldest version in this list should match what we have in our go.mod.
# Variables don't seem to be supported here, or we could have done something nice.
go: ["1.19", "1.20"]
go: ["~1.21.7", "~1.22.3"]
runs-on: ${{ matrix.runner }}
steps:
- name: Set git to use LF
@ -55,24 +61,32 @@ jobs:
git config --global core.autocrlf false
git config --global core.eol lf
- uses: actions/checkout@v3
- uses: actions/checkout@v4
- uses: actions/setup-go@v4
- uses: actions/setup-go@v5
with:
go-version: ${{ matrix.go }}
cache: true
check-latest: true
- name: Build
run: |
go run build.go
- name: Test
# Our Windows tests currently don't work on Go 1.20
# https://github.com/syncthing/syncthing/issues/8779
# https://github.com/syncthing/syncthing/issues/8778
if: "!(matrix.go == '1.20' && matrix.runner == 'windows-latest')"
- name: Install go-test-json-to-loki
run: |
go run build.go test
go install calmh.dev/go-test-json-to-loki@latest
- name: Test
run: |
go version
go run build.go test | go-test-json-to-loki
env:
GOFLAGS: "-json"
LOKI_URL: ${{ vars.LOKI_URL }}
LOKI_USER: ${{ vars.LOKI_USER }}
LOKI_PASSWORD: ${{ secrets.LOKI_PASSWORD }}
LOKI_LABELS: "go=${{ matrix.go }},runner=${{ matrix.runner }},repo=${{ github.repository }},ref=${{ github.ref }}"
#
# Meta checks for formatting, copyright, etc
@ -82,26 +96,49 @@ jobs:
name: Check correctness
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4
- uses: actions/setup-go@v4
- uses: actions/setup-go@v5
with:
go-version: ${{ env.GO_VERSION }}
cache: false
check-latest: true
- name: Check correctness
run: |
go test -v ./meta
#
# The basic checks job is a virtual one that depends on the matrix tests,
# the correctness checks, and various builds that we always do. This makes
# it easy to have the PR process have a single test as a gatekeeper for
# merging, instead of having to add all the matrix tests and update them
# each time the version changes. (The top level test is not available for
# choosing there, only the matrix "children".)
#
basics:
name: Basic checks passed
runs-on: ubuntu-latest
needs:
- build-test
- correctness
- package-linux
- package-cross
- package-source
- package-debian
- govulncheck
steps:
- uses: actions/checkout@v4
#
# Windows
#
package-windows:
name: Package for Windows
if: github.event_name == 'push' && (github.ref == 'refs/heads/release' || startsWith(github.ref, 'refs/heads/release-'))
if: (github.event_name == 'push' || github.event_name == 'workflow_dispatch') && (github.ref == 'refs/heads/release' || startsWith(github.ref, 'refs/heads/release-'))
environment: signing
needs:
- build-test
runs-on: windows-latest
steps:
- name: Set git to use LF
@ -113,15 +150,22 @@ jobs:
git config --global core.autocrlf false
git config --global core.eol lf
- uses: actions/checkout@v3
- uses: actions/checkout@v4
with:
fetch-depth: 0
- uses: actions/setup-go@v4
- uses: actions/setup-go@v5
with:
go-version: ${{ env.GO_VERSION }}
cache: false
check-latest: true
- uses: actions/cache@v3
- name: Get actual Go version
run: |
go version
echo "GO_VERSION=$(go version | sed 's#^.*go##;s# .*##')" >> $GITHUB_ENV
- uses: actions/cache@v4
with:
path: |
~\AppData\Local\go-build
@ -146,7 +190,7 @@ jobs:
CODESIGN_TIMESTAMP_SERVER: ${{ secrets.CODESIGN_TIMESTAMP_SERVER }}
- name: Archive artifacts
uses: actions/upload-artifact@v3
uses: actions/upload-artifact@v4
with:
name: packages-windows
path: syncthing-windows-*.zip
@ -157,19 +201,24 @@ jobs:
package-linux:
name: Package for Linux
needs:
- build-test
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4
with:
fetch-depth: 0
- uses: actions/setup-go@v4
- uses: actions/setup-go@v5
with:
go-version: ${{ env.GO_VERSION }}
cache: false
check-latest: true
- uses: actions/cache@v3
- name: Get actual Go version
run: |
go version
echo "GO_VERSION=$(go version | sed 's#^.*go##;s# .*##')" >> $GITHUB_ENV
- uses: actions/cache@v4
with:
path: |
~/.cache/go-build
@ -186,7 +235,7 @@ jobs:
CGO_ENABLED: "0"
- name: Archive artifacts
uses: actions/upload-artifact@v3
uses: actions/upload-artifact@v4
with:
name: packages-linux
path: syncthing-linux-*.tar.gz
@ -197,21 +246,26 @@ jobs:
package-macos:
name: Package for macOS
if: github.event_name == 'push' && (github.ref == 'refs/heads/release' || startsWith(github.ref, 'refs/heads/release-'))
if: (github.event_name == 'push' || github.event_name == 'workflow_dispatch') && (github.ref == 'refs/heads/release' || startsWith(github.ref, 'refs/heads/release-'))
environment: signing
needs:
- build-test
runs-on: macos-latest
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4
with:
fetch-depth: 0
- uses: actions/setup-go@v4
- uses: actions/setup-go@v5
with:
go-version: ${{ env.GO_VERSION }}
cache: false
check-latest: true
- uses: actions/cache@v3
- name: Get actual Go version
run: |
go version
echo "GO_VERSION=$(go version | sed 's#^.*go##;s# .*##')" >> $GITHUB_ENV
- uses: actions/cache@v4
with:
path: |
~/.cache/go-build
@ -280,21 +334,22 @@ jobs:
zip -r "../$universal.zip" "$universal"
- name: Archive artifacts
uses: actions/upload-artifact@v3
uses: actions/upload-artifact@v4
with:
name: packages-macos
path: syncthing-*.zip
notarize-macos:
name: Notarize for macOS
if: github.event_name == 'push' && (github.ref == 'refs/heads/release' || startsWith(github.ref, 'refs/heads/release-'))
if: (github.event_name == 'push' || github.event_name == 'workflow_dispatch') && (github.ref == 'refs/heads/release' || startsWith(github.ref, 'refs/heads/release-'))
environment: signing
needs:
- package-macos
- basics
runs-on: macos-latest
steps:
- name: Download artifacts
uses: actions/download-artifact@v3
uses: actions/download-artifact@v4
with:
name: packages-macos
@ -320,19 +375,24 @@ jobs:
package-cross:
name: Package cross compiled
needs:
- build-test
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4
with:
fetch-depth: 0
- uses: actions/setup-go@v4
- uses: actions/setup-go@v5
with:
go-version: ${{ env.GO_VERSION }}
cache: false
check-latest: true
- uses: actions/cache@v3
- name: Get actual Go version
run: |
go version
echo "GO_VERSION=$(go version | sed 's#^.*go##;s# .*##')" >> $GITHUB_ENV
- uses: actions/cache@v4
with:
path: |
~/.cache/go-build
@ -351,20 +411,27 @@ jobs:
| grep -v nacl/ \
| grep -v plan9/ \
| grep -v windows/ \
| grep -v /wasm \
)
# Build for each platform with errors silenced, because we expect
# some oddball platforms to fail. This avoids a bunch of errors in
# the GitHub Actions output, instead summarizing each build
# failure as a warning.
for plat in $platforms; do
goos="${plat%/*}"
goarch="${plat#*/}"
if ! go run build.go -goos "$goos" -goarch "$goarch" tar ; then
echo "*** $plat failed ***"
echo "::group ::$plat"
if ! go run build.go -goos "$goos" -goarch "$goarch" tar 2>/dev/null; then
echo "::warning ::Failed to build for $plat"
fi
echo "::endgroup::"
done
env:
CGO_ENABLED: "0"
- name: Archive artifacts
uses: actions/upload-artifact@v3
uses: actions/upload-artifact@v4
with:
name: packages-other
path: syncthing-*.tar.gz
@ -375,17 +442,17 @@ jobs:
package-source:
name: Package source code
needs:
- build-test
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4
with:
fetch-depth: 0
- uses: actions/setup-go@v4
- uses: actions/setup-go@v5
with:
go-version: ${{ env.GO_VERSION }}
cache: false
check-latest: true
- name: Package source
run: |
@ -404,38 +471,46 @@ jobs:
mv "syncthing-source-$version.tar.gz" syncthing
- name: Archive artifacts
uses: actions/upload-artifact@v3
uses: actions/upload-artifact@v4
with:
name: packages-source
path: syncthing-source-*.tar.gz
#
# Sign binaries for auto upgrade
# Sign binaries for auto upgrade, generate ASC signature files
#
sign-for-upgrade:
name: Sign for upgrade
if: github.event_name == 'push' && (github.ref == 'refs/heads/release' || startsWith(github.ref, 'refs/heads/release-'))
if: (github.event_name == 'push' || github.event_name == 'workflow_dispatch') && (github.ref == 'refs/heads/release' || startsWith(github.ref, 'refs/heads/release-'))
environment: signing
needs:
- basics
- package-windows
- package-linux
- package-macos
- package-cross
- package-source
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4
with:
fetch-depth: 0
- uses: actions/checkout@v3
- uses: actions/checkout@v4
with:
repository: syncthing/release-tools
path: tools
fetch-depth: 0
- name: Download artifacts
uses: actions/download-artifact@v3
uses: actions/download-artifact@v4
- uses: actions/setup-go@v5
with:
go-version: ${{ env.GO_VERSION }}
cache: false
check-latest: true
- name: Install signing tool
run: |
@ -450,11 +525,31 @@ jobs:
mv packages-*/* packages
pushd packages
"$GITHUB_WORKSPACE/tools/sign-only"
rm -f "$PRIVATE_KEY"
env:
STSIGTOOL_PRIVATE_KEY: ${{ secrets.STSIGTOOL_PRIVATE_KEY }}
- name: Create and sign .asc files
run: |
sudo apt update
sudo apt -y install gnupg
export SIGNING_KEY="$RUNNER_TEMP/gpg-secret.asc"
echo "$GNUPG_SIGNING_KEY_BASE64" | base64 -d > "$SIGNING_KEY"
gpg --import < "$SIGNING_KEY"
pushd packages
files=(*.tar.gz *.zip)
sha1sum "${files[@]}" | gpg --clearsign > sha1sum.txt.asc
sha256sum "${files[@]}" | gpg --clearsign > sha256sum.txt.asc
gpg --sign --armour --detach syncthing-source-*.tar.gz
popd
rm -f "$SIGNING_KEY" .gnupg
env:
GNUPG_SIGNING_KEY_BASE64: ${{ secrets.GNUPG_SIGNING_KEY_BASE64 }}
- name: Archive artifacts
uses: actions/upload-artifact@v3
uses: actions/upload-artifact@v4
with:
name: packages-signed
path: packages/*
@ -465,17 +560,22 @@ jobs:
package-debian:
name: Package for Debian
needs:
- build-test
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4
with:
fetch-depth: 0
- uses: actions/setup-go@v4
- uses: actions/setup-go@v5
with:
go-version: ${{ env.GO_VERSION }}
cache: false
check-latest: true
- name: Get actual Go version
run: |
go version
echo "GO_VERSION=$(go version | sed 's#^.*go##;s# .*##')" >> $GITHUB_ENV
- uses: ruby/setup-ruby@v1
with:
@ -485,7 +585,7 @@ jobs:
run: |
gem install fpm
- uses: actions/cache@v3
- uses: actions/cache@v4
with:
path: |
~/.cache/go-build
@ -494,14 +594,14 @@ jobs:
- name: Package for Debian
run: |
for goarch in amd64 arm64 arm ; do
go run build.go -goos linux -goarch "$goarch" deb
for arch in amd64 i386 armhf armel arm64 ; do
go run build.go -no-upgrade -installsuffix=no-upgrade -goarch "$arch" deb
done
env:
BUILD_USER: debian
- name: Archive artifacts
uses: actions/upload-artifact@v3
uses: actions/upload-artifact@v4
with:
name: debian-packages
path: "*.deb"
@ -512,25 +612,31 @@ jobs:
publish-nightly:
name: Publish nightly build
if: github.event_name == 'push' && startsWith(github.ref, 'refs/heads/release-nightly')
if: (github.event_name == 'push' || github.event_name == 'workflow_dispatch') && startsWith(github.ref, 'refs/heads/release-nightly')
environment: signing
needs:
- sign-for-upgrade
- notarize-macos
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4
with:
repository: syncthing/release-tools
path: tools
fetch-depth: 0
- name: Download artifacts
uses: actions/download-artifact@v3
uses: actions/download-artifact@v4
with:
name: packages-signed
path: packages
- uses: actions/setup-go@v5
with:
go-version: ${{ env.GO_VERSION }}
cache: false
check-latest: true
- name: Create release json
run: |
cd packages
@ -549,3 +655,194 @@ jobs:
RCLONE_CONFIG_SPACES_ACL: public-read
with:
args: sync packages spaces:syncthing/nightly
#
# Push release artifacts to Spaces
#
publish-release-files:
name: Publish release files
if: (github.event_name == 'push' || github.event_name == 'workflow_dispatch') && github.ref == 'refs/heads/release'
environment: signing
needs:
- sign-for-upgrade
- package-debian
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Download signed packages
uses: actions/download-artifact@v4
with:
name: packages-signed
path: packages
- name: Download debian packages
uses: actions/download-artifact@v4
with:
name: debian-packages
path: packages
- uses: actions/setup-go@v5
with:
go-version: ${{ env.GO_VERSION }}
cache: false
check-latest: true
- name: Set version
run: |
version=$(go run build.go version)
echo "VERSION=$version" >> $GITHUB_ENV
- name: Push to Spaces (${{ env.VERSION }})
uses: docker://docker.io/rclone/rclone:latest
env:
RCLONE_CONFIG_SPACES_TYPE: s3
RCLONE_CONFIG_SPACES_PROVIDER: DigitalOcean
RCLONE_CONFIG_SPACES_ACCESS_KEY_ID: ${{ secrets.SPACES_KEY }}
RCLONE_CONFIG_SPACES_SECRET_ACCESS_KEY: ${{ secrets.SPACES_SECRET }}
RCLONE_CONFIG_SPACES_ENDPOINT: ams3.digitaloceanspaces.com
RCLONE_CONFIG_SPACES_ACL: public-read
with:
args: sync packages spaces:syncthing/release/${{ env.VERSION }}
- name: Push to Spaces (latest)
uses: docker://docker.io/rclone/rclone:latest
env:
RCLONE_CONFIG_SPACES_TYPE: s3
RCLONE_CONFIG_SPACES_PROVIDER: DigitalOcean
RCLONE_CONFIG_SPACES_ACCESS_KEY_ID: ${{ secrets.SPACES_KEY }}
RCLONE_CONFIG_SPACES_SECRET_ACCESS_KEY: ${{ secrets.SPACES_SECRET }}
RCLONE_CONFIG_SPACES_ENDPOINT: ams3.digitaloceanspaces.com
RCLONE_CONFIG_SPACES_ACL: public-read
with:
args: sync spaces:syncthing/release/${{ env.VERSION }} spaces:syncthing/release/latest
#
# Build and push to Docker Hub
#
docker-syncthing:
name: Build and push Docker images
runs-on: ubuntu-latest
if: (github.event_name == 'push' || github.event_name == 'workflow_dispatch') && (github.ref == 'refs/heads/release' || github.ref == 'refs/heads/main' || github.ref == 'refs/heads/infrastructure' || startsWith(github.ref, 'refs/heads/release-'))
environment: docker
strategy:
matrix:
pkg:
- syncthing
- strelaysrv
- stdiscosrv
include:
- pkg: syncthing
dockerfile: Dockerfile
image: syncthing/syncthing
- pkg: strelaysrv
dockerfile: Dockerfile.strelaysrv
image: syncthing/relaysrv
- pkg: stdiscosrv
dockerfile: Dockerfile.stdiscosrv
image: syncthing/discosrv
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- uses: actions/setup-go@v5
with:
go-version: ${{ env.GO_VERSION }}
cache: false
check-latest: true
- name: Get actual Go version
run: |
go version
echo "GO_VERSION=$(go version | sed 's#^.*go##;s# .*##')" >> $GITHUB_ENV
- uses: actions/cache@v4
with:
path: |
~/.cache/go-build
~/go/pkg/mod
key: ${{ runner.os }}-go-${{ env.GO_VERSION }}-docker-${{ matrix.pkg }}-${{ hashFiles('**/go.sum') }}
- name: Build binaries
run: |
for arch in amd64 arm64 arm; do
go run build.go -goos linux -goarch "$arch" -no-upgrade build ${{ matrix.pkg }}
mv ${{ matrix.pkg }} ${{ matrix.pkg }}-linux-"$arch"
done
env:
CGO_ENABLED: "0"
BUILD_USER: docker
- name: Check if we will be able to push images
run: |
if [[ "${{ secrets.DOCKERHUB_TOKEN }}" != "" ]]; then
echo "DOCKER_PUSH=true" >> $GITHUB_ENV;
fi
- name: Login to Docker Hub
uses: docker/login-action@v3
if: env.DOCKER_PUSH == 'true'
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
- name: Set version tags
run: |
version=$(go run build.go version)
version=${version#v}
if [[ $version == @([0-9]|[0-9][0-9]).@([0-9]|[0-9][0-9]).@([0-9]|[0-9][0-9]) ]] ; then
echo Release version, pushing to :latest and version tags
major=${version%.*.*}
minor=${version%.*}
tags=${{ matrix.image }}:$version,${{ matrix.image }}:$major,${{ matrix.image }}:$minor,${{ matrix.image }}:latest
elif [[ $version == *-rc.@([0-9]|[0-9][0-9]) ]] ; then
echo Release candidate, pushing to :rc
tags=${{ matrix.image }}:rc
else
echo Development version, pushing to :edge
tags=${{ matrix.image }}:edge
fi
echo "DOCKER_TAGS=$tags" >> $GITHUB_ENV
echo "VERSION=$version" >> $GITHUB_ENV
- name: Build and push Docker image
uses: docker/build-push-action@v5
with:
context: .
file: ${{ matrix.dockerfile }}
platforms: linux/amd64,linux/arm64,linux/arm/7
push: ${{ env.DOCKER_PUSH == 'true' }}
tags: ${{ env.DOCKER_TAGS }}
labels: |
org.opencontainers.image.version=${{ env.VERSION }}
org.opencontainers.image.revision=${{ github.sha }}
#
# Check for known vulnerabilities in Go dependencies
#
govulncheck:
runs-on: ubuntu-latest
name: Run govulncheck
steps:
- uses: actions/checkout@v4
- uses: actions/setup-go@v5
with:
go-version: ${{ env.GO_VERSION }}
cache: false
check-latest: true
- name: run govulncheck
run: |
go run build.go assets
go install golang.org/x/vuln/cmd/govulncheck@latest
govulncheck ./...

View File

@ -12,7 +12,7 @@ jobs:
name: Push to release-nightly to trigger build
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4
with:
token: ${{ secrets.ACTIONS_GITHUB_TOKEN }}
fetch-depth: 0

View File

@ -10,13 +10,13 @@ jobs:
runs-on: ubuntu-latest
name: Update translations and documentation
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4
with:
fetch-depth: 0
token: ${{ secrets.ACTIONS_GITHUB_TOKEN }}
- uses: actions/setup-go@v4
- uses: actions/setup-go@v5
with:
go-version: ^1.19.6
go-version: stable
- run: |
set -euo pipefail
git config --global user.name 'Syncthing Release Automation'

4
.gitignore vendored
View File

@ -1,11 +1,10 @@
/syncthing
/stdiscosrv
syncthing.exe
stdiscosrv.exe
*.tar.gz
*.zip
*.asc
*.deb
*.exe
.jshintrc
coverage.out
files/pidx
@ -19,4 +18,3 @@ deb
/repos
/proto/scripts/protoc-gen-gosyncthing
/gui/next-gen-gui
.idea

36
AUTHORS
View File

@ -23,9 +23,11 @@ Alessandro G. (alessandro.g89) <alessandro.g89@gmail.com>
Alex Lindeman <139387+aelindeman@users.noreply.github.com>
Alex Xu <alex.hello71@gmail.com>
Alexander Graf (alex2108) <register-github@alex-graf.de>
Alexander Seiler <seileralex@gmail.com>
Alexandre Alves <alexandrealvesdb.contact@gmail.com>
Alexandre Viau (aviau) <alexandre@alexandreviau.net> <aviau@debian.org>
Aman Gupta <aman@tmm1.net>
Anatoli Babenia <anatoli@rainforce.org>
Anderson Mesquita (andersonvom) <andersonvom@gmail.com>
Andreas Sommer <andreas.sommer87@googlemail.com>
andresvia <andres.via@gmail.com>
@ -49,6 +51,7 @@ Audrius Butkevicius (AudriusButkevicius) <audrius.butkevicius@gmail.com> <github
Aurélien Rainone <476650+arl@users.noreply.github.com>
BAHADIR YILMAZ <bahadiryilmaz32@gmail.com>
Bart De Vries (mogwa1) <devriesb@gmail.com>
Beat Reichenbach <44111292+beatreichenbach@users.noreply.github.com>
Ben Curthoys (bencurthoys) <ben@bencurthoys.com>
Ben Schulz (uok) <ueomkail@gmail.com> <uok@users.noreply.github.com>
Ben Shepherd (benshep) <bjashepherd@gmail.com>
@ -67,36 +70,45 @@ Brian R. Becker (brbecker) <brbecker@gmail.com>
bt90 <btom1990@googlemail.com>
Caleb Callaway (cqcallaw) <enlightened.despot@gmail.com>
Carsten Hagemann (carstenhag) <moter8@gmail.com> <carsten@chagemann.de>
Catfriend1 <16361913+Catfriend1@users.noreply.github.com>
Cathryne Linenweaver (Cathryne) <cathryne.linenweaver@gmail.com> <Cathryne@users.noreply.github.com> <katrinleinweber@MAC.local>
Cedric Staniewski (xduugu) <cedric@gmx.ca>
chenrui <rui@meetup.com>
Chih-Hsuan Yen <yan12125@gmail.com>
Chih-Hsuan Yen <yan12125@gmail.com> <1937689+yan12125@users.noreply.github.com>
Choongkyu <choongkyu.kim+gh@gmail.com> <vapidlyrapid+gh@gmail.com>
Chris Howie (cdhowie) <me@chrishowie.com>
Chris Joel (cdata) <chris@scriptolo.gy>
Chris Tonkinson <chris@masterbran.ch>
Christian Kujau <ckujau@users.noreply.github.com>
Christian Prescott <me@christianprescott.com>
chucic <chucic@seznam.cz>
cjc7373 <niuchangcun@gmail.com>
Colin Kennedy (moshen) <moshen.colin@gmail.com>
Cromefire_ <tim.l@nghorst.net> <26320625+cromefire@users.noreply.github.com>
cui fliter <imcusg@gmail.com>
Cyprien Devillez <cypx@users.noreply.github.com>
d-volution <49024624+d-volution@users.noreply.github.com>
Dale Visser <dale.visser@live.com>
Dan <benda.daniel@gmail.com>
Daniel Barczyk <46358936+DanielBarczyk@users.noreply.github.com>
Daniel Bergmann (brgmnn) <dan.arne.bergmann@gmail.com> <brgmnn@users.noreply.github.com>
Daniel Harte (norgeous) <daniel@harte.me> <daniel@danielharte.co.uk> <norgeous@users.noreply.github.com>
Daniel Martí (mvdan) <mvdan@mvdan.cc>
Daniel Padrta <64928366+danpadcz@users.noreply.github.com>
Darshil Chanpura (dtchanpura) <dtchanpura@gmail.com> <dcprime314@gmail.com>
David Rimmer (dinosore) <dinosore@dbrsoftware.co.uk>
deepsource-autofix[bot] <62050782+deepsource-autofix[bot]@users.noreply.github.com>
DeflateAwning <11021263+DeflateAwning@users.noreply.github.com>
Denis A. (dva) <denisva@gmail.com>
Dennis Wilson (snnd) <dw@risu.io>
dependabot-preview[bot] <dependabot-preview[bot]@users.noreply.github.com> <27856297+dependabot-preview[bot]@users.noreply.github.com>
dependabot[bot] <dependabot[bot]@users.noreply.github.com> <49699333+dependabot[bot]@users.noreply.github.com>
derekriemer <derek.riemer@colorado.edu>
DerRockWolf <50499906+DerRockWolf@users.noreply.github.com>
desbma <desbma@users.noreply.github.com>
Devon G. Redekopp <devon@redekopp.com>
diemade <spamkill@posteo.ch>
digital <didev@dinid.net>
Dimitri Papadopoulos Orfanos <3234522+DimitriPapadopoulos@users.noreply.github.com>
Dmitry Saveliev (dsaveliev) <d.e.saveliev@gmail.com>
Domenic Horner <domenic@tgxn.net>
@ -104,6 +116,7 @@ Dominik Heidler (asdil12) <dominik@heidler.eu>
Elias Jarlebring (jarlebring) <jarlebring@gmail.com>
Elliot Huffman <thelich2@gmail.com>
Emil Hessman (ceh) <emil@hessman.se>
Emil Lundberg <emil@emlun.se>
Eng Zer Jun <engzerjun@gmail.com>
entity0xfe <109791748+entity0xfe@users.noreply.github.com> <entity0xfe@my.domain>
Eric Lesiuta <elesiuta@gmail.com>
@ -112,6 +125,7 @@ Erik Meitner (WSGCSysadmin) <e.meitner@willystreet.coop>
Evan Spensley <94762716+0evan@users.noreply.github.com>
Evgeny Kuznetsov <evgeny@kuznetsov.md>
Federico Castagnini (facastagnini) <federico.castagnini@gmail.com>
Felix <53702818+f-eliks@users.noreply.github.com>
Felix Ableitner (Nutomic) <me@nutomic.com>
Felix Lampe <mail@flampe.de>
Felix Unterpaintner (bigbear2nd) <bigbear2nd@gmail.com>
@ -125,6 +139,8 @@ Gleb Sinyavskiy <zhulik.gleb@gmail.com>
Graham Miln (grahammiln) <graham.miln@dssw.co.uk> <graham.miln@miln.eu>
greatroar <61184462+greatroar@users.noreply.github.com>
Greg <gco@jazzhaiku.com>
guangwu <guoguangwu@magic-shield.com>
gudvinr <gudvinr@gmail.com>
Han Boetes <han@boetes.org>
HansK-p <42314815+HansK-p@users.noreply.github.com>
Harrison Jones (harrisonhjones) <harrisonhjones@users.noreply.github.com>
@ -141,13 +157,14 @@ Jacek Szafarkiewicz (hadogenes) <szafar@linux.pl>
Jack Croft <jccroft1@users.noreply.github.com>
Jacob <jyundt@gmail.com>
Jake Peterson (acogdev) <jake@acogdev.com>
Jakob Borg (calmh) <jakob@nym.se> <jakob@kastelo.net>
Jakob Borg (calmh) <jakob@nym.se> <jakob@kastelo.net> <jborg@coreweave.com>
James O'Beirne <wild-github@au92.org>
James Patterson (jpjp) <jamespatterson@operamail.com> <jpjp@users.noreply.github.com>
janost <janost@tuta.io>
Jaroslav Lichtblau <svetlemodry@users.noreply.github.com>
Jaroslav Malec (dzarda) <dzardacz@gmail.com>
jaseg <githubaccount@jaseg.net>
Jaspitta <ste.scarpitta@gmail.com>
Jauder Ho <jauderho@users.noreply.github.com>
Jaya Chithra (jayachithra) <s.k.jayachithra@gmail.com>
Jaya Kumar <jaya.kumar@ict.nl>
@ -166,12 +183,14 @@ Jonathan Cross <jcross@gmail.com>
Jonta <359397+Jonta@users.noreply.github.com>
Jose Manuel Delicado (jmdaweb) <jmdaweb@hotmail.com> <jmdaweb@users.noreply.github.com>
jtagcat <git-514635f7@jtag.cat> <git-12dbd862@jtag.cat>
Julian Lehrhuber <jul13579@users.noreply.github.com>
Jörg Thalheim <Mic92@users.noreply.github.com>
Jędrzej Kula <kula.jedrek@gmail.com>
K.B.Dharun Krishna <kbdharunkrishna@gmail.com>
Kalle Laine <pahakalle@protonmail.com>
Karol Różycki (krozycki) <rozycki.karol@gmail.com>
Kebin Liu <lkebin@gmail.com>
Keith Harrison <keithh@protonmail.com>
Keith Turner <kturner@apache.org>
Kelong Cong (kc1212) <kc04bc@gmx.com> <kc1212@users.noreply.github.com>
Ken'ichi Kamada (kamadak) <kamada@nanohz.org>
@ -180,6 +199,7 @@ Kevin Bushiri (keevBush) <keevbush@gmail.com> <36192217+keevBush@users.noreply.g
Kevin White, Jr. (kwhite17) <kevinwhite1710@gmail.com>
klemens <ka7@github.com>
Kurt Fitzner (Kudalufi) <kurt@va1der.ca> <kurt.fitzner@gmail.com>
kylosus <33132401+kylosus@users.noreply.github.com>
Lars K.W. Gohlke (lkwg82) <lkwg82@gmx.de>
Lars Lehtonen <lars.lehtonen@gmail.com>
Laurent Arnoud <laurent@spkdev.net>
@ -190,6 +210,7 @@ Lode Hoste (Zillode) <zillode@zillode.be>
Lord Landon Agahnim (LordLandon) <lordlandon@gmail.com>
LSmithx2 <42276854+lsmithx2@users.noreply.github.com>
Lukas Lihotzki <lukas@lihotzki.de>
Luke Hamburg <1992842+luckman212@users.noreply.github.com>
luzpaz <luzpaz@users.noreply.github.com>
Majed Abdulaziz (majedev) <majed.alhajry@gmail.com>
Marc Laporte (marclaporte) <marc@marclaporte.com> <marc@laporte.name>
@ -200,6 +221,7 @@ Marcus Legendre <marcus.legendre@gmail.com>
Mario Majila <mariustshipichik@gmail.com>
Mark Pulford (mpx) <mark@kyne.com.au>
Martchus <martchus@gmx.net>
Martin Polehla <p0l0us@users.noreply.github.com>
Mateusz Naściszewski (mateon1) <matin1111@wp.pl>
Mateusz Ż <thedead4fun@live.com>
Matic Potočnik <hairyfotr@gmail.com>
@ -211,12 +233,14 @@ Max <github@germancoding.com>
Max Schulze (kralo) <max.schulze@online.de> <kralo@users.noreply.github.com>
MaximAL <almaximal@ya.ru>
Maxime Thirouin <m@moox.io>
Maximilian <maxi.rostock@outlook.de> <public@complexvector.space>
mclang <1721600+mclang@users.noreply.github.com>
Michael Jephcote (Rewt0r) <rewt0r@gmx.com> <Rewt0r@users.noreply.github.com>
Michael Ploujnikov (plouj) <ploujj@gmail.com>
Michael Rienstra <mrienstra@gmail.com>
Michael Tilli (pyfisch) <pyfisch@gmail.com>
MichaIng <micha@dietpi.com>
Migelo <miha@filetki.si>
Mike Boone <mike@boonedocks.net>
MikeLund <MikeLund@users.noreply.github.com>
MikolajTwarog <43782609+MikolajTwarog@users.noreply.github.com>
@ -224,6 +248,7 @@ Mingxuan Lin <gdlmx@users.noreply.github.com>
mv1005 <49659413+mv1005@users.noreply.github.com>
Nate Morrison (nrm21) <natemorrison@gmail.com>
Naveen <172697+naveensrinivasan@users.noreply.github.com>
nf <nf@wh3rd.net>
Nicholas Rishel (PrototypeNM1) <rishel.nick@gmail.com> <PrototypeNM1@users.noreply.github.com>
Nick Busey <NickBusey@users.noreply.github.com>
Nico Stapelbroek <3368018+nstapelbroek@users.noreply.github.com>
@ -235,6 +260,7 @@ NinoM4ster <ninom4ster@gmail.com>
Nitroretro <43112364+Nitroretro@users.noreply.github.com>
NoLooseEnds <jon.koslung@gmail.com>
Oliver Freyermuth <o.freyermuth@googlemail.com>
orangekame3 <miya.org.0309@gmail.com>
otbutz <tbutz@optitool.de>
Otiel <Otiel@users.noreply.github.com>
overkill <22098433+0verk1ll@users.noreply.github.com>
@ -273,6 +299,7 @@ Sacheendra Talluri (sacheendra) <sacheendra.t@gmail.com>
Scott Klupfel (kluppy) <kluppy@going2blue.com>
sec65 <106604020+sec65@users.noreply.github.com>
Sergey Mishin (ralder) <ralder@yandex.ru>
Sertonix <83883937+Sertonix@users.noreply.github.com>
Shaarad Dalvi <60266155+shaaraddalvi@users.noreply.github.com> <shdalv@microsoft.com>
Simon Frei (imsodin) <freisim93@gmail.com>
Simon Mwepu <simonmwepu@gmail.com>
@ -281,12 +308,15 @@ Stefan Kuntz (Stefan-Code) <stefan.github@gmail.com> <Stefan.github@gmail.com>
Stefan Tatschner (rumpelsepp) <stefan@sevenbyte.org> <rumpelsepp@sevenbyte.org> <stefan@rumpelsepp.org>
Steven Eckhoff <steven.eckhoff.opensource@gmail.com>
Suhas Gundimeda (snugghash) <suhas.gundimeda@gmail.com> <snugghash@gmail.com>
Sven Bachmann <dev@mcbachmann.de>
Syncthing Automation <automation@syncthing.net>
Syncthing Release Automation <release@syncthing.net>
Taylor Khan (nelsonkhan) <nelsonkhan@gmail.com>
Thomas <9749173+uhthomas@users.noreply.github.com>
Thomas Hipp <thomashipp@gmail.com>
Tim Abell (timabell) <tim@timwise.co.uk>
Tim Howes (timhowes) <timhowes@berkeley.edu>
Tim Nordenfur <tim@gurka.se>
Tobias Klauser <tobias.klauser@gmail.com>
Tobias Nygren (tnn2) <tnn@nygren.pp.se>
Tobias Tom (tobiastom) <t.tom@succont.de>
@ -297,6 +327,7 @@ Tully Robinson (tojrobinson) <tully@tojr.org>
Tyler Brazier (tylerbrazier) <tyler@tylerbrazier.com>
Tyler Kropp <kropptyler@gmail.com>
Unrud (Unrud) <unrud@openaliasbox.org> <Unrud@users.noreply.github.com>
vapatel2 <149737089+vapatel2@users.noreply.github.com>
Veeti Paananen (veeti) <veeti.paananen@rojekti.fi>
Victor Buinsky (buinsky) <vix_booja@tut.by>
Vik <63919734+ViktorOn@users.noreply.github.com>
@ -304,6 +335,7 @@ Vil Brekin (Vilbrekin) <vilbrekin@gmail.com>
villekalliomaki <53118179+villekalliomaki@users.noreply.github.com>
Vladimir Rusinov <vrusinov@google.com> <vladimir.rusinov@gmail.com>
wangguoliang <liangcszzu@163.com>
Will Rouesnel <wrouesnel@wrouesnel.com>
William A. Kennington III (wkennington) <william@wkennington.com>
wouter bolsterlee <wouter@bolsterl.ee>
Wulf Weich (wweich) <wweich@users.noreply.github.com> <wweich@gmx.de> <wulf@weich-kr.de>

View File

@ -1,15 +1,41 @@
ARG GOVERSION=latest
#
# Maybe build Syncthing. This is a bit ugly as we can't make an entire
# section of the Dockerfile conditional, so we end up always pulling the
# golang image as builder. Then we check if the executable we need already
# exists (pre-built) otherwise we build it.
#
FROM golang:$GOVERSION AS builder
ARG BUILD_USER
ARG BUILD_HOST
ARG TARGETARCH
WORKDIR /src
COPY . .
ENV CGO_ENABLED=0
ENV BUILD_HOST=syncthing.net
ENV BUILD_USER=docker
RUN rm -f syncthing && go run build.go -no-upgrade build syncthing
RUN if [ ! -f syncthing-linux-$TARGETARCH ] ; then \
go run build.go -no-upgrade build syncthing ; \
mv syncthing syncthing-linux-$TARGETARCH ; \
fi
#
# The rest of the Dockerfile uses the binary from the builder, prebuilt or
# not.
#
FROM alpine
ARG TARGETARCH
LABEL org.opencontainers.image.authors="The Syncthing Project" \
org.opencontainers.image.url="https://syncthing.net" \
org.opencontainers.image.documentation="https://docs.syncthing.net" \
org.opencontainers.image.source="https://github.com/syncthing/syncthing" \
org.opencontainers.image.vendor="The Syncthing Project" \
org.opencontainers.image.licenses="MPL-2.0" \
org.opencontainers.image.title="Syncthing"
EXPOSE 8384 22000/tcp 22000/udp 21027/udp
@ -17,7 +43,7 @@ VOLUME ["/var/syncthing"]
RUN apk add --no-cache ca-certificates curl libcap su-exec tzdata
COPY --from=builder /src/syncthing /bin/syncthing
COPY --from=builder /src/syncthing-linux-$TARGETARCH /bin/syncthing
COPY --from=builder /src/script/docker-entrypoint.sh /bin/entrypoint.sh
ENV PUID=1000 PGID=1000 HOME=/var/syncthing
@ -26,5 +52,6 @@ HEALTHCHECK --interval=1m --timeout=10s \
CMD curl -fkLsS -m 2 127.0.0.1:8384/rest/noauth/health | grep -o --color=never OK || exit 1
ENV STGUIADDRESS=0.0.0.0:8384
ENV STHOMEDIR=/var/syncthing/config
RUN chmod 755 /bin/entrypoint.sh
ENTRYPOINT ["/bin/entrypoint.sh", "/bin/syncthing", "-home", "/var/syncthing/config"]
ENTRYPOINT ["/bin/entrypoint.sh", "/bin/syncthing"]

View File

@ -1,6 +1,14 @@
ARG GOVERSION=latest
FROM golang:$GOVERSION
LABEL org.opencontainers.image.authors="The Syncthing Project" \
org.opencontainers.image.url="https://syncthing.net" \
org.opencontainers.image.documentation="https://docs.syncthing.net" \
org.opencontainers.image.source="https://github.com/syncthing/syncthing" \
org.opencontainers.image.vendor="The Syncthing Project" \
org.opencontainers.image.licenses="MPL-2.0" \
org.opencontainers.image.title="Syncthing Builder"
# FPM to build Debian packages
RUN apt-get update && apt-get install -y --no-install-recommends \
locales rubygems ruby-dev build-essential git \

View File

@ -1,19 +0,0 @@
FROM alpine
ARG TARGETARCH
EXPOSE 8384 22000/tcp 22000/udp 21027/udp
VOLUME ["/var/syncthing"]
RUN apk add --no-cache ca-certificates curl libcap su-exec tzdata
COPY ./syncthing-linux-$TARGETARCH /bin/syncthing
COPY ./script/docker-entrypoint.sh /bin/entrypoint.sh
ENV PUID=1000 PGID=1000 HOME=/var/syncthing
HEALTHCHECK --interval=1m --timeout=10s \
CMD curl -fkLsS -m 2 127.0.0.1:8384/rest/noauth/health | grep -o --color=never OK || exit 1
ENV STGUIADDRESS=0.0.0.0:8384
ENTRYPOINT ["/bin/entrypoint.sh", "/bin/syncthing", "-home", "/var/syncthing/config"]

View File

@ -1,18 +1,16 @@
ARG GOVERSION=latest
FROM golang:$GOVERSION AS builder
WORKDIR /src
COPY . .
ENV CGO_ENABLED=0
ENV BUILD_HOST=syncthing.net
ENV BUILD_USER=docker
RUN rm -f stcrashreceiver && go run build.go build stcrashreceiver
FROM alpine
ARG TARGETARCH
LABEL org.opencontainers.image.authors="The Syncthing Project" \
org.opencontainers.image.url="https://syncthing.net" \
org.opencontainers.image.documentation="https://docs.syncthing.net" \
org.opencontainers.image.source="https://github.com/syncthing/syncthing" \
org.opencontainers.image.vendor="The Syncthing Project" \
org.opencontainers.image.licenses="MPL-2.0" \
org.opencontainers.image.title="Syncthing Crash Receiver"
EXPOSE 8080
COPY --from=builder /src/stcrashreceiver /bin/stcrashreceiver
COPY stcrashreceiver-linux-${TARGETARCH} /bin/stcrashreceiver
ENTRYPOINT [ "/bin/stcrashreceiver" ]

View File

@ -1,15 +1,28 @@
ARG GOVERSION=latest
FROM golang:$GOVERSION AS builder
ARG BUILD_USER
ARG BUILD_HOST
ARG TARGETARCH
WORKDIR /src
COPY . .
ENV CGO_ENABLED=0
ENV BUILD_HOST=syncthing.net
ENV BUILD_USER=docker
RUN rm -f stdiscosrv && go run build.go -no-upgrade build stdiscosrv
RUN if [ ! -f stdiscosrv-linux-$TARGETARCH ] ; then \
go run build.go -no-upgrade build stdiscosrv ; \
mv stdiscosrv stdiscosrv-linux-$TARGETARCH ; \
fi
FROM alpine
ARG TARGETARCH
LABEL org.opencontainers.image.authors="The Syncthing Project" \
org.opencontainers.image.url="https://syncthing.net" \
org.opencontainers.image.documentation="https://docs.syncthing.net" \
org.opencontainers.image.source="https://github.com/syncthing/syncthing" \
org.opencontainers.image.vendor="The Syncthing Project" \
org.opencontainers.image.licenses="MPL-2.0" \
org.opencontainers.image.title="Syncthing Discovery Server"
EXPOSE 19200 8443
@ -17,7 +30,7 @@ VOLUME ["/var/stdiscosrv"]
RUN apk add --no-cache ca-certificates su-exec
COPY --from=builder /src/stdiscosrv /bin/stdiscosrv
COPY --from=builder /src/stdiscosrv-linux-$TARGETARCH /bin/stdiscosrv
COPY --from=builder /src/script/docker-entrypoint.sh /bin/entrypoint.sh
ENV PUID=1000 PGID=1000 HOME=/var/stdiscosrv

View File

@ -1,15 +1,13 @@
ARG GOVERSION=latest
FROM golang:$GOVERSION AS builder
WORKDIR /src
COPY . .
ENV CGO_ENABLED=0
ENV BUILD_HOST=syncthing.net
ENV BUILD_USER=docker
RUN rm -f strelaysrv && go run build.go -no-upgrade build strelaypoolsrv
FROM alpine
ARG TARGETARCH
LABEL org.opencontainers.image.authors="The Syncthing Project" \
org.opencontainers.image.url="https://syncthing.net" \
org.opencontainers.image.documentation="https://docs.syncthing.net" \
org.opencontainers.image.source="https://github.com/syncthing/syncthing" \
org.opencontainers.image.vendor="The Syncthing Project" \
org.opencontainers.image.licenses="MPL-2.0" \
org.opencontainers.image.title="Syncthing Relay Pool Server"
EXPOSE 8080
@ -19,8 +17,8 @@ ENV PUID=1000 PGID=1000 MAXMIND_KEY=
RUN mkdir /var/strelaypoolsrv && chown 1000 /var/strelaypoolsrv
USER 1000
COPY --from=builder /src/strelaypoolsrv /bin/strelaypoolsrv
COPY --from=builder /src/script/strelaypoolsrv-entrypoint.sh /bin/entrypoint.sh
COPY strelaypoolsrv-linux-${TARGETARCH} /bin/strelaypoolsrv
COPY script/strelaypoolsrv-entrypoint.sh /bin/entrypoint.sh
WORKDIR /var/strelaypoolsrv
ENTRYPOINT ["/bin/entrypoint.sh", "/bin/strelaypoolsrv", "-listen", ":8080"]

View File

@ -1,15 +1,28 @@
ARG GOVERSION=latest
FROM golang:$GOVERSION AS builder
ARG BUILD_USER
ARG BUILD_HOST
ARG TARGETARCH
WORKDIR /src
COPY . .
ENV CGO_ENABLED=0
ENV BUILD_HOST=syncthing.net
ENV BUILD_USER=docker
RUN rm -f strelaysrv && go run build.go -no-upgrade build strelaysrv
RUN if [ ! -f strelaysrv-linux-$TARGETARCH ] ; then \
go run build.go -no-upgrade build strelaysrv ; \
mv strelaysrv strelaysrv-linux-$TARGETARCH ; \
fi
FROM alpine
ARG TARGETARCH
LABEL org.opencontainers.image.authors="The Syncthing Project" \
org.opencontainers.image.url="https://syncthing.net" \
org.opencontainers.image.documentation="https://docs.syncthing.net" \
org.opencontainers.image.source="https://github.com/syncthing/syncthing" \
org.opencontainers.image.vendor="The Syncthing Project" \
org.opencontainers.image.licenses="MPL-2.0" \
org.opencontainers.image.title="Syncthing Relay Server"
EXPOSE 22067 22070
@ -17,7 +30,7 @@ VOLUME ["/var/strelaysrv"]
RUN apk add --no-cache ca-certificates su-exec
COPY --from=builder /src/strelaysrv /bin/strelaysrv
COPY --from=builder /src/strelaysrv-linux-$TARGETARCH /bin/strelaysrv
COPY --from=builder /src/script/docker-entrypoint.sh /bin/entrypoint.sh
ENV PUID=1000 PGID=1000 HOME=/var/strelaysrv

View File

@ -1,18 +1,16 @@
ARG GOVERSION=latest
FROM golang:$GOVERSION AS builder
WORKDIR /src
COPY . .
ENV CGO_ENABLED=0
ENV BUILD_HOST=syncthing.net
ENV BUILD_USER=docker
RUN rm -f stupgrades && go run build.go build stupgrades
FROM alpine
ARG TARGETARCH
LABEL org.opencontainers.image.authors="The Syncthing Project" \
org.opencontainers.image.url="https://syncthing.net" \
org.opencontainers.image.documentation="https://docs.syncthing.net" \
org.opencontainers.image.source="https://github.com/syncthing/syncthing" \
org.opencontainers.image.vendor="The Syncthing Project" \
org.opencontainers.image.licenses="MPL-2.0" \
org.opencontainers.image.title="Syncthing Upgrades"
EXPOSE 8080
COPY --from=builder /src/stupgrades /bin/stupgrades
COPY stupgrades-linux-${TARGETARCH} /bin/stupgrades
ENTRYPOINT [ "/bin/stupgrades" ]

16
Dockerfile.ursrv Normal file
View File

@ -0,0 +1,16 @@
FROM alpine
ARG TARGETARCH
LABEL org.opencontainers.image.authors="The Syncthing Project" \
org.opencontainers.image.url="https://syncthing.net" \
org.opencontainers.image.documentation="https://docs.syncthing.net" \
org.opencontainers.image.source="https://github.com/syncthing/syncthing" \
org.opencontainers.image.vendor="The Syncthing Project" \
org.opencontainers.image.licenses="MPL-2.0" \
org.opencontainers.image.title="Syncthing Usage Reporting Server"
EXPOSE 8080
COPY ursrv-linux-${TARGETARCH} /bin/ursrv
ENTRYPOINT [ "/bin/ursrv" ]

View File

@ -24,17 +24,17 @@ to avoid corrupting the user's files.
### 2. Secure Against Attackers
Again, protecting the user's data is paramount. Regardless of our other
goals we must never allow the user's data to be susceptible to eavesdropping
goals, we must never allow the user's data to be susceptible to eavesdropping
or modification by unauthorized parties.
> This should be understood in context. It is not necessarily reasonable to
> expect Syncthing to be resistant against well equipped state level
> attackers. We will however do our best. Note also that this is different
> attackers. We will, however, do our best. Note also that this is different
> from anonymity which is not, currently, a goal.
### 3. Easy to Use
Syncthing should be approachable, understandable and inclusive.
Syncthing should be approachable, understandable, and inclusive.
> Complex concepts and maths form the base of Syncthing's functionality.
> This should nonetheless be abstracted or hidden to a degree where
@ -52,18 +52,18 @@ User interaction should be required only when absolutely necessary.
### 5. Universally Available
Syncthing should run on every common computer. We are mindful that the
latest technology is not always available to any given individual.
latest technology is not always available to every individual.
> Computers include desktops, laptops, servers, virtual machines, small
> general purpose computers such as Raspberry Pis and, *where possible*,
> tablets and phones. NAS appliances, toasters, cars, firearms, thermostats
> tablets and phones. NAS appliances, toasters, cars, firearms, thermostats,
> and so on may include computing capabilities but it is not our goal for
> Syncthing to run smoothly on these devices.
### 6. For Individuals
Syncthing is primarily about empowering the individual user with safe,
secure and easy to use file synchronization.
secure, and easy to use file synchronization.
> We acknowledge that it's also useful in an enterprise setting and include
> functionality to support that. If this is in conflict with the

View File

@ -15,6 +15,9 @@ To grant Syncthing additional capabilities without running as root, use the
`PCAP` environment variable with the same syntax as that for `setcap(8)`.
For example, `PCAP=cap_chown,cap_fowner+ep`.
To set a different umask value, use the `UMASK` environment variable. For
example `UMASK=002`.
## Example Usage
**Docker cli**

View File

@ -2,9 +2,6 @@
---
[![Latest Linux & Cross Build](https://img.shields.io/teamcity/https/build.syncthing.net/s/Syncthing_BuildLinuxCross.svg?style=flat-square&label=linux+%26+cross+build)](https://build.syncthing.net/viewType.html?buildTypeId=Syncthing_BuildLinuxCross&guest=1)
[![Latest Windows Build](https://img.shields.io/teamcity/https/build.syncthing.net/s/Syncthing_BuildWindows.svg?style=flat-square&label=windows+build)](https://build.syncthing.net/viewType.html?buildTypeId=Syncthing_BuildWindows&guest=1)
[![Latest Mac Build](https://img.shields.io/teamcity/https/build.syncthing.net/s/Syncthing_BuildMac.svg?style=flat-square&label=mac+build)](https://build.syncthing.net/viewType.html?buildTypeId=Syncthing_BuildMac&guest=1)
[![MPLv2 License](https://img.shields.io/badge/license-MPLv2-blue.svg?style=flat-square)](https://www.mozilla.org/MPL/2.0/)
[![CII Best Practices](https://bestpractices.coreinfrastructure.org/projects/88/badge)](https://bestpractices.coreinfrastructure.org/projects/88)
[![Go Report Card](https://goreportcard.com/badge/github.com/syncthing/syncthing)](https://goreportcard.com/report/github.com/syncthing/syncthing)
@ -13,8 +10,8 @@
Syncthing is a **continuous file synchronization program**. It synchronizes
files between two or more computers. We strive to fulfill the goals below.
The goals are listed in order of importance, the most important one being
the first. This is the summary version of the goal list - for more
The goals are listed in order of importance, the most important ones first.
This is the summary version of the goal list - for more
commentary, see the full [Goals document][13].
Syncthing should be:
@ -27,12 +24,12 @@ Syncthing should be:
2. **Secure Against Attackers**
Again, protecting the user's data is paramount. Regardless of our other
goals we must never allow the user's data to be susceptible to
goals, we must never allow the user's data to be susceptible to
eavesdropping or modification by unauthorized parties.
3. **Easy to Use**
Syncthing should be approachable, understandable and inclusive.
Syncthing should be approachable, understandable, and inclusive.
4. **Automatic**
@ -41,12 +38,12 @@ Syncthing should be:
5. **Universally Available**
Syncthing should run on every common computer. We are mindful that the
latest technology is not always available to any given individual.
latest technology is not always available to every individual.
6. **For Individuals**
Syncthing is primarily about empowering the individual user with safe,
secure and easy to use file synchronization.
secure, and easy to use file synchronization.
7. **Everything Else**
@ -60,23 +57,22 @@ Take a look at the [getting started guide][2].
There are a few examples for keeping Syncthing running in the background
on your system in [the etc directory][3]. There are also several [GUI
implementations][11] for Windows, Mac and Linux.
implementations][11] for Windows, Mac, and Linux.
## Docker
To run Syncthing in Docker, see [the Docker README][16].
## Vote on features/bugs
We'd like to encourage you to [vote][12] on issues that matter to you.
This helps the team understand what are the biggest pain points for our users, and could potentially influence what is being worked on next.
## Getting in Touch
The first and best point of contact is the [Forum][8].
If you've found something that is clearly a
bug, feel free to report it in the [GitHub issue tracker][10].
If you believe that youve found a Syncthing-related security vulnerability,
please report it by emailing security@syncthing.net. Do not report it in the
Forum or issue tracker.
## Building
Building Syncthing from source is easy. After extracting the source bundle from
@ -86,11 +82,11 @@ build process.
## Signed Releases
As of v0.10.15 and onwards release binaries are GPG signed with the key
D26E6ED000654A3E, available from https://syncthing.net/security.html and
As of v0.10.15 and onwards, release binaries are GPG signed with the key
D26E6ED000654A3E, available from https://syncthing.net/security/ and
most key servers.
There is also a built in automatic upgrade mechanism (disabled in some
There is also a built-in automatic upgrade mechanism (disabled in some
distribution channels) which uses a compiled in ECDSA signature. macOS
binaries are also properly code signed.
@ -109,7 +105,6 @@ All code is licensed under the [MPLv2 License][7].
[8]: https://forum.syncthing.net/
[10]: https://github.com/syncthing/syncthing/issues
[11]: https://docs.syncthing.net/users/contrib.html#gui-wrappers
[12]: https://www.bountysource.com/teams/syncthing/issues
[13]: https://github.com/syncthing/syncthing/blob/main/GOALS.md
[14]: assets/logo-text-128.png
[15]: https://syncthing.net/

View File

@ -33,6 +33,7 @@ import (
"text/template"
"time"
_ "github.com/syncthing/syncthing/lib/automaxprocs"
buildpkg "github.com/syncthing/syncthing/lib/build"
)
@ -214,11 +215,17 @@ var targets = map[string]target{
binaryName: "stupgrades",
},
"stcrashreceiver": {
name: "stupgrastcrashreceiverdes",
name: "stcrashreceiver",
description: "Syncthing Crash Server",
buildPkgs: []string{"github.com/syncthing/syncthing/cmd/stcrashreceiver"},
binaryName: "stcrashreceiver",
},
"ursrv": {
name: "ursrv",
description: "Syncthing Usage Reporting Server",
buildPkgs: []string{"github.com/syncthing/syncthing/cmd/ursrv"},
binaryName: "ursrv",
},
}
func initTargets() {

11
c-bindings/c_bindings.c Normal file
View File

@ -0,0 +1,11 @@
#include "c_bindings.h"
libst_logging_callback_function_t libst_logging_callback_function = NULL;
void libst_invoke_logging_callback(int log_level, const char *message, size_t message_size)
{
if (!libst_logging_callback_function) {
return;
}
libst_logging_callback_function(log_level, message, message_size);
}

241
c-bindings/c_bindings.go Normal file
View File

@ -0,0 +1,241 @@
package main
import (
"context"
"fmt"
"os"
"unsafe"
"path/filepath"
_ "net/http/pprof" // Need to import this to support STPROFILER.
"github.com/syncthing/syncthing/lib/build"
"github.com/syncthing/syncthing/lib/config"
"github.com/syncthing/syncthing/lib/events"
"github.com/syncthing/syncthing/lib/logger"
"github.com/syncthing/syncthing/lib/locations"
"github.com/syncthing/syncthing/lib/protocol"
"github.com/syncthing/syncthing/lib/svcutil"
"github.com/syncthing/syncthing/lib/syncthing"
"github.com/syncthing/syncthing/cmd/syncthing/cli"
"github.com/syncthing/syncthing/cmd/syncthing"
"github.com/thejerf/suture/v4"
)
// include header for required C helper functions (so the following comment is NO comment)
// #include "c_bindings.h"
import "C"
var theApp *syncthing.App
var myID protocol.DeviceID
var cliArgs []string
const (
tlsDefaultCommonName = "syncthing"
)
//export libst_own_device_id
func libst_own_device_id() string {
return myID.String()
}
//export libst_init_logging
func libst_init_logging() {
l.AddHandler(logger.LevelVerbose, func(level logger.LogLevel, msg string) {
runes := []byte(msg)
length := len(runes)
if length <= 0 {
return
}
C.libst_invoke_logging_callback(C.int(level), (*C.char)(unsafe.Pointer(&runes[0])), C.size_t(len(runes)))
})
}
//export libst_clear_cli_args
func libst_clear_cli_args(command string) {
if command == "cli" {
cliArgs = []string{}
} else {
cliArgs = []string{command}
}
}
//export libst_append_cli_arg
func libst_append_cli_arg(arg string) {
cliArgs = append(cliArgs, arg)
}
//export libst_run_cli
func libst_run_cli() int {
if err := cli.RunWithArgs(cliArgs); err != nil {
fmt.Println(err)
return 1
}
return 0
}
//export libst_run_main
func libst_run_main() int {
if err := syncthing_main.RunWithArgs(cliArgs); err != nil {
fmt.Println(err)
return 1
}
return 0
}
//export libst_run_syncthing
func libst_run_syncthing(configDir string, dataDir string, guiAddress string, guiApiKey string, verbose bool, allowNewerConfig bool, noDefaultConfig bool, skipPortProbing bool, ensureConfigDirExists bool, ensureDataDirExists bool) int {
// return if already running (for simplicity we only allow one Syncthing instance at at time for now)
if theApp != nil {
return 0
}
// set specified GUI address and API key
if guiAddress != "" {
os.Setenv("STGUIADDRESS", guiAddress)
}
if guiApiKey != "" {
os.Setenv("STGUIAPIKEY", guiApiKey)
}
// set specified config dir
if configDir != "" {
if !filepath.IsAbs(configDir) {
var err error
configDir, err = filepath.Abs(configDir)
if err != nil {
l.Warnln("Failed to make config path absolute:", err)
return 3
}
}
if err := locations.SetBaseDir(locations.ConfigBaseDir, configDir); err != nil {
l.Warnln(err)
return 3
}
}
// set specified database dir
if dataDir != "" {
if !filepath.IsAbs(dataDir) {
var err error
dataDir, err = filepath.Abs(dataDir)
if err != nil {
l.Warnln("Failed to make database path absolute:", err)
return 3
}
}
if err := locations.SetBaseDir(locations.DataBaseDir, dataDir); err != nil {
l.Warnln(err)
return 3
}
}
// ensure that the config directory exists
if ensureConfigDirExists {
if err := syncthing.EnsureDir(locations.GetBaseDir(locations.ConfigBaseDir), 0700); err != nil {
l.Warnln("Failed to create config directory:", err)
return 4
}
}
// ensure that the database directory exists
if dataDir != "" && ensureDataDirExists {
if err := syncthing.EnsureDir(locations.GetBaseDir(locations.DataBaseDir), 0700); err != nil {
l.Warnln("Failed to create database directory:", err)
return 4
}
}
// ensure that we have a certificate and key
cert, certErr := syncthing.LoadOrGenerateCertificate(
locations.Get(locations.CertFile),
locations.Get(locations.KeyFile),
)
if certErr != nil {
l.Warnln("Failed to load/generate certificate:", certErr)
return 1
}
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
// earlyService is a supervisor that runs the services needed for or
// before app startup; the event logger, and the config service.
spec := svcutil.SpecWithDebugLogger(l)
earlyService := suture.New("early", spec)
earlyService.ServeBackground(ctx)
evLogger := events.NewLogger()
earlyService.Add(evLogger)
// load config
configLocation := locations.Get(locations.ConfigFile)
l.Infoln("Loading config from:", configLocation)
cfgWrapper, cfgErr := syncthing.LoadConfigAtStartup(configLocation, cert, evLogger, allowNewerConfig, noDefaultConfig, skipPortProbing)
if cfgErr != nil {
l.Warnln("Failed to initialize config:", cfgErr)
return 2
}
if cfgService, ok := cfgWrapper.(suture.Service); ok {
earlyService.Add(cfgService)
}
// open database
dbFile := locations.Get(locations.Database)
l.Infoln("Opening database from:", dbFile)
ldb, dbErr := syncthing.OpenDBBackend(dbFile, config.TuningAuto)
if dbErr != nil {
l.Warnln("Error opening database:", dbErr)
return 4
}
appOpts := syncthing.Options{
ProfilerAddr: os.Getenv("STPROFILER"),
NoUpgrade: true,
Verbose: verbose,
}
var err error
theApp, err = syncthing.New(cfgWrapper, ldb, evLogger, cert, appOpts)
if err != nil {
l.Warnln("Failed to start Syncthing:", err)
return svcutil.ExitError.AsInt()
}
// start Syncthing and block until it has finished
returnCode := 0
if err := theApp.Start(); err != nil {
returnCode = svcutil.ExitError.AsInt()
}
returnCode = theApp.Wait().AsInt();
theApp = nil
return returnCode
}
//export libst_stop_syncthing
func libst_stop_syncthing() int {
if theApp != nil {
return int(theApp.Stop(svcutil.ExitSuccess))
} else {
return 0;
}
}
//export libst_reset_database
func libst_reset_database() {
os.RemoveAll(locations.Get(locations.Database))
}
//export libst_syncthing_version
func libst_syncthing_version() *C.char {
return C.CString(build.Version)
}
//export libst_long_syncthing_version
func libst_long_syncthing_version() *C.char {
return C.CString(build.LongVersion)
}
func main() {
// prevent "runtime.main_main·f: function main is undeclared in the main package"
}

11
c-bindings/c_bindings.h Normal file
View File

@ -0,0 +1,11 @@
#ifndef LIBSYNCTHING_INTERNAL_H
#define LIBSYNCTHING_INTERNAL_H
#include <stddef.h>
// allow registration of callback function
typedef void (*libst_logging_callback_function_t) (int logLevel, const char *msg, size_t msgSize);
extern libst_logging_callback_function_t libst_logging_callback_function;
extern void libst_invoke_logging_callback(int log_level, const char *message, size_t message_size);
#endif // LIBSYNCTHING_INTERNAL_H

15
c-bindings/debug.go Normal file
View File

@ -0,0 +1,15 @@
// Copyright (C) 2014 The Syncthing Authors.
//
// This Source Code Form is subject to the terms of the Mozilla Public
// License, v. 2.0. If a copy of the MPL was not distributed with this file,
// You can obtain one at https://mozilla.org/MPL/2.0/.
package main
import (
"github.com/syncthing/syncthing/lib/logger"
)
var (
l = logger.DefaultLogger.NewFacility("main", "Main package")
)

View File

@ -15,6 +15,7 @@ import (
"os"
"path/filepath"
_ "github.com/syncthing/syncthing/lib/automaxprocs"
"github.com/syncthing/syncthing/lib/sha256"
)

View File

@ -12,6 +12,7 @@ import (
"context"
"io"
"log"
"math"
"os"
"path/filepath"
"sort"
@ -40,7 +41,7 @@ type currentFile struct {
}
func (d *diskStore) Serve(ctx context.Context) {
if err := os.MkdirAll(d.dir, 0750); err != nil {
if err := os.MkdirAll(d.dir, 0o700); err != nil {
log.Println("Creating directory:", err)
return
}
@ -60,7 +61,7 @@ func (d *diskStore) Serve(ctx context.Context) {
case entry := <-d.inbox:
path := d.fullPath(entry.path)
if err := os.MkdirAll(filepath.Dir(path), 0755); err != nil {
if err := os.MkdirAll(filepath.Dir(path), 0o700); err != nil {
log.Println("Creating directory:", err)
continue
}
@ -75,7 +76,7 @@ func (d *diskStore) Serve(ctx context.Context) {
log.Println("Failed to compress crash report:", err)
continue
}
if err := os.WriteFile(path, buf.Bytes(), 0644); err != nil {
if err := os.WriteFile(path, buf.Bytes(), 0o600); err != nil {
log.Printf("Failed to write %s: %v", entry.path, err)
_ = os.Remove(path)
continue
@ -147,6 +148,11 @@ func (d *diskStore) clean() {
if len(d.currentFiles) > 0 {
oldest = time.Since(time.Unix(d.currentFiles[0].mtime, 0)).Truncate(time.Minute)
}
metricDiskstoreFilesTotal.Set(float64(len(d.currentFiles)))
metricDiskstoreBytesTotal.Set(float64(d.currentSize))
metricDiskstoreOldestAgeSeconds.Set(math.Round(oldest.Seconds()))
log.Printf("Clean complete: %d files, %d MB, oldest is %v ago", len(d.currentFiles), d.currentSize>>20, oldest)
}
@ -178,6 +184,11 @@ func (d *diskStore) inventory() error {
if len(d.currentFiles) > 0 {
oldest = time.Since(time.Unix(d.currentFiles[0].mtime, 0)).Truncate(time.Minute)
}
metricDiskstoreFilesTotal.Set(float64(len(d.currentFiles)))
metricDiskstoreBytesTotal.Set(float64(d.currentSize))
metricDiskstoreOldestAgeSeconds.Set(math.Round(oldest.Seconds()))
log.Printf("Inventory complete: %d files, %d MB, oldest is %v ago", len(d.currentFiles), d.currentSize>>20, oldest)
return err
}

View File

@ -21,26 +21,25 @@ import (
"net/http"
"os"
"path/filepath"
"time"
"github.com/alecthomas/kong"
raven "github.com/getsentry/raven-go"
"github.com/prometheus/client_golang/prometheus/promhttp"
_ "github.com/syncthing/syncthing/lib/automaxprocs"
"github.com/syncthing/syncthing/lib/sha256"
"github.com/syncthing/syncthing/lib/ur"
raven "github.com/getsentry/raven-go"
)
const maxRequestSize = 1 << 20 // 1 MiB
type cli struct {
Dir string `help:"Parent directory to store crash and failure reports in" env:"REPORTS_DIR" default:"."`
DSN string `help:"Sentry DSN" env:"SENTRY_DSN"`
Listen string `help:"HTTP listen address" default:":8080" env:"LISTEN_ADDRESS"`
MaxDiskFiles int `help:"Maximum number of reports on disk" default:"100000" env:"MAX_DISK_FILES"`
MaxDiskSizeMB int64 `help:"Maximum disk space to use for reports" default:"1024" env:"MAX_DISK_SIZE_MB"`
CleanInterval time.Duration `help:"Interval between cleaning up old reports" default:"12h" env:"CLEAN_INTERVAL"`
SentryQueue int `help:"Maximum number of reports to queue for sending to Sentry" default:"64" env:"SENTRY_QUEUE"`
DiskQueue int `help:"Maximum number of reports to queue for writing to disk" default:"64" env:"DISK_QUEUE"`
Dir string `help:"Parent directory to store crash and failure reports in" env:"REPORTS_DIR" default:"."`
DSN string `help:"Sentry DSN" env:"SENTRY_DSN"`
Listen string `help:"HTTP listen address" default:":8080" env:"LISTEN_ADDRESS"`
MaxDiskFiles int `help:"Maximum number of reports on disk" default:"100000" env:"MAX_DISK_FILES"`
MaxDiskSizeMB int64 `help:"Maximum disk space to use for reports" default:"1024" env:"MAX_DISK_SIZE_MB"`
SentryQueue int `help:"Maximum number of reports to queue for sending to Sentry" default:"64" env:"SENTRY_QUEUE"`
DiskQueue int `help:"Maximum number of reports to queue for writing to disk" default:"64" env:"DISK_QUEUE"`
}
func main() {
@ -69,6 +68,10 @@ func main() {
}
mux.Handle("/", cr)
mux.HandleFunc("/ping", func(w http.ResponseWriter, req *http.Request) {
w.Write([]byte("OK"))
})
mux.Handle("/metrics", promhttp.Handler())
if params.DSN != "" {
mux.HandleFunc("/newcrash/failure", handleFailureFn(params.DSN, filepath.Join(params.Dir, "failure_reports")))
@ -82,6 +85,11 @@ func main() {
func handleFailureFn(dsn, failureDir string) func(w http.ResponseWriter, req *http.Request) {
return func(w http.ResponseWriter, req *http.Request) {
result := "failure"
defer func() {
metricFailureReportsTotal.WithLabelValues(result).Inc()
}()
lr := io.LimitReader(req.Body, maxRequestSize)
bs, err := io.ReadAll(lr)
req.Body.Close()
@ -132,6 +140,7 @@ func handleFailureFn(dsn, failureDir string) func(w http.ResponseWriter, req *ht
log.Println("Failed to send failure report:", err)
} else {
log.Println("Sent failure report:", r.Description)
result = "success"
}
}
}

View File

@ -0,0 +1,40 @@
// Copyright (C) 2023 The Syncthing Authors.
//
// This Source Code Form is subject to the terms of the Mozilla Public
// License, v. 2.0. If a copy of the MPL was not distributed with this file,
// You can obtain one at https://mozilla.org/MPL/2.0/.
package main
import (
"github.com/prometheus/client_golang/prometheus"
"github.com/prometheus/client_golang/prometheus/promauto"
)
var (
metricCrashReportsTotal = promauto.NewCounterVec(prometheus.CounterOpts{
Namespace: "syncthing",
Subsystem: "crashreceiver",
Name: "crash_reports_total",
}, []string{"result"})
metricFailureReportsTotal = promauto.NewCounterVec(prometheus.CounterOpts{
Namespace: "syncthing",
Subsystem: "crashreceiver",
Name: "failure_reports_total",
}, []string{"result"})
metricDiskstoreFilesTotal = promauto.NewGauge(prometheus.GaugeOpts{
Namespace: "syncthing",
Subsystem: "crashreceiver",
Name: "diskstore_files_total",
})
metricDiskstoreBytesTotal = promauto.NewGauge(prometheus.GaugeOpts{
Namespace: "syncthing",
Subsystem: "crashreceiver",
Name: "diskstore_bytes_total",
})
metricDiskstoreOldestAgeSeconds = promauto.NewGauge(prometheus.GaugeOpts{
Namespace: "syncthing",
Subsystem: "crashreceiver",
Name: "diskstore_oldest_age_seconds",
})
)

View File

@ -40,6 +40,7 @@ type sentryService struct {
type sentryRequest struct {
reportID string
userID string
data []byte
}
@ -52,7 +53,7 @@ func (s *sentryService) Serve(ctx context.Context) {
log.Println("Failed to parse crash report:", err)
continue
}
if err := sendReport(s.dsn, pkt, req.reportID); err != nil {
if err := sendReport(s.dsn, pkt, req.userID); err != nil {
log.Println("Failed to send crash report:", err)
}
@ -62,9 +63,9 @@ func (s *sentryService) Serve(ctx context.Context) {
}
}
func (s *sentryService) Send(reportID string, data []byte) bool {
func (s *sentryService) Send(reportID, userID string, data []byte) bool {
select {
case s.inbox <- sentryRequest{reportID, data}:
case s.inbox <- sentryRequest{reportID, userID, data}:
return true
default:
return false
@ -215,7 +216,13 @@ func crashReportFingerprint(message string) []string {
}
// syncthing v1.1.4-rc.1+30-g6aaae618-dirty-crashrep "Erbium Earthworm" (go1.12.5 darwin-amd64) jb@kvin.kastelo.net 2019-05-23 16:08:14 UTC [foo, bar]
var longVersionRE = regexp.MustCompile(`syncthing\s+(v[^\s]+)\s+"([^"]+)"\s\(([^\s]+)\s+([^-]+)-([^)]+)\)\s+([^\s]+)[^\[]*(?:\[(.+)\])?$`)
// or, somewhere along the way the "+" in the version tag disappeared:
// syncthing v1.23.7-dev.26.gdf7b56ae.dirty-stversionextra "Fermium Flea" (go1.20.5 darwin-arm64) jb@ok.kastelo.net 2023-07-12 06:55:26 UTC [Some Wrapper, purego, stnoupgrade]
var (
longVersionRE = regexp.MustCompile(`syncthing\s+(v[^\s]+)\s+"([^"]+)"\s\(([^\s]+)\s+([^-]+)-([^)]+)\)\s+([^\s]+)[^\[]*(?:\[(.+)\])?$`)
gitExtraRE = regexp.MustCompile(`\.\d+\.g[0-9a-f]+`) // ".1.g6aaae618"
gitExtraSepRE = regexp.MustCompile(`[.-]`) // dot or dash
)
type version struct {
version string // "v1.1.4-rc.1+30-g6aaae618-dirty-crashrep"
@ -257,10 +264,21 @@ func parseVersion(line string) (version, error) {
builder: m[6],
}
parts := strings.Split(v.version, "+")
// Split the version tag into tag and commit. This is old style
// v1.2.3-something.4+11-g12345678 or newer with just dots
// v1.2.3-something.4.11.g12345678 or v1.2.3-dev.11.g12345678.
parts := []string{v.version}
if strings.Contains(v.version, "+") {
parts = strings.Split(v.version, "+")
} else {
idxs := gitExtraRE.FindStringIndex(v.version)
if len(idxs) > 0 {
parts = []string{v.version[:idxs[0]], v.version[idxs[0]+1:]}
}
}
v.tag = parts[0]
if len(parts) > 1 {
fields := strings.Split(parts[1], "-")
fields := gitExtraSepRE.Split(parts[1], -1)
if len(fields) >= 2 && strings.HasPrefix(fields[1], "g") {
v.commit = fields[1][1:]
}

View File

@ -44,6 +44,20 @@ func TestParseVersion(t *testing.T) {
extra: []string{"foo", "bar"},
},
},
{
longVersion: `syncthing v1.23.7-dev.26.gdf7b56ae-stversionextra "Fermium Flea" (go1.20.5 darwin-arm64) jb@ok.kastelo.net 2023-07-12 06:55:26 UTC [Some Wrapper, purego, stnoupgrade]`,
parsed: version{
version: "v1.23.7-dev.26.gdf7b56ae-stversionextra",
tag: "v1.23.7-dev",
commit: "df7b56ae",
codename: "Fermium Flea",
runtime: "go1.20.5",
goos: "darwin",
goarch: "arm64",
builder: "jb@ok.kastelo.net",
extra: []string{"Some Wrapper", "purego", "stnoupgrade"},
},
},
}
for _, tc := range cases {

View File

@ -71,6 +71,11 @@ func (r *crashReceiver) serveHead(reportID string, w http.ResponseWriter, _ *htt
// servePut accepts and stores the given report.
func (r *crashReceiver) servePut(reportID string, w http.ResponseWriter, req *http.Request) {
result := "receive_failure"
defer func() {
metricCrashReportsTotal.WithLabelValues(result).Inc()
}()
// Read at most maxRequestSize of report data.
log.Println("Receiving report", reportID)
lr := io.LimitReader(req.Body, maxRequestSize)
@ -81,13 +86,17 @@ func (r *crashReceiver) servePut(reportID string, w http.ResponseWriter, req *ht
return
}
result = "success"
// Store the report
if !r.store.Put(reportID, bs) {
log.Println("Failed to store report (queue full):", reportID)
result = "queue_failure"
}
// Send the report to Sentry
if !r.sentry.Send(reportID, bs) {
if !r.sentry.Send(reportID, userIDFor(req), bs) {
log.Println("Failed to send report to sentry (queue full):", reportID)
result = "sentry_failure"
}
}

View File

@ -15,6 +15,7 @@ import (
"strings"
"time"
_ "github.com/syncthing/syncthing/lib/automaxprocs"
"github.com/syncthing/syncthing/lib/beacon"
"github.com/syncthing/syncthing/lib/discover"
"github.com/syncthing/syncthing/lib/protocol"

View File

@ -8,6 +8,7 @@ package main
import (
"bytes"
"compress/gzip"
"context"
"crypto/tls"
"encoding/base64"
@ -15,6 +16,7 @@ import (
"encoding/pem"
"errors"
"fmt"
io "io"
"log"
"math/rand"
"net"
@ -27,6 +29,7 @@ import (
"time"
"github.com/syncthing/syncthing/lib/protocol"
"github.com/syncthing/syncthing/lib/stringutil"
)
// announcement is the format received from and sent to clients
@ -78,18 +81,10 @@ func (s *apiSrv) Serve(_ context.Context) error {
s.listener = listener
} else {
tlsCfg := &tls.Config{
Certificates: []tls.Certificate{s.cert},
ClientAuth: tls.RequestClientCert,
SessionTicketsDisabled: true,
MinVersion: tls.VersionTLS12,
CipherSuites: []uint16{
tls.TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,
tls.TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,
tls.TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA,
tls.TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA,
tls.TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA,
tls.TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA,
},
Certificates: []tls.Certificate{s.cert},
ClientAuth: tls.RequestClientCert,
MinVersion: tls.VersionTLS12,
NextProtos: []string{"h2", "http/1.1"},
}
tlsListener, err := tls.Listen("tcp", s.addr, tlsCfg)
@ -107,6 +102,7 @@ func (s *apiSrv) Serve(_ context.Context) error {
ReadTimeout: httpReadTimeout,
WriteTimeout: httpWriteTimeout,
MaxHeaderBytes: httpMaxHeaderBytes,
ErrorLog: log.New(io.Discard, "", 0),
}
err := srv.Serve(s.listener)
@ -116,8 +112,6 @@ func (s *apiSrv) Serve(_ context.Context) error {
return err
}
var topCtx = context.Background()
func (s *apiSrv) handler(w http.ResponseWriter, req *http.Request) {
t0 := time.Now()
@ -130,10 +124,10 @@ func (s *apiSrv) handler(w http.ResponseWriter, req *http.Request) {
}()
reqID := requestID(rand.Int63())
ctx := context.WithValue(topCtx, idKey, reqID)
req = req.WithContext(context.WithValue(req.Context(), idKey, reqID))
if debug {
log.Println(reqID, req.Method, req.URL)
log.Println(reqID, req.Method, req.URL, req.Proto)
}
remoteAddr := &net.TCPAddr{
@ -142,7 +136,12 @@ func (s *apiSrv) handler(w http.ResponseWriter, req *http.Request) {
}
if s.useHTTP {
remoteAddr.IP = net.ParseIP(req.Header.Get("X-Forwarded-For"))
// X-Forwarded-For can have multiple client IPs; split using the comma separator
forwardIP, _, _ := strings.Cut(req.Header.Get("X-Forwarded-For"), ",")
// net.ParseIP will return nil if leading/trailing whitespace exists; use strings.TrimSpace()
remoteAddr.IP = net.ParseIP(strings.TrimSpace(forwardIP))
if parsedPort, err := strconv.ParseInt(req.Header.Get("X-Client-Port"), 10, 0); err == nil {
remoteAddr.Port = int(parsedPort)
}
@ -159,17 +158,17 @@ func (s *apiSrv) handler(w http.ResponseWriter, req *http.Request) {
}
switch req.Method {
case "GET":
s.handleGET(ctx, lw, req)
case "POST":
s.handlePOST(ctx, remoteAddr, lw, req)
case http.MethodGet:
s.handleGET(lw, req)
case http.MethodPost:
s.handlePOST(remoteAddr, lw, req)
default:
http.Error(lw, "Method Not Allowed", http.StatusMethodNotAllowed)
}
}
func (s *apiSrv) handleGET(ctx context.Context, w http.ResponseWriter, req *http.Request) {
reqID := ctx.Value(idKey).(requestID)
func (s *apiSrv) handleGET(w http.ResponseWriter, req *http.Request) {
reqID := req.Context().Value(idKey).(requestID)
deviceID, err := protocol.DeviceIDFromString(req.URL.Query().Get("device"))
if err != nil {
@ -213,23 +212,34 @@ func (s *apiSrv) handleGET(ctx context.Context, w http.ResponseWriter, req *http
s.db.put(key, rec)
}
w.Header().Set("Retry-After", notFoundRetryAfterString(int(misses)))
afterS := notFoundRetryAfterSeconds(int(misses))
retryAfterHistogram.Observe(float64(afterS))
w.Header().Set("Retry-After", strconv.Itoa(afterS))
http.Error(w, "Not Found", http.StatusNotFound)
return
}
lookupRequestsTotal.WithLabelValues("success").Inc()
bs, _ := json.Marshal(announcement{
Seen: time.Unix(0, rec.Seen),
w.Header().Set("Content-Type", "application/json")
var bw io.Writer = w
// Use compression if the client asks for it
if strings.Contains(req.Header.Get("Accept-Encoding"), "gzip") {
w.Header().Set("Content-Encoding", "gzip")
gw := gzip.NewWriter(bw)
defer gw.Close()
bw = gw
}
json.NewEncoder(bw).Encode(announcement{
Seen: time.Unix(0, rec.Seen).Truncate(time.Second),
Addresses: addressStrs(rec.Addresses),
})
w.Header().Set("Content-Type", "application/json")
w.Write(bs)
}
func (s *apiSrv) handlePOST(ctx context.Context, remoteAddr *net.TCPAddr, w http.ResponseWriter, req *http.Request) {
reqID := ctx.Value(idKey).(requestID)
func (s *apiSrv) handlePOST(remoteAddr *net.TCPAddr, w http.ResponseWriter, req *http.Request) {
reqID := req.Context().Value(idKey).(requestID)
rawCert, err := certificateBytes(req)
if err != nil {
@ -351,13 +361,16 @@ func certificateBytes(req *http.Request) ([]byte, error) {
bs = pem.EncodeToMemory(&pem.Block{Type: "CERTIFICATE", Bytes: hdr})
} else if hdr := req.Header.Get("X-Forwarded-Tls-Client-Cert"); hdr != "" {
// Traefik 2 passtlsclientcert
// The certificate is in PEM format with url encoding but without newlines
// and start/end statements. We need to decode, reinstate the newlines every 64
//
// The certificate is in PEM format, maybe with URL encoding
// (depends on Traefik version) but without newlines and start/end
// statements. We need to decode, reinstate the newlines every 64
// character and add statements for the PEM decoder
hdr, err := url.QueryUnescape(hdr)
if err != nil {
// Decoding failed
return nil, err
if strings.Contains(hdr, "%") {
if unesc, err := url.QueryUnescape(hdr); err == nil {
hdr = unesc
}
}
for i := 64; i < len(hdr); i += 65 {
@ -365,7 +378,7 @@ func certificateBytes(req *http.Request) ([]byte, error) {
}
hdr = "-----BEGIN CERTIFICATE-----\n" + hdr
hdr = hdr + "\n-----END CERTIFICATE-----\n"
hdr += "\n-----END CERTIFICATE-----\n"
bs = []byte(hdr)
}
@ -404,13 +417,13 @@ func fixupAddresses(remote *net.TCPAddr, addresses []string) []string {
continue
}
if remote != nil {
if host == "" || ip.IsUnspecified() {
if host == "" || ip.IsUnspecified() {
if remote != nil {
// Replace the unspecified IP with the request source.
// ... unless the request source is the loopback address or
// multicast/unspecified (can't happen, really).
if remote.IP.IsLoopback() || remote.IP.IsMulticast() || remote.IP.IsUnspecified() {
if remote.IP == nil || remote.IP.IsLoopback() || remote.IP.IsMulticast() || remote.IP.IsUnspecified() {
continue
}
@ -426,11 +439,22 @@ func fixupAddresses(remote *net.TCPAddr, addresses []string) []string {
}
host = remote.IP.String()
} else {
// remote is nil, unable to determine host IP
continue
}
// If zero port was specified, use remote port.
if port == "0" && remote.Port > 0 {
}
// If zero port was specified, use remote port.
if port == "0" {
if remote != nil && remote.Port > 0 {
// use remote port
port = strconv.Itoa(remote.Port)
} else {
// unable to determine remote port
continue
}
}
@ -438,6 +462,9 @@ func fixupAddresses(remote *net.TCPAddr, addresses []string) []string {
fixed = append(fixed, uri.String())
}
// Remove duplicate addresses
fixed = stringutil.UniqueTrimmedStrings(fixed)
return fixed
}
@ -467,13 +494,13 @@ func errorRetryAfterString() string {
return strconv.Itoa(errorRetryAfterSeconds + rand.Intn(errorRetryFuzzSeconds))
}
func notFoundRetryAfterString(misses int) string {
func notFoundRetryAfterSeconds(misses int) int {
retryAfterS := notFoundRetryMinSeconds + notFoundRetryIncSeconds*misses
if retryAfterS > notFoundRetryMaxSeconds {
retryAfterS = notFoundRetryMaxSeconds
}
retryAfterS += rand.Intn(notFoundRetryFuzzSeconds)
return strconv.Itoa(retryAfterS)
return retryAfterS
}
func reannounceAfterString() string {

View File

@ -69,6 +69,14 @@ func TestFixupAddresses(t *testing.T) {
remote: addr("123.123.123.123", 9000),
in: []string{"tcp://44.44.44.44:0"},
out: []string{"tcp://44.44.44.44:9000"},
}, { // remote ip nil
remote: addr("", 9000),
in: []string{"tcp://:22000", "tcp://44.44.44.44:9000"},
out: []string{"tcp://44.44.44.44:9000"},
}, { // remote port 0
remote: addr("123.123.123.123", 0),
in: []string{"tcp://:22000", "tcp://44.44.44.44"},
out: []string{"tcp://123.123.123.123:22000"},
},
}

View File

@ -12,9 +12,12 @@ package main
import (
"context"
"log"
"net"
"net/url"
"sort"
"time"
"github.com/syncthing/syncthing/lib/sliceutil"
"github.com/syndtr/goleveldb/leveldb"
"github.com/syndtr/goleveldb/leveldb/storage"
"github.com/syndtr/goleveldb/leveldb/util"
@ -217,7 +220,7 @@ func (s *levelDBStore) statisticsServe(trigger <-chan struct{}, done chan<- stru
cutoff24h := t0.Add(-24 * time.Hour).UnixNano()
cutoff1w := t0.Add(-7 * 24 * time.Hour).UnixNano()
cutoff2Mon := t0.Add(-60 * 24 * time.Hour).UnixNano()
current, last24h, last1w, inactive, errors := 0, 0, 0, 0, 0
current, currentIPv4, currentIPv6, last24h, last1w, inactive, errors := 0, 0, 0, 0, 0, 0, 0
iter := s.db.NewIterator(&util.Range{}, nil)
for iter.Next() {
@ -232,9 +235,35 @@ func (s *levelDBStore) statisticsServe(trigger <-chan struct{}, done chan<- stru
// If there are addresses that have not expired it's a current
// record, otherwise account it based on when it was last seen
// (last 24 hours or last week) or finally as inactice.
addrs := expire(rec.Addresses, nowNanos)
switch {
case len(expire(rec.Addresses, nowNanos)) > 0:
case len(addrs) > 0:
current++
seenIPv4, seenIPv6 := false, false
for _, addr := range addrs {
uri, err := url.Parse(addr.Address)
if err != nil {
continue
}
host, _, err := net.SplitHostPort(uri.Host)
if err != nil {
continue
}
if ip := net.ParseIP(host); ip != nil && ip.To4() != nil {
seenIPv4 = true
} else if ip != nil {
seenIPv6 = true
}
if seenIPv4 && seenIPv6 {
break
}
}
if seenIPv4 {
currentIPv4++
}
if seenIPv6 {
currentIPv6++
}
case rec.Seen > cutoff24h:
last24h++
case rec.Seen > cutoff1w:
@ -258,6 +287,8 @@ func (s *levelDBStore) statisticsServe(trigger <-chan struct{}, done chan<- stru
iter.Release()
databaseKeys.WithLabelValues("current").Set(float64(current))
databaseKeys.WithLabelValues("currentIPv4").Set(float64(currentIPv4))
databaseKeys.WithLabelValues("currentIPv6").Set(float64(currentIPv6))
databaseKeys.WithLabelValues("last24h").Set(float64(last24h))
databaseKeys.WithLabelValues("last1w").Set(float64(last1w))
databaseKeys.WithLabelValues("inactive").Set(float64(inactive))
@ -352,14 +383,7 @@ func expire(addrs []DatabaseAddress, now int64) []DatabaseAddress {
i := 0
for i < len(addrs) {
if addrs[i].Expires < now {
// This item is expired. Replace it with the last in the list
// (noop if we are at the last item).
addrs[i] = addrs[len(addrs)-1]
// Wipe the last item of the list to release references to
// strings and stuff.
addrs[len(addrs)-1] = DatabaseAddress{}
// Shorten the slice.
addrs = addrs[:len(addrs)-1]
addrs = sliceutil.RemoveAndZero(addrs, i)
continue
}
i++

View File

@ -185,7 +185,7 @@ func TestFilter(t *testing.T) {
},
{
a: []DatabaseAddress{{Address: "a", Expires: 5}, {Address: "b", Expires: 15}, {Address: "c", Expires: 5}, {Address: "d", Expires: 15}, {Address: "e", Expires: 5}},
b: []DatabaseAddress{{Address: "d", Expires: 15}, {Address: "b", Expires: 15}}, // gets reordered
b: []DatabaseAddress{{Address: "b", Expires: 15}, {Address: "d", Expires: 15}},
},
}

View File

@ -14,10 +14,12 @@ import (
"net"
"net/http"
"os"
"runtime"
"strings"
"time"
"github.com/prometheus/client_golang/prometheus/promhttp"
_ "github.com/syncthing/syncthing/lib/automaxprocs"
"github.com/syncthing/syncthing/lib/build"
"github.com/syncthing/syncthing/lib/protocol"
"github.com/syncthing/syncthing/lib/tlsutil"
@ -64,9 +66,7 @@ var levelDBOptions = &opt.Options{
WriteBuffer: 32 << 20, // default 4<<20
}
var (
debug = false
)
var debug = false
func main() {
var listen string
@ -76,20 +76,26 @@ func main() {
var replicationPeers string
var certFile string
var keyFile string
var replCertFile string
var replKeyFile string
var useHTTP bool
var largeDB bool
log.SetOutput(os.Stdout)
log.SetFlags(0)
flag.StringVar(&certFile, "cert", "./cert.pem", "Certificate file")
flag.StringVar(&keyFile, "key", "./key.pem", "Key file")
flag.StringVar(&dir, "db-dir", "./discovery.db", "Database directory")
flag.BoolVar(&debug, "debug", false, "Print debug output")
flag.BoolVar(&useHTTP, "http", false, "Listen on HTTP (behind an HTTPS proxy)")
flag.StringVar(&listen, "listen", ":8443", "Listen address")
flag.StringVar(&keyFile, "key", "./key.pem", "Key file")
flag.StringVar(&metricsListen, "metrics-listen", "", "Metrics listen address")
flag.StringVar(&replicationPeers, "replicate", "", "Replication peers, id@address, comma separated")
flag.StringVar(&replicationListen, "replication-listen", ":19200", "Replication listen address")
flag.StringVar(&replCertFile, "replication-cert", "", "Certificate file for replication")
flag.StringVar(&replKeyFile, "replication-key", "", "Key file for replication")
flag.BoolVar(&largeDB, "large-db", false, "Use larger database settings")
showVersion := flag.Bool("version", false, "Show version")
flag.Parse()
@ -98,6 +104,17 @@ func main() {
return
}
buildInfo.WithLabelValues(build.Version, runtime.Version(), build.User, build.Date.UTC().Format("2006-01-02T15:04:05Z")).Set(1)
if largeDB {
levelDBOptions.BlockCacheCapacity = 64 << 20
levelDBOptions.BlockSize = 64 << 10
levelDBOptions.CompactionTableSize = 16 << 20
levelDBOptions.CompactionTableSizeMultiplier = 2.0
levelDBOptions.WriteBuffer = 64 << 20
levelDBOptions.CompactionL0Trigger = 8
}
cert, err := tls.LoadX509KeyPair(certFile, keyFile)
if os.IsNotExist(err) {
log.Println("Failed to load keypair. Generating one, this might take a while...")
@ -111,6 +128,16 @@ func main() {
devID := protocol.NewDeviceID(cert.Certificate[0])
log.Println("Server device ID is", devID)
replCert := cert
if replCertFile != "" && replKeyFile != "" {
replCert, err = tls.LoadX509KeyPair(replCertFile, replKeyFile)
if err != nil {
log.Fatalln("Failed to load replication keypair:", err)
}
}
replDevID := protocol.NewDeviceID(replCert.Certificate[0])
log.Println("Replication device ID is", replDevID)
// Parse the replication specs, if any.
var allowedReplicationPeers []protocol.DeviceID
var replicationDestinations []string
@ -165,14 +192,14 @@ func main() {
// Start any replication senders.
var repl replicationMultiplexer
for _, dst := range replicationDestinations {
rs := newReplicationSender(dst, cert, allowedReplicationPeers)
rs := newReplicationSender(dst, replCert, allowedReplicationPeers)
main.Add(rs)
repl = append(repl, rs)
}
// If we have replication configured, start the replication listener.
if len(allowedReplicationPeers) > 0 {
rl := newReplicationListener(replicationListen, cert, allowedReplicationPeers, db)
rl := newReplicationListener(replicationListen, replCert, allowedReplicationPeers, db)
main.Add(rl)
}

View File

@ -19,8 +19,11 @@ import (
"github.com/syncthing/syncthing/lib/protocol"
)
const replicationReadTimeout = time.Minute
const replicationHeartbeatInterval = time.Second * 30
const (
replicationReadTimeout = time.Minute
replicationWriteTimeout = 30 * time.Second
replicationHeartbeatInterval = time.Second * 30
)
type replicator interface {
send(key string, addrs []DatabaseAddress, seen int64)
@ -68,6 +71,12 @@ func (s *replicationSender) Serve(ctx context.Context) error {
conn.Close()
}()
// The replication stream is not especially latency sensitive, but it is
// quite a lot of data in small writes. Make it more efficient.
if tcpc, ok := conn.NetConn().(*net.TCPConn); ok {
_ = tcpc.SetNoDelay(false)
}
// Get the other side device ID.
remoteID, err := deviceID(conn)
if err != nil {
@ -116,7 +125,7 @@ func (s *replicationSender) Serve(ctx context.Context) error {
binary.BigEndian.PutUint32(buf, uint32(n))
// Send
conn.SetWriteDeadline(time.Now().Add(5 * time.Second))
conn.SetWriteDeadline(time.Now().Add(replicationWriteTimeout))
if _, err := conn.Write(buf[:4+n]); err != nil {
replicationSendsTotal.WithLabelValues("error").Inc()
log.Println("Replication write:", err)

View File

@ -14,6 +14,14 @@ import (
)
var (
buildInfo = prometheus.NewGaugeVec(
prometheus.GaugeOpts{
Namespace: "syncthing",
Subsystem: "discovery",
Name: "build_info",
Help: "A metric with a constant '1' value labeled by version, goversion, builduser and builddate from which stdiscosrv was built.",
}, []string{"version", "goversion", "builduser", "builddate"})
apiRequestsTotal = prometheus.NewCounterVec(
prometheus.CounterOpts{
Namespace: "syncthing",
@ -90,6 +98,14 @@ var (
Help: "Latency of database operations.",
Objectives: map[float64]float64{0.5: 0.05, 0.9: 0.01, 0.99: 0.001},
}, []string{"operation"})
retryAfterHistogram = prometheus.NewHistogram(prometheus.HistogramOpts{
Namespace: "syncthing",
Subsystem: "discovery",
Name: "retry_after_seconds",
Help: "Retry-After header value in seconds.",
Buckets: prometheus.ExponentialBuckets(60, 2, 7), // 60, 120, 240, 480, 960, 1920, 3840
})
)
const (
@ -104,11 +120,13 @@ const (
)
func init() {
prometheus.MustRegister(apiRequestsTotal, apiRequestsSeconds,
prometheus.MustRegister(buildInfo,
apiRequestsTotal, apiRequestsSeconds,
lookupRequestsTotal, announceRequestsTotal,
replicationSendsTotal, replicationRecvsTotal,
databaseKeys, databaseStatisticsSeconds,
databaseOperations, databaseOperationSeconds)
databaseOperations, databaseOperationSeconds,
retryAfterHistogram)
processCollectorOpts := collectors.ProcessCollectorOpts{
Namespace: "syncthing_discovery",
@ -120,5 +138,4 @@ func init() {
prometheus.MustRegister(
collectors.NewProcessCollector(processCollectorOpts),
)
}

View File

@ -14,6 +14,8 @@ import (
"net/http"
"os"
"time"
_ "github.com/syncthing/syncthing/lib/automaxprocs"
)
type event struct {

View File

@ -13,6 +13,7 @@ import (
"os"
"path/filepath"
_ "github.com/syncthing/syncthing/lib/automaxprocs"
"github.com/syncthing/syncthing/lib/protocol"
"github.com/syncthing/syncthing/lib/scanner"
)

View File

@ -16,6 +16,7 @@ import (
"os"
"time"
_ "github.com/syncthing/syncthing/lib/automaxprocs"
"github.com/syncthing/syncthing/lib/config"
"github.com/syncthing/syncthing/lib/discover"
"github.com/syncthing/syncthing/lib/events"

View File

@ -12,6 +12,7 @@ import (
"fmt"
"os"
_ "github.com/syncthing/syncthing/lib/automaxprocs"
"github.com/syncthing/syncthing/lib/fs"
"github.com/syncthing/syncthing/lib/ignore"
)

View File

@ -15,6 +15,8 @@ import (
"os"
"path/filepath"
"time"
_ "github.com/syncthing/syncthing/lib/automaxprocs"
)
func main() {
@ -43,7 +45,7 @@ func generateFiles(dir string, files, maxexp int, srcname string) error {
}
p0 := filepath.Join(dir, string(n[0]), n[0:2])
err = os.MkdirAll(p0, 0755)
err = os.MkdirAll(p0, 0o755)
if err != nil {
log.Fatal(err)
}
@ -82,7 +84,7 @@ func generateOneFile(fd io.ReadSeeker, p1 string, s int64) error {
return err
}
os.Chmod(p1, os.FileMode(rand.Intn(0777)|0400))
os.Chmod(p1, os.FileMode(rand.Intn(0o777)|0o400))
t := time.Now().Add(-time.Duration(rand.Intn(30*86400)) * time.Second)
return os.Chtimes(p1, t, t)

View File

@ -237,7 +237,7 @@
uptimeSeconds: 0,
};
$scope.map = L.map('map').setView([40.90296, 1.90925], 2);
L.tileLayer('https://{s}.tile.openstreetmap.org/{z}/{x}/{y}.png',
L.tileLayer('https://tile.openstreetmap.org/{z}/{x}/{y}.png',
{
attribution: 'Leaflet',
maxZoom: 17

View File

@ -21,14 +21,14 @@ import (
"time"
lru "github.com/hashicorp/golang-lru/v2"
"github.com/syncthing/syncthing/lib/httpcache"
"github.com/syncthing/syncthing/lib/protocol"
"github.com/oschwald/geoip2-golang"
"github.com/prometheus/client_golang/prometheus"
"github.com/prometheus/client_golang/prometheus/promhttp"
"github.com/syncthing/syncthing/cmd/strelaypoolsrv/auto"
"github.com/syncthing/syncthing/lib/assets"
_ "github.com/syncthing/syncthing/lib/automaxprocs"
"github.com/syncthing/syncthing/lib/httpcache"
"github.com/syncthing/syncthing/lib/protocol"
"github.com/syncthing/syncthing/lib/rand"
"github.com/syncthing/syncthing/lib/relay/client"
"github.com/syncthing/syncthing/lib/sync"

View File

@ -36,8 +36,14 @@ func listener(_, addr string, config *tls.Config, token string) {
for {
conn, isTLS, err := listener.AcceptNoWrapTLS()
if err != nil {
// Conn may be nil if accept failed, or non-nil if the initial
// read to figure out if it's TLS or not failed. In the latter
// case, close the connection before moving on.
if conn != nil {
conn.Close()
}
if debug {
log.Println("Listener failed to accept connection from", conn.RemoteAddr(), ". Possibly a TCP Ping.")
log.Println("Listener failed to accept:", err)
}
continue
}

View File

@ -19,19 +19,18 @@ import (
"syscall"
"time"
_ "github.com/syncthing/syncthing/lib/automaxprocs"
"github.com/syncthing/syncthing/lib/build"
"github.com/syncthing/syncthing/lib/config"
"github.com/syncthing/syncthing/lib/events"
"github.com/syncthing/syncthing/lib/nat"
"github.com/syncthing/syncthing/lib/osutil"
_ "github.com/syncthing/syncthing/lib/pmp"
syncthingprotocol "github.com/syncthing/syncthing/lib/protocol"
"github.com/syncthing/syncthing/lib/relay/protocol"
"github.com/syncthing/syncthing/lib/tlsutil"
"golang.org/x/time/rate"
"github.com/syncthing/syncthing/lib/config"
"github.com/syncthing/syncthing/lib/nat"
_ "github.com/syncthing/syncthing/lib/pmp"
_ "github.com/syncthing/syncthing/lib/upnp"
syncthingprotocol "github.com/syncthing/syncthing/lib/protocol"
"golang.org/x/time/rate"
)
var (
@ -194,7 +193,15 @@ func main() {
cfg.Options.NATTimeoutS = natTimeout
})
natSvc := nat.NewService(id, wrapper)
mapping := mapping{natSvc.NewMapping(nat.TCP, addr.IP, addr.Port)}
var ipVersion nat.IPVersion
if strings.HasSuffix(proto, "4") {
ipVersion = nat.IPv4Only
} else if strings.HasSuffix(proto, "6") {
ipVersion = nat.IPv6Only
} else {
ipVersion = nat.IPvAny
}
mapping := mapping{natSvc.NewMapping(nat.TCP, ipVersion, addr.IP, addr.Port)}
if natEnabled {
ctx, cancel := context.WithCancel(context.Background())

View File

@ -14,6 +14,7 @@ import (
"path/filepath"
"time"
_ "github.com/syncthing/syncthing/lib/automaxprocs"
syncthingprotocol "github.com/syncthing/syncthing/lib/protocol"
"github.com/syncthing/syncthing/lib/relay/client"
"github.com/syncthing/syncthing/lib/relay/protocol"

View File

@ -12,6 +12,7 @@ import (
"log"
"os"
_ "github.com/syncthing/syncthing/lib/automaxprocs"
"github.com/syncthing/syncthing/lib/signature"
"github.com/syncthing/syncthing/lib/upgrade"
)

View File

@ -19,6 +19,7 @@ import (
"time"
"github.com/alecthomas/kong"
_ "github.com/syncthing/syncthing/lib/automaxprocs"
"github.com/syncthing/syncthing/lib/httpcache"
"github.com/syncthing/syncthing/lib/upgrade"
)
@ -57,7 +58,7 @@ type githubReleases struct {
url string
}
func (p *githubReleases) ServeHTTP(w http.ResponseWriter, req *http.Request) {
func (p *githubReleases) ServeHTTP(w http.ResponseWriter, _ *http.Request) {
log.Println("Fetching", p.url)
rels := upgrade.FetchLatestReleases(p.url, "")
if rels == nil {
@ -68,6 +69,16 @@ func (p *githubReleases) ServeHTTP(w http.ResponseWriter, req *http.Request) {
sort.Sort(upgrade.SortByRelease(rels))
rels = filterForLatest(rels)
// Move the URL used for browser downloads to the URL field, and remove
// the browser URL field. This avoids going via the GitHub API for
// downloads, since Syncthing uses the URL field.
for _, rel := range rels {
for j, asset := range rel.Assets {
rel.Assets[j].URL = asset.BrowserURL
rel.Assets[j].BrowserURL = ""
}
}
buf := new(bytes.Buffer)
_ = json.NewEncoder(buf).Encode(rels)

View File

@ -26,6 +26,7 @@ import (
"sync/atomic"
"time"
_ "github.com/syncthing/syncthing/lib/automaxprocs"
"github.com/syncthing/syncthing/lib/protocol"
)
@ -157,7 +158,7 @@ func saveCert(priv interface{}, derBytes []byte) {
os.Exit(1)
}
keyOut, err := os.OpenFile("key.pem", os.O_WRONLY|os.O_CREATE|os.O_TRUNC, 0600)
keyOut, err := os.OpenFile("key.pem", os.O_WRONLY|os.O_CREATE|os.O_TRUNC, 0o600)
if err != nil {
fmt.Println(err)
os.Exit(1)

View File

@ -13,6 +13,7 @@ import (
"os"
"time"
_ "github.com/syncthing/syncthing/lib/automaxprocs"
"github.com/syncthing/syncthing/lib/sha256"
)

View File

@ -4,7 +4,7 @@
// License, v. 2.0. If a copy of the MPL was not distributed with this file,
// You can obtain one at https://mozilla.org/MPL/2.0/.
package main
package syncthing_main
import (
"fmt"

View File

@ -13,6 +13,7 @@ import (
"reflect"
"github.com/AudriusButkevicius/recli"
"github.com/alecthomas/kong"
"github.com/syncthing/syncthing/lib/config"
"github.com/urfave/cli"
)
@ -23,9 +24,20 @@ type configHandler struct {
err error
}
func getConfigCommand(f *apiClientFactory) (cli.Command, error) {
type configCommand struct {
Args []string `arg:"" default:"-h"`
}
func (c *configCommand) Run(ctx Context, _ *kong.Context) error {
app := cli.NewApp()
app.Name = "syncthing"
app.Author = "The Syncthing Authors"
app.Metadata = map[string]interface{}{
"clientFactory": ctx.clientFactory,
}
h := new(configHandler)
h.client, h.err = f.getClient()
h.client, h.err = ctx.clientFactory.getClient()
if h.err == nil {
h.cfg, h.err = getConfig(h.client)
}
@ -38,17 +50,15 @@ func getConfigCommand(f *apiClientFactory) (cli.Command, error) {
commands, err := recli.New(recliCfg).Construct(&h.cfg)
if err != nil {
return cli.Command{}, fmt.Errorf("config reflect: %w", err)
return fmt.Errorf("config reflect: %w", err)
}
return cli.Command{
Name: "config",
HideHelp: true,
Usage: "Configuration modification command group",
Subcommands: commands,
Before: h.configBefore,
After: h.configAfter,
}, nil
app.Commands = commands
app.HideHelp = true
app.Before = h.configBefore
app.After = h.configAfter
return app.Run(append([]string{app.Name}, c.Args...))
}
func (h *configHandler) configBefore(c *cli.Context) error {

View File

@ -9,47 +9,37 @@ package cli
import (
"fmt"
"net/url"
"github.com/urfave/cli"
)
var debugCommand = cli.Command{
Name: "debug",
HideHelp: true,
Usage: "Debug command group",
Subcommands: []cli.Command{
{
Name: "file",
Usage: "Show information about a file (or directory/symlink)",
ArgsUsage: "FOLDER-ID PATH",
Action: expects(2, debugFile()),
},
indexCommand,
{
Name: "profile",
Usage: "Save a profile to help figuring out what Syncthing does.",
ArgsUsage: "cpu | heap",
Action: expects(1, profile()),
},
},
type fileCommand struct {
FolderID string `arg:""`
Path string `arg:""`
}
func debugFile() cli.ActionFunc {
return func(c *cli.Context) error {
query := make(url.Values)
query.Set("folder", c.Args()[0])
query.Set("file", normalizePath(c.Args()[1]))
return indexDumpOutput("debug/file?" + query.Encode())(c)
func (f *fileCommand) Run(ctx Context) error {
indexDumpOutput := indexDumpOutputWrapper(ctx.clientFactory)
query := make(url.Values)
query.Set("folder", f.FolderID)
query.Set("file", normalizePath(f.Path))
return indexDumpOutput("debug/file?" + query.Encode())
}
type profileCommand struct {
Type string `arg:"" help:"cpu | heap"`
}
func (p *profileCommand) Run(ctx Context) error {
switch t := p.Type; t {
case "cpu", "heap":
return saveToFile(fmt.Sprintf("debug/%vprof", p.Type), ctx.clientFactory)
default:
return fmt.Errorf("expected cpu or heap as argument, got %v", t)
}
}
func profile() cli.ActionFunc {
return func(c *cli.Context) error {
switch t := c.Args()[0]; t {
case "cpu", "heap":
return saveToFile(fmt.Sprintf("debug/%vprof", c.Args()[0]))(c)
default:
return fmt.Errorf("expected cpu or heap as argument, got %v", t)
}
}
type debugCommand struct {
File fileCommand `cmd:"" help:"Show information about a file (or directory/symlink)"`
Profile profileCommand `cmd:"" help:"Save a profile to help figuring out what Syncthing does"`
Index indexCommand `cmd:"" help:"Show information about the index (database)"`
}

View File

@ -11,36 +11,25 @@ import (
"fmt"
"strings"
"github.com/urfave/cli"
"github.com/alecthomas/kong"
)
var errorsCommand = cli.Command{
Name: "errors",
HideHelp: true,
Usage: "Error command group",
Subcommands: []cli.Command{
{
Name: "show",
Usage: "Show pending errors",
Action: expects(0, indexDumpOutput("system/error")),
},
{
Name: "push",
Usage: "Push an error to active clients",
ArgsUsage: "ERROR-MESSAGE",
Action: expects(1, errorsPush),
},
{
Name: "clear",
Usage: "Clear pending errors",
Action: expects(0, emptyPost("system/error/clear")),
},
},
type errorsCommand struct {
Show struct{} `cmd:"" help:"Show pending errors"`
Push errorsPushCommand `cmd:"" help:"Push an error to active clients"`
Clear struct{} `cmd:"" help:"Clear pending errors"`
}
func errorsPush(c *cli.Context) error {
client := c.App.Metadata["client"].(APIClient)
errStr := strings.Join(c.Args(), " ")
type errorsPushCommand struct {
ErrorMessage string `arg:""`
}
func (e *errorsPushCommand) Run(ctx Context) error {
client, err := ctx.clientFactory.getClient()
if err != nil {
return err
}
errStr := e.ErrorMessage
response, err := client.Post("system/error", strings.TrimSpace(errStr))
if err != nil {
return err
@ -59,3 +48,13 @@ func errorsPush(c *cli.Context) error {
}
return nil
}
func (*errorsCommand) Run(ctx Context, kongCtx *kong.Context) error {
switch kongCtx.Selected().Name {
case "show":
return indexDumpOutput("system/error", ctx.clientFactory)
case "clear":
return emptyPost("system/error/clear", ctx.clientFactory)
}
return nil
}

View File

@ -7,32 +7,26 @@
package cli
import (
"github.com/urfave/cli"
"github.com/alecthomas/kong"
)
var indexCommand = cli.Command{
Name: "index",
Usage: "Show information about the index (database)",
Subcommands: []cli.Command{
{
Name: "dump",
Usage: "Print the entire db",
Action: expects(0, indexDump),
},
{
Name: "dump-size",
Usage: "Print the db size of different categories of information",
Action: expects(0, indexDumpSize),
},
{
Name: "check",
Usage: "Check the database for inconsistencies",
Action: expects(0, indexCheck),
},
{
Name: "account",
Usage: "Print key and value size statistics per key type",
Action: expects(0, indexAccount),
},
},
type indexCommand struct {
Dump struct{} `cmd:"" help:"Print the entire db"`
DumpSize struct{} `cmd:"" help:"Print the db size of different categories of information"`
Check struct{} `cmd:"" help:"Check the database for inconsistencies"`
Account struct{} `cmd:"" help:"Print key and value size statistics per key type"`
}
func (*indexCommand) Run(kongCtx *kong.Context) error {
switch kongCtx.Selected().Name {
case "dump":
return indexDump()
case "dump-size":
return indexDumpSize()
case "check":
return indexCheck()
case "account":
return indexAccount()
}
return nil
}

View File

@ -10,12 +10,10 @@ import (
"fmt"
"os"
"text/tabwriter"
"github.com/urfave/cli"
)
// indexAccount prints key and data size statistics per class
func indexAccount(*cli.Context) error {
func indexAccount() error {
ldb, err := getDB()
if err != nil {
return err

View File

@ -11,13 +11,11 @@ import (
"fmt"
"time"
"github.com/urfave/cli"
"github.com/syncthing/syncthing/lib/db"
"github.com/syncthing/syncthing/lib/protocol"
)
func indexDump(*cli.Context) error {
func indexDump() error {
ldb, err := getDB()
if err != nil {
return err

View File

@ -11,12 +11,10 @@ import (
"fmt"
"sort"
"github.com/urfave/cli"
"github.com/syncthing/syncthing/lib/db"
)
func indexDumpSize(*cli.Context) error {
func indexDumpSize() error {
type sizedElement struct {
key string
size int

View File

@ -13,8 +13,6 @@ import (
"fmt"
"sort"
"github.com/urfave/cli"
"github.com/syncthing/syncthing/lib/db"
"github.com/syncthing/syncthing/lib/protocol"
)
@ -35,7 +33,7 @@ type sequenceKey struct {
sequence uint64
}
func indexCheck(*cli.Context) (err error) {
func indexCheck() (err error) {
ldb, err := getDB()
if err != nil {
return err

View File

@ -7,166 +7,72 @@
package cli
import (
"bufio"
"errors"
"fmt"
"io"
"os"
"strings"
"github.com/alecthomas/kong"
"github.com/flynn-archive/go-shlex"
"github.com/urfave/cli"
"github.com/willabides/kongplete"
"github.com/syncthing/syncthing/cmd/syncthing/cmdutil"
"github.com/syncthing/syncthing/lib/config"
)
type preCli struct {
type CLI struct {
cmdutil.CommonOptions
DataDir string `name:"data" placeholder:"PATH" env:"STDATADIR" help:"Set data directory (database and logs)"`
GUIAddress string `name:"gui-address"`
GUIAPIKey string `name:"gui-apikey"`
HomeDir string `name:"home"`
ConfDir string `name:"config"`
DataDir string `name:"data"`
Show showCommand `cmd:"" help:"Show command group"`
Debug debugCommand `cmd:"" help:"Debug command group"`
Operations operationCommand `cmd:"" help:"Operation command group"`
Errors errorsCommand `cmd:"" help:"Error command group"`
Config configCommand `cmd:"" help:"Configuration modification command group" passthrough:""`
Stdin stdinCommand `cmd:"" name:"-" help:"Read commands from stdin"`
}
func Run() error {
// This is somewhat a hack around a chicken and egg problem. We need to set
// the home directory and potentially other flags to know where the
// syncthing instance is running in order to get it's config ... which we
// then use to construct the actual CLI ... at which point it's too late to
// add flags there...
c := preCli{}
parseFlags(&c)
return runInternal(c, os.Args)
type Context struct {
clientFactory *apiClientFactory
}
func RunWithArgs(cliArgs []string) error {
c := preCli{}
parseFlagsWithArgs(cliArgs, &c)
return runInternal(c, cliArgs)
}
func runInternal(c preCli, cliArgs []string) error {
// Not set as default above because the strings can be really long.
err := cmdutil.SetConfigDataLocationsFromFlags(c.HomeDir, c.ConfDir, c.DataDir)
func (cli CLI) AfterApply(kongCtx *kong.Context) error {
err := cmdutil.SetConfigDataLocationsFromFlags(cli.HomeDir, cli.ConfDir, cli.DataDir)
if err != nil {
return fmt.Errorf("Command line options: %w", err)
return fmt.Errorf("command line options: %w", err)
}
clientFactory := &apiClientFactory{
cfg: config.GUIConfiguration{
RawAddress: c.GUIAddress,
APIKey: c.GUIAPIKey,
RawAddress: cli.GUIAddress,
APIKey: cli.GUIAPIKey,
},
}
configCommand, err := getConfigCommand(clientFactory)
context := Context{
clientFactory: clientFactory,
}
kongCtx.Bind(context)
return nil
}
type stdinCommand struct{}
func RunWithArgs(args []string) error {
var cli CLI
p, err := kong.New(&cli)
if err != nil {
// can't happen, really
return fmt.Errorf("creating parser: %w", err)
}
kongplete.Complete(p)
ctx, err := p.Parse(args)
if err != nil {
fmt.Println("Error:", err)
return err
}
// Implement the same flags at the upper CLI, but do nothing with them.
// This is so that the usage text is the same
fakeFlags := []cli.Flag{
cli.StringFlag{
Name: "gui-address",
Usage: "Override GUI address to `URL` (e.g. \"192.0.2.42:8443\")",
},
cli.StringFlag{
Name: "gui-apikey",
Usage: "Override GUI API key to `API-KEY`",
},
cli.StringFlag{
Name: "home",
Usage: "Set configuration and data directory to `PATH`",
},
cli.StringFlag{
Name: "config",
Usage: "Set configuration directory (config and keys) to `PATH`",
},
cli.StringFlag{
Name: "data",
Usage: "Set data directory (database and logs) to `PATH`",
},
}
// Construct the actual CLI
app := cli.NewApp()
app.Author = "The Syncthing Authors"
app.Metadata = map[string]interface{}{
"clientFactory": clientFactory,
}
app.Commands = []cli.Command{{
Name: "cli",
Usage: "Syncthing command line interface",
Flags: fakeFlags,
Subcommands: []cli.Command{
configCommand,
showCommand,
operationCommand,
errorsCommand,
debugCommand,
{
Name: "-",
HideHelp: true,
Usage: "Read commands from stdin",
Action: func(ctx *cli.Context) error {
if ctx.NArg() > 0 {
return errors.New("command does not expect any arguments")
}
// Drop the `-` not to recurse into self.
args := make([]string, len(cliArgs)-1)
copy(args, cliArgs)
fmt.Println("Reading commands from stdin...", args)
scanner := bufio.NewScanner(os.Stdin)
for scanner.Scan() {
input, err := shlex.Split(scanner.Text())
if err != nil {
return fmt.Errorf("parsing input: %w", err)
}
if len(input) == 0 {
continue
}
err = app.Run(append(args, input...))
if err != nil {
return err
}
}
return scanner.Err()
},
},
},
}}
return app.Run(cliArgs)
}
func parseFlags(c *preCli) error {
// kong only needs to parse the global arguments after "cli" and before the
// subcommand (if any).
if len(os.Args) <= 2 {
return nil
}
return parseFlagsWithArgs(os.Args[2:], c)
}
func parseFlagsWithArgs(args []string, c *preCli) error {
for i := 0; i < len(args); i++ {
if !strings.HasPrefix(args[i], "--") {
args = args[:i]
break
}
if !strings.Contains(args[i], "=") {
i++
}
}
// We don't want kong to print anything nor os.Exit (e.g. on -h)
parser, err := kong.New(c, kong.Writers(io.Discard, io.Discard), kong.Exit(func(int) {}))
if err != nil {
if err := ctx.Run(); err != nil {
fmt.Println("Error:", err)
return err
}
_, err = parser.Parse(args)
return err
return nil
}

View File

@ -12,48 +12,43 @@ import (
"fmt"
"path/filepath"
"github.com/alecthomas/kong"
"github.com/syncthing/syncthing/lib/config"
"github.com/syncthing/syncthing/lib/fs"
"github.com/urfave/cli"
)
var operationCommand = cli.Command{
Name: "operations",
HideHelp: true,
Usage: "Operation command group",
Subcommands: []cli.Command{
{
Name: "restart",
Usage: "Restart syncthing",
Action: expects(0, emptyPost("system/restart")),
},
{
Name: "shutdown",
Usage: "Shutdown syncthing",
Action: expects(0, emptyPost("system/shutdown")),
},
{
Name: "upgrade",
Usage: "Upgrade syncthing (if a newer version is available)",
Action: expects(0, emptyPost("system/upgrade")),
},
{
Name: "folder-override",
Usage: "Override changes on folder (remote for sendonly, local for receiveonly). WARNING: Destructive - deletes/changes your data.",
ArgsUsage: "FOLDER-ID",
Action: expects(1, foldersOverride),
},
{
Name: "default-ignores",
Usage: "Set the default ignores (config) from a file",
ArgsUsage: "PATH",
Action: expects(1, setDefaultIgnores),
},
},
type folderOverrideCommand struct {
FolderID string `arg:""`
}
func foldersOverride(c *cli.Context) error {
client, err := getClientFactory(c).getClient()
type defaultIgnoresCommand struct {
Path string `arg:""`
}
type operationCommand struct {
Restart struct{} `cmd:"" help:"Restart syncthing"`
Shutdown struct{} `cmd:"" help:"Shutdown syncthing"`
Upgrade struct{} `cmd:"" help:"Upgrade syncthing (if a newer version is available)"`
FolderOverride folderOverrideCommand `cmd:"" help:"Override changes on folder (remote for sendonly, local for receiveonly). WARNING: Destructive - deletes/changes your data"`
DefaultIgnores defaultIgnoresCommand `cmd:"" help:"Set the default ignores (config) from a file"`
}
func (*operationCommand) Run(ctx Context, kongCtx *kong.Context) error {
f := ctx.clientFactory
switch kongCtx.Selected().Name {
case "restart":
return emptyPost("system/restart", f)
case "shutdown":
return emptyPost("system/shutdown", f)
case "upgrade":
return emptyPost("system/upgrade", f)
}
return nil
}
func (f *folderOverrideCommand) Run(ctx Context) error {
client, err := ctx.clientFactory.getClient()
if err != nil {
return err
}
@ -61,7 +56,7 @@ func foldersOverride(c *cli.Context) error {
if err != nil {
return err
}
rid := c.Args()[0]
rid := f.FolderID
for _, folder := range cfg.Folders {
if folder.ID == rid {
response, err := client.Post("db/override", "")
@ -86,12 +81,12 @@ func foldersOverride(c *cli.Context) error {
return fmt.Errorf("Folder %q not found", rid)
}
func setDefaultIgnores(c *cli.Context) error {
client, err := getClientFactory(c).getClient()
func (d *defaultIgnoresCommand) Run(ctx Context) error {
client, err := ctx.clientFactory.getClient()
if err != nil {
return err
}
dir, file := filepath.Split(c.Args()[0])
dir, file := filepath.Split(d.Path)
filesystem := fs.NewFilesystem(fs.FilesystemTypeBasic, dir)
fd, err := filesystem.Open(file)

View File

@ -9,37 +9,30 @@ package cli
import (
"net/url"
"github.com/urfave/cli"
"github.com/alecthomas/kong"
)
var pendingCommand = cli.Command{
Name: "pending",
HideHelp: true,
Usage: "Pending subcommand group",
Subcommands: []cli.Command{
{
Name: "devices",
Usage: "Show pending devices",
Action: expects(0, indexDumpOutput("cluster/pending/devices")),
},
{
Name: "folders",
Usage: "Show pending folders",
Flags: []cli.Flag{
cli.StringFlag{Name: "device", Usage: "Show pending folders offered by given device"},
},
Action: expects(0, folders()),
},
},
type pendingCommand struct {
Devices struct{} `cmd:"" help:"Show pending devices"`
Folders struct {
Device string `help:"Show pending folders offered by given device"`
} `cmd:"" help:"Show pending folders"`
}
func folders() cli.ActionFunc {
return func(c *cli.Context) error {
if c.String("device") != "" {
func (p *pendingCommand) Run(ctx Context, kongCtx *kong.Context) error {
indexDumpOutput := indexDumpOutputWrapper(ctx.clientFactory)
switch kongCtx.Selected().Name {
case "devices":
return indexDumpOutput("cluster/pending/devices")
case "folders":
if p.Folders.Device != "" {
query := make(url.Values)
query.Set("device", c.String("device"))
return indexDumpOutput("cluster/pending/folders?" + query.Encode())(c)
query.Set("device", p.Folders.Device)
return indexDumpOutput("cluster/pending/folders?" + query.Encode())
}
return indexDumpOutput("cluster/pending/folders")(c)
return indexDumpOutput("cluster/pending/folders")
}
return nil
}

View File

@ -7,44 +7,36 @@
package cli
import (
"github.com/urfave/cli"
"github.com/alecthomas/kong"
)
var showCommand = cli.Command{
Name: "show",
HideHelp: true,
Usage: "Show command group",
Subcommands: []cli.Command{
{
Name: "version",
Usage: "Show syncthing client version",
Action: expects(0, indexDumpOutput("system/version")),
},
{
Name: "config-status",
Usage: "Show configuration status, whether or not a restart is required for changes to take effect",
Action: expects(0, indexDumpOutput("config/restart-required")),
},
{
Name: "system",
Usage: "Show system status",
Action: expects(0, indexDumpOutput("system/status")),
},
{
Name: "connections",
Usage: "Report about connections to other devices",
Action: expects(0, indexDumpOutput("system/connections")),
},
{
Name: "discovery",
Usage: "Show the discovered addresses of remote devices (from cache of the running syncthing instance)",
Action: expects(0, indexDumpOutput("system/discovery")),
},
pendingCommand,
{
Name: "usage",
Usage: "Show usage report",
Action: expects(0, indexDumpOutput("svc/report")),
},
},
type showCommand struct {
Version struct{} `cmd:"" help:"Show syncthing client version"`
ConfigStatus struct{} `cmd:"" help:"Show configuration status, whether or not a restart is required for changes to take effect"`
System struct{} `cmd:"" help:"Show system status"`
Connections struct{} `cmd:"" help:"Report about connections to other devices"`
Discovery struct{} `cmd:"" help:"Show the discovered addresses of remote devices (from cache of the running syncthing instance)"`
Usage struct{} `cmd:"" help:"Show usage report"`
Pending pendingCommand `cmd:"" help:"Pending subcommand group"`
}
func (*showCommand) Run(ctx Context, kongCtx *kong.Context) error {
indexDumpOutput := indexDumpOutputWrapper(ctx.clientFactory)
switch kongCtx.Selected().Name {
case "version":
return indexDumpOutput("system/version")
case "config-status":
return indexDumpOutput("config/restart-required")
case "system":
return indexDumpOutput("system/status")
case "connections":
return indexDumpOutput("system/connections")
case "discovery":
return indexDumpOutput("system/discovery")
case "usage":
return indexDumpOutput("svc/report")
}
return nil
}

View File

@ -19,7 +19,6 @@ import (
"github.com/syncthing/syncthing/lib/config"
"github.com/syncthing/syncthing/lib/db/backend"
"github.com/syncthing/syncthing/lib/locations"
"github.com/urfave/cli"
)
func responseToBArray(response *http.Response) ([]byte, error) {
@ -30,68 +29,72 @@ func responseToBArray(response *http.Response) ([]byte, error) {
return bytes, response.Body.Close()
}
func emptyPost(url string) cli.ActionFunc {
return func(c *cli.Context) error {
client, err := getClientFactory(c).getClient()
if err != nil {
return err
}
_, err = client.Post(url, "")
func emptyPost(url string, apiClientFactory *apiClientFactory) error {
client, err := apiClientFactory.getClient()
if err != nil {
return err
}
_, err = client.Post(url, "")
return err
}
func indexDumpOutputWrapper(apiClientFactory *apiClientFactory) func(url string) error {
return func(url string) error {
return indexDumpOutput(url, apiClientFactory)
}
}
func indexDumpOutput(url string) cli.ActionFunc {
return func(c *cli.Context) error {
client, err := getClientFactory(c).getClient()
if err != nil {
return err
}
response, err := client.Get(url)
if errors.Is(err, errNotFound) {
return errors.New("not found (folder/file not in database)")
}
if err != nil {
return err
}
return prettyPrintResponse(response)
}
}
func saveToFile(url string) cli.ActionFunc {
return func(c *cli.Context) error {
client, err := getClientFactory(c).getClient()
if err != nil {
return err
}
response, err := client.Get(url)
if err != nil {
return err
}
_, params, err := mime.ParseMediaType(response.Header.Get("Content-Disposition"))
if err != nil {
return err
}
filename := params["filename"]
if filename == "" {
return errors.New("Missing filename in response")
}
bs, err := responseToBArray(response)
if err != nil {
return err
}
f, err := os.Create(filename)
if err != nil {
return err
}
defer f.Close()
_, err = f.Write(bs)
if err != nil {
return err
}
fmt.Println("Wrote results to", filename)
func indexDumpOutput(url string, apiClientFactory *apiClientFactory) error {
client, err := apiClientFactory.getClient()
if err != nil {
return err
}
response, err := client.Get(url)
if errors.Is(err, errNotFound) {
return errors.New("not found (folder/file not in database)")
}
if err != nil {
return err
}
return prettyPrintResponse(response)
}
func saveToFile(url string, apiClientFactory *apiClientFactory) error {
client, err := apiClientFactory.getClient()
if err != nil {
return err
}
response, err := client.Get(url)
if err != nil {
return err
}
_, params, err := mime.ParseMediaType(response.Header.Get("Content-Disposition"))
if err != nil {
return err
}
filename := params["filename"]
if filename == "" {
return errors.New("Missing filename in response")
}
bs, err := responseToBArray(response)
if err != nil {
return err
}
f, err := os.Create(filename)
if err != nil {
return err
}
_, err = f.Write(bs)
if err != nil {
_ = f.Close()
return err
}
err = f.Close()
if err != nil {
return err
}
fmt.Println("Wrote results to", filename)
return nil
}
func getConfig(c APIClient) (config.Configuration, error) {
@ -111,19 +114,6 @@ func getConfig(c APIClient) (config.Configuration, error) {
return cfg, nil
}
func expects(n int, actionFunc cli.ActionFunc) cli.ActionFunc {
return func(ctx *cli.Context) error {
if ctx.NArg() != n {
plural := ""
if n != 1 {
plural = "s"
}
return fmt.Errorf("expected %d argument%s, got %d", n, plural, ctx.NArg())
}
return actionFunc(ctx)
}
}
func prettyPrintJSON(data interface{}) error {
enc := json.NewEncoder(os.Stdout)
enc.SetIndent("", " ")
@ -159,7 +149,3 @@ func nulString(bs []byte) string {
func normalizePath(path string) string {
return filepath.ToSlash(filepath.Clean(path))
}
func getClientFactory(c *cli.Context) *apiClientFactory {
return c.App.Metadata["clientFactory"].(*apiClientFactory)
}

View File

@ -9,8 +9,8 @@ package cmdutil
// CommonOptions are reused among several subcommands
type CommonOptions struct {
buildCommonOptions
ConfDir string `name:"config" placeholder:"PATH" help:"Set configuration directory (config and keys)"`
HomeDir string `name:"home" placeholder:"PATH" help:"Set configuration and data directory"`
ConfDir string `name:"config" placeholder:"PATH" env:"STCONFDIR" help:"Set configuration directory (config and keys)"`
HomeDir string `name:"home" placeholder:"PATH" env:"STHOMEDIR" help:"Set configuration and data directory"`
NoDefaultFolder bool `env:"STNODEFAULTFOLDER" help:"Don't create the \"default\" folder on first startup"`
SkipPortProbing bool `help:"Don't try to find free ports for GUI and listen addresses on first startup"`
}

View File

@ -4,7 +4,7 @@
// License, v. 2.0. If a copy of the MPL was not distributed with this file,
// You can obtain one at https://mozilla.org/MPL/2.0/.
package main
package syncthing_main
import (
"bytes"

View File

@ -4,7 +4,7 @@
// License, v. 2.0. If a copy of the MPL was not distributed with this file,
// You can obtain one at https://mozilla.org/MPL/2.0/.
package main
package syncthing_main
import (
"bytes"

View File

@ -4,12 +4,12 @@
// License, v. 2.0. If a copy of the MPL was not distributed with this file,
// You can obtain one at https://mozilla.org/MPL/2.0/.
package main
package syncthing_main
import (
"github.com/syncthing/syncthing/lib/logger"
)
var (
l = logger.DefaultLogger.NewFacility("main", "Main package")
l = logger.DefaultLogger.NewFacility("syncthing_main", "Syncthing package")
)

View File

@ -69,7 +69,7 @@ func Generate(l logger.Logger, confDir, guiUser, guiPassword string, noDefaultFo
return err
}
if err := syncthing.EnsureDir(dir, 0700); err != nil {
if err := syncthing.EnsureDir(dir, 0o700); err != nil {
return err
}
locations.SetBaseDir(locations.ConfigBaseDir, dir)
@ -127,7 +127,7 @@ func updateGUIAuthentication(l logger.Logger, guiCfg *config.GUIConfiguration, g
}
if guiPassword != "" && guiCfg.Password != guiPassword {
if err := guiCfg.HashAndSetPassword(guiPassword); err != nil {
if err := guiCfg.SetPassword(guiPassword); err != nil {
return fmt.Errorf("failed to set GUI authentication password: %w", err)
}
l.Infoln("Updated GUI authentication password.")

View File

@ -4,7 +4,7 @@
// License, v. 2.0. If a copy of the MPL was not distributed with this file,
// You can obtain one at https://mozilla.org/MPL/2.0/.
package main
package syncthing_main
import (
"fmt"

View File

@ -4,7 +4,7 @@
// License, v. 2.0. If a copy of the MPL was not distributed with this file,
// You can obtain one at https://mozilla.org/MPL/2.0/.
package main
package syncthing_main
import (
"bytes"
@ -22,7 +22,6 @@ import (
"path"
"path/filepath"
"regexp"
"runtime"
"runtime/pprof"
"sort"
"strconv"
@ -31,9 +30,10 @@ import (
"time"
"github.com/alecthomas/kong"
_ "github.com/syncthing/syncthing/lib/automaxprocs"
"github.com/thejerf/suture/v4"
"github.com/willabides/kongplete"
"github.com/syncthing/syncthing/cmd/syncthing/cli"
"github.com/syncthing/syncthing/cmd/syncthing/cmdutil"
"github.com/syncthing/syncthing/cmd/syncthing/decrypt"
"github.com/syncthing/syncthing/cmd/syncthing/generate"
@ -88,9 +88,6 @@ above.
STTRACE A comma separated string of facilities to trace. The valid
facility strings are listed below.
STDEADLOCKTIMEOUT Used for debugging internal deadlocks; sets debug
sensitivity. Use only under direction of a developer.
STLOCKTHRESHOLD Used for debugging internal deadlocks; sets debug
sensitivity. Use only under direction of a developer.
@ -99,6 +96,11 @@ above.
"minio" for the github.com/minio/sha256-simd implementation,
and blank (the default) for auto detection.
STVERSIONEXTRA Add extra information to the version string in logs and the
version line in the GUI. Can be set to the name of a wrapper
or tool controlling syncthing to communicate this to the end
user.
GOMAXPROCS Set the maximum number of CPU cores to use. Defaults to all
available CPU cores.
@ -131,10 +133,9 @@ var (
// commands and options here are top level commands to syncthing.
// Cli is just a placeholder for the help text (see main).
var entrypoint struct {
Serve serveOptions `cmd:"" help:"Run Syncthing"`
Generate generate.CLI `cmd:"" help:"Generate key and config, then exit"`
Decrypt decrypt.CLI `cmd:"" help:"Decrypt or verify an encrypted folder"`
Cli struct{} `cmd:"" help:"Command line interface for Syncthing"`
Serve serveOptions `cmd:"" help:"Run Syncthing"`
Generate generate.CLI `cmd:"" help:"Generate key and config, then exit"`
Decrypt decrypt.CLI `cmd:"" help:"Decrypt or verify an encrypted folder"`
}
// serveOptions are the options for the `syncthing serve` command.
@ -144,9 +145,9 @@ type serveOptions struct {
Audit bool `help:"Write events to audit file"`
AuditFile string `name:"auditfile" placeholder:"PATH" help:"Specify audit file (use \"-\" for stdout, \"--\" for stderr)"`
BrowserOnly bool `help:"Open GUI in browser"`
DataDir string `name:"data" placeholder:"PATH" help:"Set data directory (database and logs)"`
DataDir string `name:"data" placeholder:"PATH" env:"STDATADIR" help:"Set data directory (database and logs)"`
DeviceID bool `help:"Show the device ID"`
GenerateDir string `name:"generate" placeholder:"PATH" help:"Generate key and config in specified dir, then exit"` //DEPRECATED: replaced by subcommand!
GenerateDir string `name:"generate" placeholder:"PATH" help:"Generate key and config in specified dir, then exit"` // DEPRECATED: replaced by subcommand!
GUIAddress string `name:"gui-address" placeholder:"URL" help:"Override GUI address (e.g. \"http://192.0.2.42:8443\")"`
GUIAPIKey string `name:"gui-apikey" placeholder:"API-KEY" help:"Override GUI API key"`
LogFile string `name:"logfile" default:"${logFile}" placeholder:"PATH" help:"Log file name (see below)"`
@ -168,7 +169,6 @@ type serveOptions struct {
// Debug options below
DebugDBIndirectGCInterval time.Duration `env:"STGCINDIRECTEVERY" help:"Database indirection GC interval"`
DebugDBRecheckInterval time.Duration `env:"STRECHECKDBEVERY" help:"Database metadata recalculation interval"`
DebugDeadlockTimeout int `placeholder:"SECONDS" env:"STDEADLOCKTIMEOUT" help:"Used for debugging internal deadlocks"`
DebugGUIAssetsDir string `placeholder:"PATH" help:"Directory to load GUI assets from" env:"STGUIASSETS"`
DebugPerfStats bool `env:"STPERFSTATS" help:"Write running performance statistics to perf-$pid.csv (Unix only)"`
DebugProfileBlock bool `env:"STBLOCKPROFILE" help:"Write block profiles to block-$pid-$timestamp.pprof every 20 seconds"`
@ -207,23 +207,10 @@ func defaultVars() kong.Vars {
return vars
}
func main() {
// The "cli" subcommand uses a different command line parser, and e.g. help
// gets mangled when integrating it as a subcommand -> detect it here at the
// beginning.
if len(os.Args) > 1 && os.Args[1] == "cli" {
if err := cli.Run(); err != nil {
fmt.Println(err)
os.Exit(1)
}
return
}
func RunWithArgs(args []string) error {
// First some massaging of the raw command line to fit the new model.
// Basically this means adding the default command at the front, and
// converting -options to --options.
args := os.Args[1:]
switch {
case len(args) == 0:
// Empty command line is equivalent to just calling serve
@ -244,16 +231,26 @@ func main() {
// Create a parser with an overridden help function to print our extra
// help info.
parser, err := kong.New(&entrypoint, kong.Help(helpHandler), defaultVars())
parser, err := kong.New(
&entrypoint,
kong.ConfigureHelp(kong.HelpOptions{
NoExpandSubcommands: true,
Compact: true,
}),
kong.Help(helpHandler),
defaultVars(),
)
if err != nil {
log.Fatal(err)
}
kongplete.Complete(parser)
ctx, err := parser.Parse(args)
parser.FatalIfErrorf(err)
ctx.BindTo(l, (*logger.Logger)(nil)) // main logger available to subcommands
err = ctx.Run()
parser.FatalIfErrorf(err)
return err
}
func helpHandler(options kong.HelpOptions, ctx *kong.Context) error {
@ -354,7 +351,7 @@ func (options serveOptions) Run() error {
}
// Ensure that our home directory exists.
if err := syncthing.EnsureDir(locations.GetBaseDir(locations.ConfigBaseDir), 0700); err != nil {
if err := syncthing.EnsureDir(locations.GetBaseDir(locations.ConfigBaseDir), 0o700); err != nil {
l.Warnln("Failure on home directory:", err)
os.Exit(svcutil.ExitError.AsInt())
}
@ -621,7 +618,6 @@ func syncthingMain(options serveOptions) {
}
appOpts := syncthing.Options{
DeadlockTimeoutS: options.DebugDeadlockTimeout,
NoUpgrade: options.NoUpgrade,
ProfilerAddr: options.DebugProfilerListen,
ResetDeltaIdxs: options.DebugResetDeltaIdxs,
@ -632,10 +628,6 @@ func syncthingMain(options serveOptions) {
if options.Audit {
appOpts.AuditWriter = auditWriter(options.AuditFile)
}
if t := os.Getenv("STDEADLOCKTIMEOUT"); t != "" {
secs, _ := strconv.Atoi(t)
appOpts.DeadlockTimeoutS = secs
}
if dur, err := time.ParseDuration(os.Getenv("STRECHECKDBEVERY")); err == nil {
appOpts.DBRecheckInterval = dur
}
@ -655,10 +647,6 @@ func syncthingMain(options serveOptions) {
setupSignalHandling(app)
if os.Getenv("GOMAXPROCS") == "" {
runtime.GOMAXPROCS(runtime.NumCPU())
}
if options.DebugProfileCPU {
f, err := os.Create(fmt.Sprintf("cpu-%d.pprof", os.Getpid()))
if err != nil {
@ -722,7 +710,6 @@ func setupSignalHandling(app *syncthing.App) {
func loadOrDefaultConfig() (config.Wrapper, error) {
cfgFile := locations.Get(locations.ConfigFile)
cfg, _, err := config.Load(cfgFile, protocol.EmptyDeviceID, events.NoopLogger)
if err != nil {
newCfg := config.New(protocol.EmptyDeviceID)
return config.Wrap(cfgFile, newCfg, protocol.EmptyDeviceID, events.NoopLogger), nil
@ -750,7 +737,7 @@ func auditWriter(auditFile string) io.Writer {
} else {
auditFlags = os.O_WRONLY | os.O_CREATE | os.O_APPEND
}
fd, err = os.OpenFile(auditFile, auditFlags, 0600)
fd, err = os.OpenFile(auditFile, auditFlags, 0o600)
if err != nil {
l.Warnln("Audit:", err)
os.Exit(svcutil.ExitError.AsInt())
@ -870,6 +857,7 @@ func cleanConfigDirectory() {
"backup-of-v0.8": 30 * 24 * time.Hour, // these neither
"tmp-index-sorter.*": time.Minute, // these should never exist on startup
"support-bundle-*": 30 * 24 * time.Hour, // keep old support bundle zip or folder for a month
"csrftokens.txt": 0, // deprecated, remove immediately
}
for pat, dur := range patterns {

View File

@ -4,7 +4,7 @@
// License, v. 2.0. If a copy of the MPL was not distributed with this file,
// You can obtain one at https://mozilla.org/MPL/2.0/.
package main
package syncthing_main
import (
"bufio"
@ -346,7 +346,7 @@ func restartMonitor(binary string, args []string) error {
}
func restartMonitorUnix(binary string, args []string) error {
return syscall.Exec(args[0], args, os.Environ())
return syscall.Exec(binary, args, os.Environ())
}
func restartMonitorWindows(binary string, args []string) error {
@ -521,7 +521,7 @@ func (f *autoclosedFile) ensureOpenLocked() error {
// We open the file for write only, and create it if it doesn't exist.
flags := os.O_WRONLY | os.O_CREATE | os.O_APPEND
fd, err := os.OpenFile(f.name, flags, 0644)
fd, err := os.OpenFile(f.name, flags, 0o644)
if err != nil {
return err
}

View File

@ -4,7 +4,7 @@
// License, v. 2.0. If a copy of the MPL was not distributed with this file,
// You can obtain one at https://mozilla.org/MPL/2.0/.
package main
package syncthing_main
import (
"io"

View File

@ -7,7 +7,7 @@
//go:build !windows
// +build !windows
package main
package syncthing_main
import (
"os/exec"

View File

@ -7,7 +7,7 @@
//go:build windows
// +build windows
package main
package syncthing_main
import "os/exec"

View File

@ -7,7 +7,7 @@
//go:build !solaris && !windows
// +build !solaris,!windows
package main
package syncthing_main
import (
"fmt"

View File

@ -7,7 +7,7 @@
//go:build solaris || windows
// +build solaris windows
package main
package syncthing_main
func startPerfStats() {
}

View File

@ -7,7 +7,7 @@
//go:build go1.7
// +build go1.7
package main
package syncthing_main
import "runtime/debug"

View File

@ -4,10 +4,11 @@
// License, v. 2.0. If a copy of the MPL was not distributed with this file,
// You can obtain one at https://mozilla.org/MPL/2.0/.
package main
package aggregate
import (
"database/sql"
"fmt"
"log"
"os"
"time"
@ -15,26 +16,21 @@ import (
_ "github.com/lib/pq"
)
var dbConn = getEnvDefault("UR_DB_URL", "postgres://user:password@localhost/ur?sslmode=disable")
func getEnvDefault(key, def string) string {
if val := os.Getenv(key); val != "" {
return val
}
return def
type CLI struct {
DBConn string `env:"UR_DB_URL" default:"postgres://user:password@localhost/ur?sslmode=disable"`
}
func main() {
func (cli *CLI) Run() error {
log.SetFlags(log.Ltime | log.Ldate)
log.SetOutput(os.Stdout)
db, err := sql.Open("postgres", dbConn)
db, err := sql.Open("postgres", cli.DBConn)
if err != nil {
log.Fatalln("database:", err)
return fmt.Errorf("database: %w", err)
}
err = setupDB(db)
if err != nil {
log.Fatalln("database:", err)
return fmt.Errorf("database: %w", err)
}
for {
@ -87,16 +83,6 @@ func setupDB(db *sql.DB) error {
return err
}
_, err = db.Exec(`CREATE TABLE IF NOT EXISTS UserMovement (
Day TIMESTAMP NOT NULL,
Added INTEGER NOT NULL,
Bounced INTEGER NOT NULL,
Removed INTEGER NOT NULL
)`)
if err != nil {
return err
}
_, err = db.Exec(`CREATE TABLE IF NOT EXISTS Performance (
Day TIMESTAMP NOT NULL,
TotFiles INTEGER NOT NULL,
@ -136,11 +122,6 @@ func setupDB(db *sql.DB) error {
_, _ = db.Exec(`CREATE INDEX VersionDayIndex ON VersionSummary (Day)`)
}
row = db.QueryRow(`SELECT 'MovementDayIndex'::regclass`)
if err := row.Scan(&t); err != nil {
_, _ = db.Exec(`CREATE INDEX MovementDayIndex ON UserMovement (Day)`)
}
row = db.QueryRow(`SELECT 'PerformanceDayIndex'::regclass`)
if err := row.Scan(&t); err != nil {
_, _ = db.Exec(`CREATE INDEX PerformanceDayIndex ON Performance (Day)`)
@ -185,87 +166,6 @@ func aggregateVersionSummary(db *sql.DB, since time.Time) (int64, error) {
return res.RowsAffected()
}
func aggregateUserMovement(db *sql.DB) (int64, error) {
rows, err := db.Query(`SELECT
DATE_TRUNC('day', Received) AS Day,
Report->>'uniqueID'
FROM ReportsJson
WHERE
Report->>'uniqueID' IS NOT NULL
AND Received < DATE_TRUNC('day', NOW())
AND Report->>'version' like 'v_.%'
ORDER BY Day
`)
if err != nil {
return 0, err
}
defer rows.Close()
firstSeen := make(map[string]time.Time)
lastSeen := make(map[string]time.Time)
var minTs time.Time
minTs = minTs.In(time.UTC)
for rows.Next() {
var ts time.Time
var id string
if err := rows.Scan(&ts, &id); err != nil {
return 0, err
}
if minTs.IsZero() {
minTs = ts
}
if _, ok := firstSeen[id]; !ok {
firstSeen[id] = ts
}
lastSeen[id] = ts
}
type sumRow struct {
day time.Time
added int
removed int
bounced int
}
var sumRows []sumRow
for t := minTs; t.Before(time.Now().Truncate(24 * time.Hour)); t = t.AddDate(0, 0, 1) {
var added, removed, bounced int
old := t.Before(time.Now().AddDate(0, 0, -30))
for id, first := range firstSeen {
last := lastSeen[id]
if first.Equal(t) && last.Equal(t) && old {
bounced++
continue
}
if first.Equal(t) {
added++
}
if last == t && old {
removed++
}
}
sumRows = append(sumRows, sumRow{t, added, removed, bounced})
}
tx, err := db.Begin()
if err != nil {
return 0, err
}
if _, err := tx.Exec("DELETE FROM UserMovement"); err != nil {
tx.Rollback()
return 0, err
}
for _, r := range sumRows {
if _, err := tx.Exec("INSERT INTO UserMovement (Day, Added, Removed, Bounced) VALUES ($1, $2, $3, $4)", r.day, r.added, r.removed, r.bounced); err != nil {
tx.Rollback()
return 0, err
}
}
return int64(len(sumRows)), tx.Commit()
}
func aggregatePerformance(db *sql.DB, since time.Time) (int64, error) {
res, err := db.Exec(`INSERT INTO Performance (
SELECT

File diff suppressed because it is too large Load Diff

View File

@ -4,7 +4,7 @@
// License, v. 2.0. If a copy of the MPL was not distributed with this file,
// You can obtain one at https://mozilla.org/MPL/2.0/.
package main
package serve
import (
"regexp"
@ -145,7 +145,7 @@ func statsForFloats(data []float64) [4]float64 {
return res
}
func group(by func(string) string, as []analytic, perGroup int) []analytic {
func group(by func(string) string, as []analytic, perGroup int, otherPct float64) []analytic {
var res []analytic
next:
@ -170,6 +170,25 @@ next:
}
sort.Sort(analyticList(res))
if otherPct > 0 {
// Groups with less than otherPCt go into "Other"
other := analytic{
Key: "Other",
}
for i := 0; i < len(res); i++ {
if res[i].Percentage < otherPct || res[i].Key == "Other" {
other.Count += res[i].Count
other.Percentage += res[i].Percentage
res = append(res[:i], res[i+1:]...)
i--
}
}
if other.Count > 0 {
res = append(res, other)
}
}
return res
}

View File

@ -4,7 +4,7 @@
// License, v. 2.0. If a copy of the MPL was not distributed with this file,
// You can obtain one at https://mozilla.org/MPL/2.0/.
package main
package serve
import "testing"

View File

@ -4,7 +4,7 @@
// License, v. 2.0. If a copy of the MPL was not distributed with this file,
// You can obtain one at https://mozilla.org/MPL/2.0/.
package main
package serve
import (
"bytes"

View File

@ -0,0 +1,26 @@
// Copyright (C) 2023 The Syncthing Authors.
//
// This Source Code Form is subject to the terms of the Mozilla Public
// License, v. 2.0. If a copy of the MPL was not distributed with this file,
// You can obtain one at https://mozilla.org/MPL/2.0/.
package serve
import (
"github.com/prometheus/client_golang/prometheus"
"github.com/prometheus/client_golang/prometheus/promauto"
)
var metricReportsTotal = promauto.NewCounterVec(prometheus.CounterOpts{
Namespace: "syncthing",
Subsystem: "ursrv",
Name: "reports_total",
}, []string{"version"})
func init() {
metricReportsTotal.WithLabelValues("fail")
metricReportsTotal.WithLabelValues("duplicate")
metricReportsTotal.WithLabelValues("v1")
metricReportsTotal.WithLabelValues("v2")
metricReportsTotal.WithLabelValues("v3")
}

1135
cmd/ursrv/serve/serve.go Normal file

File diff suppressed because it is too large Load Diff

View File

Before

Width:  |  Height:  |  Size: 4.8 KiB

After

Width:  |  Height:  |  Size: 4.8 KiB

View File

Before

Width:  |  Height:  |  Size: 61 KiB

After

Width:  |  Height:  |  Size: 61 KiB

View File

@ -197,7 +197,7 @@ found in the LICENSE file.
};
var baseLayer = L.tileLayer(
'https://{s}.tile.openstreetmap.org/{z}/{x}/{y}.png',{
'https://tile.openstreetmap.org/{z}/{x}/{y}.png',{
attribution: '...',
maxZoom: 18
}
@ -454,13 +454,13 @@ found in the LICENSE file.
<table class="table table-striped">
<thead>
<tr>
<th>Builder</th>
<th>Distribution Channel</th>
<th class="text-right">Devices</th>
<th class="text-right">Share</th>
</tr>
</thead>
<tbody>
{{range .builders}}
{{range .distributions}}
<tr>
<td>{{.Key}}</td>
<td class="text-right">{{.Count}}</td>
@ -475,13 +475,13 @@ found in the LICENSE file.
<table class="table table-striped">
<thead>
<tr>
<th>Distribution Channel</th>
<th>Builder</th>
<th class="text-right">Devices</th>
<th class="text-right">Share</th>
</tr>
</thead>
<tbody>
{{range .distributions}}
{{range .builders}}
<tr>
<td>{{.Key}}</td>
<td class="text-right">{{.Count}}</td>
@ -611,6 +611,7 @@ found in the LICENSE file.
</div>
<hr>
<p>
<a href="https://github.com/syncthing/syncthing/tree/main/cmd/ursrv">Source code</a>.
This product includes GeoLite2 data created by MaxMind, available from
<a href="http://www.maxmind.com">http://www.maxmind.com</a>.
</p>

Some files were not shown because too many files have changed in this diff Show More