Commit Graph

278 Commits

Author SHA1 Message Date
Jakob Borg f7fc0c1d3e all: Update license url to https (ref #3976) 2017-02-09 08:04:16 +01:00
Simon Frei 237893ead3 cmd/syncthing: Only delay next scan (via REST) if the scan succeeds
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3927
2017-02-05 18:17:44 +00:00
Jakob Borg 3655c97850 cmd/syncthing, lib/fs, lib/sync: Spelling in comments 2017-02-05 18:51:52 +01:00
Antony Male ac510b26e2 cmd/syncthing, lib/events, lib/sync: Add timeout to REST event API, remove Ping (fixes #3933)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3941
2017-01-31 12:04:29 +00:00
Jakob Borg 1c9361a818 cmd/syncthing: Implement "release candidate" logic
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3943
LGTM: AudriusButkevicius
2017-01-30 21:33:07 +00:00
Jakob Borg 35e87e23fd cmd/syncthing, gui, lib/config, lib/upgrade: Add option to upgrade to pre-releases
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3939
2017-01-27 12:17:06 +00:00
Mark Pulford 69ae4aa024 cmd/syncthing: Avoid Keepalive/GUI refresh race
This avoids unnecessary browser request failures and retries. Eg:
- Browser reuses existing HTTP connection for GUI refresh request
- Server closes connection with request in flight
- Browser retries GET request.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3854
2017-01-01 12:38:31 +00:00
Audrius Butkevicius bab7c8ebbf all: Add folder pause, make pauses permanent (fixes #3407, fixes #215, fixes #3001)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3520
2016-12-21 18:41:25 +00:00
Jakob Borg d41c131364 build: Enable gometalinter "gosimple" check, improve build.go 2016-12-21 14:53:45 +01:00
Audrius Butkevicius f60b424d70 lib/config: Raw() -> RawCopy() 2016-11-13 09:29:35 +01:00
Jakob Borg 4e8c8d7e2c cmd/syncthing, lib/db, lib/model: Track more detailed file/dirs/links/deleted counts 2016-10-17 23:57:43 +02:00
Jakob Borg 215503b4f7 cmd/syncthing: Delay browser start until the GUI is ready (fixes #3619) 2016-10-07 12:10:26 +09:00
Jakob Borg d1d565e58b cmd/syncthing: Localhost header comparison should be case insensitive 2016-10-03 17:34:13 +09:00
Nathan Morrison d322ebd0b9 Add API service for local disk changes
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3626
LGTM: calmh, AudriusButkevicius
2016-09-28 15:54:13 +00:00
Jakob Borg 467c1b26fb cmd/syncthing, lib/config: Log errors replacing or saving config (ref #3567) 2016-09-24 09:59:09 +02:00
Laurent Etiemble 3990014073 cmd/syncthing: Conditionally enable CORS
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3541
LGTM: AudriusButkevicius
2016-09-06 22:16:50 +00:00
Jakob Borg 49910a1d85 lib/config, cmd/syncthing: Enforce localhost only connections
When the GUI/API is bound to localhost, we enforce that the Host header
looks like localhost. This can be disabled by setting
insecureSkipHostCheck in the GUI config.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3558
2016-09-03 08:33:34 +00:00
Jakob Borg 69b7f26e4c lib/model, cmd/syncthing: Also account for deleted files in folder summary events (ref #3496)
This should probably be reflected in the GUI somewhere as well...
2016-09-02 10:45:39 +02:00
Jakob Borg 5b37d0356c lib/model, gui: Correct completion percentages when there are lots of deletes (fixes #3496)
We used to consider deleted files & directories 128 bytes large. After
the delta indexes change a bug slipped in where deleted files would be
weighted according to their old non-deleted size. Both ways are
incorrect (but the latest change made it worse), as if there are more
files deleted than remaining data in the repo the needSize can be
greater than the globalSize, resulting in a negative completion
percentage.

This change makes it so that deleted items are zero bytes large, which
makes more sense. Instead we expose the number of files that we need to
delete as a separate field in the Completion() result, and hack the
percentage down to 95% complete if it was 100% complete but we need to
delete files. This latter part is sort of ugly, but necessary to give
the user some sort of feedback.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3556
2016-09-02 06:45:46 +00:00
Jakob Borg 7776839c82 cmd/syncthing, gui: Improve completion calculation (fixes #3492)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3493
2016-08-12 06:41:43 +00:00
Jakob Borg ea87bcefd6 lib/protocol, lib/model: Implement high precision time stamps (fixes #3305)
This adds a new nanoseconds field to the FileInfo, populates it during
scans and sets the non-truncated time in Chtimes calls.

The actual file modification time is defined as modified_s seconds +
modified_ns nanoseconds. It's expected that the modified_ns field is <=
1e9 (that is, all whole seconds should go in the modified_s field) but
not really enforced. Given that it's an int32 the timestamp can be
adjusted += ~2.9 seconds by the modified_ns field...

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3431
2016-08-06 13:05:59 +00:00
Jakob Borg ffe7a2fcd7 cmd/syncthing, lib/config: Enable HTTP CPU/heap profile collection for users
This adds a config to enable debug functions on the API server, which is
by default disabled. When enabled, the /rest/debug things become
available and become available without requiring a CSRF token (although
authentication is required if configured).

We also add a new endpoint /rest/debug/cpuprof?duration=15s (with the
duration being configurable, defaulting to 30s). This runs a CPU profile
for the duration and returns it as a file. It sets headers so that a
browser will save the file with an informative name.

The same is done for heap profiles, /rest/debug/heapprof, which does not
take any parameters.

The purpose of this is that any user can enable debugging under
advanced, then point their browser to the endpoint above and get a file
that contains a CPU or heap profile we can use, with the filename
telling us what version and architecture the profile is from.

On the command line, this becomes

    curl -O -J http://localhost:8082/rest/debug/cpuprof?duration=5s
    curl: Saved to filename
    'syncthing-cpu-darwin-amd64-v0.14.3+4-g935bcc0-110307.pprof'

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3467
2016-08-02 11:06:45 +00:00
Jakob Borg 297240facf all: Rename LocalVersion to Sequence (fixes #3461)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3462
2016-07-29 19:54:24 +00:00
Jakob Borg 44d30c83bf lib/config, cmd/syncthing: Handle committing configuration better (fixes #3077)
This slightly changes the interface used for committing configuration
changes. The two parts are now:

 - VerifyConfiguration, which runs synchronously and locked, and can
   abort the config change. These callbacks shouldn't *do* anything
   apart from looking at the config changes and saying yes or no. No
   change from previously.

 - CommitConfiguration, which runs asynchronously (one goroutine per
   call) *after* replacing the config and releasing any locks. Returning
   false from these methods sets the "requires restart" flag, which now
   lives in the config.Wrapper.

This should be deadlock free as the CommitConfiguration calls can take
as long as they like and can wait for locks to be released when they
need to tweak things. I think this should be safe compared to before as
the CommitConfiguration calls were always made from a random background
goroutine (typically one from the HTTP server), so it was always
concurrent with everything else anyway.

Hence the CommitResponse type is gone, instead you get an error back on
verification failure only, and need to explicitly check
w.RequiresRestart() afterwards if you care.

As an added bonus this fixes a bug where we would reset the "requires
restart" indicator if a config that did not require restart was saved,
even if we already were in the requires-restart state.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3386
2016-07-04 20:32:34 +00:00
Jakob Borg fa0101bd60 lib/protocol, lib/discover, lib/db: Use protocol buffer serialization (fixes #3080)
This changes the BEP protocol to use protocol buffer serialization
instead of XDR, and therefore also the database format. The local
discovery protocol is also updated to be protocol buffer format.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3276
LGTM: AudriusButkevicius
2016-07-04 10:40:29 +00:00
Jakob Borg 6d357211b2 lib/config: Remove "Invalid" attribute (fixes #2471)
This contains the following behavioral changes:

 - Duplicate folder IDs is now fatal during startup
 - Invalid folder flags in the ClusterConfig is fatal for the connection
   (this will go away soon with the proto changes, as we won't have any
   unknown flags any more then)
 - Empty path is a folder error reported at runtime

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3370
2016-07-02 19:38:39 +00:00
Lars K.W. Gohlke af0bc95de5 lib/model: Refactor encapsulation of the folder scanning
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3017
2016-06-29 06:37:34 +00:00
Audrius Butkevicius 502bee9a09 lib/osutil: Return "/" as filesystem root on non-windows (fixes #3321)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3332
2016-06-20 20:25:00 +00:00
Daniel Harte e71c78ae84 cmd/syncthing: Remove folder limit on /rest/system/browse
Previously limited to 10 results, now unlimited.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3279
LGTM: calmh, AudriusButkevicius
2016-06-08 07:09:50 +00:00
Jakob Borg 03a8027efc cmd/syncthing: Refactor out staticsServer (prev. embeddedStatic) a bit
The purpose of this operation is to separate the serving of GUI assets a
bit from the serving of the REST API. It's by no means complete. The end
goal is something like a combined server type that embeds a statics
server and an API server and wraps it in authentication and HTTPS and
stuff, plus possibly a named pipe server that only provides the API and
does not wrap in the same authentication etc.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3273
2016-06-07 07:46:45 +00:00
Jakob Borg 5aacfd1639 cmd/syncthing: Make API serve loop more robust (fixes #3136)
This sacrifices the ability to return an error when creating the service
for being more persistent in keeping it running.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3270
LGTM: AudriusButkevicius, canton7
2016-06-06 22:12:23 +00:00
Audrius Butkevicius 1612baca92 gui: /rest/system/browse with no arguments returns drives on Windows (ref #3201)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3203
2016-05-31 19:27:07 +00:00
Antony Male 6361172bea cmd/syncthing: Be more explicit about how assets should be cached (fixes #3182)
With the previous setup, browsers were free to use a local cache for any
length of time they pleased: we didn't set an 'Expires' header (or max-age
directive), and Cache-Control just said "you're free to cache this".

Therefore be more explicit: we don't mind if browsers cache things, but they
MUST revalidate everything on every request.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3221
2016-05-30 13:54:55 +00:00
Jakob Borg da5010d37a cmd/syncthing: Use API to generate API Key and folder ID (fixes #3179)
Expose a random string generator in the API and use it when the GUI
needs random strings for API key and folder ID.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3192
2016-05-26 07:25:34 +00:00
Jakob Borg e6b78e5d56 lib/rand: Break out random functions into separate package
The intention for this package is to provide a combination of the
security of crypto/rand and the convenience of math/rand. It should be
the first choice of random data unless ultimate performance is required
and the usage is provably irrelevant from a security standpoint.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3186
2016-05-26 07:02:56 +00:00
Jakob Borg 98734375f2 cmd/syncthing: Correctly set, parse and compare modified time HTTP headers (fixes #3165)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3167
2016-05-23 12:16:14 +00:00
Jakob Borg 4bc2b3f369 gui: Set CSRF stuff earlier (fixes #3138)
We need to set these properties *before* Angular starts making requests,
and doing that from the response to a request is too late. The obvious
choice (to me) would be to use the angular $cookies service, but that
service isn't available until after initialization so we can't use it.
Instead, add a special file that is loaded by index.html and includes
the info we need before the JS app even starts running.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3152
2016-05-22 10:26:09 +00:00
Jakob Borg bf7fcc612d cmd/syncthing: Enforce stricter CSRF policy on /rest GET requests (fixes #3134)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3137
2016-05-21 13:48:55 +00:00
klemens bd41e21c26 all: Correct spelling in comments
Skip-check: authors pr-build-mac

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3056
2016-05-08 10:54:22 +00:00
Jakob Borg 39899e40bf cmd/syncthing: Use ReadAll + json.Unmarshal in places were we care about consuming the reader
Because json.NewDecoder(r).Decode(&v) doesn't necessarily consume all
data on the reader, that means an HTTP connection can't be reused. We
don't do a lot of HTTP traffic where we read JSON responses, but the
discovery is one such place. The other two are for POSTs from the GUI,
where it's not exactly critical but still nice if the connection still
can be keep-alive'd after the request as well.

Also ensure that we call req.Body.Close() for clarity, even though this
should by all accounts not really be necessary.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3050
2016-05-06 22:01:56 +00:00
Audrius Butkevicius 674fc566bb lib/connections: Refactor
1. Removes separate relay lists and relay clients/services, just makes it a listen address
2. Easier plugging-in of other transports
3. Allows "hot" disabling and enabling NAT services
4. Allows "hot" listen address changes
5. Changes listen address list with a preferable "default" value just like for discovery
6. Debounces global discovery announcements as external addresses change (which it might alot upon starting)
7. Stops this whole "pick other peers relay by latency". This information is no longer available,
   but I don't think it matters as most of the time other peer only has one relay.
8. Rename ListenAddress to ListenAddresses, as well as in javascript land.
9. Stop serializing deprecated values to JSON

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/2982
2016-05-04 19:38:12 +00:00
AudriusButkevicius 1a5f524ae4 lib/model, lib/protocol: Implement temporary indexes (fixes #950)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/2252
2016-04-15 10:59:41 +00:00
Jakob Borg 52c7804f32 lib/connections: Silence vet and lint warnings
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/2971
2016-04-13 11:50:51 +00:00
Laurent Etiemble 7b4e1e9055 cmd/syncthing: Fix handler ordering so CORS middleware wraps all the others but the debug one 2016-04-03 13:24:55 +02:00
Jakob Borg 4c3cd4c9e3 lib/ignore: Replace lib/fnmatch with github.com/gobwas/glob
Because it's literally ten times faster:



	benchmark                  old ns/op     new ns/op     delta

	BenchmarkMatch-8           13842         1200          -91.33%

	BenchmarkMatchCached-8     139           147           +5.76%



	benchmark                  old allocs     new allocs     delta

	BenchmarkMatch-8           0              0              +0.00%

	BenchmarkMatchCached-8     0              0              +0.00%



	benchmark                  old bytes     new bytes     delta

	BenchmarkMatch-8           12            0             -100.00%

	BenchmarkMatchCached-8     0             0             +0.00%
2016-04-02 20:03:24 +01:00
Audrius Butkevicius 29913dd1e4 lib/connections: Refactor address listing into connection service 2016-03-25 07:35:18 +00:00
Jakob Borg a492cfba13 cmd/syncthing: Extract interfaces for things the API depends on
Enables testing of the API service, in the long run.
2016-03-21 19:36:08 +00:00
Jakob Borg 9dec6f1324 cmd/syncthing: Return 500 with an error object instead of empty 200 on marshalling failure in REST response 2016-03-20 11:54:53 +01:00
Jakob Borg ba5231dc89 apiService should not reference global variable 'locations' (hinders testing) 2016-03-13 11:03:00 +01:00
Jakob Borg 032365d57c Fix STGUIASSETS search paths & order (fixes #2827) 2016-03-12 12:17:25 +00:00