Commit Graph

187 Commits

Author SHA1 Message Date
Jakob Borg f4372710bf
all: Remove crypto/md5 (#7493)
This is a mostly pointless change to make security scanners and static
analysis tools happy, as they all hate seeing md5. None of our md5 uses
were security relevant, but still. Only visible effect of this change is
that our temp file names for very long file names become slightly longer
than they were previously...
2021-03-17 22:22:49 +01:00
Simon Frei 3dd13c3994
test, lib/model: Various integration test updates & improvements (#6956) 2020-09-07 09:35:37 +02:00
Simon Frei 932d8c69de lib/fs: Properly handle case insensitive systems (fixes #1787, fixes #2739, fixes #5708)
With this change we emulate a case sensitive filesystem on top of
insensitive filesystems. This means we correctly pick up case-only renames
and throw a case conflict error when there would be multiple files differing
only in case.

This safety check has a small performance hit (about 20% more filesystem
operations when scanning for changes). The new advanced folder option
`caseSensitiveFS` can be used to disable the safety checks, retaining the
previous behavior on systems known to be fully case sensitive.

Co-authored-by: Jakob Borg <jakob@kastelo.net>
2020-07-28 11:15:11 +02:00
Simon Frei dbacef35c4
build: Update integration tests and add to build.go (#6744) 2020-06-16 14:33:17 +02:00
Jakob Borg dd92b2b8f4
all: Tweak error creation (#6391)
- In the few places where we wrap errors, use the new Go 1.13 "%w"
  construction instead of %s or %v.

- Where we create errors with constant strings, consistently use
  errors.New and not fmt.Errorf.

- Remove capitalization from errors in the few places where we had that.
2020-03-03 22:40:00 +01:00
Jakob Borg d91c4b010b
lib/config, lib/model: Limit concurrent pulls (fixes #5914) (#6290)
Adds a new folder state "Waiting to Sync" in the same vein as the
existing "Waiting to Scan". This vastly improves performances in the
rare cases when there are lots and lots of folders operating.
2020-01-27 17:31:17 +01:00
Simon Frei 2d124e053c
test: Get integration tests up to speed (config, build and test fixes) (#5962) 2019-08-20 10:17:11 +02:00
Simon Frei 1df8701c46 test: Report time per MiB on transfer benchs (#5711) 2019-05-18 08:46:08 +02:00
Simon Frei bc53782f88 test: Update conflict integration test (ref #5511) (#5618)
The change in 225c0dda80 (#5511) results in
conflicts being immediately scanned and synced, while the test expects just one
conflict on either side. Change it to expect the same conflict on both sides.
2019-03-25 12:53:06 +01:00
Jakob Borg c2ddc83509 all: Revert the underscore sillyness 2019-02-02 12:16:27 +01:00
Simon Frei 657be162dd test, lib/rc: Integration test fixes and polish (#5488) 2019-01-29 16:59:00 +01:00
Simon Frei c3b3e02f21 test: Update configs and revert changes during testing (#5393) 2018-12-21 11:50:28 +01:00
Jakob Borg 797a999585 test: Terminology only 2018-09-17 12:39:28 +02:00
Jakob Borg 4781e1d86f test: Migrate test configs to current version 2018-06-08 12:39:17 +02:00
Jakob Borg e187a5f5cb test: Add another variant of API timeout to skip when benchmarking 2018-05-30 13:23:52 +02:00
Jakob Borg 19c7cd99f5 all: Implement variable sized blocks (fixes #4807) 2018-04-16 19:08:50 +01:00
Jakob Borg 74b820f287 test: Update conflict tests 2018-03-04 14:52:09 +01:00
Jakob Borg 42cc64e2ed
lib/config, lib/model: Auto adjust pullers based on desired amount of pending data (#4748)
This makes the number of pullers vary with the desired amount of outstanding requests instead of being a fixed number.
2018-02-25 10:14:02 +01:00
Jakob Borg b97d5bcca8
Remove KCP (fixes #4737) (#4741) 2018-02-09 11:40:57 +01:00
Jakob Borg 2fdf9bc55d test: Mend the transfer benchmark 2018-02-03 10:29:05 +01:00
Jakob Borg e589e6c19d test: Forgot a resume 2017-11-21 08:21:25 +01:00
Jakob Borg 5f4ed66aa1 lib/model: Properly schedule pull on reconnect (fixes #4504)
We need to reset prevSeq so that we force a full check when someone
reconnects - the sequence number may not have changed due to the
reconnect. (This is a regression; we did this before f6ea2a7.)

Also add an optimization: we schedule a pull after scanning, but there
is no need to do so if no changes were detected. This matters now
because the scheduled pull actually traverses the database which is
expensive.

This, however, makes the pull not happen on initial scan if there were
no changes during the initial scan. Compensate by always scheduling a
pull after initial scan in the rwfolder itself.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/4508
LGTM: imsodin, AudriusButkevicius
2017-11-17 12:11:45 +00:00
Jakob Borg 80031c59da test: Clean & unflake cli & conflict tests 2017-11-13 01:06:16 +01:00
Jakob Borg 6e35592e9e test: Clean & unflake HTTP / filetype tests 2017-11-13 01:00:49 +01:00
Jakob Borg 9e11d7b201 test: Clean and unflake ignore / manupeers / reconnect / override tests 2017-11-13 01:00:36 +01:00
Jakob Borg fd5f9f8968 test: Fix reconnect test 2017-11-13 00:36:37 +01:00
Jakob Borg d8a0a477ca test: Unflake symlink/scan tests 2017-11-13 00:25:07 +01:00
Jakob Borg 6dd6ecde95 test: Clean up and unflake sync cluster test 2017-11-13 00:25:07 +01:00
Jakob Borg 6e148e20cd test: Mend tests for latest event changes etc 2017-11-13 00:25:07 +01:00
Jakob Borg 72c5f2e5c6 test: Updates for fs/osutil package changes 2017-11-13 00:25:07 +01:00
HairyFotr 7cbd92e1b1 all: Fix comment typos
Skip-check: authors

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/4481
2017-11-04 07:20:11 +00:00
HairyFotr 0ad10b0fee all: Typos
Skip-check: authors

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/4263
2017-07-20 13:10:46 +00:00
Audrius Butkevicius 0da0774ce4 lib/connections: Add KCP support (fixes #804)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3489
2017-03-07 12:44:16 +00:00
Jakob Borg f7fc0c1d3e all: Update license url to https (ref #3976) 2017-02-09 08:04:16 +01:00
Jakob Borg c4ba580cbb all: Remove symlink support on Windows, SymlinksEnabled config
After this change,

- Symlinks on Windows are always unsupported. Sorry.

- Symlinks are always enabled on other platforms. They are just a small
  file like anything else. There is no need to special case them. If you
  don't want to sync some symlinks, ignore them.

- The protocol doesn't differentiate between different "types" of
  symlinks. If that distinction ever does become relevant the individual
  devices can figure it out by looking at the destination when they
  create the link.

It's backwards compatible in that all the old symlink types are still
understood to be symlinks, and the new SYMLINK type is equivalent to the
old SYMLINK_UNKNOWN which was always a valid way to do it.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3962
LGTM: AudriusButkevicius
2017-02-07 08:34:24 +00:00
Jakob Borg dd7bb6c4b8 lib/model: Fix tests, clean up pool usage in protocol 2016-12-21 14:53:45 +01:00
Jakob Borg d41c131364 build: Enable gometalinter "gosimple" check, improve build.go 2016-12-21 14:53:45 +01:00
Audrius Butkevicius 0582836820 lib/model, lib/scanner: Efficient inserts/deletes in the middle of the file
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3527
2016-12-14 23:30:29 +00:00
Jakob Borg 6ec9b84674 test: Fix test config 2016-11-09 09:02:55 +08:00
Jakob Borg 6152eb6d6d gui: Hide "Failed Items" unless there is an actual failure (fixes #3647)
Since delta indexes it's perfectly normal for us to need files that are
currently unavailable due to devices being disconnected. This doesn't
imply a failure, so we should not show the "Failed Items" line and
corresponding eternal spinner (since it would never be filled in, since
there is no failure).

We still show state "Out of Sync" (correct) and the list of files we
need (correct).
2016-10-17 23:57:58 +02:00
Nathan Morrison d322ebd0b9 Add API service for local disk changes
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3626
LGTM: calmh, AudriusButkevicius
2016-09-28 15:54:13 +00:00
Jakob Borg 18cc7a663b lib: Remove osutil.Remove & osutil.RemoveAll (fixes #3513)
These are no longer required with Go 1.7. Change made by removing the
functions, doing a global s/osutil.Remove/os.Remove/.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3514
2016-08-16 10:01:58 +00:00
Jakob Borg ea87bcefd6 lib/protocol, lib/model: Implement high precision time stamps (fixes #3305)
This adds a new nanoseconds field to the FileInfo, populates it during
scans and sets the non-truncated time in Chtimes calls.

The actual file modification time is defined as modified_s seconds +
modified_ns nanoseconds. It's expected that the modified_ns field is <=
1e9 (that is, all whole seconds should go in the modified_s field) but
not really enforced. Given that it's an int32 the timestamp can be
adjusted += ~2.9 seconds by the modified_ns field...

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3431
2016-08-06 13:05:59 +00:00
Jakob Borg ffe7a2fcd7 cmd/syncthing, lib/config: Enable HTTP CPU/heap profile collection for users
This adds a config to enable debug functions on the API server, which is
by default disabled. When enabled, the /rest/debug things become
available and become available without requiring a CSRF token (although
authentication is required if configured).

We also add a new endpoint /rest/debug/cpuprof?duration=15s (with the
duration being configurable, defaulting to 30s). This runs a CPU profile
for the duration and returns it as a file. It sets headers so that a
browser will save the file with an informative name.

The same is done for heap profiles, /rest/debug/heapprof, which does not
take any parameters.

The purpose of this is that any user can enable debugging under
advanced, then point their browser to the endpoint above and get a file
that contains a CPU or heap profile we can use, with the filename
telling us what version and architecture the profile is from.

On the command line, this becomes

    curl -O -J http://localhost:8082/rest/debug/cpuprof?duration=5s
    curl: Saved to filename
    'syncthing-cpu-darwin-amd64-v0.14.3+4-g935bcc0-110307.pprof'

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3467
2016-08-02 11:06:45 +00:00
Jakob Borg 3176629410 cmd, lib: Fix ineffectual assignments (ineffasign) and comment spelling
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3405
2016-07-15 14:23:20 +00:00
Jakob Borg 137894348b test: Update test configs to latest format 2016-05-31 10:36:33 +02:00
Jakob Borg 2c1323ece6 lib/connections: Un-deprecate relaysEnabled (fixes #3074)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3098
2016-05-17 00:05:38 +00:00
Jakob Borg 09832abe50 lib/config: Change folder type attribute to a FolderType type
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3032
2016-05-04 11:26:36 +00:00
Michael Ploujnikov 66f480519b lib/model: Refactor out scanning method from rwfolder.Serve loop 2016-03-30 06:53:47 +00:00
Laurent Etiemble fed374fcb6 Add a CORS handler to deal with preflight OPTIONS requests 2016-01-26 21:55:51 +01:00