Commit Graph

5827 Commits

Author SHA1 Message Date
Jakob Borg 04ff890263
build: Clean up build.sh, add build.ps1 (#6689) 2020-05-28 12:42:15 +02:00
greatroar 9c0825c0d9
lib/scanner: Simplify, optimize and document Validate (#6674) (#6688) 2020-05-27 22:23:12 +02:00
Jakob Borg f78133b8e9
lib/db: Adjust transaction flush sizes downwards (#6686)
This reduces the size of our write batches before we flush them. This
has two effects: reducing the amount of data lost if we crash when
updating the database, and reducing the amount of memory used when we do
large updates without checkpoint (e.g., deleting a folder).

I ran our SyncManyFiles benchmark as it is the one doing most
transactions, however there was no relevant change in any metric (it's
limited by our fsync I expect). This is good as any visible change would
just be a decrease in performance.

I don't have a benchmark on deleting a large folder, taking that part on
trust for now...
2020-05-27 12:15:00 +02:00
Simon Frei b784f5b9e3
lib/config, lib/model: Commit auto-accepted folders all at once (#6684) 2020-05-27 08:05:26 +02:00
Jakob Borg 1c089a4d11 gui, man, authors: Update docs, translations, and contributors 2020-05-27 07:45:24 +02:00
greatroar baa38eea7a
lib/assets: Allow assets to remain uncompressed (#6661) 2020-05-25 08:51:27 +02:00
Simon Frei c3b5eba205
lib/model: Fix checking children when trying to delete a dir (fixes #6646) (#6647) 2020-05-25 08:46:24 +02:00
Simon Frei cf75329067
lib/model: Handle concurrently removed device in ClusterConfig (#6666) 2020-05-20 11:13:55 +02:00
Jakob Borg 8343db6766 Merge branch 'release'
* release:
  lib/db: Initialize need meta on first dev occurrence (fixes #6668) (#6669)
  lib/db: Don't panic on seq. coruption when debugging (#6662)
2020-05-20 11:05:53 +02:00
Simon Frei 8c74177699 lib/db: Initialize need meta on first dev occurrence (fixes #6668) (#6669) 2020-05-20 11:04:45 +02:00
Simon Frei 9a5f7fbadf lib/db: Don't panic on seq. coruption when debugging (#6662) 2020-05-20 11:04:38 +02:00
Simon Frei 8f344d0915
lib/db: Initialize need meta on first dev occurrence (fixes #6668) (#6669) 2020-05-20 11:01:27 +02:00
Jakob Borg 77dd874383 gui, man, authors: Update docs, translations, and contributors 2020-05-20 07:45:28 +02:00
Simon Frei 5b34c31cb3
lib/db: Don't panic on seq. coruption when debugging (#6662) 2020-05-18 10:42:51 +02:00
André Colomb 668979605b
cmd/stindex: Add missing KeyType values in stindex dump code (#6659)
* cmd/stindex: Unify access to key from cached variable.

Avoid calling the Key() method from the iterator each time the value
is needed.  Just reuse the cache variable already assigned before the
switch block.

* cmd/stindex: Display the prefix byte value for unknown key types.

Make it easier to diagnose corrupt / unknown key type entries by
showing their decimal value, correlating with the definitions in
keyer.go.

* cmd/stindex: Add missing KeyType values in stindex dump code.

Recently added DB key prefixes KeyTypeBlockListMap and KeyTypeVersion
were unknown to the stindex dumping tool.  Add basic parsing to dump
their key structure.
2020-05-17 11:06:14 +02:00
Jakob Borg 5ffa012410
lib/model: Don't crash when taking rename shortcut (fixes #6654) (#6657)
If we fail to take the rename shortcut we may crash on a later loop,
because we do trickiness with the indexes but the original buckets[key]
in "range buckets[key]" isn't re-evaluated so i exceeds the max index.
2020-05-17 08:48:35 +01:00
Jakob Borg 5c54d879a1
cmd/syncthing: Don't crash when failing to create default config (fixes #6655) (#6658)
This is not an ignorable error, because it can happen if we fail to
allocate a free port for the GUI or sync port on first startup.
2020-05-17 07:56:24 +02:00
Jakob Borg 1c5af3a4bd Merge branch 'release'
* release:
  lib/model: Partial revert of rename fix (fixes #6653) (#6656)
2020-05-16 23:08:43 +02:00
Jakob Borg 22c222bf75 lib/model: Partial revert of rename fix (fixes #6653) (#6656)
We can't just drop the snap because it's in use elsewhere. This should
be equally functional.
2020-05-16 23:06:25 +02:00
Jakob Borg 651ee2ce74
lib/model: Partial revert of rename fix (fixes #6653) (#6656)
We can't just drop the snap because it's in use elsewhere. This should
be equally functional.
2020-05-16 23:05:33 +02:00
Jakob Borg 5c9dc4c883 Merge branch 'release'
* release:
  lib/model: Fix rename handling (ref #6650) (#6652)
  lib/db: Filter repeat files in one update (ref #6650) (#6651)
2020-05-16 14:42:11 +02:00
Audrius Butkevicius 258341f8bf lib/model: Fix rename handling (ref #6650) (#6652) 2020-05-16 14:40:20 +02:00
Simon Frei f5ca213682 lib/db: Filter repeat files in one update (ref #6650) (#6651) 2020-05-16 14:40:20 +02:00
Audrius Butkevicius a1c5b44c74
lib/model: Fix rename handling (ref #6650) (#6652) 2020-05-16 14:39:27 +02:00
Simon Frei de9489585f
lib/db: Filter repeat files in one update (ref #6650) (#6651) 2020-05-16 14:34:53 +02:00
Jakob Borg 438f687591 docker: Add tzdata for local time log entries 2020-05-16 11:34:46 +02:00
Shaarad Dalvi 7a8cc5fc99
docker: Improved health check for host networks (#6649) 2020-05-15 12:59:22 +02:00
xarx00 f05ccd775a
lib/fs: Set execute bits on junctions converted to dirs (ref #6606) (#6645) 2020-05-14 08:09:58 +02:00
Simon Frei e5cc55ce09
lib/model: Close conns when devices are removed (fixes #6564) (#6641) 2020-05-14 07:50:53 +02:00
Simon Frei 299b9d8883
lib/model: Adjust remote-rename-test to timer-based versions (fixes #6625) (#6644) 2020-05-14 00:31:05 +02:00
Simon Frei 074097a8e7
lib/fs: Prevent race-detector triggering in tests (fixes #6608) (#6642) 2020-05-14 00:30:09 +02:00
xarx00 ee445e35a0
lib/fs: Treat Windows junctions as normal directories (#6606)
Fixes #1830, presumably.
2020-05-13 21:46:24 +02:00
Jakob Borg 3ad049184e Merge branch 'release'
* release:
  lib/db: Dont add symlinks to blocks map (fixes #6637) (#6639)
2020-05-13 20:48:34 +02:00
Simon Frei 6768daec07 lib/db: Dont add symlinks to blocks map (fixes #6637) (#6639) 2020-05-13 20:47:05 +02:00
Simon Frei 974551375e
lib/db: Dont add symlinks to blocks map (fixes #6637) (#6639) 2020-05-13 20:38:21 +02:00
Jakob Borg 531ceb2b0f
Add indirection for large version vectors. (#6376)
This adds indirection of large version vectors in the same manner as we
already to block lists. The effect is the same: less duplicated data in
some situations.

To mitigate the impact for when this indirection
wouldn't be needed I've added an indirection cutoff for both blocks and
the new version vector stuff: we don't do the indirection at all for
small block lists or small version vectors, instead storing it directly
like we used to do. This is faster for small files and small setups.
2020-05-13 14:28:42 +02:00
Simon Frei ba4462a70b
lib/db: Fix updating/removing from global (ref #6413) (#6635) 2020-05-13 12:56:49 +02:00
Jakob Borg 8419c05794 gui, man, authors: Update docs, translations, and contributors 2020-05-13 07:45:31 +02:00
NinoM4ster c84f60f949 etc: Add RestartSec=5 to linux-systemd units
Might help avoiding the 'Start request repeated too quickly.' error.

Fixes #6633, fixes #6634
2020-05-12 10:02:32 +02:00
Audrius Butkevicius 6201eebc98
lib/model: Add support for different puller block ordering (#6587)
* WIP

* Tests

* Header and format

* WIP

* Fix tests

* Goland you disappoint me

* Remove CC storage

* Update blockpullreorderer.go
2020-05-11 22:44:04 +01:00
Audrius Butkevicius decb967969
all: Reorder sequences for better rename detection (#6574) 2020-05-11 20:15:11 +02:00
Jakob Borg aac3750298 build: Enable hardened runtime in Mac codesigning
syncthing/syncthing-macos#111
2020-05-11 18:19:39 +02:00
Simon Frei a94951becd
lib/db, lib/model: Keep need stats in metadata (ref #5899) (#6413) 2020-05-11 15:07:06 +02:00
Audrius Butkevicius 7dc290c3ed
lib/connections: React to listeners going up and down faster (#6590) 2020-05-11 15:02:22 +02:00
Jakob Borg 50faa8f7ef build: Check new assets location for rebuild 2020-05-11 12:20:21 +02:00
Simon Frei 5c87ceb392
build: Update .gitignores for new asset locs (ref #6624) (#6631) 2020-05-11 11:04:47 +01:00
dependabot-preview[bot] 6ffc8255b6
build(deps): bump github.com/lucas-clemente/quic-go (#6630)
Bumps [github.com/lucas-clemente/quic-go](https://github.com/lucas-clemente/quic-go) from 0.15.6 to 0.15.7.
- [Release notes](https://github.com/lucas-clemente/quic-go/releases)
- [Changelog](https://github.com/lucas-clemente/quic-go/blob/master/Changelog.md)
- [Commits](https://github.com/lucas-clemente/quic-go/compare/v0.15.6...v0.15.7)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>

Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>
2020-05-11 11:04:23 +01:00
dependabot-preview[bot] 3354e60461
build(deps): bump github.com/go-ldap/ldap/v3 from 3.1.7 to 3.1.10 (#6629)
Bumps [github.com/go-ldap/ldap/v3](https://github.com/go-ldap/ldap) from 3.1.7 to 3.1.10.
- [Release notes](https://github.com/go-ldap/ldap/releases)
- [Commits](https://github.com/go-ldap/ldap/compare/v3.1.7...v3.1.10)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>

Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>
2020-05-11 10:14:31 +01:00
greatroar 06365e5635
cmd/strelaypoolsrv, lib/api: Factor out static asset serving (#6624) 2020-05-10 11:44:34 +02:00
Audrius Butkevicius da99203dcd
lib/db: Fix non-truncated fileinfo loading (#6621) 2020-05-08 20:31:43 +01:00