Skip to content

Commit 4b9d8eb

Browse files
committed
Fix README for PATCH
1 parent 92ddeb8 commit 4b9d8eb

File tree

1 file changed

+30
-22
lines changed

1 file changed

+30
-22
lines changed

README.md

Lines changed: 30 additions & 22 deletions
Original file line numberDiff line numberDiff line change
@@ -93,18 +93,19 @@ See also [Common Issues](#common-issues).
9393

9494
\* xattrs without extra RTT only work with Yandex S3 (--list-type=ext-v1).
9595

96-
\* Partial object updates only work with Yandex S3.
96+
\* Partial object updates (PATCH) only work with Yandex S3.
9797

98-
## Partial object updates
98+
## Partial object updates (PATCH)
9999

100-
With Yandex S3 it is possible to do partial object updates (data only) without server-side copy or reupload.
101-
Currently the feature can be enabled by the flag `--enable-patch` and will be enabled by default for YC S3 in the future.
100+
With Yandex S3 it is possible to do partial object updates (data only) without server-side copy or reupload.
101+
102+
Currently the feature can be enabled with `--enable-patch` and will be enabled by default for YC S3 in the future.
102103

103104
Enabling patch uploads has the following benefits:
104105
- Fast [fsync](#fsync): since nothing needs to be copied, fsync is now much cheaper
105106
- Support for [concurrent updates](#concurrent-updates)
106107
- Better memory utilization: less intermediate state needs to be cached, so more memory can be used for (meta)data cache
107-
- Better performace for big files
108+
- Better performance for big files
108109

109110
Note: new files, metadata changes and renames are still flushed to S3 as multipart uploads.
110111

@@ -231,23 +232,6 @@ fio -name=test -ioengine=libaio -direct=1 -bs=4M -iodepth=1 -fallocate=none \
231232

232233
## Concurrent Updates
233234

234-
### Yandex S3
235-
236-
When using Yandex S3, it is possible to concurrently update a single object/file from multiple hosts
237-
using PATCH method (`--enable-patch`). However, concurrent changes are not reported back to the clients,
238-
so in order to see the actual object contents you need to stop all writes and refresh the inode cache (see below).
239-
240-
It is strongly advised that clients from different hosts write data by non-overlapping offsets and
241-
the writes are aligned with object parts borders to avoid conflicts. If it impossible to avoid conflicts entirely,
242-
the conflicts are resolved by the LWW strategy. In case the conflict can't be resolved,
243-
you can choose to drop the cached update (`--drop-patch-conflicts`), otherwise the write will be retried later.
244-
245-
The conflicts are reported in the log as following:
246-
247-
```
248-
main.WARNING Failed to patch range %d-%d of file %s (inode %d) due to concurrent updates
249-
```
250-
251235
### Other clouds
252236

253237
GeeseFS doesn't support concurrent updates of the same file from multiple hosts. If you try to
@@ -261,6 +245,30 @@ you may encounter lost updates (conflicts) which are reported in the log in the
261245
main.WARNING File xxx/yyy is deleted or resized remotely, discarding local changes
262246
```
263247

248+
### PATCH
249+
250+
When using Yandex S3, it is possible to concurrently update a single object/file from multiple hosts
251+
using PATCH method (`--enable-patch`). However, concurrent changes are not reported back to the clients,
252+
so you have to stop all writes and refresh the inode cache to see the changes made by other hosts.
253+
254+
It is strongly advised to write data in non-overlapping ranges to avoid conflicts.
255+
If you do overlapping writes then you have to guarantee that your hosts coordinate
256+
and serialize their updates by doing an `fsync()` before the next overlapping write.
257+
258+
Also note that even with PATCH, best performance is achieved when writes are aligned
259+
with object part boundaries (i.e. with 5 MB chunks for the first 5 GB by default),
260+
because the server may do internal read-modify-write for patched object parts, and
261+
it also may return an error if too many parallel requests try to modify the same
262+
object part. Such "PATCH failures" are reported in the log in the following way:
263+
264+
```
265+
main.WARNING Failed to patch range %d-%d of file %s (inode %d) due to concurrent updates
266+
```
267+
268+
Normally, GeeseFS retries requests failed in this way, but if you don't want it
269+
to retry them, you can also choose to drop the cached update by enabling
270+
`--drop-patch-conflicts`.
271+
264272
## Asynchronous Write Errors
265273

266274
GeeseFS buffers updates in memory (or disk cache, if enabled) and flushes them asynchronously,

0 commit comments

Comments
 (0)