flaxo/ghost

public

Source code of Ghost - On Ghost

ghostdocs0010-orphans-are-garbage.md

it now supports push and pull50ad2a1a month ago37 lines · 1.5 KB

1	# 0010 — Orphaned entries are garbage, never corruption
2
3	Status: adopted (collector not yet built)
4
5	## Decision
6
7	The entry upload happens outside the retry loop and is never rolled back. An
8	entry that no index references is dead weight to be swept later, and is not an
9	error condition.
10
11	## Why
12
13	This is what makes [0005](0005-single-commit-point.md) work. The expensive,
14	slow, unbounded part of a push (uploading a packfile) is deliberately
15	uncoordinated: content-addressed, idempotent, order-independent. Only the tiny
16	index write is contended. If the expensive part needed rollback, the design
17	would need a distributed transaction; because it does not, it needs one
18	conditional PUT.
19
20	An unreferenced packfile is invisible to every reader. Nothing resolves it,
21	nothing serves it, it costs storage and nothing else.
22
23	## The trap in the collector
24
25	The obvious sweep — "delete entries no index references" — is **wrong on its
26	own**. An entry being uploaded right now is legitimately unreferenced for the
27	duration of the upload plus the CAS. Deleting it destroys an in-flight push.
28
29	The sweep must be: unreferenced and older than a generous grace period,
30	hours rather than minutes, exceeding the longest plausible push. The ULID's
31	embedded timestamp supplies the age with no extra bookkeeping.
32
33	## Consequence
34
35	Storage grows with failed pushes until the collector exists. At current volume
36	this is not urgent, but it is unbounded, so it is tracked in
37	[0011](0011-deferred.md).

it now supports push and pull50ad2a1a month ago37 lines · 1.5 KB

1	# 0010 — Orphaned entries are garbage, never corruption
2
3	Status: adopted (collector not yet built)
4
5	## Decision
6
7	The entry upload happens outside the retry loop and is never rolled back. An
8	entry that no index references is dead weight to be swept later, and is not an
9	error condition.
10
11	## Why
12
13	This is what makes [0005](0005-single-commit-point.md) work. The expensive,
14	slow, unbounded part of a push (uploading a packfile) is deliberately
15	uncoordinated: content-addressed, idempotent, order-independent. Only the tiny
16	index write is contended. If the expensive part needed rollback, the design
17	would need a distributed transaction; because it does not, it needs one
18	conditional PUT.
19
20	An unreferenced packfile is invisible to every reader. Nothing resolves it,
21	nothing serves it, it costs storage and nothing else.
22
23	## The trap in the collector
24
25	The obvious sweep — "delete entries no index references" — is **wrong on its
26	own**. An entry being uploaded right now is legitimately unreferenced for the
27	duration of the upload plus the CAS. Deleting it destroys an in-flight push.
28
29	The sweep must be: unreferenced and older than a generous grace period,
30	hours rather than minutes, exceeding the longest plausible push. The ULID's
31	embedded timestamp supplies the age with no extra bookkeeping.
32
33	## Consequence
34
35	Storage grows with failed pushes until the collector exists. At current volume
36	this is not urgent, but it is unbounded, so it is tracked in
37	[0011](0011-deferred.md).