flaxo/ghost

public

Source code of Ghost - On Ghost

it now supports push and pull50ad2a1a month ago78 lines · 3.3 KB

1	# 0012 — Materialization is forward-only replay
2
3	Status: adopted
4
5	## Decision
6
7	`RepositoryMaterializerService` brings a cached bare repository up to the state
8	the log describes, before every read and before every push. The cache records
9	its position in `<repoDirectory>/ghost-wal-seq`.
10
11	```
12	read index → compare to cached seq → index-pack each missing layer in order
13	→ reconcile refs to the snapshot → repoint HEAD → write cached seq
14	```
15
16	This is what turns [0004](0004-object-storage-is-the-truth.md
17	intention into an enforced rule. Before it, the read path served whatever
18	happened to be on local disk and a cold node served an empty repository.
19
20	## Why replay is only ever forward
21
22	The cache can be behind the log but never ahead of it — that is the ordering
23	guarantee [0005](0005-single-commit-point.md
24	needs to undo anything: it applies missing packfiles and moves refs to the
25	snapshot. There is no rollback path to get wrong.
26
27	## Packs from `
28
29	A pushed packfile may contain deltas whose base objects the client knew the
30	server already had, so it omitted them. Replaying such a pack into an empty
31	object store fails.
32
33	Two things make this work:
34
35	- Sequence order is mandatory. Layer
36	so replay walks `layers` in index order and never in parallel.
37	- **`git index-pack --fix-thin --stdin`
38	already present, rather than `unpack-objects`, which cannot.
39
40	## Refs are reconciled, not replayed
41
42	Entries carry transitions, but materialization applies the index's snapshot
43	via a single `git update-ref --stdin` batch, then deletes any cached ref the
44	snapshot no longer carries. Replaying transitions one at a time would produce
45	identical final state at more cost, and would fail on a cache that had drifted.
46	Reconciling to the snapshot is idempotent from any starting state, which is what
47	makes a stale or hand-edited cache self-healing.
48
49	## HEAD
50
51	A bare repository whose `HEAD` names a branch that does not exist clones as
52	empty with no error — one of git's least helpful behaviours. After reconciling,
53	`ensureHead` repoints `HEAD` at `main
54
55	## The push path deliberately leaves the marker stale
56
57	`GitService.receivePack` materializes, commits to the log, then lets
58	`git receive-pack` apply the push to the cache — but does not
59	`ghost-wal-seq`. If another node's push interleaved, the local cache is now a
60	fork; leaving the marker behind guarantees the next materialize replays over it
61	and reconciles refs to the log. The cost is one redundant `index-pack` of
62	objects already present; the benefit is that no interleaving can leave the cache
63	silently wrong.
64
65	## Concurrency
66
67	`materialize` collapses concurrent calls for the same repository through an
68	in-flight promise map, so parallel fetches on one node replay once. Across
69	nodes there is no coordination and none is needed: replay is idempotent, and
70	every node converges on the same log.
71
72	## Consequences
73
74	- A node can serve any repository it has never seen.
75	- Local disk can be wiped as an operational action.
76	- `repository-materializer.service.spec.ts` exercises this against real `git`
77	binaries and real packfiles with an in-memory store, so push → materialize →
78	`git clone` is covered without object storage.