GLM-5.3 · Heavy
dual-storage-architecture-grafana-codex
Grafana · Go · L · 300–799 lines · 4 files
The generalist plus 9 specialist lens agents in parallel, then the same verify and dedup steps. Judge: Claude Haiku 5.5.
- Gold bugs found
- 8 of 9
- Comments published
- 10
- Comments matching a bug
- 8
- Comments not matching
- 2
- Cost of this PR
- $2.623
Published comments (10)
Async legacy writes use request context that cancels when handler returns — This defeats mode 3's stated purpose (legacy write as a safe measure) and only passes tests because local file/unified storage finishes within the response-write window. Apply the timeout to a detached context instead, e.g. `context.WithTimeoutCause(context.WithoutCancel(ctx), time.Second*10, ...)` (the same pattern Delete at line 110, Update at 135, and DeleteCollection at 163 must use).
Fire-and-forget goroutines break legacy write ordering across dual stores — Mode 3 now performs every legacy write in a fire-and-forget goroutine (same pattern repeated at lines 108-114, 134-141 and 161-167). Dual-write correctness assumes operations reach LegacyStorage in the same order they were applied to Storage, but goroutine execution order is not guaranteed: a delete(name) followed by a create(name) can leave the object present in legacy storage while unified storage reports it deleted, and two sequential updates (A then B) can leave legacy holding A's stale state. **Also found in:** - `pkg/apiserver/rest/dualwriter_mode3.go:134-141`
Delete returns early on NotFound from unified storage, orphaning legacy-only rows — Delete now returns immediately when `d.Storage.Delete` fails with any error, including NotFound, whereas the removed code tolerated NotFound and still called `d.Legacy.Delete` to clean up legacy-only rows. Objects that exist only in legacy storage (the migration drift this dual writer exists to reconcile) become undeletable: the caller gets a 404 and the legacy row is orphaned forever, reappearing if reads ever fall back to legacy.
Data race: async legacy Create mutates obj during response serialization — Mode 1 explicitly guards this exact scenario with `created.DeepCopyObject()` before its async write (dualwriter_mode1.go:51). Pass `obj.DeepCopyObject()` into the goroutine.
Mode 3 tests never assert the asynchronous legacy half of the dual write — dualwriter_mode3.go now performs Legacy Create/Update/Delete/DeleteCollection in fire-and-forget goroutines, but every TestMode3_* function returns after the synchronous Storage call and asserts only on its result — no AssertExpectations/AssertCalled or call-count check exists anywhere in the package, and nothing waits for the goroutines. If all the `go func() { d.Legacy.X(...) }` blocks were deleted from the implementation, every one of these tests would still pass (TestMode3_Delete/TestMode3_DeleteCollection don't even configure a legacy expectation).
Shared mock.Mock makes Get/List tests unable to pin reads to Storage — TestMode3_Get (and TestMode3_List) cannot detect which backend served the read, because legacyStoreMock and storageMock wrap the same shared mock.Mock and dispatch to the same method names — the `m.On("Get", mock.Anything, name, ...)` expectation set via setupStorageFn is matched identically by d.Legacy.Get. If Mode3's Get/List were changed to read from LegacyStorage instead of Storage (mode 3's core read contract, and the difference between mode 2 and mode 3), these tests still pass.
Async legacy Update re-derives object instead of writing unified storage result — Mode 2 still enforces replication with `&updateWrapper{upstream: objInfo, updated: obj}` (dualwriter_mode2.go:298), as did this file's previous implementation. Wrap the unified-storage result: `d.Legacy.Update(ctx, name, &updateWrapper{upstream: objInfo, updated: res}, ...)`.
Create/Update storage error paths record duration under legacy metric — On unified-storage failure, Create records the failure with `d.recordLegacyDuration(true, ...)` using the storage start time — the error and elapsed time land in grafana_dual_writer_legacy_duration_seconds for a call that never touched legacy storage, and grafana_dual_writer_storage_duration_seconds never sees its own failures. Update's error path has the identical defect at line 129. **Also found in:** - `pkg/apiserver/rest/dualwriter_mode3.go:127-131` - `pkg/apiserver/rest/dualwriter_mode3.go:129-132`
DeleteCollection legacy goroutine records latency on the storage metric instead of the legacy metric — It must call `d.recordLegacyDuration(err != nil, ...)` like the legacy goroutines in Create (line 56), Delete (line 113), and Update (line 140).
Delete success path records object name instead of kind in storage duration metric — The Delete success path passes the object `name` where every other call site in this file passes `options.Kind`, so the kind label of grafana_dual_writer_storage_duration_seconds receives the object's name (e.g. "foo") instead of the resource kind. Per-kind aggregation breaks, and each distinct object name creates a new histogram series (buckets × is_error × mode × method), growing label cardinality without bound as objects are deleted. **Also found in:** - `pkg/apiserver/rest/dualwriter_mode3.go:2-7`