DeepSeek V4.1 Flash · Balanced

dual-storage-architecture-grafana-codex

Grafana · Go · L · 300–799 lines · 4 files

The normal review, with one generalist agent. Judge: Claude Haiku 5.5.

Gold bugs found
6 of 9
Comments published
6
Comments matching a bug
6
Comments not matching
0
Cost of this PR
$0.127

Published comments (6)

highbugpkg/apiserver/rest/dualwriter_mode3.go:50-57

Async legacy writes use request context canceled after response — The legacy `Create`/`Update`/`Delete`/`DeleteCollection` run under the request context, which Go cancels as soon as `ServeHTTP` returns, so each aborts with `context.Canceled` right after the response and the dual write never happens. Detach the goroutine with `context.WithoutCancel(ctx)` while keeping request values.

highbugpkg/apiserver/rest/dualwriter_mode3.go:134-141

Async legacy update reuses raw objInfo instead of persisted result — `Legacy.Update` re-invokes `objInfo.UpdatedObject` on the legacy store's own old object, so the legacy copy can diverge from unified storage. Wrap the persisted `res` in `updateWrapper` as modes 1 and 2 do and pass that to `d.Legacy.Update`.

highbugpkg/apiserver/rest/dualwriter_mode3.go:108-114

Async legacy writes in mode 3 can be applied out of order per object — Fire-and-forget goroutines drop the ordering guarantee: a create-then-delete, as in `playlist_test.go:506-576`, can reach legacy reversed and leave the row orphaned. Serialize the async legacy writes per object with a single worker queue keyed by kind+name.

mediumbugpkg/apiserver/rest/dualwriter_mode3.go:165-166

Legacy DeleteCollection records to storage histogram instead of legacy — Legacy latency lands in the storage histogram and legacy failures stay invisible in `grafana_dual_writer_legacy_duration_seconds`. Call `d.recordLegacyDuration`, as the other legacy calls do.

mediumbugpkg/apiserver/rest/dualwriter_mode3.go:106

Delete storage duration metric uses object name as kind label — `recordStorageDuration` gets the object `name` in the `kind` label, so every deleted object adds a new `grafana_dual_writer_storage_duration_seconds` series (unbounded growth, unusable filtering). Pass `options.Kind` instead.

mediumbugpkg/apiserver/rest/dualwriter_mode3.go:45

Create error path records legacy duration instead of storage duration — The storage failure goes to the legacy histogram, so `grafana_dual_writer_storage_duration_seconds` never shows `is_error=true` and legacy gets a false spike. Record it with `d.recordStorageDuration` instead. **Also found in:** - `pkg/apiserver/rest/dualwriter_mode3.go:125-129`

Gold bugs found (6)

mediumThe legacy DeleteCollection goroutine records its duration with recordStorageDuration instead of recordLegacyDuration, misattributing legacy latency and errors to the storage metric.
mediumDelete's success path records the storage metric with the object name in the kind label instead of options.Kind, creating per-object series and inconsistent labels.
mediumOn storage Create/Update failure the duration is recorded with recordLegacyDuration, attributing unified-store errors to the legacy metric.
highMode 3 legacy writes now run in background goroutines derived from the request context, which is cancelled when the request completes, so legacy writes can be cancelled and legacy storage falls out of sync.
highLegacy.Update now receives the original objInfo instead of a wrapper carrying the object stored in unified storage, so legacy recomputes the update and can diverge.
highIndependent per-request goroutines remove ordering between legacy writes to the same resource, so legacy can end up with an older version or a resurrected object.

Gold bugs missed (3)

mediumThe Mode 3 tests never wait for or assert the asynchronous legacy writes, and legacy and storage share one mock, so removing the legacy calls would not fail them.
mediumMode 3 legacy-storage write errors (create/update/delete/delete-collection) are no longer logged after moving to async goroutines; only a duration metric with an error flag is recorded.
highDualWriterMode3.Delete now returns on any unified-storage error, including NotFound, before deleting from legacy, so objects that exist only in legacy can no longer be deleted.