Kimi K3.1 Pro · Balanced

dual-storage-architecture-grafana-codex

Grafana · Go · L · 300–799 lines · 4 files

The normal review, with one generalist agent. Judge: Claude Haiku 5.5.

Gold bugs found
7 of 9
Comments published
9
Comments matching a bug
7
Comments not matching
2
Cost of this PR
$1.108

Published comments (9)

mediumbugpkg/tests/apis/playlist/playlist_test.go:140-142

Mode3 dual-write tests silently exercise Mode1 instead — `InstallAPIs` resolves this desired value through `SetDualWritingMode`, which defaults a fresh key to Mode1 and only implements Mode1/Mode2 transitions, so these new cases silently exercise Mode1 and Mode3 cannot be enabled through this configuration. Add the Mode3 transition and assert the selected mode.

mediumbugpkg/apiserver/rest/dualwriter_mode3.go:50-56

Async legacy writes inherit canceled request contexts — The timeout inherits the incoming HTTP request context, which is canceled when the handler returns, so the asynchronous legacy call can start with an already-canceled context and skip the safety write; delete, update, and delete-collection repeat this pattern. Build the timeout from `context.WithoutCancel(ctx)` so request values remain but request cancellation does not propagate.

mediumbugpkg/apiserver/rest/dualwriter_mode3.go:134-140

Async legacy writes can reorder creates and updates, leaving stale state — Independent legacy goroutines allow sequential API operations to execute out of order, so an older update can finish after a newer one or an update can run before its create, leaving legacy stale or missing after all writes finish. Serialize legacy operations per object/key while keeping the unified write synchronous.

mediumbugpkg/apiserver/rest/dualwriter_mode3.go:139

Legacy update recomputes the object instead of using the storage result — Passing the original `objInfo` makes legacy updaters such as playlist's call `UpdatedObject` a second time against legacy state instead of persisting the object produced by unified storage, allowing non-idempotent transforms and server-populated fields to diverge. Capture a deep copy of `res` and pass it through `updateWrapper` as the previous implementation did.

mediumperformancepkg/apiserver/rest/dualwriter_mode3.go:50-57

Unbounded legacy-write goroutines accumulate when the backend is slow — Every successful write creates an untracked goroutine with no concurrency bound, so a slow legacy backend lets completed requests accumulate goroutines and backend operations without applying backpressure. Replace per-request goroutines in all four write paths with a bounded worker queue.

mediumperformancepkg/apiserver/rest/dualwriter_mode3.go:106

Delete metrics use object names as kind labels, causing unbounded cardinality — Passing `name` as the `kind` label creates a persistent Prometheus histogram series per deleted object, causing unbounded cardinality and memory growth while preventing per-kind aggregation. Pass `options.Kind` instead.

mediumbugpkg/apiserver/rest/dualwriter_mode3.go:166

Legacy delete-collection duration is misattributed to unified storage — Call `recordLegacyDuration` here.

mediumbugpkg/apiserver/rest/dualwriter_mode3.go:101-104

Returning on every unified-storage error skips the legacy delete when unified storage returns… — This leaves stale legacy data that can reappear after a rollback; preserve the `NotFound` exception by scheduling `d.Legacy.Delete` and returning the original not-found error, while failing fast only for other errors.

mediumbugpkg/apiserver/rest/dualwriter_mode3.go:45

Unified create failures are recorded in the legacy histogram — Call `recordStorageDuration` here. **Also found in:** - `pkg/apiserver/rest/dualwriter_mode3.go:129-129`

Gold bugs found (7)

mediumThe legacy DeleteCollection goroutine records its duration with recordStorageDuration instead of recordLegacyDuration, misattributing legacy latency and errors to the storage metric.
mediumDelete's success path records the storage metric with the object name in the kind label instead of options.Kind, creating per-object series and inconsistent labels.
mediumOn storage Create/Update failure the duration is recorded with recordLegacyDuration, attributing unified-store errors to the legacy metric.
highMode 3 legacy writes now run in background goroutines derived from the request context, which is cancelled when the request completes, so legacy writes can be cancelled and legacy storage falls out of sync.
highLegacy.Update now receives the original objInfo instead of a wrapper carrying the object stored in unified storage, so legacy recomputes the update and can diverge.
highIndependent per-request goroutines remove ordering between legacy writes to the same resource, so legacy can end up with an older version or a resurrected object.
highDualWriterMode3.Delete now returns on any unified-storage error, including NotFound, before deleting from legacy, so objects that exist only in legacy can no longer be deleted.

Gold bugs missed (2)

mediumThe Mode 3 tests never wait for or assert the asynchronous legacy writes, and legacy and storage share one mock, so removing the legacy calls would not fail them.
mediumMode 3 legacy-storage write errors (create/update/delete/delete-collection) are no longer logged after moving to async goroutines; only a duration metric with an error flag is recorded.