diff --git a/doc/rfc/index.md b/doc/rfc/index.md index 4122aff7..f27eaaf0 100644 --- a/doc/rfc/index.md +++ b/doc/rfc/index.md @@ -10,6 +10,7 @@ Design documents and technical proposals, grouped by scope. Shared/cross-cutting - [Consumer Gate](consumer-gate.md) - Stopping and starting individual queue controllers at runtime via a consumer-side check: blocked deliveries are recorded as parked and postponed back to the queue (re-checked on redelivery), gate state as a separate extension with a file-based first implementation shared by tests and operators - [Consumer Hold](consumer-hold.md) - Fourth delivery outcome letting a controller postpone its delivery: the message becomes a partition barrier that pauses consumption for a chosen delay, redelivers in order, and does not count as a failure toward dead-lettering - [Change URIs](change-uri.md) - Identity of a code change: `scheme://{host[:port]}/{path}` per provider (GitHub PR, Phabricator Diff, git ref/commit) and canonical-form rules +- [Scoped Sequential Resource IDs](scoped-resource-ids.md) - Queue-scoped positive numeric IDs allocated by durable per-domain, per-kind counters, stored without queue/kind prefixes, and rendered directly in resource URL segments - [Hooks Framework](hook-framework.md) - Implemented fire-and-forget side effects: one shared `HookEvent` contract (`api/base/hook/`) on a durable per-domain hook topic, dispatched by `platform/hook` to `platform/extension/hook`. Stovepipe `process` and `record` publish repository events; the SubmitQueue orchestrator registers the stage and does not publish events yet - [Service-Scoped Extensions](service-scoped-extensions.md) - Implemented for SubmitQueue storage: gateway and orchestrator aggregates, schemas, and the core packages that serve one service have moved, while store contracts stay at `submitqueue/extension/storage`. Domain-level `buildrunner`, `conflict`, and `speculation` have not moved, and `changeset` still declares its own store slice diff --git a/doc/rfc/scoped-resource-ids.md b/doc/rfc/scoped-resource-ids.md new file mode 100644 index 00000000..e408459e --- /dev/null +++ b/doc/rfc/scoped-resource-ids.md @@ -0,0 +1,68 @@ +# Scoped Sequential Resource IDs + +## Decision + +A generated resource ID is the canonical decimal string for a positive value returned by a durable counter scoped to `(owner domain, queue, resource kind)`. + +| Resource | Current ID | Proposed ID | Complete identity | +|---|---:|---:|---| +| SubmitQueue request | `demo-queue/42` | `"42"` | `(submitqueue, demo-queue, request, "42")` | +| SubmitQueue batch | `demo-queue/batch/7` | `"7"` | `(submitqueue, demo-queue, batch, "7")` | +| Stovepipe request | `request/monorepo/main/42` | `"42"` | `(stovepipe, monorepo/main, request, "42")` | + +The decimal ID is unique only within its scope. The same value may appear in another queue, resource kind, or domain. APIs and messages therefore carry the queue separately; their typed field or message type supplies the resource kind. + +Do not embed scope into the ID. Forms such as `demo-queue/42`, `demo-queue/batch/7`, `request.42`, and ARN-like resource names are not stored or accepted as IDs. + +## Counter + +The counter backend persists one high-water mark per `(owner domain, queue, resource kind)`. For example: + +```text +(submitqueue, demo-queue, request) -> 42 +(submitqueue, demo-queue, batch) -> 7 +(stovepipe, demo-queue, request) -> 11 +``` + +Controllers allocate an ID before creating the resource; stores accept the caller-supplied ID and never generate one. + +- The first ID is `1`; `0` is the unset value. +- Allocation is atomic across replicas and durable across restarts. +- Allocated values are never reused. Failed writes may leave gaps. +- Overflow fails instead of wrapping. +- Numeric order is allocation order only within the same scope. + +The counter contract requires an atomic durable increment, not MySQL specifically. MySQL remains the initial implementation. + +## Storage and contracts + +Resource tables keep IDs as strings. Queue remains the leading key: + +```text +request(queue, id VARCHAR(...), ...) PRIMARY KEY (queue, id) +batch(queue, id VARCHAR(...), ...) PRIMARY KEY (queue, id) +``` + +Reference columns use the same string type. Domain entities may use distinct named string types such as `RequestID` and `BatchID`; protobuf resource fields remain `string`. The counter backend may store its high-water marks as integers, and controllers convert allocated values to canonical decimal strings before creating resources. + +This proposal applies only to counter-generated resources. Provider build IDs, message and hook IDs, change URIs, and content hashes keep their existing contracts. + +## URLs and display + +| Before — base64url or percent-encoded | After — readable | +|---|---| +| `/queues/demo-queue/requests/ZGVtby1xdWV1ZS80Mg`
or `/queues/demo-queue/requests/demo-queue%2F42` | `/queues/demo-queue/requests/42` | +| `/queues/demo-queue/batches/ZGVtby1xdWV1ZS9iYXRjaC83`
or `/queues/demo-queue/batches/demo-queue%2Fbatch%2F7` | `/queues/demo-queue/batches/7` | +| `/queues/demo-queue/changes/Z2l0aHViOi8v…`
or `/queues/demo-queue/changes/github%3A%2F%2Fgithub.com%2Fuber%2Frepo%2Fpull%2F123%2F{sha}` | `/queues/demo-queue/changes/github/github.com/uber/repo/pull/123` | +| `/queues/demo-queue/changes/cGhhYjovL3BoYWIuZXhhbXBsZS5jb20vRDEyMzQ1LzY3ODkw`
or `/queues/demo-queue/changes/phab%3A%2F%2Fphab.example.com%2FD12345%2F67890` | `/queues/demo-queue/changes/phab/phab.example.com/D12345` | + +Both columns use the same queue prefix for comparison. The before batch/change routes and percent-encoded alternatives are illustrative, not implemented pages; GitHub base64url is abbreviated, and `{sha}` stands for a full lowercase commit SHA. + +## Rejected alternatives + +- **Queue or kind prefixes:** duplicate explicit context, lengthen keys, require parsing, and introduce URL separators. +- **ARN-like names:** solve global lookup, which current APIs neither provide nor require. +- **UUIDs or a global counter:** provide global uniqueness at the cost of unnecessary encoding or coordination. +- **Integer resource fields:** couple the persisted and wire contracts to the current counter representation without adding identity semantics. +- **SQL auto-increment or `MAX(id) + 1`:** move allocation into one storage implementation or fail under concurrency. +- **Process-local counters:** reuse IDs after restart and collide across replicas.