You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
## Summary
Third of five PRs porting the session-pool infrastructure from
`feat/bigtable-sessionz-debug` into upstream. Introduces
`internal/session/` — a proto-native SessionClient + SessionTable
API sitting on top of PR-2's SessionPoolImpl.
- `internal/session/api.go` — public interfaces (`ChannelPool`,
`Config`, `SessionClient`, `SessionTableAPI`, `DebugAccess`).
- `internal/session/client.go` — `SessionClient` impl: dedicated
channel pool (no primer), `ClientConfigurationManager` wiring,
`OpenSessionTable` / `OpenAuthorizedView` / `OpenMaterializedView`
factories that mint lazily-opened per-resource pools keyed by
`{resource, permission}`.
- `internal/session/table.go` — `SessionTable` impl. Two `*lazyPool`
(read + write); MV is read-only (write pool nil, `MutateRow`
returns `ErrWriteNotSupported`). `stampAttempt` sources per-attempt
`cluster_id` / `zone_id` / peer fields from typed
`InvokeResult.ClusterInfo` and `InvokeResult.PeerInfo` per
CLIENT_SIDE_METRICS_SPEC #1.
- `internal/session/lazy_pool.go` — `Invoker` + `SessionPool`
interfaces + open-on-first-use lazy wrapper. Failed opens are NOT
cached; the next call retries.
- `internal/session/debug.go` — `DebugAccess` impl surfacing pool
snapshots for sessionz/loadz/channelz/configz.
### Transport additions to support the above
- `transport/debug_api.go` (new) — `SessionDebugProvider` /
`ChannelDebugProvider` / `ConfigDebugProvider` interfaces +
`ChannelPoolDebug` / `SessionRef` DTOs. Lives in transport (not
bigtable) so `bigtable.Client` and `internal/session.SessionClient`
can implement without an import cycle.
- `transport/diverter.go` — `sessionPicks` / `classicPicks` counters,
`DiverterSnapshot`, `Snapshot()`.
- `transport/connpool.go` — `ChannelSnapshot` +
`ChannelPoolSnapshot` type + method, `WithInstanceName` /
`WithAppProfile` options for channelz labelling.
- `transport/debug_tracer.go` — exported `DebugTag` +
`RecordDebugTag` + `TagSessionAttemptNilClusterInfo` /
`TagSessionAttemptEmptyClusterID` catalog constants.
- `transport/session_descriptors.go` — `SessionType.ProtoName()` for
human-readable pool identifiers.
- `transport/direct_access_checker.go` — renames
`newPingAndWarmDirectAccessChecker` →
`NewPingAndWarmDirectAccessChecker` (constructor exported) and
nil-guards `primer.Prime` so session-based clients can pass a nil
primer. Session clients warm channels on-demand via `OpenSession`,
not eagerly at pool-init.
### Lifecycle correctness
`sessionClient.Close()` snapshots owned resources under `poolsMu` and
releases the lock before running `Close` / `Shutdown` / `Cancel`
calls, so a snapshot method holding `poolsMu` never deadlocks
teardown. Post-Close Opens surface a distinct
`ErrSessionClientClosed` sentinel rather than misleading
`errReadPoolNil` / `ErrWriteNotSupported`.
## Stack
- **PR-1** #20224 (sessionList) — merged
- **PR-2** #20225 (SessionPoolImpl) — open; this PR stacks on it
- **PR-4** (this PR)
- PR-5 will follow with the bigtable-package integration + debugview.
The diff on this PR includes PR-2's commits until #20225 merges into
main.
## Test plan
- [x] `go build ./internal/session/ ./internal/transport/ ./...`
- [x] `go test ./internal/session/ -race -count=1 -short -timeout=180s`
- [x] `go test ./internal/transport/ -race -count=1 -short -skip
'AfeLbSim|TestHighQpsSession' -timeout=240s`
- [x] `gofmt -l ./internal/session/ ./internal/transport/` — clean
- [x] `go vet ./internal/session/ ./internal/transport/` — clean
- [x] Reviewed against the 4 behavioral specs (SESSION_SPEC,
SESSION_CLIENT_SPEC, SESSION_POOL_SPEC, CLIENT_SIDE_METRICS_SPEC)
and SESSION_COMPONENT_SPEC (boundary rules) — PASS.
-**Classic source:**`ExtractServerLatency(headers, trailers)` reads `x-goog-cbt-*` server-timing metadata; a t4t7 fallback tracker fills in when the header is absent (`internal/metrics/tracer.go:822-828`).
58
-
-**Session source:**`result.Stats.BackendLatency` — a typed `google.protobuf.Duration` on `SessionRequestStats` (part of the vRPC response frame). `stampAttempt` converts and stamps: `att.SetServerLatency(ConvertToMs(result.Stats.GetBackendLatency().AsDuration()))` (`internal/session/table.go:245-247`).
59
-
-**Nil-guard:** stamp is skipped when `Stats == nil` OR `Stats.BackendLatency == nil` — server MAY omit either (typically on error frames). The session path has NO t4t7-equivalent fallback; if a fallback becomes necessary, add it here rather than on the classic side.
58
+
-**Session source:**`stampAttempt` on the session path (`internal/session/table.go`) does NOT stamp `server_latencies` — the field is not set. `result.Stats.BackendLatency` (a typed `google.protobuf.Duration` on `SessionRequestStats`) captures server-side compute time, but it is NOT wire-comparable to classic's server-timing-header value — session-path AFE overhead is measured separately via `transport_latencies`. Populating `server_latencies` from BackendLatency would produce a wrong-shape sample that dashboards cross-comparing session/classic would misread.
59
+
-**connectivity_error_count interlock:** because session-path `serverLatency` is never stamped (default 0, `serverLatencyErr` nil), the `connectivity_error_count` classifier at `internal/metrics/tracer.go` MUST additionally check `transportType != ""` (populated by `stampAttempt` from `result.PeerInfo`). PeerInfo presence proves the `OpenSessionResponse` arrived, i.e., we reached a server; without this prong every session attempt would misclassify. See the tracer's connectivity_error block for the three-prong OR gate (PeerInfo || server-timing header || location header).
60
+
-**Classic-path side-effect (intended):** the same three-prong OR runs on classic attempts. A classic attempt with sideband PeerInfo but no server-timing header now correctly counts as "reached server" instead of "connectivity error" — the sideband is definitive proof the server responded. Any tests that pinned the old two-prong classifier need updating.
- Fully covered by **invariant #1**. Recap: `stampAttempt` reads `result.PeerInfo` (a pointer to `Session.peerInfo`, set once by `handleOpenSession`) and calls `SetTransportType`/`SetTransportRegion`/`SetTransportZone`/`SetTransportSubZone` (`internal/session/table.go:248-253`). Same PeerInfo pointer on every attempt on session S — semantic difference from classic where `grpc.Peer` can vary per attempt.
Copy file name to clipboardExpand all lines: bigtable/docs/specs/SESSION_CLIENT_SPEC.md
+8-8Lines changed: 8 additions & 8 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -16,17 +16,17 @@
16
16
17
17
### 1. One `Client` owns exactly one `SessionClient`; one `SessionClient` owns many session pools, all lazily created
18
18
19
-
**`Client` → `SessionClient` is strictly 1:1.**`bigtable.Client` holds `sessionImpl session.SessionClient` as a single field (`client.go:53`). When `EnableSessionPool` is set on `ClientConfig`, `NewClientWithConfig` calls `session.NewSessionClient(ctx, project, instance, appProfile, metricsProvider, o...)`**exactly once** during construction and stores the returned handle (`client.go:224`). There is no per-request `SessionClient`; every session-eligible operation on that `Client` fans through the same `SessionClient` instance. Standalone `NewSessionClient` consumers likewise get one `SessionClient` per logical client — the type is not intended to be pooled or shared across logical clients.
19
+
**`Client` → `SessionClient` is strictly 1:1.**`bigtable.Client` holds `sessionImpl session.Client` as a single field (`client.go:53`). When `EnableSessionPool` is set on `ClientConfig`, `NewClientWithConfig` calls `session.NewClient(ctx, project, instance, appProfile, metricsProvider, o...)`**exactly once** during construction and stores the returned handle (`client.go:224`). There is no per-request `SessionClient`; every session-eligible operation on that `Client` fans through the same `SessionClient` instance. Standalone `NewSessionClient` consumers likewise get one `SessionClient` per logical client — the type is not intended to be pooled or shared across logical clients.
20
20
21
-
**`SessionClient` → many `*SessionPoolImpl` is 1:N**, keyed by `{resource, direction}`. The `SessionClient` maintains an internal `pools map[string]*ManagedPool` guarded by `poolsMu`. Each `OpenSessionTable` / `OpenAuthorizedView` / `OpenMaterializedView` call registers up to two entries (read + write; MaterializedView is read-only, no write entry). Keys use fixed prefixes so different resource kinds cannot collide:
21
+
**`SessionClient` → many `*SessionPoolImpl` is 1:N**, keyed by `{resource, direction}`. The `SessionClient` maintains an internal `sessionPools map[string]*managedSessionPool` guarded by `sessionPoolsMu`. Each `OpenTable` / `OpenAuthorizedView` / `OpenMaterializedView` call registers up to two entries (read + write; MaterializedView is read-only, no write entry). Keys use fixed prefixes so different resource kinds cannot collide:
Two `OpenSessionTable("t")` calls dedup on this key and return handles backed by the same underlying pool pair. The dedup is per-`SessionClient`; two different `SessionClient` instances would open independent pools even for the same resource — which is why the 1:1 `Client → SessionClient` rule matters (it's what guarantees one Bigtable `Client` never fans out to duplicate session infra for the same table).
29
+
Two `OpenTable("t")` calls dedup on this key and return handles backed by the same underlying pool pair. The dedup is per-`SessionClient`; two different `SessionClient` instances would open independent pools even for the same resource — which is why the 1:1 `Client → SessionClient` rule matters (it's what guarantees one Bigtable `Client` never fans out to duplicate session infra for the same table).
30
30
31
31
**`SessionClient` does NOT cache the returned `SessionTableApi`.** The consumer does — mixed-mode `bigtable.Client` holds `sessionTables map[string]session.SessionTableApi` guarded by `sessionTablesMu` (`client.go:59-64`). Rationale: `NewSessionClient` returns fresh `SessionTableApi` handles that share the underlying pool via the `SessionClient`-level pool cache; the per-`Client` cache is only for user-facing identity (same `Open*` call returns the same handle so callers can compare by pointer). Standalone `NewSessionClient` consumers MUST implement equivalent caching if they need handle identity — the pool dedup is only inside `SessionClient`.
32
32
@@ -38,7 +38,7 @@ Two `OpenSessionTable("t")` calls dedup on this key and return handles backed by
38
38
**Consequences worth calling out.**
39
39
-**Deferred failure surface:** marshal failures on session payloads surface on first `ReadRow`/`Apply`, not at `OpenTable`. Callers that expected eager failure (matching classic Bigtable's behavior) will see it later.
40
40
-**Zero-traffic Clients open zero streams.** A `Client` that never issues a session-eligible call opens exactly zero `OpenSession` streams. Only `GetClientConfiguration` polls run from client construction time (they travel through the shared session channel pool — `SESSION_CLIENT_SPEC.md #2`).
41
-
-**Config listener registration is deferred to pool open.** Per-pool `UpdateConfig` listeners (`SESSION_CLIENT_SPEC.md #3`) only register after the pool is actually opened by `getOrCreatePool` (`internal/session/client.go:536+`). Config polls run from `SessionClient` construction — the first poll's result is stored on the `ClientConfigurationManager` and used to seed the pool at open time.
41
+
-**Config listener registration is deferred to pool open.** Per-pool `UpdateConfig` listeners (`SESSION_CLIENT_SPEC.md #3`) only register after the pool is actually opened by `getOrCreateSessionPool` (`internal/session/client.go:536+`). Config polls run from `SessionClient` construction — the first poll's result is stored on the `ClientConfigurationManager` and used to seed the pool at open time.
42
42
43
43
### 2. All Session Pools on a `SessionClient` share one channel pool
44
44
@@ -89,13 +89,13 @@ The wire-level `bigtable.v2.OpenSessionRequest` is deliberately resource-agnosti
89
89
90
90
**`Permission` is baked into the pool key** (`SESSION_CLIENT_SPEC.md #1`). For Table and AuthorizedView, `Permission ∈ {PERMISSION_READ, PERMISSION_WRITE}` is encoded into the inner proto at pool construction time. This is *the* reason read and write pools are separate `*SessionPoolImpl` instances (`SESSION_POOL_SPEC.md #1`) — the `OpenSessionRequest.Payload` bytes for the read pool encode `Permission=READ` and the write pool encodes `Permission=WRITE`, so the server issues a session scoped to the right permission. MaterializedView has no `Permission` field on its inner proto and no write pool — attempting `MutateRow` on a MatView returns `ErrWriteNotSupported` client-side without any wire traffic (`SESSION_POOL_SPEC.md #1`).
91
91
92
-
**Payload construction happens once per pool.**`createPoolForPayload` (`internal/session/client.go:481-531`) marshals the inner proto into `Payload` bytes at pool creation time and stashes the resulting `OpenSessionRequest` on the pool as its `handshake`. The same `handshake` is re-used **verbatim** for every session that pool opens over its lifetime — the payload bytes are immutable per pool. This is what keeps `OpenSession` cheap: no per-session proto encoding.
92
+
**Payload construction happens once per pool.**`createSessionPoolForPayload` (`internal/session/client.go:481-531`) marshals the inner proto into `Payload` bytes at pool creation time and stashes the resulting `OpenSessionRequest` on the pool as its `handshake`. The same `handshake` is re-used **verbatim** for every session that pool opens over its lifetime — the payload bytes are immutable per pool. This is what keeps `OpenSession` cheap: no per-session proto encoding.
93
93
94
-
**Routing metadata is derived from the same inner proto** via `SessionDescriptor.MetadataFn` (`internal/transport/session_descriptors.go:79-172`). The per-resource `MetadataFn` walks the inner proto to build `open_session.payload.*` header values, URL-escapes each, and joins them into the `x-goog-request-params` header alongside the `x-goog-request-params` resource prefix. This is what lets AFEs route the `OpenSession` stream without deserializing `Payload` — the resource identity is duplicated as headers so the transport tier can route on them directly. Headers and payload MUST agree; they are constructed from the same proto instance, so drift is not possible unless one code path bypasses `createPoolForPayload`.
94
+
**Routing metadata is derived from the same inner proto** via `SessionDescriptor.MetadataFn` (`internal/transport/session_descriptors.go:79-172`). The per-resource `MetadataFn` walks the inner proto to build `open_session.payload.*` header values, URL-escapes each, and joins them into the `x-goog-request-params` header alongside the `x-goog-request-params` resource prefix. This is what lets AFEs route the `OpenSession` stream without deserializing `Payload` — the resource identity is duplicated as headers so the transport tier can route on them directly. Headers and payload MUST agree; they are constructed from the same proto instance, so drift is not possible unless one code path bypasses `createSessionPoolForPayload`.
95
95
96
96
**`SessionType` (`session_descriptors.go:38-77`) is the compile-time discriminator** carried on `*SessionPoolImpl` and each `Session` so the internal transport tier can tag its debug output and log names — e.g., `"OpenTablePool-3 (sushanb) [READ]"` — without re-parsing the payload bytes. The enum is `{SessionTypeTable, SessionTypeAuthorizedView, SessionTypeMaterializedView}`, set when the pool is constructed from the descriptor. `SessionDescriptor.ProtoName()` returns the bare inner-proto name (`"OpenTable"` / `"OpenAuthorizedView"` / `"OpenMaterializedView"`) used to bake the human-readable pool identifier.
97
97
98
-
**A marshal failure on the inner proto is fatal to that specific pool open**, not to the SessionClient. `createPoolForPayload` returns `fmt.Errorf("proto.Marshal session payload: %w", err)`; `lazyPool.get()` surfaces it to the caller and does NOT cache the failure (`SESSION_CLIENT_SPEC.md #1`). The `SessionClient` itself, its channel pool, and its config manager are untouched — other pools remain openable.
98
+
**A marshal failure on the inner proto is fatal to that specific pool open**, not to the SessionClient. `createSessionPoolForPayload` returns `fmt.Errorf("proto.Marshal session payload: %w", err)`; `lazyPool.get()` surfaces it to the caller and does NOT cache the failure (`SESSION_CLIENT_SPEC.md #1`). The `SessionClient` itself, its channel pool, and its config manager are untouched — other pools remain openable.
99
99
100
100
**`SessionRefreshConfig.OptimizedOpenRequest`** (`session.pb.go:2764+`) is the server's mechanism for supplying a *replacement*`OpenSessionRequest` the client should use on session refresh — an AFE may pre-encode a cheaper handshake (e.g., a pre-planned query for BTQL) and hand it to the client via the refresh path. When set, it overrides the pool's cached `handshake` for the next `OpenSession` call on that specific replacement session. This does not violate the "payload bytes are immutable per pool" rule — the refresh config replaces the handshake atomically on receipt, and subsequent sessions minted by the same pool use the new bytes.
Copy file name to clipboardExpand all lines: bigtable/docs/specs/SESSION_COMPONENT_SPEC.md
+1-1Lines changed: 1 addition & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -219,7 +219,7 @@ Every rule is a MUST. Violations are bugs, not preferences.
219
219
| Per-attempt transport peer labels (classic) | grpc.Peer at attempt time | can vary across attempts within an operation |
220
220
| Per-attempt transport peer labels (session) |`InvokeResult.PeerInfo` (== `Session.peerInfo`, set once) | fixed for the session's lifetime; all attempts on session S share it (spec #16) |
221
221
| Per-attempt `client_blocking_latencies` (session path) |`InvokeResult.SentAt` (captured by `Session.Invoke` immediately before `s.Send`, `session_vrpc.go:124-127`) | stamped once by `sessionTable.stampAttempt` (`internal/session/table.go:242-243`); MUST NOT be recomputed from any other timestamp; classic path uses `blockingLatencyTracker.messageSentNanos` instead (`CLIENT_SIDE_METRICS_SPEC.md #2`) |
222
-
| Per-attempt `server_latencies` (session path) |`InvokeResult.Stats.BackendLatency`(typed proto duration on the vRPC response frame) | stamped once by `sessionTable.stampAttempt` (`internal/session/table.go:245-247`); MUST NOT be read from `x-goog-cbt-*` headers on this path; classic path uses `ExtractServerLatency` + t4t7 fallback (`CLIENT_SIDE_METRICS_SPEC.md #2`)|
222
+
| Per-attempt `server_latencies` (session path) |NOT stamped on session path — `sessionTable.stampAttempt` no longer calls `SetServerLatency`; the field is left at its zero value (`internal/session/table.go`) |`InvokeResult.Stats.BackendLatency`is deliberately unused — it's not wire-comparable to classic's server-timing value; session AFE overhead lives in `transport_latencies` separately. The tracer's emission gate at `internal/metrics/tracer.go:682` still records the zero-value sample today; a follow-up will gate emission on a `serverLatencySet` flag so the histogram truly reflects session-path absence. The `connectivity_error_count` classifier gates on `transportType != ""` (PeerInfo prong), so the zero-value serverLatency does not misclassify session attempts — see `CLIENT_SIDE_METRICS_SPEC.md #2` "server_latencies" + interlock bullets.|
223
223
| Session-tracer OTel histograms (`session.durations`, `session.open_latencies`, `session.uptime`, `transport_latencies`) |`sessionTracer` (`internal/transport/session_tracer.go:121`); registered once by `InitializeSessionMetrics` (`:67-114`) | Sole writer; `transport_latencies` recorded only from `session_pool.go:650` under the positive-delta gate at `:646-647`; `session_name` label is pool-scoped (bounded), NOT `Session.LogName` (unbounded); do NOT record from any other site or add a second registration path (`CLIENT_SIDE_METRICS_SPEC.md #3`) |
224
224
| Debug snapshot lock discipline |`SessionPoolImpl`/`Session`/etc. — snapshot methods only | MUST take at most RLock, release before returning value; z-pages hold no lock across HTTP write (spec #15) |
0 commit comments