Skip to content

fix(bigtable): session.durations / session.uptime — set explicit histogram bucket boundaries - #20276

Merged
sushanb merged 3 commits into
googleapis:mainfrom
sushanb:feat/bigtable-session-metric-buckets
Jul 31, 2026
Merged

fix(bigtable): session.durations / session.uptime — set explicit histogram bucket boundaries#20276
sushanb merged 3 commits into
googleapis:mainfrom
sushanb:feat/bigtable-session-metric-buckets

Conversation

@sushanb

@sushanb sushanb commented Jul 31, 2026

Copy link
Copy Markdown
Contributor

Summary

The OTel SDK default explicit-bucket boundaries cap at 10s
({0, 5, 10, 25, 50, ..., 5000, 7500, 10000}). Session lifetimes and
per-session uptimes are minutes-to-hours (an idle session lives until
server-driven scale-down, typically 4+ minutes). Every sample past 10s
lands in the last bucket, so p50/p95/p99 all render as 10000ms on
Cloud Monitoring dashboards — the metric silently pins.

Same bug applied to session.durations, session.uptime, and
session.open_latencies — all three histograms were registered
without explicit bucket boundaries.

Fix

Register both hour-scale histograms with explicit boundaries:
{0} + geometric doubling from 1ms to ~17.5min. Matches the
cross-language client convention for session-scale buckets.

Introduce SessionLifetimeBounds next to the existing
FineGrainLatencyBounds so future additions have a clear pick between
"sub-ms latency" (FineGrain) and "session-scale lifetime"
(SessionLifetime) sets.

session.open_latencies gets FineGrainLatencyBounds here too — same
clip-at-10s hazard, smaller visible impact since pre-open latency past
10s is uncommon. Included for consistency with the other three session
histograms and with transport_latencies (which already had bounds
set).

Verification

  • go build ./... + go vet ./internal/transport/ clean.
  • Confirmed against a live sandbox before the fix: sessionz "Session lifetimes" showed n=74 · p50=4m21s · p95=4m50s · p99=4m54s (correct, via internal HDR histogram) while the OTel session.durations metric reported p50/p95/p99 = 10000ms for the same window. After the fix, the OTel metric should display the true percentiles.

Test plan

  • Verify OTel session.durations p50/p95/p99 match sessionz "Session lifetimes" on the same sandbox run
  • session.uptime — same check against sessionz's uptime samples
  • Confirm existing transport_latencies unaffected (already had explicit bounds; this change doesn't touch that call site)

…et boundaries

The OTel SDK default explicit-bucket boundaries cap at 10s
{0, 5, 10, 25, 50, ..., 5000, 7500, 10000}. Session lifetimes and
per-session uptimes are minutes-to-hours (an idle session lives
until server-driven scale-down, typically 4+ minutes). Every sample
past 10s lands in the last bucket, so p50/p95/p99 all render as
"10000ms" on dashboards — which is what you see today when
comparing the OTel session.durations metric against sessionz's
"Session lifetimes" (n=74, p50=4m21s, p95=4m50s, p99=4m54s):
the sessionz HDR histogram gets it right, the OTel metric flattens
everything to 10s.

Fix: register both histograms with explicit boundaries that match
java-bigtable's ClientSessionDuration / ClientSessionUptime
BUCKETS_MS — {0} + geometric doubling from 1ms to ~17.5min. Same
constant used for both, matching Java's shared BUCKETS_MS layout.
Introduces SessionLifetimeBounds next to the existing
FineGrainLatencyBounds so future metric additions have a clear
pick between "sub-ms latency" and "session-scale lifetime" bucket
sets.

session.open_latencies also gets explicit boundaries in this
change (FineGrainLatencyBounds) — same clip-at-10s bug technically
applies, but pre-open latency past 10s is rare so the visible
impact was smaller. Setting it here for consistency with the other
three session metrics and with transport_latencies.

No behaviour change for any other subsystem — bucket selection is
per-histogram at registration time.
@sushanb
sushanb requested review from a team as code owners July 31, 2026 03:57
@product-auto-label product-auto-label Bot added the api: bigtable Issues related to the Bigtable API. label Jul 31, 2026

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request configures explicit bucket boundaries for OpenTelemetry metrics in the Bigtable session tracer, applying custom boundaries to session durations, uptime, and open latencies. The review feedback suggests unexporting the newly introduced SessionLifetimeBounds variable to sessionLifetimeBounds to improve encapsulation, as it is only used internally within this package.

Comment on lines +46 to +58
// SessionLifetimeBounds matches java-bigtable's ClientSessionDuration
// / ClientSessionUptime BUCKETS_MS: {0} + geometric doubling from
// 1ms to ~17.5min. Sessions live minutes-to-hours; the OTel SDK
// default explicit-bucket boundaries cap at 10s, so p50/p95/p99 all
// clip to "10000ms" on the dashboard when session lifetimes exceed
// 10s. Shared by session.durations and session.uptime.
var SessionLifetimeBounds = func() []float64 {
b := []float64{0}
for v := float64(1); v <= 1_200_000; v *= 2 {
b = append(b, v)
}
return b
}()

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Since SessionLifetimeBounds is only used within this package (and specifically within this file), it should be unexported (i.e., renamed to sessionLifetimeBounds) to prevent external packages from accessing or accidentally mutating this global slice. This improves encapsulation and avoids potential side effects.

Suggested change
// SessionLifetimeBounds matches java-bigtable's ClientSessionDuration
// / ClientSessionUptime BUCKETS_MS: {0} + geometric doubling from
// 1ms to ~17.5min. Sessions live minutes-to-hours; the OTel SDK
// default explicit-bucket boundaries cap at 10s, so p50/p95/p99 all
// clip to "10000ms" on the dashboard when session lifetimes exceed
// 10s. Shared by session.durations and session.uptime.
var SessionLifetimeBounds = func() []float64 {
b := []float64{0}
for v := float64(1); v <= 1_200_000; v *= 2 {
b = append(b, v)
}
return b
}()
// sessionLifetimeBounds matches java-bigtable's ClientSessionDuration
// / ClientSessionUptime BUCKETS_MS: {0} + geometric doubling from
// 1ms to ~17.5min. Sessions live minutes-to-hours; the OTel SDK
// default explicit-bucket boundaries cap at 10s, so p50/p95/p99 all
// clip to "10000ms" on the dashboard when session lifetimes exceed
// 10s. Shared by session.durations and session.uptime.
var sessionLifetimeBounds = func() []float64 {
b := []float64{0}
for v := float64(1); v <= 1_200_000; v *= 2 {
b = append(b, v)
}
return b
}()

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in b895f044e6 — renamed to sessionLifetimeBounds. Confirmed grep of bigtable/ returns only session_tracer.go itself.

"session.durations",
metric.WithDescription("Duration a session was alive (startTime → close)"),
metric.WithUnit("ms"),
metric.WithExplicitBucketBoundaries(SessionLifetimeBounds...),

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Update the reference to use the unexported sessionLifetimeBounds variable.

Suggested change
metric.WithExplicitBucketBoundaries(SessionLifetimeBounds...),
metric.WithExplicitBucketBoundaries(sessionLifetimeBounds...),

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in b895f044e6 — renamed to sessionLifetimeBounds. Confirmed grep of bigtable/ returns only session_tracer.go itself.

"session.uptime",
metric.WithDescription("Age of currently-active sessions, sampled periodically"),
metric.WithUnit("ms"),
metric.WithExplicitBucketBoundaries(SessionLifetimeBounds...),

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Update the reference to use the unexported sessionLifetimeBounds variable.

Suggested change
metric.WithExplicitBucketBoundaries(SessionLifetimeBounds...),
metric.WithExplicitBucketBoundaries(sessionLifetimeBounds...),

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in b895f044e6 — renamed to sessionLifetimeBounds. Confirmed grep of bigtable/ returns only session_tracer.go itself.

Addresses gemini-code-assist review on PR googleapis#20276. The variable is only
consumed within session_tracer.go (session.durations + session.uptime
histogram registration). No external caller reaches for it, so keeping
it exported broadens the internal/transport API surface without a
reader.

Mechanical rename SessionLifetimeBounds → sessionLifetimeBounds across
all four occurrences in session_tracer.go. Build + vet clean; no other
package touches the symbol.
@sushanb

sushanb commented Jul 31, 2026

Copy link
Copy Markdown
Contributor Author

Addressed in b895f044e6SessionLifetimeBoundssessionLifetimeBounds. Confirmed no external callers reach for it (grep of bigtable/ for the exported name returns only session_tracer.go itself).

Note on the failing bigtable_conformance (1.26, bigtable) check: TestReadRow_NoRetry_CommitInSeparateChunk is unrelated to this PR — the change touches only session_tracer.go histogram bucket boundaries, not any ReadRow path. The failure is a data-comparison mismatch in the conformance test suite on Go 1.26 (passes on 1.25) and matches known flake patterns for the pinned conformance-test tag.

@sushanb
sushanb merged commit 97eee22 into googleapis:main Jul 31, 2026
19 checks passed
sushanb pushed a commit that referenced this pull request Aug 3, 2026
🤖 I have created a release *beep* *boop*
---


##
[1.52.0](bigtable/v1.51.0...bigtable/v1.52.0)
(2026-08-03)


### Features

* **bigtable:** Add AFE picker (Simple / LeastInFlight / LeastLatency)
([#20204](#20204))
([bcbf714](bcbf714))
* **bigtable:** Add ClientConfig.DisableSession to opt out of session
backend
([#20297](#20297))
([7ee5e44](7ee5e44))
* **bigtable:** Add getClientConfigDirectAccessChecker for session pools
([#20209](#20209))
([3b8d30a](3b8d30a))
* **bigtable:** Add NoOpChannelPrimer for session channel pools
([#20208](#20208))
([d055a8a](d055a8a))
* **bigtable:** Add per-AFE sessionList for the two-tier session pool
([#20224](#20224))
([dbf0c3f](dbf0c3f))
* **bigtable:** Add protoRowToRow conversion helper for TableShim
([#20257](#20257))
([1297143](1297143))
* **bigtable:** Add Session debug surface (observability fields +
methods)
([#20211](#20211))
([d8d3e16](d8d3e16))
* **bigtable:** Add Session lifecycle (Start, Close, ForceClose,
readLoop, heartBeatLoop)
([#20215](#20215))
([b9e53c6](b9e53c6))
* **bigtable:** Add Session struct + state machine
([#20117](#20117))
([09acbb3](09acbb3))
* **bigtable:** Add session.Config.EnableDebug to gate sessionz debug
state
([#20247](#20247))
([ce74c31](ce74c31))
* **bigtable:** Add SessionClient + SessionTable + lazyPool
([#20228](#20228))
([ab2c96c](ab2c96c))
* **bigtable:** Add SessionPoolImpl (two-tier pool + scaling + debug)
([#20225](#20225))
([683eda8](683eda8))
* **bigtable:** Rename session pool display to
&lt;resource-id&gt;-&lt;PERM&gt;
([#20248](#20248))
([35e146e](35e146e))
* **bigtable:** Route Client.Open()-returned *Table through the Diverter
([#20273](#20273))
([2b81c7d](2b81c7d))
* **bigtable:** State-based classification for abnormal session close
([#20243](#20243))
([f2905b7](f2905b7))
* **bigtable:** TableShim fallback to classic on session UNIMPLEMENTED
([#20269](#20269))
([36540af](36540af))
* **bigtable:** TTL-on-idle cache for per-resource session.TableAPI
([#20263](#20263))
([00b2a49](00b2a49))
* **bigtable:** Wire Diverter on Client and route Open* via TableShim
([#20256](#20256))
([b32fbd7](b32fbd7))


### Bug Fixes

* **bigtable:** AFE picker latency signal — subtract poolWait and
compute TransportLatency = wire − backend at source
([#20281](#20281))
([bb8c4d5](bb8c4d5))
* **bigtable:** Guard NewStream OnFinish against grpc-go double-fire
([#20295](#20295))
([b51da29](b51da29))
* **bigtable:** Real per-resource pool teardown on sessionTable.Close +
cache close-race gate
([#20264](#20264))
([599aea9](599aea9))
* **bigtable:** Session.durations / session.uptime — set explicit
histogram bucket boundaries
([#20276](#20276))
([97eee22](97eee22))
* **bigtable:** SessionTableHandle self-heals across cache eviction
([#20296](#20296))
([0dd98cd](0dd98cd))
* **bigtable:** Translate ctx errors to gRPC status on session vRPC
([#20299](#20299))
([0f3b2a5](0f3b2a5))
* **bigtable:** Treat PingAndWarm NotFound as a successful prime
([#20219](#20219))
([a1557ad](a1557ad))


### Performance Improvements

* **bigtable:** Delete periodic Tick loop; sizing is event-driven
([#20285](#20285))
([2c096bd](2c096bd))
* **bigtable:** Drop pick_lost_race debug tag from CheckoutSession hot
path
([#20280](#20280))
([bd0e400](bd0e400))

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).

Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

api: bigtable Issues related to the Bigtable API.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants