Repository navigation
Conversation
|
@charmander @brianc I closed the previous PR to clean up a few things. I would love to get feedback on this. I got a few points worth noting/discussing: Argument sanitizationAPMs usually sanitize arguments, the redis implementation for tracing channels actually takes this into account but I don't think this is needed for SQL since parameterized queries already separate the statement from the values. This PR emits query.text but not query.values, which is the same approach taken by OTel. DocumentationI haven't yet documented any of this, would something like this work here? Happy to add a doc covering channel names, payload shapes, and usage examples once the API is agreed on. |
There was a problem hiding this comment.
Pull request overview
Adds first-class diagnostics_channel / TracingChannel instrumentation to pg and pg-pool so query/connection/pool lifecycles can be observed via subscribers instead of monkey-patching.
Changes:
- Introduces
pg:queryandpg:connectionTracingChannels and wires them intoClient#queryandClient#connect. - Introduces
pg:pool:connectTracingChannelpluspg:pool:release/pg:pool:removechannels and wires them into pool lifecycle paths. - Adds unit tests validating emitted events and context payloads (skipping unstable Node versions).
Reviewed changes
Copilot reviewed 6 out of 6 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| packages/pg/lib/diagnostics.js | Creates/query/connection tracing channels and shouldTrace() helper. |
| packages/pg/lib/client.js | Wraps connect() / query() callbacks with TracingChannel#traceCallback. |
| packages/pg/test/unit/client/diagnostics-tests.js | Adds unit tests for pg:query and pg:connection tracing. |
| packages/pg-pool/diagnostics.js | Creates pool diagnostics channels and shouldTrace() helper. |
| packages/pg-pool/index.js | Publishes pool connect/release/remove lifecycle events. |
| packages/pg-pool/test/diagnostics.js | Adds tests for pool tracing + publish-only channels. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
ce2eaaf to
f722fef
Compare
|
@brianc friendly ping. I added docs and adjusted some areas for OTel alignment. Since last change here we shipped this in GraphQL and Nitro with more libraries in the ecosystem adopting tracing channels. is there anything I can help with or clarify? |
fed4ead to
cf9ce13
Compare
cf9ce13 to
5d396e0
Compare
Publishes pg:query and pg:connection tracing channels from the client, and pg:pool:connect, pg:pool:release and pg:pool:remove from the pool, so instrumentation libraries can subscribe instead of monkey-patching. All publishing is skipped when there are no subscribers. Closes brianc#3619
5d396e0 to
de87909
Compare
diagnostics_channel tracing
|
@brianc Sorry to ping again. At the moment all those frameworks and libraries have adopted tracing channels, and Since April, these have adopted it as well:
Node.js also marked Is there a specific blocker I can help with or address? Perhaps I could split this into smaller PRs (one for the query channels and one for the pool)? |
…ch PRs 16 git cases pinned to commit SHAs: - sequelize, jsonparser, cJSON, Cachet security fixes (from their GitHub advisories), each as fix (approve), honest revert (reject), and revert with a misleading description (reject) - ReviewBench k1LoW/gh-copilot-review#9, AA-Factory/aafactory-prototype#17, brexhq/CrabTrap#15, brianc/node-postgres#3650 (reject); target findings from ReviewBench golden/ Checked when added: all repos fetch and build, every reversal exactly undoes its fix, no target_issue text appears in any prompt. Dry run: 32 calls (16 cases x 2 templates), ~439k prompt tokens, worst case ~$0.47 on GLM-4.6. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
#73) The driver now publishes diagnostics_channel events that APM tools can subscribe to without patching it. Nothing is published, and nothing is allocated, while a channel has no subscriber. TracingChannels, each following tracePromise order (start, end, error, asyncStart, asyncEnd) and settling once: - postgres:query, one per query, started in Query.handle in the caller's async context so a bindStore transform sees it. The text and statement name are filled in when the query is built for a connection. - postgres:connect, one per connection acquisition, from the socket attempt through the first ReadyForQuery. - postgres:pool:connect, one per wait for a connection: every plain query inside its query span, and every sql.reserve() and sql.begin(). - postgres:transaction, for sql.begin() and sql.savepoint(), with result.committed once known. Plain channels: postgres:pool:release when a reserved connection or transaction lets go of its connection, and postgres:pool:remove when a connection closes. The payload follows the field names of brianc/node-postgres#3650: query { text, name } and client { database, host, port, user, ssl, processID } on queries, connection {...} on connect, pool { totalCount, idleCount, waitingCount, maxSize } then client { processID, reused } on the pool span, and { client: { processID }, error } on release and remove. The pool span splits the wait. It starts when the request is routed. assign() records handoverTime, the moment a connection took the request over, and queuedBehind, how many queries that connection already had running or waiting. A query's span ends when the connection makes it its active query, so the time from handoverTime to the end is spent behind those queries, and the time before it is the wait for a connection. A request that is refused, cancelled or lost before then fails the pool span first and then itself. Not published: parameter values, in any payload; a send event; a channel for LISTEN or subscriptions. Internal queries (type fetching, session checks) are not traced. Each hook sits on a single transition of the request lifecycle: Query.handle, build, succeed/fail, assign, activate, the connection's acquisition and close, and the lease release. Types, docs/api.md, a workerd scenario, an A/B input for the Workers CPU job and tests/tracing.js cover it.
#73) The driver now publishes diagnostics_channel events that APM tools can subscribe to without patching it. Nothing is published, and nothing is allocated, while a channel has no subscriber. TracingChannels, each following tracePromise order (start, end, error, asyncStart, asyncEnd) and settling once: - postgres:query, one per query, started in Query.handle in the caller's async context so a bindStore transform sees it. The text and statement name are filled in when the query is built for a connection. - postgres:connect, one per connection acquisition, from the socket attempt through the first ReadyForQuery. - postgres:pool:connect, one per wait for a connection: every plain query inside its query span, and every sql.reserve() and sql.begin(). - postgres:transaction, for sql.begin() and sql.savepoint(), with result.committed once known. Plain channels: postgres:pool:release when a reserved connection or transaction lets go of its connection, and postgres:pool:remove when a connection closes. The payload follows the field names of brianc/node-postgres#3650: query { text, name } and client { database, host, port, user, ssl, processID } on queries, connection {...} on connect, pool { totalCount, idleCount, waitingCount, maxSize } then client { processID, reused } on the pool span, and { client: { processID }, error } on release and remove. The pool span splits the wait. It starts when the request is routed. assign() records handoverTime, the moment a connection took the request over, and queuedBehind, how many queries that connection already had running or waiting. A query's span ends when the connection makes it its active query, so the time from handoverTime to the end is spent behind those queries, and the time before it is the wait for a connection. A request that never reached a connection (refused, cancelled in the queue, connection attempt failed) fails the pool span first and then itself. Once handed over, the pool span resolves, and a later failure fails only the query span. Not published: parameter values, in any payload; a send event; a channel for LISTEN or subscriptions. Internal queries (type fetching, session checks) are not traced. Each hook sits on a single transition of the request lifecycle: Query.handle, build, succeed/fail, assign, activate, the connection's acquisition and close, and the lease release. Types, docs/api.md, a workerd scenario, an A/B input for the Workers CPU job and tests/tracing.js cover it.
#73) The driver now publishes diagnostics_channel events that APM tools can subscribe to without patching it. Nothing is published, and nothing is allocated, while a channel has no subscriber. TracingChannels, each following tracePromise order (start, end, error, asyncStart, asyncEnd) and settling once: - postgres:query, one per query, started in Query.handle in the caller's async context so a bindStore transform sees it. The text and statement name are filled in when the query is built for a connection. - postgres:connect, one per connection acquisition, from the socket attempt through the first ReadyForQuery. - postgres:pool:connect, one per wait for a connection: every plain query inside its query span, and every sql.reserve() and sql.begin(). - postgres:transaction, for sql.begin() and sql.savepoint(), with result.committed once known. Plain channels: postgres:pool:release when a reserved connection or transaction lets go of its connection, and postgres:pool:remove when a connection closes. Release carries an error only when the connection is terminated instead of returned healthy. The payload follows the field names of brianc/node-postgres#3650: query { text, name } and client { database, host, port, user, ssl, processID } on queries, connection {...} on connect, pool { totalCount, idleCount, waitingCount, maxSize } then client { processID, reused } on the pool span, and { client: { processID }, error } on release and remove. The pool span splits the wait. It starts when the request is routed. assign() records handoverTime, the moment a connection took the request over, and queuedBehind, how many queries that connection already had running or waiting. A query's span ends when the connection makes it its active query, so the time from handoverTime to the end is spent behind those queries, and the time before it is the wait for a connection. A request that never reached a connection (refused, cancelled in the queue, connection attempt failed) fails the pool span first and then itself. Once handed over, the pool span resolves, and a later failure fails only the query span. Not published: parameter values, in any payload; a send event; a channel for LISTEN or subscriptions. Internal queries (type fetching, session checks) are not traced. Each hook sits on a single transition of the request lifecycle: Query.handle, build, succeed/fail, assign, activate, the connection's acquisition and close, and the lease release. Types, docs/api.md, a workerd scenario, an A/B input for the Workers CPU job and tests/tracing.js cover it.
#73) The driver now publishes diagnostics_channel events that APM tools can subscribe to without patching it. Nothing is published, and nothing is allocated, while a channel has no subscriber. TracingChannels, each following tracePromise order (start, end, error, asyncStart, asyncEnd) and settling once: - postgres:query, one per query, started in Query.handle in the caller's async context so a bindStore transform sees it. The text and statement name are filled in when the query is built for a connection. - postgres:connect, one per connection acquisition, from the socket attempt through the first ReadyForQuery. - postgres:pool:connect, one per wait for a connection: every plain query inside its query span, and every sql.reserve() and sql.begin(). - postgres:transaction, for sql.begin() and sql.savepoint(), with result.committed once known. Plain channels: postgres:pool:release when a reserved connection or transaction lets go of its connection, and postgres:pool:remove when a connection closes. Release carries an error only when the connection is terminated instead of returned healthy. The payload follows the field names of brianc/node-postgres#3650: query { text, name } and client { database, host, port, user, ssl, processID } on queries, connection {...} on connect, pool { totalCount, idleCount, waitingCount, maxSize } then client { processID, reused } on the pool span, and { client: { processID }, error } on release and remove. The pool span splits the wait. It starts when the request is routed. assign() records handoverTime, the moment a connection took the request over, and queuedBehind, how many queries that connection already had running or waiting. A query's span ends when the connection makes it its active query, so the time from handoverTime to the end is spent behind those queries, and the time before it is the wait for a connection. A request that never reached a connection (refused, cancelled in the queue, connection attempt failed) fails the pool span first and then itself. Once handed over, the pool span resolves, and a later failure fails only the query span. Not published: parameter values, in any payload; a send event; a channel for LISTEN or subscriptions. Internal queries (type fetching, session checks) are not traced. Each hook sits on a single transition of the request lifecycle: Query.handle, build, succeed/fail, assign, activate, the connection's acquisition and close, and the lease release. Types, docs/api.md, a workerd scenario, an A/B input for the Workers CPU job and tests/tracing.js cover it.
This PR introduces tracing channels support to
pg-poolandpgquerying methods as discussed in #3619Without subscribers there is no measurable overhead, so existing users shouldn't be affected. The benchmark run keeps every scenario within its noise (-2.1% to +3.6%) and flags none of them. The overhead would come from the consumers of the diag channels published here which is done anyways via monkey patching.
I added the following channels:
pg:querypgpg:connectpgpg:pool:connectpg-poolpg:pool:releasepg-poolpg:pool:removepg-poolUsage
Instrumentations will only need to subscribe to tracing channels to create traces, logs or metrics:
This is part of a broader initiative to bring TracingChannel support to the most widely used Node.js database and cache libraries. The same pattern has already been merged and shipped in:
I have been directly involved in those prior implementations and I'm happy to own this through to release, address any feedback, and provide whatever support is needed. Would love to hear your thoughts.
Supersedes #3624.