Skip to content

feat(telemetry): add the telemetry-3 conformance handler - #916

Merged
hln33 merged 3 commits into
mainfrom
feat/conformance-telemetry-3
Nov 9, 2026
Merged

hln33 merged 3 commits into
mainfrom
feat/conformance-telemetry-3

Conversation

@hln33

@hln33 hln33 commented Nov 9, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Adds the Python handler for conformance requirement telemetry-3 (per-operation result selection). The requirement is defined in aws/aws-durable-execution-conformance-tests#183 and was raised in aws/aws-durable-execution-conformance-tests#182.

Step fetch-user returns {userId, email, token}. An OperationOverride(operation_name="fetch-user", result=select_user_id) returns {userId} alone. content.input=False and content.output=False omit the execution input and output from the record. So the exported record has no input, no output, and operationsByName.fetch-user.result == {"userId": "u-123"}. The execution result still holds all three keys, which the suite's core ExpectedResult check verifies.

Design and alternatives

Problem: Verify that telemetry exports only userId from a step result while the execution still returns userId, email, and token.

Coverage gap: The existing telemetry scenarios do not exercise per-operation result selection.

Alternatives evaluated during review:

  • Add no handler. This leaves the result-selection behavior untested by the deployed conformance suite.
  • Reuse the existing success handler unchanged. It exercises an increment step and includes execution input and output. It cannot produce the required result-selection record.
  • Extend that handler to serve both scenarios. The scenarios require different step results and content settings. Combining them adds scenario selection to the handler and configuration.
  • Return only userId from the step. This changes the execution result. It also avoids testing whether the telemetry transform selects the field correctly.

Chosen approach: Keep a dedicated handler with one step and one result transform. Disable execution input and output in its telemetry configuration. Return the complete step result unchanged. This preserves the existing scenarios without adding scenario-selection logic.

Removals: None. This adds conformance coverage using the existing SDK API.

Please confirm this approach before further implementation changes.

Changes

  • src/telemetry_3_result_select.py: new handler.
  • template.yaml: Telemetry3ResultSelect → TestDescription: [telemetry-3], function name ${AWS::StackName}-telemetry-3.
  • src/Makefile: build-Telemetry3ResultSelect target.
  • tests/test_telemetry_examples.py: module/function sets include telemetry-3; new local-runner test asserts the one-key record result and the absent input/output.
  • README.md: coverage and layout rows.
  • .github/workflows/telemetry-conformance-tests.yml: pinned to the fix: Change default key 'name' for logger extra field #183 merge commit on main (cf13ef2).

Verification

  • hatch run test:all packages/aws-durable-execution-sdk-python-conformance-tests-telemetry/tests: 20 passed.
  • hatch fmt --check on the package: clean.
  • hatch run types:check: clean.
  • Telemetry conformance e2e: telemetry-1, telemetry-2, and telemetry-3 passed. 3 passed, 0 failed. Run 42691679335 at 3efaa4be. Re-run after the re-pin: Run 42713583152 at c1943e80, pinned to cf13ef27, 3 passed, 0 failed.

hln33 added 2 commits November 9, 2026 13:18
Requirement telemetry-3 checks that an OperationOverride.result transform
exports only the keys it returns. The new handler runs step fetch-user,
which returns {userId, email, token}. The override returns {userId} alone.
The handler also sets content.input=False and content.output=False, so the
record carries no input and no output key.

The template maps Telemetry3ResultSelect to telemetry-3. The Makefile gains
its build target. The package test runs the handler through
DurableFunctionTestRunner with a fake logs client and asserts the record's
operationsByName.fetch-user.result is exactly {"userId": "u-123"}.
The telemetry-3 requirement lives in aws-durable-execution-conformance-tests
PR #183. The shared workflow at the old pin does not know the test, so the
new function would be reported UNCOVERED. Pin to the PR head so CI validates
telemetry-3 now. Re-pin to the #183 merge commit before leaving draft.
@hln33
hln33 force-pushed the feat/conformance-telemetry-3 branch from 8ddc1a1 to 3efaa4b Compare November 9, 2026 20:18
@hln33
hln33 marked this pull request as ready for review November 9, 2026 20:18
@hln33
hln33 deployed to ai-pr-review-runtime November 9, 2026 20:18 — with GitHub Actions Active
@hln33
hln33 had a problem deploying to ai-pr-review-runtime November 9, 2026 20:19 — with GitHub Actions Error
@hln33
hln33 deployed to ai-pr-review-runtime November 9, 2026 20:19 — with GitHub Actions Active
@github-actions

This comment has been minimized.

@hln33

hln33 commented Nov 9, 2026

Copy link
Copy Markdown
Contributor Author

/ai review

@github-actions

This comment has been minimized.

@ParidelPooya ParidelPooya left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. The handler matches telemetry-3. _apply_result_override decodes the checkpoint before it calls the transform, so result["userId"] is right. CI ran telemetry-3 against cb3753e and it passed.

A few small things:

  1. The PR body ("Draft until") and the workflow comment at telemetry-conformance-tests.yml:46 say "before leaving draft", but the PR is not a draft. Suggest rewording it to "before this PR merges".
  2. README.md:23: the new row is 4 characters shorter than the other rows. The layout comment on line 31 starts at column 35, while the other lines use column 33.
  3. Not from this PR: mypy reports Too few arguments [call-arg] on durable_execution(plugins=...) at telemetry_3_result_select.py:66. The sibling handlers on main have the same error. types:check does not cover this package, so CI does not catch it. A separate change could add @overloads to durable_execution.

Conformance PR #183 merged as cf13ef27. The workflow now pins that commit
and the TODO to re-pin is removed.

README: the telemetry-3 coverage row is padded to the table width, and the
layout comments share one column.
@hln33
hln33 deployed to ai-pr-review-runtime November 9, 2026 22:38 — with GitHub Actions Active
@github-actions

github-actions Bot commented Nov 9, 2026

Copy link
Copy Markdown
Contributor

Codex AI review

No actionable findings across the six changed files and contribution rules 1–9. The updated design note resolves the prior finding. Residual risk: tests were not run, and the newly pinned external workflow was not independently inspected.

Reviewed commit c1943e806da15b0efd6ae767709981599399ece2. Workflow run

@hln33

hln33 commented Nov 9, 2026

Copy link
Copy Markdown
Contributor Author

Thanks. Addressed in c1943e8:

  1. The workflow pin now points at the fix: Change default key 'name' for logger extra field #183 merge commit cf13ef27 and the TODO is removed. The PR body no longer has a "Draft until" section.
  2. README.md:23: the telemetry-3 row is padded to the table width. The layout block's comments all start at column 32, since telemetry_3_result_select.py is longer than the column the other lines used.
  3. The mypy call-arg error is tracked in [Bug]: durable_execution(plugins=...) return type fails mypy with "Too few arguments" #921. The declared return type is the two-argument handler in both call forms, so the decorator-factory form fails; two @overloads fix it.

Telemetry e2e on the re-pinned head: run 42713583152, telemetry-1, -2, -3 passed.

@ParidelPooya ParidelPooya left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks. The README and the pin comment are fixed. The suite is pinned to the #183 merge commit cf13ef2, and telemetry-3 passes on it. The mypy point is from main, so it can go in a separate change. Approving.

@hln33
hln33 merged commit f7492a1 into main Nov 9, 2026
23 checks passed
@hln33
hln33 deleted the feat/conformance-telemetry-3 branch November 9, 2026 23:10

This branch was successfully deployed

1 active deployment
ai-pr-review-runtime — c1943e80 Deployed Nov 9, 2026 by hln33 via ai-pr-review / Codex review / Generate Codex review #3003
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants