Repository navigation
feat(telemetry): add the telemetry-3 conformance handler - #916
Conversation
Requirement telemetry-3 checks that an OperationOverride.result transform
exports only the keys it returns. The new handler runs step fetch-user,
which returns {userId, email, token}. The override returns {userId} alone.
The handler also sets content.input=False and content.output=False, so the
record carries no input and no output key.
The template maps Telemetry3ResultSelect to telemetry-3. The Makefile gains
its build target. The package test runs the handler through
DurableFunctionTestRunner with a fake logs client and asserts the record's
operationsByName.fetch-user.result is exactly {"userId": "u-123"}.
8ddc1a1 to
3efaa4b
Compare
This comment has been minimized.
This comment has been minimized.
|
/ai review |
This comment has been minimized.
This comment has been minimized.
ParidelPooya
left a comment
There was a problem hiding this comment.
Looks good. The handler matches telemetry-3. _apply_result_override decodes the checkpoint before it calls the transform, so result["userId"] is right. CI ran telemetry-3 against cb3753e and it passed.
A few small things:
- The PR body ("Draft until") and the workflow comment at
telemetry-conformance-tests.yml:46say "before leaving draft", but the PR is not a draft. Suggest rewording it to "before this PR merges". README.md:23: the new row is 4 characters shorter than the other rows. The layout comment on line 31 starts at column 35, while the other lines use column 33.- Not from this PR: mypy reports
Too few arguments [call-arg]ondurable_execution(plugins=...)attelemetry_3_result_select.py:66. The sibling handlers on main have the same error.types:checkdoes not cover this package, so CI does not catch it. A separate change could add@overloads todurable_execution.
Conformance PR #183 merged as cf13ef27. The workflow now pins that commit and the TODO to re-pin is removed. README: the telemetry-3 coverage row is padded to the table width, and the layout comments share one column.
Codex AI reviewNo actionable findings across the six changed files and contribution rules 1–9. The updated design note resolves the prior finding. Residual risk: tests were not run, and the newly pinned external workflow was not independently inspected. Reviewed commit |
|
Thanks. Addressed in c1943e8:
Telemetry e2e on the re-pinned head: run 42713583152, telemetry-1, -2, -3 passed. |
ParidelPooya
left a comment
There was a problem hiding this comment.
Thanks. The README and the pin comment are fixed. The suite is pinned to the #183 merge commit cf13ef2, and telemetry-3 passes on it. The mypy point is from main, so it can go in a separate change. Approving.
Summary
Adds the Python handler for conformance requirement telemetry-3 (per-operation result selection). The requirement is defined in aws/aws-durable-execution-conformance-tests#183 and was raised in aws/aws-durable-execution-conformance-tests#182.
Step
fetch-userreturns{userId, email, token}. AnOperationOverride(operation_name="fetch-user", result=select_user_id)returns{userId}alone.content.input=Falseandcontent.output=Falseomit the execution input and output from the record. So the exported record has noinput, nooutput, andoperationsByName.fetch-user.result == {"userId": "u-123"}. The execution result still holds all three keys, which the suite's coreExpectedResultcheck verifies.Design and alternatives
Problem: Verify that telemetry exports only
userIdfrom a step result while the execution still returnsuserId,email, andtoken.Coverage gap: The existing telemetry scenarios do not exercise per-operation result selection.
Alternatives evaluated during review:
userIdfrom the step. This changes the execution result. It also avoids testing whether the telemetry transform selects the field correctly.Chosen approach: Keep a dedicated handler with one step and one result transform. Disable execution input and output in its telemetry configuration. Return the complete step result unchanged. This preserves the existing scenarios without adding scenario-selection logic.
Removals: None. This adds conformance coverage using the existing SDK API.
Please confirm this approach before further implementation changes.
Changes
src/telemetry_3_result_select.py: new handler.template.yaml:Telemetry3ResultSelect→TestDescription: [telemetry-3], function name${AWS::StackName}-telemetry-3.src/Makefile:build-Telemetry3ResultSelecttarget.tests/test_telemetry_examples.py: module/function sets include telemetry-3; new local-runner test asserts the one-key record result and the absentinput/output.README.md: coverage and layout rows..github/workflows/telemetry-conformance-tests.yml: pinned to the fix: Change default key 'name' for logger extra field #183 merge commit on main (cf13ef2).Verification
hatch run test:all packages/aws-durable-execution-sdk-python-conformance-tests-telemetry/tests: 20 passed.hatch fmt --checkon the package: clean.hatch run types:check: clean.3efaa4be. Re-run after the re-pin: Run 42713583152 atc1943e80, pinned tocf13ef27, 3 passed, 0 failed.