Skip to content

Share Python extension localization tooling and submission action - #125

Merged
Bill Schnurr (bschnurr) merged 1 commit into
mainfrom
users/bschnurr/shared-localization
Nov 3, 2026
Merged

Bill Schnurr (bschnurr) merged 1 commit into
mainfrom
users/bschnurr/shared-localization

Conversation

@bschnurr

@bschnurr Bill Schnurr (bschnurr) commented Oct 30, 2026 •

Copy link
Copy Markdown
Member

Summary

  • Centralize English export/submission and staged release imports in one standard-library Python CLI.
  • Expose a pinned composite action for thin public extension workflows; Pylance-Eng invokes the same importer from an explicit tooling checkout.
  • Derive canonical IDs and optional runtime metadata. Support manifest-only extensions generically; the autopep8 caller now explicitly enables runtime localization as requested.
  • Require translation evidence independently in every required real-language manifest/runtime table; allow individual untranslated entries and validate everything before placement.

Verification

  • 29 dependency-free regression tests, including actual bare-Git dev-only submission/no-op fixtures, failure cases, and bounded push retry.
  • Black, isort, flake8; Python 3.10 syntax and executable guards reading the actual action/Azure YAML.
  • Executed export/submission commands read from the composite action against a real bare remote.
  • Actual pinned-CLI English exports succeeded for all eight extensions (debugger 75 runtime strings, environments 456; autopep8 now extracts 3 runtime strings).

Dependencies and release blockers

This is the shared dependency for Pylance-Eng PR 801431, vscode-isort PR 777, and seven sibling migrations. Consumers pin the same reviewed tooling/action revision; no runtime library adoption is required.

Keep the rollout draft until credentials/onboarding and handbacks are ready. Current drop validation rejects English-only isort, missing environments output, and missing runtime tables for flake8/mypy/black. Debugger, pylint, and autopep8 pass current manifest/runtime handback readiness checks. Autopep8 now enables its three existing runtime code strings. Do not merge or publish as part of this change.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@bschnurr

Copy link
Copy Markdown
Member Author

Live non-publishing release verification is complete for the three extensions whose current handbacks are ready:

All three exact-feature builds succeeded with publishing explicitly disabled and the publication stage skipped. Four actual VSIXs were inspected: universal artifacts for all three and debugger win32-x64. Each contains all 28 localization files and genuine German manifest/runtime translations. Debugger's full eight-platform build succeeded; autopep8's new runtime l10n wiring is verified.

Release blockers remain for isort (English-only handbacks), environments (missing canonical handback; native definitions disabled), and flake8/mypy/black (missing runtime tables). Seven siblings' GitHub localization credential access still needs owner confirmation/setup. Autopep8 also has the documented pre-existing Python 3.10 hashed-requirements CI failure.

Merge order: common-LSP #125, Pylance-Eng #801431, then update all eight consumer PRs to merged refs and merge only after their prerequisites are ready. No merges or extension publication were performed.

@bschnurr
Bill Schnurr (bschnurr) marked this pull request as ready for review October 30, 2026 18:03
@rchiodo

Rich Chiodo (rchiodo) commented Oct 30, 2026 •

Copy link
Copy Markdown
Contributor

🔒 Automated review in progress — Rich Chiodo (@rchiodo) is auto-reviewing this PR.

@rchiodo

Copy link
Copy Markdown
Contributor

Result: ⚠️ partially-verified

Verification details

Verification: Automated verification is best effort. The PR author remains responsible for testing.

Summary: All 29 localization regression tests passed in the container, including real bare-Git submission and no-op fixtures. Flake8, Black, and isort passed; three additional checks passed for genuine competing pushes, retry limits, and action/Python 3.10 syntax guards. No PR-caused failure was observed. Verification is partial because the pinned localization CLI was unavailable in the offline cache, preventing real extraction/import verification and end-to-end action execution. **Reviewer dependency follow-up (nonblocking):** Suggest a separate PR adding `@vscode/l10n-dev@0.0.35` to the official reviewer dependency declarations, using `scripts/localization.py` as the dependency/version evidence. Do not automatically create that PR. This is reviewer environment maintenance, not a required change to the reviewed PR.

Test runs: 4 passed, 1 not run

  • ⚠️ Not run | Pinned localization CLI offline preparation | npm exec --offline --yes --package=@vscode/l10n-dev@0.0.35 -- l10n-dev --help
  • ✅ Passed | Localization regression suite | python -m unittest discover -s scripts -p '*_test.py' -v
  • ✅ Passed | Localization lint and formatting checks | python -m flake8 scripts/localization.py scripts/localization_test.py && python -m black --check scripts/localization.py scripts/localization_test.py && python -m isort --check scripts/localization.py scripts/localization_test.py
  • ✅ Passed | Dependency and test discovery | printf 'Sandbox profile: %s\n' "$AUTOMATION_SANDBOX_PROFILE"; python --version; git --version; node --version; npm --version; git status --short; git log -1 --oneline; python - <<'PY'
    import importlib.util
    import pathlib
    for name in ('unittest', 'black', 'isort', 'flake8', 'yaml'):
    print(f'{name}: {importlib.util.find_spec(name) is not None}')
    print('Declared Python environment:')
    print(pathlib.Path('python/pyproject.toml').read_text())
    print('Localization test methods:')
    import ast
    tree = ast.parse(pathlib.Path('scripts/localization_test.py').read_text())
    for cls in tree.body:
    if isinstance(cls, ast.ClassDef):
    for method in cls.body:
    if isinstance(method, ast.FunctionDef) and method.name.startswith('test_'):
    print(f'{cls.name}.{method.name}')
    PY
    npm cache ls '@vscode/l10n-dev'
  • ✅ Passed | Additional concurrency, retry, and action guard verification | python - <<'PY'
    import ast
    import os
    import pathlib
    import subprocess
    import sys
    import unittest
    from unittest import mock
    import yaml
    sys.path.insert(0, str(pathlib.Path('scripts').resolve()))
    import localization
    from localization_test import SubmissionTests

class AdditionalVerification(SubmissionTests):
def test_real_competing_push(self):
competitor = self.base / 'competitor'
subprocess.run(['git', 'clone', '--quiet', '--branch', 'dev', str(self.remote), str(competitor)], check=True)
localization.git(competitor, 'config', 'user.name', 'Competitor')
localization.git(competitor, 'config', 'user.email', 'competitor@example.invalid')
original_run = subprocess.run
raced = False
rejected = []
def concurrent_push(command, **kwargs):
nonlocal raced
if command[:3] == ['git', 'push', 'origin'] and kwargs.get('cwd') == self.drop and not raced:
raced = True
(competitor / 'another-extension.xlf').write_text('<xliff/>\n')
localization.git(competitor, 'add', 'another-extension.xlf')
localization.git(competitor, 'commit', '-m', 'Concurrent extension update')
localization.git(competitor, 'push', 'origin', 'dev')
result = original_run(command, **kwargs)
rejected.append(result)
return result
return original_run(command, **kwargs)
with mock.patch.object(localization.subprocess, 'run', side_effect=concurrent_push):
self.submit()
self.assertTrue(raced)
self.assertEqual(len(rejected), 1)
self.assertNotEqual(rejected[0].returncode, 0)
self.assertIn('[rejected]', rejected[0].stderr)
self.assertEqual(self.git('rev-parse', 'origin/dev'), self.git('rev-parse', 'HEAD'))
self.assertTrue((self.drop / 'another-extension.xlf').exists())
self.assertTrue((self.drop / 'src/en/ms-python.tool/ms-python.tool-new.xlf').exists())
self.assertEqual(self.git('rev-parse', 'origin/main'), self.main)

def test_retry_limit_and_non_rejection_failure(self):
    original_run = subprocess.run
    for reason, expected in [(&#x27;! [rejected] HEAD -&gt; dev (fetch first)&#x27;, 3), (&#x27;fatal: authentication failed&#x27;, 1)]:
        with self.subTest(reason=reason):
            attempts = []
            def failed_push(command, **kwargs):
                if command[:3] == [&#x27;git&#x27;, &#x27;push&#x27;, &#x27;origin&#x27;]:
                    attempts.append(command)
                    return subprocess.CompletedProcess(command, 1, &#x27;&#x27;, reason)
                return original_run(command, **kwargs)
            (self.output / &#x27;ms-python.tool-new.xlf&#x27;).write_text(f&#x27;&lt;xliff&gt;{expected}&lt;/xliff&gt;\n&#x27;)
            with mock.patch.object(localization.subprocess, &#x27;run&#x27;, side_effect=failed_push):
                with self.assertRaises(subprocess.CalledProcessError):
                    self.submit()
            self.assertEqual(len(attempts), expected)
            self.git(&#x27;reset&#x27;, &#x27;--hard&#x27;, &#x27;origin/dev&#x27;)

class WiringVerification(unittest.TestCase):
def test_python_310_and_action_guards(self):
for path in ('scripts/localization.py', 'scripts/localization_test.py'):
ast.parse(pathlib.Path(path).read_text(), feature_version=(3, 10))
path = pathlib.Path('.github/actions/localization/action.yml').resolve()
action = yaml.safe_load(path.read_text())
self.assertEqual(action['runs']['using'], 'composite')
self.assertTrue((path.parent / '../../../scripts/localization.py').resolve().is_file())
steps = {step['name']: step for step in action['runs']['steps']}
validation = steps['Validate localization credentials']['run']
for app_id, key in [('', ''), ('123', ''), ('', 'secret')]:
with mock.patch.dict(os.environ, {'LOCALIZATION_APP_ID': app_id, 'LOCALIZATION_PRIVATE_KEY': key}):
with self.assertRaises(ValueError):
exec(validation, {})
with mock.patch.dict(os.environ, {'LOCALIZATION_APP_ID': '123', 'LOCALIZATION_PRIVATE_KEY': 'secret'}):
exec(validation, {})
for name, operation in [('Extract English strings', 'export'), ('Submit changed English XLF', 'submit')]:
command = steps[name]['run']
self.assertIn(f'--operation {operation}', command)
self.assertIn('"$LOCALIZATION_ACTION_PATH/../../../scripts/localization.py"', command)
self.assertNotIn('${{', command)
print('Python 3.10 syntax, action helper path, credential guards, and command wiring validated.')

suite = unittest.TestSuite([
AdditionalVerification('test_real_competing_push'),
AdditionalVerification('test_retry_limit_and_non_rejection_failure'),
WiringVerification('test_python_310_and_action_guards'),
])
result = unittest.TextTestRunner(verbosity=2).run(suite)
sys.exit(not result.wasSuccessful())
PY

⚠️ Pinned localization CLI offline preparation diagnostic output
Selected Node v24.17.0/npm 11.13.0; Python checks used python 3.13.18. scripts/localization.py:15 declares CLI = "@vscode/l10n-dev@0.0.35". Offline preparation exited 1:
npm error code ENOTCACHED
npm error request to https://packagefeedproxy.microsoft.io/npm/@vscode%2fl10n-dev failed: cache mode is 'only-if-cached' but no cached response is available.
Network installation is prohibited by sandbox policy. Real CLI extraction/import checks could not proceed.

@rchiodo Rich Chiodo (rchiodo) left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved via Review Center.

@rchiodo Rich Chiodo (rchiodo) added the review-auto:approved Automated review: no blocking findings (approval posted). label Oct 30, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved via Review Center.

Eduardo Villalpando Mello (edvilme) pushed a commit to microsoft/vscode-python-ci that referenced this pull request Oct 30, 2026
…lation checkout support

## Shared pull-only release localization
This PR replaces the unused unmerged Azure English extraction/submission machinery with an opt-in pullLocalization interface on both public release wrappers. It invokes the same Python core as the public GitHub composite action in microsoft/vscode-common-python-lsp#125 rather than maintaining a second importer.

- Consumers declare translations (private drop dev) and localizationTools (reviewed immutable common-LSP SHA), with literal Github-Pylance endpoints.
- Source stays at workspace/s; tools and translations are sibling checkouts outside extension source.
- Run the shared importer after caller build/pre-built overlays and before final VSIX packaging.
- Keep the default self SDL scan, explicitly include executable tooling, and exclude translation data. Additional caller exclusions remain supported; sourceAnalysisPool preserves original caller pools.
- Derive canonical IDs and optional runtime metadata. Require all locales and per-table real-language translation evidence before copying; allow individual untranslated strings.
- No release-side English extraction, app write-token acquisition, Git pushes, or translation-update PRs. Existing signing/setup PowerShell is intentionally unchanged.
- Preserve opt-out callers, platform/version parameters, stable manual validation, and publication gates. Azure expansion can persist checkout credentials; no claim is made otherwise.

## Verification and rollout
Actual YAML-reading guards and shared Python/Git fixtures passed. Both release entry points for seven enabled repositories expanded successfully with publishing explicitly disabled. Both environments definitions are disabled; their exact YAML also expanded through the enabled isort definition without enabling or modifying those definitions. All expanded platform jobs preserve import order, source paths, SDL scanning, and publication gates. Shared core CI and the actual isort GitHub submission action passed.

Shared dependency: microsoft/vscode-common-python-lsp#125
Pilot: microsoft/vscode-isort#777
Sibling draft PRs:
- debugger: microsoft/vscode-python-debugger#1134
- environments: microsoft/vscode-python-environments#1971
- pylint: microsoft/vscode-pylint#915
- flake8: microsoft/vscode-flake8#609
- autopep8 (including runtime l10n): microsoft/vscode-autopep8#415
- mypy: microsoft/vscode-mypy#651
- black-formatter: microsoft/vscode-black-formatter#833

Keep this PR and callers draft pending dependencies, credential setup, and real handbacks. Current drop blocks isort (English-only), environments (missing canonical output), and flake8/mypy/black (missing runtime tables). Debugger, pylint, and autopep8 passed actual manifest/runtime imports from current drop. Autopep8 runtime ...
@bschnurr
Bill Schnurr (bschnurr) merged commit f6cd2fb into main Nov 3, 2026
24 of 25 checks passed
@bschnurr
Bill Schnurr (bschnurr) deleted the users/bschnurr/shared-localization branch November 3, 2026 17:26
Bill Schnurr (bschnurr) added a commit to microsoft/vscode-python-environments that referenced this pull request Nov 10, 2026
## Summary
- Use the shared Python localization core and composite action rather
than copying helper scripts into this extension.
- Keep GitHub English submission separate from pull-only DevDiv
stable/pre-release imports.
- Use drop branch dev and canonical extension ID
ms-python.vscode-python-envs; remove tracked generated locale snapshots
where present and keep generated files in VSIX selection.
- Preserve existing runtime metadata, extraction sources, feeds,
platform matrices, and version/validation behavior.
- Require complete, nonempty manifest/runtime tables for all 14
supported locales. Allow untranslated strings and wholly untranslated
tables with English fallback. Reject missing, empty, malformed, or
incomplete handbacks before packaging.

## Shared dependencies
- microsoft/vscode-common-python-lsp#125
-
https://dev.azure.com/devdiv/DevDiv/_git/Pylance-Eng/pullrequest/801431
- Pilot: microsoft/vscode-isort#777

The GitHub action and Azure tooling checkout are pinned to
`6b4a3573e7e7a9fe1e468e129544432278aec2d6`, which implements the
approved English-fallback policy in
microsoft/vscode-common-python-lsp#127. Both
release pipelines use the GitHub-hosted `microsoft/pylance-eng`
templates at `refs/heads/main` through `Github-Pylance`.

## Verification
Executable guards read the actual YAML and preserve existing triggers,
build steps, publication defaults, validation contacts, version flags,
and matrices. Both release YAMLs expanded with publishing explicitly
disabled (environments used the enabled validation definition because
its native definitions are disabled). Actual English export used the
pinned CLI through the internal feed. Changed YAML/JSON formatting and
whitespace checks passed.

## Setup and release prerequisites
The localization drop must contain complete, nonempty manifest/runtime
handbacks for all 14 supported locales; translated content is not a
readiness requirement. Both native DevDiv release definitions remain
disabled; the exact YAML was expanded through the enabled isort
validation definition without enabling them.

Configure the `PYTHON_LOCALIZATION_APP_ID` Actions variable and
`PYTHON_LOCALIZATION_PRIVATE_KEY` secret, or grant access to
organization-level credentials. Credential access was not rechecked for
this feedback update. Configure access through the owners; do not copy
credentials between repositories. Authorize `Github-Pylance` for
template, translation, and shared-tooling repository resources in both
release pipelines.

Merge and publication remain owner-controlled. No merge or publication
is requested.

## GitHub pipeline templates
- Use `microsoft/pylance-eng` on GitHub instead of `DevDiv/Pylance-Eng`
in both stable and pre-release template resources, through the existing
`Github-Pylance` service connection.
- Preserve `refs/heads/main`, template paths, build steps, localization
pins, release settings, and publication defaults. Document
service-connection access for template, translation, and shared-tooling
resources.
- Follow the verified isort change:
microsoft/vscode-isort#793 (test run 15971864
succeeded with publishing disabled and Publish skipped).

### Verification
- Existing executable actual-YAML localization/release guards passed.
- Exact parsed-YAML comparisons against the pre-change commit confirmed
that the template resource is the only release configuration change.
- All fourteen changed sibling release YAMLs passed formatting;
CRLF-aware whitespace checks passed.
- Both native release definitions remain disabled. The exact stable and
pre-release YAMLs expanded successfully through enabled isort definition
27670, covering all 10 configured build platforms. No pipeline was
enabled.
- Expanded pipeline checks preserved localization import ordering,
source scanning, and disabled publishing/validation-stage conditions.

These are preview expansions, not full sibling build runs. No new
release builds were queued, no extension was published, and no PR was
merged.


## Review feedback follow-up
- Runtime environment routing is unchanged from main. Removed the
PR-added routing change, explanatory comments, and added global-refresh
test to keep this PR focused on localization/pipeline work.
- Reconciled the acceptance contract with the README and exact pinned
importer: wholly untranslated tables are allowed with English fallback;
complete/nonempty locale coverage and structural validation remain
required.
- TypeScript compilation, targeted ESLint, whitespace checks, and all 76
focused environment-routing/global-selection tests passed. Verified that
the GitHub action and both release imports use the approved
English-fallback revision.

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: d6166fde-f261-4e66-abe4-3ef5c1313713
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

debt Code Cleanup, Refactorings and Repo health. Not feature related review-auto:approved Automated review: no blocking findings (approval posted).

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants