Skip to content

PROF-15982 Skip in-memory ELF parse of a library musl is still mapping - #996

Open
zhengyu123 wants to merge 7 commits into
mainfrom
zgu/PROF-15982-musl-partial-load
Open

zhengyu123 wants to merge 7 commits into
mainfrom
zgu/PROF-15982-musl-partial-load

Conversation

@zhengyu123

@zhengyu123 zhengyu123 commented Nov 14, 2026 •

Copy link
Copy Markdown
Contributor

What does this PR do?:

On musl, UnloadProtection now passes only for a library the dynamic loader has finished mapping. Symbols::parseLibraries() then skips the in-memory ELF parse (ElfParser::parseProgramHeaders()) of a library that another thread's dlopen() is still loading.

  • UnloadProtection on musl: _valid requires dladdr(image_base) to return a library whose dli_fbase equals image_base. The main executable and the loader are still accepted without a check.
  • New gtest unloadProtection_ut.cpp: rejects a file mapping the loader does not know, accepts a loaded image, and rejects an address inside a loaded library that is not its base.
  • New test header testTempFile.h: FdOwner and copyToTempFile(), shared by unloadProtection_ut and libraryPatcherUnload_ut (whose own copy loop it replaces).

Motivation:

PROF-15982: SIGSEGV in ElfParser::parseDynamicSection() during Profiler::start() on musl aarch64 (tracer 1.66.0, which bundles ddprof 1.50.0).

  • In the 1.50.0 linux-arm64-musl library, parseDynamicSection()+0x158 is ldr x16, [x8]: the d_tag read in the first line of the dynamic-section loop. At the fault, x8 points inside libverify.so and x9 is the dynamic-section end. The PT_DYNAMIC segment itself was unreadable.
  • musl's map_library() first maps the whole address span from file offset 0, then maps each later segment over it with MAP_FIXED. It links the library into its loaded list only after that. In between, /proc/self/maps already lists the library, but its later segments hold the wrong file bytes or SIGBUS past EOF.
  • On glibc, UnloadProtection's dlopen(RTLD_NOLOAD) waits for a load in progress. On musl the class returned valid without any check, because musl never unloads.

Additional Notes:

  • Why dladdr(), not dlopen(RTLD_NOLOAD) on musl. musl's dladdr() searches only libraries already linked into its list, that is, fully mapped. musl looks up an absolute path by opening the file, so RTLD_NOLOAD would reject libraries whose file was deleted after loading. Those keep working with dladdr().
  • Scope. All UnloadProtection callers on musl now get the check, including the GOT patching in MallocHooker and applyPatch(). That matches what the class already guarantees on glibc.
  • Not fixed here. A library rejected this way is still recorded as parsed and is not parsed again later. Its file symbols are still loaded by ElfParser::parseFile(); only its in-memory imports and unwind tables are missing. glibc behaves the same way when UnloadProtection is invalid. Tracked in PROF-16675.
  • Review. Pre-PR review: Sphinx (standard effort) on the full range, then /code-review medium on each of five fix ranges, including the fix for the Codex RAII comment. No Critical or High finding is open. Open, Low or nit only, all in test code:
    • libraryPatcherUnload_ut copies its DSO to /tmp, so it fails on a host where /tmp is noexec. It used /tmp before this PR as well.
    • "/tmp" appears three times in libraryPatcherUnload_ut instead of one named constant.
    • A failed dlopen(), chdir() or symlink() in loadUnlinkedLib() exits without printing the cause.
    • After ENAMETOOLONG, copyToTempFile() leaves the truncated template in path_out.
  • Human review recommended. The change alters shared-library handling on musl, which AGENTS.md lists as an area that needs engineer steering.

How to test the change?:

Run on the head commit unless noted; aarch64 containers, debug build, utils/run-containers-tests.sh --mount.

Check State
gtestDebug_unloadProtection_ut on musl without the fix failed as intended: RejectsImageNotLoadedByTheLoader, RejectsLoadedImageAtWrongBase
gtestDebug_unloadProtection_ut on musl, 3/3 tests run passed
gtestDebug_unloadProtection_ut on glibc passed
gtestDebug_libraryPatcherUnload_ut on musl (6 passed, 20 skipped by design on musl) and glibc passed
gtestDebug_elfparser_ut on musl and glibc inconclusive for the head commit: passed on the first fix commit; production code has since changed only in a comment
qualityChecks (first fix commit), check-copyright.py, check-claude-md.py passed
PROF-15982 crash signature absent from crash tracking after release pending (production-check)

The race itself is not reproduced: the gtest checks the guard's decision on a raw file mapping that the loader never registered.

For Datadog employees:

  • If this PR touches code that signs or publishes builds or packages, or handles
    credentials of any kind, I've requested a security review (run the dd:platform-security-review
    skill, or file a request via the PSEC review form).
    bewaire also runs automatically on every PR.
  • This PR doesn't touch any of that.
  • JIRA: PROF-15982

🤖 Generated with Claude Code

zhengyu123 and others added 4 commits November 13, 2026 19:55
On musl, UnloadProtection accepted every library, so Symbols::parseLibraries()
read the dynamic section of a library that another thread's dlopen() had
mapped only partly. musl maps the whole span from file offset 0 first and the
later segments afterwards, so those reads saw the wrong bytes or faulted.

On musl the guard now requires dladdr() to find a library that starts at the
image base. musl's dladdr() searches only libraries whose mapping is done.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… leaks

Map the copy read-only, honour TMPDIR, treat read errors as copy failures
and remove the copy on every path. State on the musl branch that it checks
load completion, as dlopen() does on glibc.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…test

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@zhengyu123
zhengyu123 requested a review from a team as a code owner November 14, 2026 02:01
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 14, 2026 •

Copy link
Copy Markdown
Codex Review Summary: ✅ Completed

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-14T02:46:49.538175Z fe9ad99 New commits
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@datadog-prod-us1-3 datadog-prod-us1-3 Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bits Code Review: PASS

More details

Static review supports the musl guard’s use of loader registration and image-base equality before allowing in-memory ELF parsing. The concurrent dlopen race itself was not reproduced.

Was this helpful? React 👍 or 👎

Open Bits AI session

🤖 Bits Code Review · Commit 01b2a3d

@dd-octo-sts

dd-octo-sts Bot commented Nov 14, 2026 •

Copy link
Copy Markdown
Contributor

Scan-Build Report

User:runner@runnervmccnms
Working Directory:/home/runner/work/java-profiler/java-profiler/ddprof-lib/src/test/make
Command Line:make -j4 all
Clang Version:Ubuntu clang version 21.1.8 (6ubuntu1)
Date:Sat Nov 14 02:38:46 2026

Bug Summary

Bug TypeQuantityDisplay?
All Bugs2
Logic error
Dereference of null pointer1
Unused code
Dead initialization1

Filter Results by File

Reports

Bug Group Bug Type ▾ File Function/Method Line Path Length
Unused codeDead initializationreferenceChains.cppapplyOomRamp4971
Logic errorDereference of null pointerperfEvents_linux.cppunregisterThread72026

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 01b2a3d5dc

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +68 to +70
int out = mkstemps(to, 3);
if (out < 0) return false;
int in = open(from, O_RDONLY);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Use RAII for the test's file and mapping resources

The new test manually pairs mkstemps/close and unlink, plus open/close and mmap/munmap, rather than using owner guards. This violates the repository’s C++ RAII rule; use scoped owners for these resources or explain why a manual pair cannot be scoped.

AGENTS.md reference: AGENTS.md:L469-L474

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 686846a. Three scoped owners in the test now release every resource on every exit path: FdOwner (fd), FileMapping (mmap/munmap) and TempCopy (mkstemps copy, unlinked in its destructor). The ASSERTs run inline again. isLoaded() closes the dlopen(RTLD_NOLOAD) handle right where it opens it.

zhengyu123 and others added 3 commits November 13, 2026 21:21
…guards

Scoped owners release the temp copy, descriptors and mapping on every exit,
so the ASSERTs run inline instead of after hand-ordered cleanup.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…rrno

copyToTempFile() in testTempFile.h replaces the copies in unloadProtection_ut
and libraryPatcherUnload_ut. It honours TMPDIR, returns the errno of the
failing step (EIO for a short write) and removes a partial copy. The tests
print that errno and the failing step instead of a later errno.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…rrors

The helper takes the directory; a null one means TMPDIR, or /tmp when unset.
libraryPatcherUnload_ut dlopen()s its copy, so it keeps /tmp rather than a
TMPDIR that may be noexec, and prints the copy error before exiting. The
helper returns ENAMETOOLONG instead of truncating the path.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@dd-octo-sts

dd-octo-sts Bot commented Nov 14, 2026

Copy link
Copy Markdown
Contributor

CI Test Results

Run: #43547292494 | Commit: 98b75cd | Duration: 20m 29s (longest job)

✅ All 32 test jobs passed

Status Overview

JDK glibc-aarch64/debug glibc-amd64/debug musl-aarch64/debug musl-amd64/debug
8 - ✅ - -
8-ibm - ✅ - -
8-j9 ✅ ✅ - -
8-librca - - ✅ ✅
8-orcl - ✅ - -
11 - ✅ - -
11-j9 ✅ ✅ - -
11-librca - - ✅ ✅
17 ✅ ✅ - -
17-graal ✅ ✅ - -
17-j9 ✅ ✅ - -
17-librca - - ✅ ✅
21 ✅ ✅ - -
21-graal ✅ ✅ - -
21-librca - - ✅ ✅
25 ✅ ✅ - -
25-graal ✅ ✅ - -
25-librca - - ✅ ✅

Legend: ✅ passed | ❌ failed | ⚪ skipped | 🚫 cancelled

Summary: Total: 32 | Passed: 32 | Failed: 0


Updated: 2026-11-14 03:00:56 UTC

@dd-octo-sts

dd-octo-sts Bot commented Nov 14, 2026

Copy link
Copy Markdown
Contributor

✅ All 40 integration tests passed

📊 Dashboard · 👷 Pipeline · 📦 fe9ad99d

@datadog-prod-us1-3

Copy link
Copy Markdown

Pipelines

⚠️ Warnings

Your PR has warnings. Please review the issues below.

🚦 1 Pipeline job failed

DDCI - DataDog/java-profiler | renaissance-benchmarks: [naive-bayes, 25]

View more details · View in DDCI

Useful? React with 👍 / 👎

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: fe9ad99 | Docs | View more details | Give us feedback!

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant