Repository navigation
Unwrap CDATA sections in entity replacement text, not in the expanded text - #40
Merged
FlorianRappl merged 1 commit intoNov 1, 2026
Merged
Conversation
… text The fallback expansion replaced <!\[CDATA\[(.*?)\]\]> in the whole text of a character token whenever the document declared any entity. That stripped the markers from escaped text such as <![CDATA[b]]> (giving 'b'), and took quadratic time on many unterminated starts: 240 KB of <![CDATA[ took 33 s. The CDATA sections of each entity's replacement text are now unwrapped once, in a single pass, when the entity is registered, and that text is used where an entity contributes to element content (tokenizer and nested expansion). Attribute values keep the replacement text as before. A 60,000-result differential fuzz against the previous code differs only for escaped CDATA-looking text (the fix), CDATA sections split across entities, recursive entities, and CDATA entities whose content references another CDATA entity, where the previous code left CDATA markers behind.
michaelthyregod
force-pushed
the
fix/cdata-in-entities
branch
from
November 1, 2026 17:19
844d769 to
1bad6e8
Compare
Contributor
Author
|
#38 is merged now and this is rebased onto devel, so it only shows its own change: 1 commit, 4 files (+138/-14). |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Builds on #38 (branched from it; until #38 is merged, this PR also shows its commits).
Problem
When a document declared any entity, the fallback expansion replaced
<!\[CDATA\[(.*?)\]\]>in the whole text of a character token. That caused two problems:a <![CDATA[b]]> cbecamea b c. Escaped text and CDATA have to stay distinguishable (Preserve CDATA sections as first-class DOM nodes #30).<![CDATA[made the regex scan to the end from each one. 240 KB took 33 s; without a DOCTYPE the same text takes 43 ms.Change
The regex exists for
XmlValidSa114, where an entity's replacement text holds a CDATA section (<!ENTITY e "<![CDATA[&foo;]]>">must give&foo;). So:XmlValidSa114still passes. The 240 KB case takes 13 ms.Evidence
A differential fuzz of 60,000 parse results against the previous code (entities with CDATA, nesting, escaped text, attributes, strict and recovery modes): 43,711 are identical, and every difference falls into one of these categories:
<![CDATA[markers in the textNew tests in
XmlEntityCDataSections. The previous code fails only the escaped-text test. 607 tests pass.🤖 Generated with Claude Code