Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
16 commits
Select commit Hold shift + click to select a range
a37f319
Prepared next version
FlorianRappl Oct 31, 2026
7084caf
Adopt AngleSharp 1.9, deep-nesting hardening and an entity expansion cap
michaelthyregod Oct 31, 2026
b42f734
Drop the declaration lookup from the DTD validation CHANGELOG entry
michaelthyregod Nov 1, 2026
17202f0
Merge pull request #38 from michaelthyregod/feature/anglesharp-1.9
FlorianRappl Nov 1, 2026
1bad6e8
Unwrap CDATA sections in entity replacement text, not in the expanded…
michaelthyregod Nov 1, 2026
42e35b4
Merge pull request #40 from michaelthyregod/fix/cdata-in-entities
FlorianRappl Nov 1, 2026
b949841
Limit how deep the parser nests elements, as the HTML parser does
michaelthyregod Nov 1, 2026
60785eb
Use trusted publishing
FlorianRappl Nov 1, 2026
cb4e551
Drop the breaking-change note from the MaximumDepth CHANGELOG entry
michaelthyregod Nov 2, 2026
fddbe52
Merge pull request #42 from michaelthyregod/feature/xml-maximum-depth
FlorianRappl Nov 2, 2026
4e8f59c
Load the external DTD subset through the resource loader and requesters
michaelthyregod Nov 2, 2026
1c75fa7
Keep the benchmark's DTD path as written, shorten the CHANGELOG entry
michaelthyregod Nov 2, 2026
1fefce2
Read the units BenchmarkDotNet writes into the CSV cells
michaelthyregod Nov 2, 2026
f956887
Merge pull request #44 from michaelthyregod/fix/compare-builds-units
FlorianRappl Nov 2, 2026
07d4409
Resolve entity references at the end of the parse
michaelthyregod Nov 2, 2026
68d437c
Merge pull request #41 from michaelthyregod/fix/external-dtd-loading
FlorianRappl Nov 2, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 18 additions & 5 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,6 @@ on: [push, pull_request]

env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
NUGET_API_KEY: ${{ secrets.NUGET_API_KEY }}
DOCS_PATH: ${{ secrets.DOCS_PATH }}
DOCS_BRANCH: ${{ secrets.DOCS_BRANCH }}

Expand Down Expand Up @@ -57,10 +56,13 @@ jobs:
10.0.x

- name: Build
run: ./build.sh -AngleSharpVersion 1.5.0
run: ./build.sh -AngleSharpVersion 1.9.0

windows:
runs-on: windows-latest
permissions:
contents: write # create the GitHub release
id-token: write # request the OIDC token for NuGet trusted publishing

steps:
- uses: actions/checkout@v5
Expand All @@ -71,12 +73,23 @@ jobs:
dotnet-version: |
10.0.x

# Exchanges the GitHub OIDC token for a short-lived (1 hour) NuGet API key.
# Requires a trusted publishing policy on nuget.org for this repository and workflow file.
- name: NuGet login
id: nuget_login
if: github.event_name == 'push' && (github.ref == 'refs/heads/main' || github.ref == 'refs/heads/devel')
uses: NuGet/login@v1
with:
user: ${{ secrets.NUGET_USER }}

- name: Build
env:
NUGET_API_KEY: ${{ steps.nuget_login.outputs.NUGET_API_KEY }}
run: |
if ($env:GITHUB_REF -eq "refs/heads/main") {
.\build.ps1 -Target Publish -AngleSharpVersion 1.5.0
.\build.ps1 -Target Publish -AngleSharpVersion 1.9.0
} elseif ($env:GITHUB_REF -eq "refs/heads/devel") {
.\build.ps1 -Target PrePublish -AngleSharpVersion 1.5.0
.\build.ps1 -Target PrePublish -AngleSharpVersion 1.9.0
} else {
.\build.ps1 -AngleSharpVersion 1.5.0
.\build.ps1 -AngleSharpVersion 1.9.0
}
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -199,3 +199,6 @@ pip-log.txt

# Fallout build tool
.fallout/temp

# BenchmarkDotNet
BenchmarkDotNet.Artifacts/
62 changes: 51 additions & 11 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,8 +19,8 @@ The public README describes the intended positioning: XML behavior that integrat

- Primary development branch is `devel`.
- The repository currently uses the Fallout build orchestrator, version 10.4.x in the checked-in build tooling.
- The current project version in `src/Directory.Build.props` is `1.2.0`.
- The package's AngleSharp dependency defaults to version `1.5.0` and allows versions below `2.0.0`.
- The current project version in `src/Directory.Build.props` is `1.3.0`.
- The package's AngleSharp dependency defaults to version `1.9.0` and allows versions below `2.0.0`.
- There is no repository-level AGENTS.md or copilot instruction file other than this one.
- The worktree should be checked with `git status --short` before making assumptions about local changes. Never discard changes that are already present.

Expand Down Expand Up @@ -67,7 +67,7 @@ The scripts:
The default local target runs restore, compile, tests, and package creation. The build reads the version from `CHANGELOG.md` and supports parameters such as:

```sh
./build.sh -AngleSharpVersion 1.5.0
./build.sh -AngleSharpVersion 1.9.0
./build.sh -Target RunUnitTests
./build.sh -Target Compile
./build.sh -Target Package
Expand Down Expand Up @@ -105,7 +105,7 @@ The solution is `src/AngleSharp.Xml.sln`; it contains the library, tests, and bu

`.github/workflows/ci.yml` runs on push and pull request.

- Linux uses .NET `10.0.x` and runs `./build.sh -AngleSharpVersion 1.5.0`.
- Linux uses .NET `10.0.x` and runs `./build.sh -AngleSharpVersion 1.9.0`.
- Windows uses .NET `10.0.x` and runs `build.ps1`.
- Windows publishes on `main`, pre-publishes on `devel`, and performs the default build for other refs.
- Documentation deployment is conditional on repository secrets and the configured docs branch/path.
Expand All @@ -117,7 +117,7 @@ The solution is `src/AngleSharp.Xml.sln`; it contains the library, tests, and bu
- `net10.0`
- `net462` and `net472` on Windows only

The project enables XML documentation generation, strong-name signing, SourceLink, symbols, and package metadata. Warnings are treated as errors through `src/Directory.Build.props`.
The project enables XML documentation generation, strong-name signing, SourceLink, symbols, and package metadata. The `net8.0` and `net10.0` targets set `IsAotCompatible`, so trim and AOT analyzer warnings fail the build. Warnings are treated as errors through `src/Directory.Build.props`.

## Repository Layout

Expand Down Expand Up @@ -157,7 +157,7 @@ Parser files:
- `Parser/XmlParser.cs`: parser facade and document creation.
- `Parser/XmlDomBuilder.cs`: tree construction, DOCTYPE handling, DTD validation, and entity expansion.
- `Parser/XmlTokenizer.cs`: XML lexical tokenizer and character/entity reference handling.
- `Parser/XmlParserOptions.cs`: suppress-errors, source-reference, and element-created callback options.
- `Parser/XmlParserOptions.cs`: suppress-errors, source-reference, element-created callback, entity expansion limit, and nesting depth limit options.
- `Parser/XmlParseError*.cs`: parse error definitions and exception mapping.
- `Parser/Tokens/`: token models for declarations, DOCTYPE, tags, text, CDATA, comments, processing instructions, and EOF.

Expand All @@ -179,16 +179,26 @@ DTD files:
- `Parser/XmlNotWfDocuments.cs`: not-well-formed XML fixtures.
- `Parser/XmlNotWfExtDtd.cs`: not-well-formed external DTD fixtures.
- `Parser/XmlDtdImplementedCases.cs`: focused, currently supported DTD validation cases.
- `Parser/XmlExternalDtdSupport.cs`: local-file external DTD/entity regression tests.
- `Parser/XmlExternalDtdSupport.cs`: external DTD loading through the resource loader, with `Mocks/MockRequester.cs`.
- `Tokenizer/XmlDTD.cs`: DTD tokenizer/parser samples, including historical commented assertions.
- `Tokenizer/XmlTokenization.cs`: XML tokenizer tests.
- `Dom/`: DOM samples and tree behavior.
- `Xhtml/`: XHTML formatter and preservation tests.
- `Mocks/MockEntityProvider.cs`: entity provider test double.
- `Mocks/MockRequester.cs`: requester serving fixed resources for `mock:` and `http://*.test`, registered as the only requester so tests never reach the network.
- `TestExtensions.cs`: test parsing helpers, including permissive conformance fallback behavior.

The tests use NUnit 3, NUnit3TestAdapter, Microsoft.NET.Test.Sdk, and target `net8.0`.

### Benchmarks: `src/AngleSharp.Xml.Benchmarks`

BenchmarkDotNet project (`net8.0`, `net10.0`), not part of the solution or the Fallout build. See *Performance* below.

- `ParseValidationBenchmark.cs`: `ParseDocument` including the validation walks, per document shape and DOCTYPE kind (none, internal subset, local external subset).
- `TextContentBenchmark.cs`: `TextContent` of XML elements, from a single-text leaf to deep chains.
- `Documents.cs`: the shared document shapes and their DTDs.
- `Compare-Builds.ps1`: measures the working tree against a baseline commit.

### Documentation

Markdown documentation is under `docs/`:
Expand Down Expand Up @@ -235,6 +245,8 @@ The parser also supports streams, fragments, and asynchronous overloads. `XmlPar
- `IsSuppressingErrors`: recovery mode; the parser attempts to return a document instead of throwing. The resulting DOM may be incomplete or surprising. Do not treat this as strict validation.
- `IsKeepingSourceReferences`: keeps source token references on created elements for diagnostics/tooling.
- `OnCreated`: callback receiving each created element and its `TextPosition`.
- `MaxCharactersFromEntities`: cap on the characters produced by expanding DTD-declared entities (default `DefaultMaxCharactersFromEntities`, 10,000,000; 0 = no limit), counted in `XmlDomBuilder.ResolveEntityReferences` through `XmlTokenizer.ClaimEntityCharacters`. Any new entity expansion path must claim its characters there too.
- `MaximumDepth`: the deepest level the parser nests elements at (root = level 1; default `DefaultMaximumDepth`, 512, as core's `HtmlParserOptions.MaximumDepth`; zero or negative restores the default, `Int32.MaxValue` lifts it). Beyond it the parser throws `MaximumDepthExceeded`; with `IsSuppressingErrors` the element is attached at the limit (`XmlDomBuilder.GetInsertionParent`). Tests and benchmarks that parse deeper documents must lift it, as `XmlDeepNesting` does.

### Validity

Expand All @@ -251,12 +263,12 @@ Currently supported or partially supported behavior includes:
- Common element models such as `ANY`, `EMPTY`, mixed `(#PCDATA|name|...)*`, and simple ordered sequences.
- Internal attribute checks, including undeclared attributes, `#REQUIRED`, and `#FIXED` cases.
- Internal general entity replacement.
- Local file-based external `SYSTEM` subset loading.
- Local external general entity replacement in supported cases.
- External `SYSTEM` subset loading through the browsing context's `IResourceLoader` (only with resource loading enabled; the requesters decide the protocols). `XmlDomBuilder.LoadExternalSubsetAsync` starts at the DOCTYPE and the parse goes on; `CompleteAsync` waits for it at the end (the synchronous parse blocks there), builds the DTD, and resolves the entity references the tokenizer kept as written (`XmlEntityReference`, offsets per text node or attribute) before validating. Resolve by recorded offset, never by searching the text: escaped `&name;` reads as `&name;`. Never read files or URLs a document names directly.
- Replacement of entities declared in a loaded external subset.

Known limitations:

- No HTTP/network retrieval for external DTDs/entities.
- External entities are not loaded; the external subset only through the resource loader.
- No complete PUBLIC identifier or XML catalog resolution workflow.
- Parameter entities and external-subset semantics are not full XML conformance.
- Complex content-model grammar and all quantifier combinations are not guaranteed by fallback validation paths.
Expand Down Expand Up @@ -309,6 +321,32 @@ For parser changes, include tests for:

Conformance fixture suites are broad and include historical XML test cases that depend on external resources, complex DTD features, or behavior not fully implemented in this repository. Treat `ToXmlDocumentConformance` and similar helpers as compatibility/recovery harnesses, not as a replacement for strict parser assertions.

## Performance

A change to the parser, the validation walks, or DOM members such as `TextContent` must not make them slower on ordinary documents, and should make them faster where it can. Prove it with the benchmarks, not with reasoning:

```sh
dotnet run -c Release --project src/AngleSharp.Xml.Benchmarks -f net8.0 -- --list flat
dotnet run -c Release --project src/AngleSharp.Xml.Benchmarks -f net8.0 -- --check
dotnet run -c Release --project src/AngleSharp.Xml.Benchmarks -f net8.0 -- --filter '*TextContent*'
```

```powershell
./src/AngleSharp.Xml.Benchmarks/Compare-Builds.ps1 -BaselineRef <commit>[, <commit>...] -AngleSharpVersion <version> -Rounds 3 -Affinity <mask>
```

- Pass the same `-AngleSharpVersion` to every build when the compared commits default to different AngleSharp versions; otherwise core's own changes end up in the comparison.
- The script writes `summary.md` to its output directory: per case, each build's median of the rounds, its ratio to the working tree, the spread of the rounds, and the allocations.

- `--check` runs every setup and benchmark body once without measuring. Run it on both builds before a long run; `Compare-Builds.ps1` does.
- Compare builds with `Compare-Builds.ps1`: two separately built copies, alternated round by round. BenchmarkDotNet rebuilds the benchmark project it finds from the current directory, so the script runs each build from its own source tree and fails a round whose logs name the other build's AngleSharp.Xml module.
- `MemoryDiagnoser` is on for every class; report the allocation column with the timings.
- Benchmarks need an otherwise idle machine; on hybrid CPUs pin them to the fast cores (`--affinity`, or `-Affinity` on the script).
- Keep the shapes within what the oldest compared build survives. Before the explicit-stack walks, the old code overflowed the call stack parsing at about 2600 levels with a DTD (2850 without) and in `TextContent` at about 5000 (net8.0, 1 MB stack), which is why the deep shapes stop at 2000 and 4000.
- Every shape must stay valid against its DTD: an invalid document stops the walks at the first error and measures an early exit. `GlobalSetup` throws if one is not.
- A parse allocates megabytes, so when the garbage collector runs during it can move a row by 5-10% either way, and the direction changes with the document size. That is why the catalog comes in three sizes (`catalog-500`, `catalog`, `catalog-2000`). A difference at one size alone is not proof. A change that really costs work shows at every size; one that flips sign between sizes is GC timing. To confirm, rerun with BenchmarkDotNet's `--envVars DOTNET_GCgen0size:20000000` (512 MB Gen0, so no Gen0 or Gen1 collection runs during a parse) and compare the work alone.
- Parsing builds the tree through AngleSharp's `AppendChild`, which costs O(depth) per insert in AngleSharp up to 1.9.0, so the deep parse rows include core's quadratic tree building.

## Coding and Editing Conventions

- C# uses four spaces; project files use two spaces.
Expand All @@ -319,6 +357,8 @@ Conformance fixture suites are broad and include historical XML test cases that
- Use explicit, descriptive variable names; follow the surrounding style.
- Keep comments concise and explanatory only where needed.
- Do not add license headers.
- Stay trim- and AOT-clean. The library sets `IsAotCompatible` on its .NETCoreApp targets, so the trim and AOT analyzers run on every build and a warning breaks it. Reflection over a runtime type (`GetType().GetProperties()`, `Assembly.GetTypes()`) is what they reject: read members off an annotated generic parameter or use a fixed table instead. The analyzers only see the library itself; publishing a small `PublishAot` console app against it is the end-to-end check.
- Walk trees with an explicit stack, not recursion. Documents can nest thousands of levels deep and a stack overflow kills the process; see `XmlDomBuilder.ValidateElements` and `XmlElement.TextContent`.
- Do not commit, reset, checkout, or create branches unless explicitly requested.
- Do not remove user changes from a dirty worktree.
- Keep generated `bin/`, `obj/`, `.fallout/`, and test-result artifacts out of source changes unless the task explicitly concerns them.
Expand All @@ -332,7 +372,7 @@ Conformance fixture suites are broad and include historical XML test cases that
- XML entity names passed to the built-in provider include the trailing semicolon (`amp;`, not `amp`).
- `IsSuppressingErrors` changes control flow and can produce a non-null but structurally unreliable document.
- A passing broad conformance helper test does not prove strict XML conformance.
- External DTD support is local-file-oriented and must not be described as network/catalog-aware.
- External DTD support depends on the configured resource loader and requesters; it has no PUBLIC identifier or catalog resolution.
- DTD `IsValid` is not XSD validation and does not replace domain-specific validation.
- The docs index and DTD support matrix should be updated when DTD behavior changes.

Expand Down
18 changes: 18 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,21 @@
# 1.3.0

Released on Sunday, November 1 2026.

- Updated to use a minimum of AngleSharp 1.9
- Improved performance of `TextContent` on XML elements, which no longer allocates for an element holding a single text or CDATA section
- Improved performance of DTD validation: content models are parsed once per document, and documents without ID declarations skip the identity walk
- Fixed parsing deeply nested documents overflowing the stack and ending the process; the DTD and identity validation walks now use an explicit stack
- Fixed `TextContent` of deeply nested XML elements overflowing the stack
- Fixed escaped text that looks like a CDATA section (`&lt;![CDATA[...]]&gt;`) losing its markers in documents that declare entities, which also took quadratic time; only the CDATA sections in an entity's replacement text are unwrapped now
- Fixed unbounded expansion of nested entities ("billion laughs"), which let a small document exhaust memory; expansion now stops at `XmlParserOptions.MaxCharactersFromEntities`
- Added trimming and Native AOT compatibility (`IsAotCompatible`) for the .NET 8 and .NET 10 targets
- Added `XmlParserOptions.MaxCharactersFromEntities` (default 10,000,000, 0 for no limit), like `System.Xml`'s setting of the same name
- Updated loading of the external DTD subset to go through the resource loader and requesters (only with resource loading enabled)
- Updated entity references to be resolved at the end of the parse, so loading the external subset does not interrupt it
- Fixed escaped entity references (`&amp;name;`) being expanded when the entity is declared, and empty entities throwing
- Added `XmlParserOptions.MaximumDepth` (default 512), like `HtmlParserOptions.MaximumDepth`: deeper elements throw, or are attached at the limit when errors are suppressed

# 1.2.0

Released on Friday, August 21 2026.
Expand Down
2 changes: 1 addition & 1 deletion build/Build.cs
Original file line number Diff line number Diff line change
Expand Up @@ -197,9 +197,9 @@ protected override void OnBuildInitialized()
.DependsOn(RunUnitTests)
.Executes(() =>
{
// In CI this is the short-lived key issued by NuGet trusted publishing (NuGet/login).
var apiKey = Environment.GetEnvironmentVariable("NUGET_API_KEY");


if (apiKey.IsNullOrEmpty())
{
throw new BuildAbortedException("Could not resolve the NuGet API key.");
Expand Down
28 changes: 27 additions & 1 deletion docs/tutorials/01-API.md
Original file line number Diff line number Diff line change
Expand Up @@ -65,9 +65,35 @@ var parser = new XmlParser(new XmlParserOptions

If true, elements keep token source references. This is useful for diagnostics and tooling.

### MaxCharactersFromEntities

The maximum number of characters that expanding the entities declared in the document's DTD may produce in total, like `System.Xml.XmlReaderSettings.MaxCharactersFromEntities`. Zero (0) means no limit. Unlike `System.Xml`, which prohibits DTD processing by default, this parser always processes the DTD, so the limit defaults to 10,000,000 characters (`XmlParserOptions.DefaultMaxCharactersFromEntities`) to stop entities that expand exponentially ("billion laughs") from exhausting memory.

Exceeding the limit throws an `XmlParseException`. With `IsSuppressingErrors`, the references beyond the limit are kept as text instead. The built-in entities (`&amp;`, `&lt;`, ...) and character references do not count.

```cs
var parser = new XmlParser(new XmlParserOptions
{
MaxCharactersFromEntities = 100_000,
});
```

### MaximumDepth

The deepest level at which the parser nests elements, counting the root element as level 1, like `HtmlParserOptions.MaximumDepth` in AngleSharp. It defaults to 512 (`XmlParserOptions.DefaultMaximumDepth`); zero or a negative value restores the default, and `Int32.MaxValue` lifts the limit. XML puts no limit on nesting, but code walking a tree that is thousands of levels deep recursively can overflow the call stack, which ends the process.

An element beyond the limit throws an `XmlParseException`. With `IsSuppressingErrors`, it is attached at the limit instead, as a sibling of the deepest element, as the HTML parser does; text that follows a closed sibling then stays in its own element.

```cs
var parser = new XmlParser(new XmlParserOptions
{
MaximumDepth = 2000,
});
```

### OnCreated

Callback invoked when an element has been created, including source position.
Callback invoked when an element has been created, including source position. References to DTD-declared entities are only resolved at the end of the parse, so the element's attribute values still hold them as written.

```cs
var parser = new XmlParser(new XmlParserOptions
Expand Down
Loading
Loading