Skip to content

GH-1471: Pair dense union children with their type ids - #1472

Merged
jbonofre merged 1 commit into
apache:mainfrom
jbonofre:dense-union-child-order
Nov 13, 2026
Merged

jbonofre merged 1 commit into
apache:mainfrom
jbonofre:dense-union-child-order

Conversation

@jbonofre

Copy link
Copy Markdown
Member

What's Changed

Follow-up to #1419. DenseUnionVector.getField() lists the children in the order their type ids were registered, while the vector of a child is created when it is first used, which can be in another order. Code that paired the two by position mixed up the children:

  • getChildrenFromFields() returned the vectors in the order they were created, so VectorUnloader and Data.exportVector wrote the buffers of one child under the field of another. It now returns them in the order of getField(), followed by any child added with addVector() without registering a type id.
  • The transfer pair paired each type id with the vector at the same position in the internal struct. It now uses the vector of the type id, and the child of the target with the same name.
  • registerNewTypeId() on a vector without declared type ids, such as the target of a transfer pair, assigned the next type id by count, which can already be in use when the type ids of the source are not 0..n-1. It now skips the type ids in use.

TestDenseUnionVector covers a union whose double child is created before its int child (child order, transfer, split, and a VectorUnloader / VectorLoader round trip, which read back [2.08E-322, 0] for [1.5, 42] before), and registering a type id on a transfer target with type ids [2, 0].

Closes #1471.

DenseUnionVector.getField() lists the children in the order their type
ids were registered, while the child vectors are created when first used.
getChildrenFromFields() and the transfer pair paired them by creation
order, which wrote the buffers of one child under the field of another
in exports and mixed up children in transfers. Both now follow the
registered type ids. registerNewTypeId() on a vector without declared
type ids, such as a transfer target, also skips the type ids in use.
@jbonofre jbonofre added the bug-fix PRs that fix a big. label Nov 13, 2026
@github-actions github-actions Bot added this to the 20.0.0 milestone Nov 13, 2026
@jbonofre
jbonofre merged commit f515e2e into apache:main Nov 13, 2026
22 of 23 checks passed
@jbonofre
jbonofre deleted the dense-union-child-order branch November 13, 2026 14:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug-fix PRs that fix a big.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Dense union children are paired with the wrong type id when created out of registration order

2 participants