Skip to content

[Bug] [Table model] Multi-value IN on INT32/INT64 fields returns wrong rows for out-of-range literals and throws ClassCastException for decimal literals #19276

Open

Description

@Karman580

Search before asking

  • I searched in the issues and found nothing similar.

Version

master, commit e78f7366134e (2.0.11-SNAPSHOT), standalone, table model.

Describe the bug and provide the minimal reproduce step

A multi-value IN list on an INT32 or INT64 field converts its literals without the range and decimal handling that = already has. This shows up in two ways:

  1. On INT32 fields, an out-of-range integer literal is narrowed with an unchecked (int) cast. It wraps around and matches an unrelated row.
  2. On INT32 and INT64 fields, a decimal literal is cast to LongLiteral, so the query fails with a ClassCastException.

Both the pushed-down scan filter and the projection evaluator are affected. x = a OR x = b is normalized into x IN (a, b), so the OR form returns the same wrong result.

create database castdb;
create table castdb.p(dev string tag, i int32 field);
insert into castdb.p(time, dev, i) values (1,'d1',1),(2,'d1',2),(3,'d1',3),(4,'d1',-2147483648),(5,'d1',2147483647),(6,'d1',null);
create table castdb.q(dev string tag, l int64 field);
insert into castdb.q(time, dev, l) values (1,'d1',1),(2,'d1',2),(3,'d1',3),(4,'d1',null);

Wrong rows (INT32)

Query Actual rows (time) Expected
select time, i from castdb.p where i in (1, 4294967298) 1, 2 1
select time, i from castdb.p where i in (1, -4294967293) 1, 3 1
select time, i from castdb.p where i in (1, 2147483648) 1, 4 (-2147483648) 1
select time, i from castdb.p where i = 1 or i = 4294967298 1, 2 1
select time, i from castdb.p where i not in (1, 4294967298) 3, 4, 5 (row 2 missing) 2, 3, 4, 5

explain for the OR query shows the rewritten predicate reaching the scan:

│PushDownPredicate: ("i" IN (1, 4294967298))│

ClassCastException (INT32 and INT64, filter and projection)

These all fail with the same message:

select time, i from castdb.p where i in (2, 2.5)
select time, i from castdb.p where i in (2.0, 2.4)
select time, i from castdb.p where i in (2.5, 3.5)
select time, i from castdb.p where i not in (1, 2.5)
select time, i from castdb.p where i = 2 or i = 2.5
select time, l from castdb.q where l in (2, 2.5)
select time, l from castdb.q where l not in (1, 2.5)
select time, i in (2, 2.5) as m from castdb.p
select time, l in (2, 2.5) as m from castdb.q

Msg: org.apache.iotdb.jdbc.IoTDBSQLException: 301: class org.apache.iotdb.commons.queryengine.plan.relational.sql.ast.DoubleLiteral cannot be cast to class org.apache.iotdb.commons.queryengine.plan.relational.sql.ast.LongLiteral

Filter pushdown path (DataNode log):

java.lang.ClassCastException: class ...DoubleLiteral cannot be cast to class ...LongLiteral
	at org.apache.iotdb.db.utils.TypeServices.getLongValue(TypeServices.java:4249)
	at org.apache.iotdb.db.utils.TypeServices$Predicate.lambda$static$5(TypeServices.java:3432)
	at org.apache.iotdb.db.queryengine.plan.relational.analyzer.predicate.ConvertPredicateToFilterVisitor.getValue(ConvertPredicateToFilterVisitor.java:364)
	at org.apache.iotdb.db.queryengine.plan.relational.analyzer.predicate.ConvertPredicateToFilterVisitor.constructInSet(ConvertPredicateToFilterVisitor.java:143)
	at org.apache.iotdb.db.queryengine.plan.relational.analyzer.predicate.ConvertPredicateToFilterVisitor.constructInFilter(ConvertPredicateToFilterVisitor.java:136)
	at org.apache.iotdb.db.queryengine.plan.relational.analyzer.predicate.ConvertPredicateToFilterVisitor.visitInPredicate(ConvertPredicateToFilterVisitor.java:125)
	...
	at org.apache.iotdb.db.queryengine.plan.analyze.PredicateUtils.convertPredicateToFilter(PredicateUtils.java:424)
	at org.apache.iotdb.db.queryengine.plan.planner.DataNodeTableOperatorGenerator.buildSeriesScanOptions(...)

Expression (projection) path:

java.lang.ClassCastException: class ...DoubleLiteral cannot be cast to class ...LongLiteral
	at org.apache.iotdb.calc.utils.TypeServices.lambda$static$136(TypeServices.java:1141)
	at org.apache.iotdb.calc.execution.relational.ColumnTransformerBuilder.constructInColumnTransformer(ColumnTransformerBuilder.java:1531)
	at org.apache.iotdb.calc.execution.relational.ColumnTransformerBuilder.visitInPredicate(ColumnTransformerBuilder.java:1520)
	...

For comparison, the single-value and equality forms already behave as expected on the same data:

Query Result
where i = 4294967298 empty
where i in (4294967298) empty
where i = 2.5 empty
where i in (2.0) row 2
select time, i in (1, 4294967298) as m true only for row 1, null for the NULL row

What did you expect to see?

x IN (a, b) should give the same result as x = a OR x = b under the existing = semantics for numeric literals on INT32/INT64 fields:

  • an integer outside the column range never matches;
  • an integral decimal such as 2.0 matches 2;
  • a non-integral decimal such as 2.5 never matches;
  • NOT IN returns the complement over non-NULL rows.

What did you see instead?

Wrong rows for out-of-range INT32 literals in IN, NOT IN and the equivalent OR form, and a ClassCastException for any decimal literal in a multi-value IN on INT32/INT64, in both the filter and the projection paths.

Anything else?

Root cause

  • ConvertPredicateToFilterVisitor.visitInPredicate sends a single-value list to constructCompareFilter. Since fixed the bug that caused incompatible behavior when comparing int64 or int32 column with double literals in query filter conditions #16917 that path handles decimal literals and out-of-range bounds.
  • A list with two or more values goes to constructInFilter → constructInSet → getValue. That uses TypeServices.Predicate.RELATIONAL_CONVERT_PREDICATE_VALUE_PARSER_SERVICE, which is (int) ((LongLiteral) v).getParsedValue() for INT32 and ((LongLiteral) v).getParsedValue() for INT64.
  • The analyzer does not insert casts for IN lists. It only checks that the types are comparable, so DoubleLiteral and oversized LongLiteral values reach this code unchanged.
  • NormalizeOrExpressionRewriter merges x = a OR x = b into the same InPredicate.
  • The projection path, calc-commons TypeServices.IN_MULTI_COLUMN_TRANSFORMER_SERVICE (used by ColumnTransformerBuilder.constructInColumnTransformer), already skips out-of-range INT32 values. It still casts every literal to LongLiteral, though, so decimals fail there too.

Not covered by existing work

The tree model is not affected in the same way: it rejects these lists with "2.5" cannot be cast to [INT32] instead of returning rows.

How this was verified

  • Standalone server: distribution built from master cd31721c14. The files on this path are identical to e78f7366134e: ConvertPredicateToFilterVisitor, both TypeServices classes, ColumnTransformerBuilder, NormalizeOrExpressionRewriter, ExpressionAnalyzer (apart from an unrelated UDF check), and PredicateUtils.
  • Unit level: calling PredicateUtils.convertPredicateToFilter with an InPredicate against the e78f7366134e jars gives measurements[0] IN [1, 2] for i IN (1, 4294967298), and a ClassCastException for decimal lists on INT32 and INT64.
  • Environment: macOS (Darwin 25.6.0, arm64), JDK 27 for the server, MEMORY_SIZE=2G.

Are you willing to submit a PR?

  • I'm willing to submit a PR.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions