Repository navigation
[mosaic] Reject TIMESTAMP(0) when validating Mosaic data fields - #11241
Open
LuciferYang wants to merge 1 commit into
Open
LuciferYang wants to merge 1 commit into
LuciferYang wants to merge 1 commit into
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Purpose
close #11228
Precision 0 maps to the Arrow
SECONDunit, which the Mosaic native writer does not accept. A Mosaic table with aTIMESTAMP(0)orTIMESTAMP(0) WITH LOCAL TIME ZONEcolumn can be created, but every write fails withwriter open failed: unsupported Timestamp unit: Second. MySQL CDC creates such columns fromDATETIME.MosaicFileFormat.validateDataFieldsnow rejects precision 0, also insideARRAYandMAP, with the same message style as the other types Mosaic rejects, and suggestsTIMESTAMP(3).This check also runs when a table is loaded. An existing Mosaic table with a
TIMESTAMP(0)column could not be written before and now cannot be read either, until the column is changed toTIMESTAMP(3)orfile.formatis set back. Both ALTERs work on such a table.Tests
MosaicFileFormatTest:testValidateDataFieldsTimestampPrecisionZeroUnsupportedchecks the full message forTIMESTAMP(0),ARRAY<TIMESTAMP(0)>,TIMESTAMP(0) WITH LOCAL TIME ZONEandMAP<STRING, TIMESTAMP(0) WITH LOCAL TIME ZONE>. It fails without the change.testValidateDataFieldsTimestampPositivePrecisionSupportedchecks that precisions 1 and 9 of both types are still accepted.