Repository navigation
Conversation
readCsvFile returns a CSVParser that owns the underlying FileInputStream, but
the import call sites never closed it: early returns on an empty file or a bad
header leaked the descriptor immediately, and importing a large directory of
CSV files could exhaust file descriptors ("Too many open files"). Wrap the
parser in try-with-resources at each call site (ImportData, ImportDataTree,
ImportDataTable, ImportSchemaTree) so it and its FileInputStream are closed on
every exit; the record stream is fully consumed inside the block, so closing on
scope exit is safe. Add a test asserting the parser is closed after use.
Signed-off-by: Zihan Dai <99155080+PDGGK@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
This PR fixes a file-descriptor leak in the CLI CSV import paths by ensuring the CSVParser returned from readCsvFile(...) is always closed, including on early-return paths (empty file / invalid header). This prevents bulk directory imports from exhausting the process open-file limit (“Too many open files”) and adds a regression test to guard the behavior.
Changes:
- Wrap CSV parsing at CLI import call sites in try-with-resources so the
CSVParser(and underlyingFileInputStream) is reliably closed. - Apply the same resource-management fix to schema CSV import (
ImportSchemaTree), which uses its own localreadCsvFile. - Add a unit test asserting the parser is closed after leaving the try-with-resources scope.
Reviewed changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
| iotdb-client/cli/src/main/java/org/apache/iotdb/tool/data/ImportData.java | Closes CSVParser via try-with-resources in single-file CSV import. |
| iotdb-client/cli/src/main/java/org/apache/iotdb/tool/data/ImportDataTree.java | Closes CSVParser via try-with-resources in tree-model CSV import. |
| iotdb-client/cli/src/main/java/org/apache/iotdb/tool/data/ImportDataTable.java | Closes CSVParser via try-with-resources in table-model CSV import. |
| iotdb-client/cli/src/main/java/org/apache/iotdb/tool/schema/ImportSchemaTree.java | Closes CSVParser via try-with-resources in schema CSV import (local readCsvFile). |
| iotdb-client/cli/src/test/java/org/apache/iotdb/tool/ReadCsvFileResourceTest.java | Adds regression test verifying parser closure after try-with-resources scope. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
|
A mechanical note rather than anything about the change itself. Every workflow run on this PR is sitting in Separately: the branch is now about 106 commits behind master, though still No urgency from my side; flagging it in case it is simply queued and unnoticed. |
The previous test wrapped readCsvFile in its own try-with-resources and then asserted the parser was closed, so it passed with or without the fix: it exercised CSVParser.close(), not the four import methods changed here. Drive each import path directly instead - ImportDataTree and ImportDataTable importFromCsvFile, ImportSchemaTree importSchemaFromCsvFile and ImportData importFromSingleFile - on an empty CSV, which takes the "Empty file!" early return without needing a session, and assert that the number of open file descriptors in the process does not grow with the number of calls. With the try-with-resources changes reverted, all four tests fail, each reporting 50 descriptors leaked over 50 calls; with them applied, all four pass. Platforms without /proc/self/fd or /dev/fd skip rather than fail. Signed-off-by: Zihan Dai <1436286758@qq.com>
Description
AbstractDataTool.readCsvFilereturns aCSVParserthat owns the underlyingFileInputStream, but the CLI import call sites (ImportData,ImportDataTree,ImportDataTable,ImportSchemaTree) never closed it. The empty-file / invalid-header early returns leaked the file descriptor immediately, and importing a directory of many CSV files could exhaust the process limit withToo many open files.This wraps the parser in a try-with-resources at each call site so it (and the
FileInputStreamit wraps) is closed on every exit path. The recordStreamis fully consumed inside the block before the method returns — every consumer is a synchronousforEachorcollect— so closing on scope exit is safe.The test drives each of the four import paths on an empty CSV, which takes the "Empty file!" early return without needing a session, and asserts that the number of open file descriptors in the process does not grow with the number of calls. With the try-with-resources changes reverted, all four tests fail, each reporting 50 descriptors leaked over 50 calls; with them applied, all four pass.
This closes #18237.