Search before asking
Paimon version
master @ 4af1470
Compute Engine
Flink
Minimal reproduce step
- Start the query service on any table:
CALL sys.query_service('default.t', 1).
- Watch the
FileMonitor-<table> source task in the Flink Web UI while no new files are committed.
QueryFileMonitor.Reader.pollNext() (paimon-flink/paimon-flink-common/src/main/java/org/apache/paimon/flink/service/QueryFileMonitor.java, line 97-105) calls Thread.sleep(monitorInterval) on the mailbox thread and then returns InputStatus.MORE_AVAILABLE. It does not override isAvailable(), so AbstractNonCoordinatedSourceReader.isAvailable() keeps returning an already completed future.
The sibling MonitorSource.Reader in the same module was changed in #6396 to await the interval asynchronously, override isAvailable() and return NOTHING_AVAILABLE. QueryFileMonitor was not covered by that change.
What doesn't meet your expectations?
Expected: the source is measured as idle while waiting for the next discovery interval.
Actual: busyTimeMsPerSecond stays at 1000, so the Web UI reports this source as a permanent bottleneck. Checkpoint triggering is also delayed by up to continuous.discovery-interval (default 10s) while the mailbox thread is blocked.
Anything else?
N/A
Are you willing to submit a PR?
Search before asking
Paimon version
master @ 4af1470
Compute Engine
Flink
Minimal reproduce step
CALL sys.query_service('default.t', 1).FileMonitor-<table>source task in the Flink Web UI while no new files are committed.QueryFileMonitor.Reader.pollNext()(paimon-flink/paimon-flink-common/src/main/java/org/apache/paimon/flink/service/QueryFileMonitor.java, line 97-105) callsThread.sleep(monitorInterval)on the mailbox thread and then returnsInputStatus.MORE_AVAILABLE. It does not overrideisAvailable(), soAbstractNonCoordinatedSourceReader.isAvailable()keeps returning an already completed future.The sibling
MonitorSource.Readerin the same module was changed in #6396 to await the interval asynchronously, overrideisAvailable()and returnNOTHING_AVAILABLE.QueryFileMonitorwas not covered by that change.What doesn't meet your expectations?
Expected: the source is measured as idle while waiting for the next discovery interval.
Actual:
busyTimeMsPerSecondstays at 1000, so the Web UI reports this source as a permanent bottleneck. Checkpoint triggering is also delayed by up tocontinuous.discovery-interval(default 10s) while the mailbox thread is blocked.Anything else?
N/A
Are you willing to submit a PR?