[feat](job) show first error message for routine load#65553
Conversation
|
Thank you for your contribution to Apache Doris. Please clearly describe your PR:
|
|
run buildall |
|
/review |
|
Codex automated review failed and did not complete. Error: You've hit your usage limit. Visit https://chatgpt.com/codex/settings/usage to purchase more credits or try again at Jul 20th, 2026 12:23 AM. Please inspect the workflow logs and rerun the review after the underlying issue is resolved. |
46cf915 to
9f85868
Compare
|
run buildall |
Cloud UT Coverage ReportIncrement line coverage Increment coverage report
|
0f76348 to
43cb1e2
Compare
|
run buildall |
Cloud UT Coverage ReportIncrement line coverage Increment coverage report
|
|
PR approved by at least one committer and no changes requested. |
|
PR approved by anyone and no changes requested. |
TPC-H: Total hot run time: 29969 ms |
FE UT Coverage ReportIncrement line coverage |
TPC-DS: Total hot run time: 180463 ms |
ClickBench: Total hot run time: 25 s |
There was a problem hiding this comment.
Static review found three remaining issues in the current head. The feature goal is clear: expose routine-load FirstErrorMsg across SHOW, information_schema, transaction attachments, cloud abort, replay, and tests. The public schema ordering now looks appended and aligned, and the OK cloud abort path is improved, but there are still gaps at the BE-to-FE RPC boundary, cloud abort retry/idempotency, and regression-test row selection.
Critical checkpoint conclusions: protocol compatibility is mostly append-only for thrift/proto/public result schemas, but the routine-load transaction RPC still carries an unbounded first-error payload before FE truncates it. Cloud abort retry handling is not idempotent for the new attachment field. Test coverage exists for display, replay, truncation, and OK cloud abort, but it misses the already-aborted cloud retry and the regression test can read a stale history row. No builds or tests were run because the review prompt forbids builds, and this checkout also lacks thirdparty/protoc setup. User focus: no additional user-provided focus points were supplied.
Note: an earlier transient diagnostic retention comment was removed after the PR head advanced and the timestamp-based patch it referenced was replaced.
|
/review |
|
run buildall |
|
Codex automated review failed and did not complete. Error: Codex completed, but no new pull request review was submitted for the current head SHA. Please inspect the workflow logs and rerun the review after the underlying issue is resolved. |
|
/review |
There was a problem hiding this comment.
Review complete. I found one additional correctness regression: the current-error reset now clears the pre-existing ERROR_LOG_URLS diagnostic.
Checkpoint conclusions:
- Goal/test: first-error message exposure is mostly wired through SHOW/information_schema and attachments, but the accepted issue means the existing error-log URL display behavior regresses; test coverage does not protect that case.
- Scope: current diff is otherwise focused and public fields are appended.
- Concurrency/lifecycle/persistence: no new lock or concurrency issue found; replay/cloud/payload concerns are already covered by existing review threads;
ERROR_LOG_URLSlive lifecycle reset is the remaining issue. - Compatibility/protocol: Thrift/protobuf fields are optional tail fields; SHOW and information_schema ordinals are aligned in current head.
- Parallel paths/tests: commit/rollback share attachment collection; missing cloud retry/replay/BE payload/stale regression cases are already covered by existing threads.
No additional user focus was provided. Builds/tests were not run because the review prompt explicitly forbids builds or code modifications in this runner.
Cloud UT Coverage ReportIncrement line coverage Increment coverage report
|
There was a problem hiding this comment.
I completed the review pass and found no additional non-duplicate inline comments to add.
Critical checkpoint conclusions:
- Goal: the PR adds Routine Load first-error-message propagation through BE attachments and FE SHOW/information_schema surfaces. The main live path is implemented, but several correctness gaps are already covered by existing review threads.
- Lifecycle and parallel paths: replay/restart, cloud progress refresh/reset, cloud abort retry, and the rolling current-statistics diagnostic reset were all reviewed. The reset-path diagnostic loss was valid, but it is now duplicate-covered by live comment 3575915355 on
RoutineLoadJob.java:991, so I did not repeat it. - Compatibility and surfaces: current head appends
FIRST_ERROR_MSGafterCOMPUTE_GROUPfor information_schema and appendsFirstErrorMsgto SHOW output; Thrift/proto fields are optional tail additions. The earlier ordinal/stale-test concerns are already covered by existing threads. - Tests: the added Kafka regression and FE unit tests cover the non-cloud happy path and some attachment/display behavior. Missing replay/cloud/retry/overlong-payload coverage is already called out in existing comments.
- Validation: static review only. The prompt forbids builds/code modifications, and this runner is missing
.worktree_initializedplusthirdparty/installed/bin/protoc.
No extra user focus was provided. All substantiated concerns from this pass were either already raised in the bundled review context or became duplicate-covered by the live PR comment set before submission.
FE UT Coverage ReportIncrement line coverage |
TPC-H: Total hot run time: 29437 ms |
TPC-DS: Total hot run time: 178000 ms |
ClickBench: Total hot run time: 24.76 s |
BE Regression && UT Coverage ReportIncrement line coverage Increment coverage report
|
FE Regression Coverage ReportIncrement line coverage |
|
PR approved by at least one committer and no changes requested. |
What problem does this PR solve?
PR #55666 added the first data quality error to load responses, but Routine Load was not covered. Routine Load transaction attachments only propagated the error log URL, so users still had to open the URL to inspect the actual error.
Design
This PR:
first_error_msgthrough the Routine Load Thrift transaction attachment and cloud transaction attachment.first_error_msg_max_lengthwhen constructing the FE transaction attachment.FirstErrorMsgcolumn toSHOW ROUTINE LOAD.FIRST_ERROR_MSGtoinformation_schema.routine_load_jobswithout changing existing column ordinals.ERROR_LOG_URLSunchanged instead of embedding the message in the URL field.Example
first_error_msgoutput:The first error can be queried directly:
or