Search before asking
Paimon version
1.3.1
Compute Engine
Flink 1.20.3
Minimal reproduce step
'num-sorted-run.stop-trigger' = '2147483647',
'compaction.size-ratio' = '1',
'compaction.max-size-amplification-percent' = '50000',
'partition.expiration-check-interval' = '1 d',
'write-only' = 'false',
'snapshot.num-retained.min' = '2',
'write-buffer-spillable' = 'true',
'path' = 's3://prd-datalake/ods_ewallet.db/TransactionDetails',
'snapshot.time-retained' = '30min',
'target-file-size' = '128mb',
'num-sorted-run.compaction-trigger' = '20',
'partition.timestamp-formatter' = 'yyyyMMdd',
'sink.parallelism' = '1',
'changelog-producer' = 'none',
'write-buffer-spill.max-disk-size' = '2gb',
'compaction.optimization-interval' = '60min',
'sort-spill-threshold' = '100',
'partition.expiration-time' = '3650 d',
'bucket' = '-1',
'changelog-producer.lookup-wait' = 'false',
'scan.plan-sort-partition' = 'true',
'snapshot.num-retained.max' = '5',
'partition.timestamp-pattern' = '$pt_createdtime',
'write-buffer-size' = '128mb',
'commit.force-compact' = 'false'
Flink SQL> CALL dl_ods.sys.compact(table => 'ods_ewallet.Payments', compact_strategy => 'full');
[ERROR] Could not execute SQL statement. Reason:
java.lang.UnsupportedOperationException: Unsupported bucket mode: KEY_DYNAMIC
What doesn't meet your expectations?
Writing Speed was very slowly and blocked, and when tried to do a dedicated compation, the client raised an UnsupportedOperationException
Anything else?
I cannot reproduce this issue. It more seems like an index_metadata definition wrong. If u know it why and how to trouble-shooting or u meet the same issue as me, please tell me. Otherwise, this BUG report will be closed by me after a week, thanks
Are you willing to submit a PR?
Search before asking
Paimon version
1.3.1
Compute Engine
Flink 1.20.3
Minimal reproduce step
'num-sorted-run.stop-trigger' = '2147483647',
'compaction.size-ratio' = '1',
'compaction.max-size-amplification-percent' = '50000',
'partition.expiration-check-interval' = '1 d',
'write-only' = 'false',
'snapshot.num-retained.min' = '2',
'write-buffer-spillable' = 'true',
'path' = 's3://prd-datalake/ods_ewallet.db/TransactionDetails',
'snapshot.time-retained' = '30min',
'target-file-size' = '128mb',
'num-sorted-run.compaction-trigger' = '20',
'partition.timestamp-formatter' = 'yyyyMMdd',
'sink.parallelism' = '1',
'changelog-producer' = 'none',
'write-buffer-spill.max-disk-size' = '2gb',
'compaction.optimization-interval' = '60min',
'sort-spill-threshold' = '100',
'partition.expiration-time' = '3650 d',
'bucket' = '-1',
'changelog-producer.lookup-wait' = 'false',
'scan.plan-sort-partition' = 'true',
'snapshot.num-retained.max' = '5',
'partition.timestamp-pattern' = '$pt_createdtime',
'write-buffer-size' = '128mb',
'commit.force-compact' = 'false'
Flink SQL> CALL dl_ods.sys.compact(
table=> 'ods_ewallet.Payments',compact_strategy=> 'full');[ERROR] Could not execute SQL statement. Reason:
java.lang.UnsupportedOperationException: Unsupported bucket mode: KEY_DYNAMIC
What doesn't meet your expectations?
Writing Speed was very slowly and blocked, and when tried to do a dedicated compation, the client raised an UnsupportedOperationException
Anything else?
I cannot reproduce this issue. It more seems like an index_metadata definition wrong. If u know it why and how to trouble-shooting or u meet the same issue as me, please tell me. Otherwise, this BUG report will be closed by me after a week, thanks
Are you willing to submit a PR?