Skip to content

[Bug] Unsupported to CALL a dedicated minor/full compaction #9256

Description

@suntectec

Search before asking

  • I searched in the issues and found nothing similar.

Paimon version

1.3.1

Compute Engine

Flink 1.20.3

Minimal reproduce step

'num-sorted-run.stop-trigger' = '2147483647',
'compaction.size-ratio' = '1',
'compaction.max-size-amplification-percent' = '50000',
'partition.expiration-check-interval' = '1 d',
'write-only' = 'false',
'snapshot.num-retained.min' = '2',
'write-buffer-spillable' = 'true',
'path' = 's3://prd-datalake/ods_ewallet.db/TransactionDetails',
'snapshot.time-retained' = '30min',
'target-file-size' = '128mb',
'num-sorted-run.compaction-trigger' = '20',
'partition.timestamp-formatter' = 'yyyyMMdd',
'sink.parallelism' = '1',
'changelog-producer' = 'none',
'write-buffer-spill.max-disk-size' = '2gb',
'compaction.optimization-interval' = '60min',
'sort-spill-threshold' = '100',
'partition.expiration-time' = '3650 d',
'bucket' = '-1',
'changelog-producer.lookup-wait' = 'false',
'scan.plan-sort-partition' = 'true',
'snapshot.num-retained.max' = '5',
'partition.timestamp-pattern' = '$pt_createdtime',
'write-buffer-size' = '128mb',
'commit.force-compact' = 'false'

Flink SQL> CALL dl_ods.sys.compact(table => 'ods_ewallet.Payments', compact_strategy => 'full');
[ERROR] Could not execute SQL statement. Reason:
java.lang.UnsupportedOperationException: Unsupported bucket mode: KEY_DYNAMIC

Image

What doesn't meet your expectations?

Writing Speed was very slowly and blocked, and when tried to do a dedicated compation, the client raised an UnsupportedOperationException

Anything else?

I cannot reproduce this issue. It more seems like an index_metadata definition wrong. If u know it why and how to trouble-shooting or u meet the same issue as me, please tell me. Otherwise, this BUG report will be closed by me after a week, thanks

Are you willing to submit a PR?

  • I'm willing to submit a PR!

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions