[python] Fix Arrow offset overflow in merge condition filtering

[XiaoHongbo-Hope]

Purpose

Fix conditional self-merge failures when the combined size of Arrow string, binary, or other 32-bit-offset columns exceeds INT32_MAX.

filter_batch previously registered every input RecordBatch in one DataFusion partition. DataFusion could concatenate those batches while evaluating the condition, causing an offset overflow even though each individual Arrow batch was valid.

Changes

• Register each input RecordBatch as a separate DataFusion partition.
• Use collect_partitioned() and rebuild the Arrow table in partition order, preserving both chunk boundaries and input row order.
• Add low-memory offset-overflow and deterministic partition-order regression tests.

The fix preserves the existing Arrow schema instead of casting columns to large_string or large_binary.

Tests

• 134 passed in ray_data_evolution_merge_into_test.py
• 26 passed in table_merge_into_test.py
• 6 passed in concat_batch_reader_test.py
• DataFusion 54 ordering stress check: 500/500 ordered after the fix
• flake8, py_compile, and git diff --check