Skip to content

[Python] Validate FileBasedSink num_shards type - #40331

Open
goutamadwant wants to merge 1 commit into
apache:masterfrom
goutamadwant:validate-filebasedsink-num-shards
Open

goutamadwant wants to merge 1 commit into
apache:masterfrom
goutamadwant:validate-filebasedsink-num-shards

Conversation

@goutamadwant

Copy link
Copy Markdown
Contributor

Fixes #25860

FileBasedSink currently accepts a side input for num_shards, then fails later while processing the pipeline.

This validates num_shards in the constructor and raises a clear TypeError when it is not an int. The regression test covers AsSingleton.

Before: the constructor accepts AsSingleton as num_shards.

After: the constructor raises TypeError: num_shards must be an int.

Tests:

  • python -m unittest -q apache_beam.io.filebasedsink_test — 15 passed
  • python -m unittest -q apache_beam.io.textio_test — 117 passed

@github-actions

Copy link
Copy Markdown
Contributor

Checks are failing. Will not request review until checks are succeeding. If you'd like to override that behavior, comment assign set of reviewers

@goutamadwant

Copy link
Copy Markdown
Contributor Author

retest this please

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: ValueError: Invalid DisplayDataItem when using AsSingleton for side input (need better parameter validation before it hits this)

1 participant