Skip to content

Fix #2390: don't silently truncate CSV columns, handle preamble row - #2410

Closed
Mohamed Elfeky (FekyBaz) wants to merge 1 commit into
microsoft:mainfrom
FekyBaz:fix-2390-csv-no-silent-truncate
Closed

Fix #2390: don't silently truncate CSV columns, handle preamble row#2410
Mohamed Elfeky (FekyBaz) wants to merge 1 commit into
microsoft:mainfrom
FekyBaz:fix-2390-csv-no-silent-truncate

Conversation

@FekyBaz

Copy link
Copy Markdown

Fixes #2390.CsvConverter treated rows[0] as header unconditionally and silently truncated wider data rows via row[:len(rows[0])].Adds two behaviors:- Preamble heuristic: lone single-field first row followed by a wider row is emitted as plain text and the second row becomes the header (covers WizTree exports whose line 1 is a banner and line 2 is the real header).- No silent loss otherwise: table widens to max row width with a UserWarning instead of truncating.Also avoids mutating parsed rows in place.Short rows are still padded; empty inner rows keep existing behavior.Verified with WizTree repro plus existing CSV expectations (BOM, blank rows, pipe escaping).

@FekyBaz

Copy link
Copy Markdown
Author

@microsoft-github-policy-service agree

@afourney

afourney commented Sep 9, 2026

Copy link
Copy Markdown
Member

I agree that we should expand the columns to match the widest row. However, I don't want to overfit to WizTree. I will address the more general case, and recommend that people implement a custom plugin for WizTree specifically.

@afourney

afourney commented Sep 9, 2026

Copy link
Copy Markdown
Member

Addressed on #2430

@afourney afourney closed this Sep 9, 2026
@FekyBaz

Copy link
Copy Markdown
Author

Thanks afourney — the general fix in #2430 covers the core issue (no more silent truncation), and I agree the WizTree preamble is better left to a custom plugin rather than overfitting the converter. Closing this in favor of the merged approach makes sense.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

CSV converter silently truncates data columns when rows have more fields than the header row

2 participants