Skip to content

fix(rss): drop layout whitespace from feed and entry titles - #2428

Merged
afourney merged 4 commits into
microsoft:mainfrom
kevin9327:fix/rss-title-layout-whitespace
Sep 9, 2026
Merged

fix(rss): drop layout whitespace from feed and entry titles#2428
afourney merged 4 commits into
microsoft:mainfrom
kevin9327:fix/rss-title-layout-whitespace

Conversation

@kevin9327

Copy link
Copy Markdown
Contributor

What this changes

A pretty-printed feed loses its heading structure. Every title is emitted with the indentation
of the element it was written in, which ends the Markdown heading before the text starts:

<channel>
  <title>
    Example feed
  </title>
  <description>
    Example feed description
  </description>
  <item>
    <title>
      A story about things
    </title>
    <pubDate>
      Mon, 01 Jan 2024 00:00:00 GMT
    </pubDate>
#
    Example feed

Example feed description

##
      A story about things

Published on:
      Mon, 01 Jan 2024 00:00:00 GMT

The # and ## are bare, the titles become indented body text, and result.title comes back
as '\n Example feed\n '.

Why

_parse_rss_type and _parse_atom_type interpolate the raw element text straight into the
heading:

md_text += f"# {channel_title}\n"
md_text += f"\n## {title}\n"
md_text += f"Published on: {pubDate}\n"

_get_data_by_tag_name returns exactly what the XML holds, and for a value on its own line
that is "\n Example feed\n ". XML has no notion of "the text minus the layout", so the
caller has to drop it -- and every one of these four call sites is a single-line construct
where a newline breaks the syntax.

The same shape was fixed for the Wikipedia converter in #1990 ("Treat whitespace-only titles
as if they were absent"); the feed converter still has it.

Fix

Read heading and metadata values through a small helper that strips the layout whitespace and
reports a whitespace-only value as absent, and guard the Atom feed heading the way the RSS one
already is.

That guard is not cosmetic: _parse_atom_type builds its heading unconditionally, so a feed
whose title is missing renders the literal # None today, and stripping would newly produce
it for a whitespace-only title:

# None
Body text.

Item bodies are untouched. description, summary and content still go through
_get_data_by_tag_name and _get_atom_content exactly as before -- their whitespace is
content, and _get_atom_content already has its own dedent rule for Atom plain text.

Tests

New file packages/markitdown/tests/test_rss_titles.py:

  • a pretty-printed RSS feed produces # Example feed, ## A story about things and a
    one-line Published on:, asserted line by line
  • the same for an Atom feed, including <subtitle> and <updated>
  • a whitespace-only channel title reports result.title is None and emits no heading
  • an Atom feed with no title anywhere does not render None
  • pinned: an ordinary single-line feed produces exactly the same markdown as before, asserted
    against the full string

Against unmodified main:

E  AssertionError: assert '\n    Example feed\n  ' == 'Example feed'
E  AssertionError: assert '\n    Example feed\n  ' == 'Example feed'
E  AssertionError: assert '   ' is None
E  AssertionError: assert 'None' not in '# None\nBody text.'
4 failed, 1 passed

With the fix:

5 passed

How I tested

Windows 11, Python 3.12, editable install of packages/markitdown[all].

pytest tests/test_rss_titles.py      ->  4 failed, 1 passed  (before)
pytest tests/test_rss_titles.py      ->  5 passed            (after)
pytest tests/test_rss_converter.py   ->  12 passed           (unchanged, before and after)
pytest tests/                        ->  18 failed, 431 passed, 4 skipped

Those 18 are unchanged by this PR: main gives 18 failed, 426 passed, 4 skipped on this
machine before any edit, and the 5 added tests account for the difference. They are the CLI
stdout-encoding, Windows file-URI and speech-transcription tests that need a UTF-8 console, a
case-sensitive path and network/ffmpeg.

black --check clean on both files.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Internal newlines remain unhandled, and channel descriptions are altered contrary to the stated scope.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

Fixes malformed Markdown headings caused by pretty-printed RSS and Atom metadata.

Changes:

  • Normalizes title and date metadata.
  • Omits missing Atom headings.
  • Adds RSS/Atom regression tests.
File summaries
File Description
_rss_converter.py Adds metadata normalization and heading guards.
test_rss_titles.py Tests pretty-printed and absent titles.
Review details
  • Files reviewed: 2/2 changed files
  • Comments generated: 2
  • Review effort level: Balanced

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread packages/markitdown/src/markitdown/converters/_rss_converter.py Outdated
Comment thread packages/markitdown/src/markitdown/converters/_rss_converter.py Outdated
afourney and others added 2 commits September 9, 2026 10:01
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
@afourney
afourney merged commit a04fd8b into microsoft:main Sep 9, 2026
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants