This is a merge queue draft opened by Mergify. Its pull request has merged, and nobody needs to act on it.
Mergify opened it for batch flying-gratin-5erixd of the default queue, to test #85 before it merged into main.
🎉 What landed
Mergify merged its pull request into main, then closed this draft:
#85 Port morss reader options and widen feed discovery (ashcoft), as 1a50c14
Required conditions of queue rule default for merge
No extra conditions.
Required conditions to stay in the queue
None
Summary by cubic
This merge queue draft tests PR #85, which ports morss CLI options and improves feed discovery and link recovery. Adds --post, --order, --firstlink, --nolink, --noref, and --silent to the CLI and /read endpoint. Expands well-known feed path discovery to try page-relative paths first, then site root, adding JSON feed paths. Unwraps Google News, Google Translate, and Pocket redirects, decodes feedsportal links, and prefers feedburner origLink. Normalizes shouting titles and recovers Wikipedia and Reddit item links. Keeps POST responses out of the URL-keyed HTTP cache.
Written for commit 3f23751d9a7106dc3215f9765c496fcfeff1f6ef. Summary will update on new commits.
Confidence Score: 2/5
Fix the decoder crash, skipped root feeds, and unstable item IDs before merging.
### Issue 1
pol/reader/urls.py:169-172
`decode_feedsportal` raises `IndexError` when a matching token contains consecutive zeros or ends in zero. Splitting on `0` creates an empty segment, which `pair[0]` cannot read. `_fix_items` calls this for every item link without checking the host, so one such link breaks the whole feed read. Return the original URL when the token is malformed.
### Issue 2
pol/reader/pipeline.py:144
For `https://example.com/news/post` with no advertised feeds, 15 page guesses fill the first slots in `candidates[:20]`. That leaves room for only five root guesses. A feed at `/feed.xml`, the sixth root guess, is never fetched, so `process` reports no feed even though this worked before the change. Give the root fallback its own request budget.
### Issue 3
pol/reader/pipeline.py:260-261
`noref` clears the link before preserving the item's fallback ID. For RSS items without a `guid`, JSON output changes from a stable URL-based `id` to the item's list position. When a new post arrives, an old `id` points to a different post, so feed readers can miss new posts or overwrite old ones. Preserve `item.link` in `item.id` when the ID is empty before clearing the link.
```suggestion
if options.noref:
item.id = item.id or item.link
item.link = ''
```
### Issue 4
pol/reader/pipeline.py:258-259
`nolink` strips anchors only from `item.content`. RSS items with only a description keep all their links, and `clip` can add summary links back above stripped content. Strip links from `item.summary` too, since `Item.body` uses it as output content.
```suggestion
if options.nolink:
item.content = strip_links(item.content)
item.summary = strip_links(item.summary)
```
Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!
---
For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.
Summary
Adds morss-style reader options, more link repairs, page-first feed guesses, and GET-only cache use.
Feed discovery checks paths beneath a page before trying the site root.
Feed items recover publisher links from common wrappers and feed formats.
Readers can choose which items to enrich and which links to keep.
The CLI can POST a body without mixing it into GET cache entries.
The CLI can finish quietly without printing the feed.
admin changed the title from merge queue: checking #85 on main (6235080) to merge queue: merged #85: Port morss reader options and widen feed discovery6h ago
admin closed this pull request6h ago
sourcery-ai[bot]6h ago
Reviewer's Guide
Data structures
Add the new processing and output options to the shared Options structure so the CLI, web endpoint, and pipeline can use them.
pol/reader/pipeline.py L44-49
API
Expose the morss-compatible options in the CLI, pass them into Options, and implement --silent by suppressing successful output.
pol/reader/cli.py L30-38, L99-104 (+2 small edits)
Accept the item-processing options as /read parameters and map them into the shared options structure.
pol/reader/web.py L132-135
Implementation
Extend feed discovery to try page-relative well-known paths before root paths, deduplicate candidates, and fall back to root candidates when page candidates do not verify.
pol/reader/discover.py L49-58, L100-133, L207-216 (+2 small edits)
pol/reader/pipeline.py L5-8, L14-20 (+3 small edits)
Keep POST requests out of the URL-keyed GET cache, and send the configured POST body when processing a feed URL.
pol/reader/http.py L185-188, L224
pol/reader/pipeline.py L306-312
Recover and normalize item links and titles, support item ordering and link actions, and use Feedburner’s original link when parsing RSS items.
pol/reader/parse.py L225-238, L384-385
pol/reader/pipeline.py L171-265 (+3 small edits)
pol/reader/urls.py L41-52, L122-138, L163-180 (+2 small edits)
Documentation
Document the expanded discovery behavior, link recovery, web parameters, and morss-compatible CLI options.
readme.md L68-72, L97-103 (+2 small edits)
Tests
Add CLI coverage for POST, silent output, item link actions, and selecting items for enrichment.
tests/test_reader_cli.py L5, L73-166
Test page-first discovery, root fallback, and JSON feed well-known paths.
tests/test_reader_discover.py L1-9, L140-172
Verify POST responses are not reused as cached GET responses.
tests/test_reader_http.py L187-206
Test Feedburner link preference and link stripping behavior.
tests/test_reader_parse.py L13, L419-440
Cover item ordering, title normalization, and recovery of Wikipedia, Reddit, wrapper, and Feedsportal links.
tests/test_reader_pipeline.py L164-242
Test added redirect-wrapper handling and Feedsportal URL decoding.
tests/test_reader_urls.py L6, L78-110
Tips and commands
Interacting with Sourcery
Trigger a new review: Comment @sourcery-ai review on the pull request.
Continue discussions: Reply directly to Sourcery's review comments.
Generate a GitHub issue from a review comment: Ask Sourcery to create an
issue from a review comment by replying to it. You can also reply to a
review comment with @sourcery-ai issue to create an issue from it.
Generate a pull request title: Write @sourcery-ai anywhere in the pull
request title to generate a title at any time. You can also comment
@sourcery-ai title on the pull request to (re-)generate the title at any time.
Generate a pull request summary: Write @sourcery-ai summary anywhere in
the pull request body to generate a PR summary at any time exactly where you
want it. You can also comment @sourcery-ai summary on the pull request to
(re-)generate the summary at any time.
Generate reviewer's guide: Comment @sourcery-ai guide on the pull
request to (re-)generate the reviewer's guide at any time.
Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
pull request to resolve all Sourcery comments. Useful if you've already
addressed all the comments and don't want to see them anymore.
Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
request to dismiss all existing Sourcery reviews. Especially useful if you
want to start fresh with a new review - don't forget to comment
@sourcery-ai review to trigger a new review!
NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewerTIP This summary will be updated as you push new changes.
greptile-apps[bot]reviewed6h ago
greptile-apps[bot]reviewed6h ago
greptile-apps[bot]reviewed6h ago
greptile-apps[bot]reviewed6h ago
greptile-apps[bot]reviewed6h ago
This pull request was closed without being merged. Branch mergify/merge-queue/0797c29ddf is still available.
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.
Share