Commit Graph

13 Commits

Author SHA1 Message Date
fkrebs 9a4b2788e4 feat: inject archive.ph manual-check link for /🚫/ states
User can click "Check archive.ph manually →" from inside Miniflux to see if a snapshot exists or to trigger one. Useful when the automated archive.ph lookup hit rate limits or no snapshot was ready yet.
2026-05-28 09:06:45 +02:00
fkrebs 48f7c2588a feat: 📰 state for free articles + crawl4ai-first ordering
- crawl4ai direct first (detect if article actually needs paywall bypass)
- Only hit archive.ph when paywall detected (saves rate-limit budget)
- New 📰 icon for free articles enriched with full text
- 📖 reserved for actual paywall bypass via archive.ph

Final state set:
- 🔓 OA via Unpaywall
- 📰 Free article enriched
- 📖 Paywall bypassed via archive.ph
-  Paywalled, retrying
- 🚫 Paywalled, gave up after 3h
-  Fetch error
2026-05-28 07:21:47 +02:00
fkrebs d9489095bf feat: archive.ph fallback + paywall detection + 3h retry/giveup + markdown→HTML
- archive.ph timemap → snapshot URL → crawl4ai fetch (low rate-limit endpoint)
- Unpaywall OA lookup for DOIs (science)
- crawl4ai direct as final fallback
- Detect paywall markers (Spiegel S+, generic abonnieren) + content length
- Status icons: 🔓 OA · 📖 fetched ·  retry · 🚫 gave up (3h) ·  error
- mdToHtml inline converter (lists/links/headings/bold/code) + strips nav/share lines
- regex hostname extractor (URL constructor undefined in n8n task runner sandbox)
2026-05-28 07:13:33 +02:00
fkrebs 813d3b899b fix: use this.helpers.httpRequest (fetch and $helpers undefined in n8n task runner sandbox) 2026-05-28 06:46:44 +02:00
fkrebs 0bfc5497d2 auto: sync n8n workflows 2026-05-28 2026-05-28 03:30:06 +02:00
fkrebs 58db1f0099 fix: use fetch instead of $helpers.httpRequest in paywall-bypass code node 2026-05-28 01:43:52 +02:00
fkrebs 6004183d4b paywall-bypass: remove time cutoff, skip already-processed entries
Replace 6-min published_at cutoff with emoji-suffix guard so the
scheduled flow processes all unread paywalled articles regardless of age.
Fixes badge text: "Wallabag" -> "crawl4ai".
2026-05-28 01:12:56 +02:00
fkrebs b62581bccf paywall-bypass: replace Wallabag with crawl4ai, add Unpaywall OA lookup
Remove Wallabag token node; fetch full text directly via crawl4ai /md.
Scientific articles DOI-resolved via Unpaywall API for open-access URLs.
Adds title emoji suffix: 🔓 (OA), 📖 (crawl4ai),  (failed).
Includes backfill variant (no time cutoff) for processing existing unread entries.
2026-05-28 00:31:09 +02:00
fkrebs ce0113f3c1 add paywall-bypass-backfill: manual trigger, no time cutoff 2026-05-28 00:20:05 +02:00
fkrebs f42b9eab52 paywall-bypass: add title emoji suffix per state
| 🔓 = open access via Unpaywall
| 📖 = paywalled, fetched via Wallabag
|  = fetch attempted, no content returned
2026-05-28 00:11:07 +02:00
fkrebs bc6ad24f91 add paywall-bypass flow
Runs every 5 min. Detects paywalled articles (known domains + short content).
For scientific: tries Unpaywall API (DOI-based OA lookup) first, falls back to
Wallabag. Writes full text back to Miniflux entry so reader shows it inline.
2026-05-28 00:10:30 +02:00
fkrebs 9d682dedbc auto: sync n8n workflows 2026-05-27 2026-05-27 03:30:03 +02:00
fkrebs cc4df25754 init: export 3 workflows, add IDEAS.md + README 2026-05-23 09:40:42 +02:00