One redesign, and the parser is dead
A renamed field, one extra wrapper div, a list that became infinite scroll. The script that worked last night returns an empty array this morning.
We fetch finance, social, developer and media sources, line up the fields and drop the duplicates. Pull it into your own system over the JSON API: everything if you want it, only what's new if you pass a cursor. Same schema everywhere, so switching sources doesn't mean rewriting code.
$ curl -s "https://api.databridge.world/v1/feeds/fin-cn-macro/items?limit=2" \ -H "Authorization: Bearer dbw_live_••••"
// 200 OK · application/json { "feed": "fin-cn-macro", "updated_at": "2026-11-04T09:12:40Z", "items": [ { "id": "fin-cn-macro:9f21c8d4", "title": "央行公开市场操作:7 天逆回购 1320 亿元", "published_at": "2026-11-04T01:00:00Z", "lang": "zh", "hash": "9f21c8d4" }, { /* ... */ } ], "next_cursor": null }
By the fifth, you're spending more time fixing parsers than building the product.
A renamed field, one extra wrapper div, a list that became infinite scroll. The script that worked last night returns an empty array this morning.
User agents get banned, rates have to come down, captchas show up, exit IPs need rotating. None of it is ever finished.
Dedupe, incremental fetch, timezones, encodings, nulls. Source twelve breaks in the same places as source one.
Pull when you need it, or let us push. Same token, same fields, same quota either way.
REST with Bearer auth, cursor paging, fixed fields. Full pull the first time, only new items after that.
GET /v1/feeds/{feed_id}/items ?since=2026-11-01T00:00:00Z &limit=100 &cursor=eyJvIjoyNDB9 GET /v1/sources # sources & status GET /v1/feeds/{id} # single source meta
Premium can point us at an endpoint. New items get POSTed there, signed, so you verify before you act on them.
# we send, you don't poll POST /hooks/dbw X-DBW-Signature: sha256=6b1f...e0 { "feed_id": "tech-github-releases", "items": [ { "id": "...", "title": "..." } ] }
The domains differ. The fields don't, so you write the parser once.
Policy releases, exchange filings, market snapshots, earnings calendar
Trending lists, public feeds, forums, public channels, podcast updates
Releases, trending repos, papers, changelogs, job boards
Trade press, government notices, independent blogs, newsletters
Dedupe, field alignment, timezones: done before an item reaches you.
id, title, url, summary, published_at, lang, hash. The same on every source, so merging needs no adapter layer.
Cursors are position-based, not page numbers, so deep paging doesn't get slower.
Ten outlets republish the same story; you get one. Matched on normalized URL plus a title fingerprint.
If a source edits or retracts a post, the item keeps a revision number and a deleted flag. It stays in history.
Every source reports its last fetch time and failure streak. Broken shows up as broken, before a user emails you.
Give us an endpoint. New items POST straight to it, signed so you can verify them.
No trial clock on the free tier. Upgrading changes three things: how many sources, how many calls a day, and how fast data lands. Endpoints stay the same.
One or two sources, to see if the data holds up
A product or internal tool maintained by one person
Production, with push delivery and history
Two minutes after signup you have a token. Your first feed can be live today.