SEC EDGAR rate limits and the User-Agent rule
EDGAR is free and has no key. It has two rules instead, and both are easy to break by accident in a way that looks like a bug on your side.
The two rules
Stay at or under ten requests per second, and send a User-Agent that identifies you with a working contact address. There is no key, no signup and no quota beyond that, which is unusual and worth not abusing.
The ceiling is per requester across ALL sec.gov hosts, not per host. www.sec.gov, data.sec.gov and efts.sec.gov draw on one budget, so a limiter instantiated per client quietly triples your real rate while every component looks compliant on its own. Share one limiter across the process.
We run at eight per second rather than ten. The headroom costs almost nothing on a nightly job and means a burst of retries cannot put us over.
import threading
import time
class SecRateLimiter:
"""One token bucket for every sec.gov request the process makes."""
def __init__(self, per_second: float = 8.0):
self._interval = 1.0 / per_second
self._lock = threading.Lock()
self._next = 0.0
def wait(self) -> None:
with self._lock:
now = time.monotonic()
delay = self._next - now
if delay > 0:
time.sleep(delay)
now += delay
self._next = now + self._intervalWhich failures are worth retrying
Retry transient TRANSPORT faults: a dropped connection, a timeout, a DNS blip. Do not retry on an HTTP status. A 403 means your User-Agent was rejected and a 404 means the document is not there, so retrying either just burns your rate budget to receive the same answer.
Retry idempotent methods only. GET and HEAD are safe to replay; nothing else is, and that distinction matters more the moment your pipeline grows a webhook.
A 404 on a daily index is not a failure at all. Weekends and market holidays have no index, and the current day's index does not exist until the following day.
Where to fetch what
The daily form index lists one row per dissemination for a single day, sorted by form type. It is the right entry point for "what was filed today", and it is small.
The bulk reference files are stable and worth caching rather than re-deriving: company_tickers.json and its exchange companion are both under a megabyte. Those sizes are knowable, which is exactly why they are safe to buffer whole.
A full-submission .txt is NOT knowable. Its size is decided by whoever filed it, so it is the one fetch that needs a cap rather than trust. That is its own guide.
Any fetch whose size a third party decides, buffered whole, in a memory-limited container, is the same bug waiting to happen. The three conditions together are the thing to look for.
Pinning a date range does not stop the pages shifting
This one falsified our own documentation, so it is worth stating plainly. Filings arrive BACKDATED. EDGAR publishes a day's index the following day, and any pipeline that rescans a rolling window will store filings dated inside a range you already walked.
Measured against our own API on 2026-08-18: with the range ending today, the total climbed across 22 pages and one record came back twice. Ending it yesterday still produced five duplicates, because that morning's pass stored a batch of filings all dated the previous day. With the range ending a week back, 25 pages and zero duplicates.
So do not rely on a pinned window for stability. Deduplicate on the accession number, which is unique per filing and holds whatever window you picked. The same advice applies to EDGAR directly and to us.
seen = set()
page = 1
while True:
response = requests.get(
"https://filingwire.io/risks/v1/events",
params={"since": "2026-08-01", "page": page, "page_size": 100},
headers={"X-API-Key": KEY}, timeout=30,
).json()
for event in response["items"]:
# Dedupe on the accession number, which is unique per filing and holds
# whatever window you picked. Pinning the date range does NOT make the
# pages stable, because filings arrive backdated.
if event["accession_number"] not in seen:
seen.add(event["accession_number"])
handle(event)
if not response["has_more"]:
break
page += 1Common questions
What is the SEC EDGAR rate limit?
Ten requests per second, counted across every sec.gov host together rather than per host. There is no key and no registration; fair access is enforced by that ceiling and by the User-Agent requirement.
What User-Agent does SEC EDGAR require?
One that identifies you and carries a working contact address, for example a company or project name followed by an email. Requests with a generic or absent User-Agent are refused with 403, which reads like a block rather than a configuration mistake.
Why am I getting 403 from sec.gov?
Almost always the User-Agent. It is not worth retrying, because the answer will not change until the header does. The other cause is sustained traffic over the ceiling, which is why one shared limiter beats one per component.
Everything above is real work, and it does not stop.
The parsing is the easy half. The rest is the daily pass, the rescan window, the filings that arrive backdated and the ones that never produce a row. We run it and serve the result as JSON, and the free key needs no card. If you would rather own the pipeline, the code above is what we would write.