SAPL pages fire 12-45 parallel requests; the old 30r/m nginx zone and
35/m Django threshold blocked normal navigation. Key changes:
nginx (nginx.conf / sapl.conf / start.sh):
- Split sapl_general (30r/m) into four dedicated zones:
sapl_general 90r/m burst=180 (HTML pages)
sapl_media 180r/m burst=180 (/media/ — own bucket, no longer drains general)
sapl_api 60r/m burst=120 (/api/ — quota layer is the real constraint)
sapl_heavy 10r/m burst=20 (/relatorios/ — unchanged, nodelay kept)
- /media/ and /api/ location blocks now reference their own zones
Django (settings.py):
- RATE_LIMITER_RATE: 35/m → 120/m
- RATE_LIMITER_RATE_AUTHENTICATED: 120/m → 240/m
- RATE_LIMIT_404_THRESHOLD: 10 → 20
- API_QUOTA_ANON_DAILY: 50 → 500 / weekly 350 → 3500
- API_QUOTA_AUTH_DAILY: 1000 → 5000 / weekly 7000 → 35000
Middleware (ratelimit.py):
- Authenticated users no longer receive a persistent 300s block key on
rate breach — they get 429 for the over-limit request and the window
resets naturally after 60s. A 5-minute lockout is wrong for a logged-in
user who clicked too fast.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Multiple councilmembers behind a shared NAT IP were getting 429s during
live votes because nginx burst exhaustion fired before Django's per-user
rate counter could run. Exempt /voto-individual/ and /sessao/<pk>/ from
nginx limit_req; mirror in RATE_LIMIT_BYPASS_PATHS as defense-in-depth.
Add docs/rate-limiter-incidents.md with root-cause analysis, architecture
diagrams, tradeoff discussion, and pending investigations for the
PatoBranco-PR 2026-05-06 incident.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- nginx: exempt /painel/<pk>/dados from rate limiting (polling endpoint,
will become WebSocket); dedicated location block with no limit_req
- ratelimit.py: bypass RATE_LIMIT_BYPASS_PATHS paths before _evaluate;
add layer=django to block log; increment daily Redis metrics counter
rl:metrics:{ns}:{date}:blocked:{reason} (TTL 8 days) on every block
- ratelimit.py: add quiltbot and AwarioBot to BOT_UA_FRAGMENTS
- ratelimit.py: fix _is_suspicious_headers to require missing UA before blocking
- settings: add RATE_LIMIT_BYPASS_PATHS with /painel/<pk>/dados pattern
- plan: extend UA blocklist SADD seed command with missing bot tokens
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Renames /private/media/ to /internal/media/ in nginx and serve_media().
Adds Content-Type and Content-Disposition to the X-Accel-Redirect response.
Replaces manual file reads in proposicao_texto and doc_texto_integral with
redirects to the media URL, removing the unused get_mime_type_from_file_extension helper.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- ConditionalGetMiddleware added to MIDDLEWARE (ETag/304 for all views)
- @condition(etag_func, last_modified_func) on MateriaLegislativa and
NormaJuridica detail views — skips view execution on cache hit via
data_ultima_atualizacao (auto_now=True) as freshness signal
- nginx /static/: expires 90m + Cache-Control public, max-age=5400
- nginx: removed upload-endpoint special-casing (location ~* ^/(protocoloadm/criar-protocolo|...))
- plan/RATE-LIMITER-PLAN.md updated to reflect all Phase 7 changes
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
nginx:
- /media/ proxied through Gunicorn (sapl_general rate limit) instead of
direct alias — Django middleware now runs on every media request
- /_accel/media/ internal location serves file bytes via X-Accel-Redirect
sapl/base/media.py (new):
- serve_media() gate: path traversal guard, auth redirect for
documentos_privados/, per-path Redis counter, content-type metadata
cache, X-Accel-Redirect response; falls back to Django serve() in DEBUG
sapl/middleware/ratelimit.py:
- RL_PATH_REQUESTS, RL_UA_BLOCKLIST, FILE_META_KEY constants
- _incr_with_ttl() extracted to module level (reused by media.py)
- Runtime UA deny list: _refresh_ua_blocklist() fetches rl:bot:ua:blocked
SET from Redis (SMEMBERS, cached per worker, TTL=RATE_LIMITER_UA_BLOCKLIST_REFRESH);
_is_redis_blocked_ua() tokenises UA and checks sha256 of each token
sapl/settings.py:
- RATE_LIMITER_UA_BLOCKLIST_REFRESH, MEDIA_PATH_COUNTER_TTL,
MEDIA_FILE_CACHE_TTL added (all env-tunable via config())
plan/RATE_LIMITER_PLAN.md:
- Key schema table updated; media file serving section added;
decision flow documented; UA deny list seed section expanded
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add AnonCachePageMixin (sapl/middleware/page_cache.py) that stores full
view responses in the default Redis cache for anonymous (unauthenticated)
GET requests only. Authenticated users always bypass the cache so CSRF
tokens and user-specific UI controls are never served stale.
Applied to:
- ParlamentarCrud.ListView / DetailView — TTL 600 s (changes each term)
- AudienciaCrud.ListView — TTL 120 s (hearings added infrequently)
- ComissaoCrud.ListView — TTL 300 s (committees change rarely)
Also:
- Add PAGE_CACHE_TTL_LIST/DETAIL/STABLE settings (env-configurable)
- Add bingbot + SERankingBacklinksBot to nginx UA blocklist (were already
in BOT_UA_FRAGMENTS / robots.txt; nginx map was the only gap)
- Remove unused ratelimit/method_decorator/RATE_LIMITER_RATE imports from
audiencia/views.py that crept in during Phase 2
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Remove a analise de vínculos cíclicos na construção inicial
do form do filterset.
- O item anterior deve resolver o timeout causado na abertura da
anexação em lote, no entanto os timeouts do nginx e gunicorn foram
aumentados.
Edward Ribeiro
Edward
Leandro Roberto
Vinícius Cantuária