SPDT/sapl - sapl - Gitea: Git with a cup of tea

Author	SHA1	Message	Date
Edward Ribeiro	39593f2284	Remove whitelist references from rate limiter plan docs continuous-integration/drone/push Build is passing Details Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>	4 weeks ago
Edward Ribeiro	74d5873426	Fix Redis configmap inline comment, clean cache key format, add blocked-IP scan to plan - redis-configmap: move inline comment to its own line (Redis fatal parse error) - settings: add CACHE_MIDDLEWARE_KEY_PREFIX='p' to remove double-dot in cache_page keys - settings: monkey-patch _i18n_cache_key_suffix to strip pt-br/timezone suffix from keys - ratelimit.py, settings: update example namespace from patobranco-pr to sapl31demo-df - robots.txt: add AwarioSmartBot block - plan: add rl:ip:*:blocked scan commands with TTL/value output Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>	2 months ago
Edward Ribeiro	bea9d84373	Phase 5: X-Accel-Redirect for /media/, UA Redis deny list, per-path counters nginx: - /media/ proxied through Gunicorn (sapl_general rate limit) instead of direct alias — Django middleware now runs on every media request - /_accel/media/ internal location serves file bytes via X-Accel-Redirect sapl/base/media.py (new): - serve_media() gate: path traversal guard, auth redirect for documentos_privados/, per-path Redis counter, content-type metadata cache, X-Accel-Redirect response; falls back to Django serve() in DEBUG sapl/middleware/ratelimit.py: - RL_PATH_REQUESTS, RL_UA_BLOCKLIST, FILE_META_KEY constants - _incr_with_ttl() extracted to module level (reused by media.py) - Runtime UA deny list: _refresh_ua_blocklist() fetches rl:bot:ua:blocked SET from Redis (SMEMBERS, cached per worker, TTL=RATE_LIMITER_UA_BLOCKLIST_REFRESH); _is_redis_blocked_ua() tokenises UA and checks sha256 of each token sapl/settings.py: - RATE_LIMITER_UA_BLOCKLIST_REFRESH, MEDIA_PATH_COUNTER_TTL, MEDIA_FILE_CACHE_TTL added (all env-tunable via config()) plan/RATE_LIMITER_PLAN.md: - Key schema table updated; media file serving section added; decision flow documented; UA deny list seed section expanded Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>	2 months ago
Edward Ribeiro	b321b57b78	GeoIP offline build; Redis inspection tools; smart_rate/smart_key; cache KEY_PREFIX GeoIP (docker/Dockerfile): Remove at-build-time MaxMind download (required BuildKit secrets, caused cache-miss issues). Replace with COPY from docker/geoip/GeoLite2-ASN.mmdb (git-ignored binary). If absent, build succeeds with ASN blocking disabled. Add docker/geoip/update_geoip.sh — run before each build to refresh the database from MaxMind using MAXMIND_LICENSE_KEY from env or .env file. Redis inspection / synthetic test data: Add docker/scripts/redis_populate_test_data.py — injects synthetic rl:* entries into Redis DB1 to validate key schema and blocking thresholds without waiting for real traffic. Supports DRY_RUN and CLEAR modes. Add §4.5 (Redis CLI quick-reference + RedisInsight guide) to rate-limiter-v2.md. Auth-aware @ratelimit decorators (smart_rate / smart_key): All 51 @ratelimit decorators across 9 files used rate=RATE_LIMITER_RATE (35/m) regardless of authentication, silently over-throttling logged-in users compared to what RateLimitMiddleware allows (120/m). Add smart_key() and smart_rate() to sapl/middleware/ratelimit.py: - smart_key: user pk for authenticated requests, masked IP for anon - smart_rate: RATE_LIMITER_RATE_AUTHENTICATED (120/m) for auth, RATE_LIMITER_RATE (35/m) for anon — mirrors middleware thresholds Update all 51 decorators across crud/base.py + 8 view files. Remove now-unused RATE_LIMITER_RATE imports from those files. Cache KEY_PREFIX (settings.py): Change KEY_PREFIX from POD_NAMESPACE ("sapl") to f"cache:{POD_NAMESPACE}" so DB0 cache keys are unambiguously prefixed cache:{ns}:* — distinct from any future static or file cache key patterns. Update key schema table and code examples in rate-limiter-v2.md to match. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>	2 months ago
Edward Ribeiro	e85f5e0847	Fix remaining get_client_ip stale imports; split Phase 5 to work_queues.md Import fixes (all three imported get_client_ip/ratelimit_ip from sapl.utils which no longer exports them — causing the ImportError at startup): - sapl/materia/forms.py: move get_client_ip to sapl.middleware.ratelimit - sapl/materia/views.py: move get_client_ip + ratelimit_ip; keep RATE_LIMITER_RATE in sapl.settings (used by @ratelimit decorators) - sapl/base/views.py: same pattern as materia/views.py Docs: - rate-limiter-v2.md: remove Phase 5 section (§8); renumber Open Questions to §8; update Table of Contents - work_queues.md (new): Async PDF via Celery + Django Channels WebSocket voting panel, with full context, Redis B topology rationale, k8s manifest list, and open questions. Planned start: after rate-limiter-2026 is stable. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>	2 months ago
Edward Ribeiro	8df2861799	Phase 0 hardening: nginx GeoIP2, rate limits, Gunicorn tuning, N+1 fix - nginx: sendfile on, tcp_nopush, reduced keepalive/proxy timeouts - nginx: GeoIP2 ASN-based bot blocking (cloud providers + known scrapers) - nginx: UA blocklist (GPTBot, ClaudeBot, Chrome/98.0.4758 impersonator, etc.) - nginx: rate-limit zones (30r/m general, 10r/m heavy/relatorios), 429/500 error pages - nginx: proper ETags + Cache-Control on /media/ to stop 30GB logo re-transfers - Dockerfile: install libnginx-mod-http-geoip2; download GeoLite2-ASN.mmdb via BuildKit secret (key never baked into image layers); ARG GEOIP_CACHE_BUST for forced re-download without --no-cache - Gunicorn: workers 3->2, threads 8->4, timeout 300->120, max_memory 300->400MB - Django: FILE_UPLOAD_MAX_MEMORY_SIZE=2MB, FILE_UPLOAD_TEMP_DIR for large uploads - relatorios/views.py: fix N+1 in get_etiqueta_protocolos with bulk-fetch MateriaLegislativa + DocumentoAdministrativo using select_related + dict lookups - Add robots.txt, 429.html, 500.html static pages - docker-compose.yaml: use sapl:local for local dev - docker/README.md: build instructions with MAXMIND_LICENSE_KEY - rate-limiter-v2.md: canonical planning document (Architecture through Phase 5) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>	2 months ago