SPDT/sapl - sapl - Gitea: Git with a cup of tea

Author	SHA1	Message	Date
Edward Ribeiro	d4738a20a9	Raise rate limits and split nginx zones for legitimate traffic SAPL pages fire 12-45 parallel requests; the old 30r/m nginx zone and 35/m Django threshold blocked normal navigation. Key changes: nginx (nginx.conf / sapl.conf / start.sh): - Split sapl_general (30r/m) into four dedicated zones: sapl_general 90r/m burst=180 (HTML pages) sapl_media 180r/m burst=180 (/media/ — own bucket, no longer drains general) sapl_api 60r/m burst=120 (/api/ — quota layer is the real constraint) sapl_heavy 10r/m burst=20 (/relatorios/ — unchanged, nodelay kept) - /media/ and /api/ location blocks now reference their own zones Django (settings.py): - RATE_LIMITER_RATE: 35/m → 120/m - RATE_LIMITER_RATE_AUTHENTICATED: 120/m → 240/m - RATE_LIMIT_404_THRESHOLD: 10 → 20 - API_QUOTA_ANON_DAILY: 50 → 500 / weekly 350 → 3500 - API_QUOTA_AUTH_DAILY: 1000 → 5000 / weekly 7000 → 35000 Middleware (ratelimit.py): - Authenticated users no longer receive a persistent 300s block key on rate breach — they get 429 for the over-limit request and the window resets naturally after 60s. A 5-minute lockout is wrong for a logged-in user who clicked too fast. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>	3 weeks ago
Edward Ribeiro	9140f3e4b8	Phase 1-4: Redis infra, rate limiter middleware, cache layer, nginx hardening docker/Dockerfile: - GeoIP offline build with MaxMind secret; optional build args for graphviz, poppler, psql client; envsubst for nginx burst vars docker/docker-compose.yaml: - saplredis service (redis:7-alpine, allkeys-lru, 512 MB) - REDIS_URL + CACHE_BACKEND wired into sapl service docker/startup_scripts/start.sh: - configure_redis_cache(): builds CACHES dict, sets REDIS_CACHE waffle switch, falls back to file cache gracefully - POD_NAMESPACE resolution (k8s Downward API → hostname fallback) - DATABASE_URL exported before migrate docker/k8s/redis/ (moved from docker/k8s/): - redis-configmap.yaml, redis-deployment.yaml, redis-service.yaml - ClusterIP service on port 6379, sapl-redis namespace docker/k8s/sapl-k8s.yaml: - REDIS_URL env var injected; app.kubernetes.io/name=sapl label for fleet-wide discovery sapl/middleware/test_ratelimiter.py: - Unit tests for RateLimitMiddleware with mocked Redis scripts/test_ratelimiter.py: - CLI smoke-test: fires N requests and reports first 429 Removed: rate-limiter-v2.md (content migrated to plan/RATE_LIMITER_PLAN.md), scripts/test_ratelimiter.sh (replaced by .py), docker/k8s/README.md (merged into plan/RATE_LIMITER_PLAN.md), docker/scripts/redis_populate_test_data.py (renamed to redis_inject_test_data.py) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>	1 month ago
Edward Ribeiro	45baa27d7f	Fix configure_redis_cache overwriting operator-set waffle switch on restart waffle_switch REDIS_CACHE off --create always wrote 'off' even when the operator had previously enabled it, making docker compose restart always reset the switch and leaving Redis cache permanently disabled. Replace with a Django shell get_or_create call that only inserts the row with active=False on first boot; subsequent restarts leave the existing value untouched. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>	2 months ago
Edward Ribeiro	c1d97e2c99	Fix DATABASE_URL not exported before migrate_db in start.sh wait_for_pg() set DATABASE_URL via := default syntax but never exported it, so Python child processes (manage.py migrate, waffle_switch) could not read it from the environment. The .env file does not exist yet at that point — write_env_file runs later — so decouple raised UndefinedValueError. Add 'export DATABASE_URL' immediately after the default is resolved in wait_for_pg(), which is the earliest point in the startup sequence where the value is known and already used by configure_pg_timezone right after. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>	2 months ago
Edward Ribeiro	bbc9323c4a	Fix 5 code-quality issues in RateLimitMiddleware 1. user_id: str(request.user.pk) — pk is int, lower()/strip() were no-ops 2. Redis key constants: RL_IP_REQUESTS, RL_IP_BLOCKED, RL_USER_REQUESTS, RL_USER_BLOCKED, RL_NS_WINDOW — no more inline f-string literals 3. Tenant namespace: _NAMESPACE resolved once at module load from POD_NAMESPACE env var (K8s Downward API) → service-account namespace file → 'global' fallback. No per-request getattr(request, 'tenant'). 4. KEY_PREFIX in CACHES['default'] set to POD_NAMESPACE (e.g. patobranco-pr) so each tenant's cache keys are isolated in shared Redis. 5. Logger extra: replaced getattr(request, 'tenant', 'unknown') with _NAMESPACE (the actual resolved constant). settings.py: add POD_NAMESPACE = config('POD_NAMESPACE', default='sapl'); use it as KEY_PREFIX. start.sh: add resolve_pod_namespace() (Downward API → SA file → fallback); call it before resolve_redis_url(); write POD_NAMESPACE into .env. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>	2 months ago
Edward Ribeiro	911b13540d	Phase 1: shared Redis pod — Django dual-backend cache + startup wiring django/settings.py: - REDIS_URL / CACHE_BACKEND env vars read at startup (written by start.sh) - CACHES['default'] (DB0, KEY_PREFIX='sapl') switches between django-redis and FileBasedCache transparently; IGNORE_EXCEPTIONS=True for graceful degradation on Redis failure - CACHES['ratelimit'] (DB1, no prefix) for cross-pod rate-limit counters - RATELIMIT_USE_CACHE = 'ratelimit' - Connection pool capped at 6/worker (1,200 pods × 2 workers × 6 = 14,400 peak connections; maxclients=20,000 gives 40% headroom) start.sh: - resolve_redis_url(): reads REDIS_URL from local namespace Secret (envFrom) or falls back to global cluster Secret via k8s API - configure_redis_cache(): ensures REDIS_CACHE waffle switch row exists (off) - resolve_cache_backend(): reads waffle switch; sets CACHE_BACKEND=redis\|file - wait_for_redis(): blocks until Redis reachable; falls back gracefully - write_env_file() now persists REDIS_URL + CACHE_BACKEND into pod .env k8s manifests (docker/k8s/): - redis-configmap.yaml: no persistence, allkeys-lru, maxmemory=5gb, maxclients=20000, activedefrag, 4 databases - redis-deployment.yaml: redis:7-alpine, 1 replica, liveness/readiness probes, 1Gi request / 6Gi limit - redis-service.yaml: ClusterIP on port 6379 requirements: add django-redis==5.4.0 Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>	2 months ago
Edward Ribeiro	8df2861799	Phase 0 hardening: nginx GeoIP2, rate limits, Gunicorn tuning, N+1 fix - nginx: sendfile on, tcp_nopush, reduced keepalive/proxy timeouts - nginx: GeoIP2 ASN-based bot blocking (cloud providers + known scrapers) - nginx: UA blocklist (GPTBot, ClaudeBot, Chrome/98.0.4758 impersonator, etc.) - nginx: rate-limit zones (30r/m general, 10r/m heavy/relatorios), 429/500 error pages - nginx: proper ETags + Cache-Control on /media/ to stop 30GB logo re-transfers - Dockerfile: install libnginx-mod-http-geoip2; download GeoLite2-ASN.mmdb via BuildKit secret (key never baked into image layers); ARG GEOIP_CACHE_BUST for forced re-download without --no-cache - Gunicorn: workers 3->2, threads 8->4, timeout 300->120, max_memory 300->400MB - Django: FILE_UPLOAD_MAX_MEMORY_SIZE=2MB, FILE_UPLOAD_TEMP_DIR for large uploads - relatorios/views.py: fix N+1 in get_etiqueta_protocolos with bulk-fetch MateriaLegislativa + DocumentoAdministrativo using select_related + dict lookups - Add robots.txt, 429.html, 500.html static pages - docker-compose.yaml: use sapl:local for local dev - docker/README.md: build instructions with MAXMIND_LICENSE_KEY - rate-limiter-v2.md: canonical planning document (Architecture through Phase 5) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>	2 months ago
Edward Ribeiro	3323f2779f	Fix recibo proposição e adiciona rate limiter em matéria e norma	8 months ago
Edward Ribeiro	b49c7916a3	Fix read-only mount on k8s	9 months ago
Edward	6ac5e419e3	Atualização da imagem base Docker (#3787 ) Update de imagem based Docker and libs Python Co-authored-by: Edward <9326037+edwardoliveira@users.noreply.github.com>	9 months ago