SPDT/sapl - sapl - Gitea: Git with a cup of tea

5873 Commits

62 Branches

269 Tags

232 MiB

Tree: bea9d84373

Commit Graph

Author	SHA1	Message	Date
Edward Ribeiro	bea9d84373	Phase 5: X-Accel-Redirect for /media/, UA Redis deny list, per-path counters nginx: - /media/ proxied through Gunicorn (sapl_general rate limit) instead of direct alias — Django middleware now runs on every media request - /_accel/media/ internal location serves file bytes via X-Accel-Redirect sapl/base/media.py (new): - serve_media() gate: path traversal guard, auth redirect for documentos_privados/, per-path Redis counter, content-type metadata cache, X-Accel-Redirect response; falls back to Django serve() in DEBUG sapl/middleware/ratelimit.py: - RL_PATH_REQUESTS, RL_UA_BLOCKLIST, FILE_META_KEY constants - _incr_with_ttl() extracted to module level (reused by media.py) - Runtime UA deny list: _refresh_ua_blocklist() fetches rl:bot:ua:blocked SET from Redis (SMEMBERS, cached per worker, TTL=RATE_LIMITER_UA_BLOCKLIST_REFRESH); _is_redis_blocked_ua() tokenises UA and checks sha256 of each token sapl/settings.py: - RATE_LIMITER_UA_BLOCKLIST_REFRESH, MEDIA_PATH_COUNTER_TTL, MEDIA_FILE_CACHE_TTL added (all env-tunable via config()) plan/RATE_LIMITER_PLAN.md: - Key schema table updated; media file serving section added; decision flow documented; UA deny list seed section expanded Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>	1 month ago
Edward Ribeiro	b321b57b78	GeoIP offline build; Redis inspection tools; smart_rate/smart_key; cache KEY_PREFIX GeoIP (docker/Dockerfile): Remove at-build-time MaxMind download (required BuildKit secrets, caused cache-miss issues). Replace with COPY from docker/geoip/GeoLite2-ASN.mmdb (git-ignored binary). If absent, build succeeds with ASN blocking disabled. Add docker/geoip/update_geoip.sh — run before each build to refresh the database from MaxMind using MAXMIND_LICENSE_KEY from env or .env file. Redis inspection / synthetic test data: Add docker/scripts/redis_populate_test_data.py — injects synthetic rl:* entries into Redis DB1 to validate key schema and blocking thresholds without waiting for real traffic. Supports DRY_RUN and CLEAR modes. Add §4.5 (Redis CLI quick-reference + RedisInsight guide) to rate-limiter-v2.md. Auth-aware @ratelimit decorators (smart_rate / smart_key): All 51 @ratelimit decorators across 9 files used rate=RATE_LIMITER_RATE (35/m) regardless of authentication, silently over-throttling logged-in users compared to what RateLimitMiddleware allows (120/m). Add smart_key() and smart_rate() to sapl/middleware/ratelimit.py: - smart_key: user pk for authenticated requests, masked IP for anon - smart_rate: RATE_LIMITER_RATE_AUTHENTICATED (120/m) for auth, RATE_LIMITER_RATE (35/m) for anon — mirrors middleware thresholds Update all 51 decorators across crud/base.py + 8 view files. Remove now-unused RATE_LIMITER_RATE imports from those files. Cache KEY_PREFIX (settings.py): Change KEY_PREFIX from POD_NAMESPACE ("sapl") to f"cache:{POD_NAMESPACE}" so DB0 cache keys are unambiguously prefixed cache:{ns}:* — distinct from any future static or file cache key patterns. Update key schema table and code examples in rate-limiter-v2.md to match. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>	2 months ago

Author

SHA1

Message

Date

Edward Ribeiro

bea9d84373

Phase 5: X-Accel-Redirect for /media/, UA Redis deny list, per-path counters

nginx:
- /media/ proxied through Gunicorn (sapl_general rate limit) instead of
  direct alias — Django middleware now runs on every media request
- /_accel/media/ internal location serves file bytes via X-Accel-Redirect

sapl/base/media.py (new):
- serve_media() gate: path traversal guard, auth redirect for
  documentos_privados/, per-path Redis counter, content-type metadata
  cache, X-Accel-Redirect response; falls back to Django serve() in DEBUG

sapl/middleware/ratelimit.py:
- RL_PATH_REQUESTS, RL_UA_BLOCKLIST, FILE_META_KEY constants
- _incr_with_ttl() extracted to module level (reused by media.py)
- Runtime UA deny list: _refresh_ua_blocklist() fetches rl:bot:ua:blocked
  SET from Redis (SMEMBERS, cached per worker, TTL=RATE_LIMITER_UA_BLOCKLIST_REFRESH);
  _is_redis_blocked_ua() tokenises UA and checks sha256 of each token

sapl/settings.py:
- RATE_LIMITER_UA_BLOCKLIST_REFRESH, MEDIA_PATH_COUNTER_TTL,
  MEDIA_FILE_CACHE_TTL added (all env-tunable via config())

plan/RATE_LIMITER_PLAN.md:
- Key schema table updated; media file serving section added;
  decision flow documented; UA deny list seed section expanded

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

1 month ago

Edward Ribeiro

b321b57b78

GeoIP offline build; Redis inspection tools; smart_rate/smart_key; cache KEY_PREFIX

GeoIP (docker/Dockerfile):
  Remove at-build-time MaxMind download (required BuildKit secrets, caused
  cache-miss issues). Replace with COPY from docker/geoip/GeoLite2-ASN.mmdb
  (git-ignored binary). If absent, build succeeds with ASN blocking disabled.
  Add docker/geoip/update_geoip.sh — run before each build to refresh the
  database from MaxMind using MAXMIND_LICENSE_KEY from env or .env file.

Redis inspection / synthetic test data:
  Add docker/scripts/redis_populate_test_data.py — injects synthetic rl:*
  entries into Redis DB1 to validate key schema and blocking thresholds
  without waiting for real traffic. Supports DRY_RUN and CLEAR modes.
  Add §4.5 (Redis CLI quick-reference + RedisInsight guide) to rate-limiter-v2.md.

Auth-aware @ratelimit decorators (smart_rate / smart_key):
  All 51 @ratelimit decorators across 9 files used rate=RATE_LIMITER_RATE
  (35/m) regardless of authentication, silently over-throttling logged-in
  users compared to what RateLimitMiddleware allows (120/m).
  Add smart_key() and smart_rate() to sapl/middleware/ratelimit.py:
    - smart_key: user pk for authenticated requests, masked IP for anon
    - smart_rate: RATE_LIMITER_RATE_AUTHENTICATED (120/m) for auth,
      RATE_LIMITER_RATE (35/m) for anon — mirrors middleware thresholds
  Update all 51 decorators across crud/base.py + 8 view files.
  Remove now-unused RATE_LIMITER_RATE imports from those files.

Cache KEY_PREFIX (settings.py):
  Change KEY_PREFIX from POD_NAMESPACE ("sapl") to f"cache:{POD_NAMESPACE}"
  so DB0 cache keys are unambiguously prefixed cache:{ns}:* — distinct from
  any future static or file cache key patterns.
  Update key schema table and code examples in rate-limiter-v2.md to match.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

2 months ago

1 Commits (bea9d84373c8ff15e5cd1a17087e0d89c8a50f70)