From a33fcc2fe624ea9b4a4693aeac37d8ef0cc9fd3e Mon Sep 17 00:00:00 2001 From: Edward Oliveira Date: Mon, 11 May 2026 17:47:32 -0300 Subject: [PATCH] Add API-specific rate limiter and remove emergency block middleware MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - Delete ApiEmergencySameSiteOnlyMiddleware (api_emergency_block.py); replace with permanent _handle_api logic inside RateLimitMiddleware. - _handle_api decision chain: OPTIONS pass → same-origin pass → global IP block check → API-specific block check → quota → API per-minute counter → anon pass / auth _evaluate. - New Redis keys: rl:api:ip::reqs, rl:api:ip::blocked, rl:index:api_blocked_ips. Global rl:ip::blocked is never written because of /api/ abuse — prevents NAT lockout. - _is_same_origin: strips port, lowercases, checks Origin first then Referer (sequential, not OR — wrong Origin blocks even if Referer matches). - Five new settings: API_RATE_LIMIT_{ENABLED,THRESHOLD,WINDOW_SECONDS, BLOCK_SECONDS,SAME_ORIGIN_BYPASS} with safe defaults. - 16 new tests; _make_middleware extended with explicit setting values. - RATE-LIMITER-PLAN.md updated with new key schema rows and _handle_api section. Co-Authored-By: Claude Sonnet 4.6 --- plan/RATE-LIMITER-PLAN.md | 67 +++++++++ sapl/middleware/ratelimit.py | 180 ++++++++++++++++++++---- sapl/middleware/test_ratelimiter.py | 205 +++++++++++++++++++++++++++- sapl/settings.py | 9 +- 4 files changed, 433 insertions(+), 28 deletions(-) diff --git a/plan/RATE-LIMITER-PLAN.md b/plan/RATE-LIMITER-PLAN.md index f00e55aaf..80d1d5532 100644 --- a/plan/RATE-LIMITER-PLAN.md +++ b/plan/RATE-LIMITER-PLAN.md @@ -1207,6 +1207,9 @@ Redis PDF caching would solve "high request volume reaching the file layer" — | 1 | API weekly quota (anon) | `quota:{ns}:weekly:{week}:ip:{ip}` | 7 d | 3 500 (`API_QUOTA_ANON_WEEKLY`) | `QUOTA_IP_WEEKLY` | | 1 | API daily quota (auth) | `quota:{ns}:daily:{date}:user:{uid}` | 24 h | 5 000 (`API_QUOTA_AUTH_DAILY`) | `QUOTA_USER_DAILY` | | 1 | API weekly quota (auth) | `quota:{ns}:weekly:{week}:user:{uid}` | 7 d | 35 000 (`API_QUOTA_AUTH_WEEKLY`) | `QUOTA_USER_WEEKLY` | +| 1 | API IP rate counter (anon external) | `rl:api:ip:{ip}:reqs` | 60 s (`API_RATE_LIMIT_WINDOW_SECONDS`) | 60 (`API_RATE_LIMIT_THRESHOLD`) | `RL_API_IP_REQUESTS` | +| 1 | API IP block marker (anon external) | `rl:api:ip:{ip}:blocked` | 300 s (`API_RATE_LIMIT_BLOCK_SECONDS`) | — | `RL_API_IP_BLOCKED` | +| 1 | API blocked-IP ZSET index | `rl:index:api_blocked_ips` | permanent ZSET, score=expiry ts | — | `RL_INDEX_API_BLOCKED_IPS` | | 2 | Django Channels | `channels:*` | session TTL | — | *Future* | ### What each counter catches — and misses @@ -1332,6 +1335,70 @@ bots that impersonate a valid browser UA completely (no known fragment to match) --- +## /api/ rate limiting — `_handle_api` (2026-05-11) + +### Problem + +Three concurrent problems made the old anonymous-/api/-passes-immediately design +insufficient: + +1. **SAPL itself polls /api/** from the browser (same-origin). It must not be counted or blocked. +2. **Legitimate user scripts behind NAT** poll `/api/` aggressively. The old design had no per-minute cap; quota (daily/weekly) was the only gate, and quota exhaustion wrote `rl:ip::blocked`, locking every person behind the same NAT out of the entire application. +3. **Bots** hammer `/api/` with no constraint other than the nginx `sapl_api` zone (60 r/m burst 120). + +### Solution — `_handle_api` + +`RateLimitMiddleware.__call__` delegates all `/api/` requests to `_handle_api`, which applies a separate, scoped decision chain: + +| Step | Condition | Action | +|------|-----------|--------| +| 1 | `OPTIONS` method | Pass — CORS preflight must never be blocked | +| 2 | Same-origin (`_is_same_origin`) | Pass — SAPL's own browser polling; no counter | +| 3 | `rl:ip::blocked` exists | 429 `global_ip_blocked` — global block also covers `/api/` | +| 4 | `rl:api:ip::blocked` exists | 429 `api_ip_blocked` — API-only block | +| 5 | Daily/weekly quota exceeded | 429 `quota_daily` / `quota_weekly` (unchanged) | +| 6 | Anon: API counter ≥ threshold | Write `rl:api:ip::blocked`; 429 `api_threshold_exceeded` | +| 6 | Anon: under threshold | Pass | +| 7 | Authenticated | Delegate to `_evaluate` (per-user counter unchanged) | + +**Key invariant**: `rl:ip::blocked` is **never written** because of `/api/` abuse. +`rl:api:ip::blocked` blocks only `/api/` — page requests from the same NAT continue. + +### Same-origin detection — `_is_same_origin` + +Replaces `ApiEmergencySameSiteOnlyMiddleware._came_from_same_host` (deleted). + +| Aspect | Emergency block | `_is_same_origin` | +|--------|-----------------|-------------------| +| Normalization | strip port, lowercase (both sides) | same | +| Origin + Referer | `origin_match OR referer_match` | sequential: Origin first, Referer only if Origin absent | +| Wrong Origin with matching Referer | pass (Referer wins) | block (explicit wrong Origin = cross-origin) | +| Both absent | block | block | + +The sequential check is stricter and matches the spec: an explicit wrong `Origin` +header means the browser knows this is cross-origin, regardless of what `Referer` says. + +### Settings + +| Setting | Env var | Default | Purpose | +|---------|---------|---------|---------| +| `API_RATE_LIMIT_ENABLED` | same | `True` | Master switch; set False to revert to quota-only | +| `API_RATE_LIMIT_THRESHOLD` | same | `60` | Requests per window before API block | +| `API_RATE_LIMIT_WINDOW_SECONDS` | same | `60` | Counter TTL (seconds) | +| `API_RATE_LIMIT_BLOCK_SECONDS` | same | `300` | `rl:api:ip::blocked` TTL | +| `API_RATE_LIMIT_SAME_ORIGIN_BYPASS` | same | `True` | Disable to test without same-origin pass | + +### Files changed + +| File | Change | +|------|--------| +| `sapl/middleware/api_emergency_block.py` | Deleted | +| `sapl/settings.py` | Removed `ApiEmergencySameSiteOnlyMiddleware` from `MIDDLEWARE`; added 5 new `API_RATE_LIMIT_*` settings | +| `sapl/middleware/ratelimit.py` | Added `RL_API_IP_REQUESTS`, `RL_API_IP_BLOCKED`, `RL_INDEX_API_BLOCKED_IPS` constants; added `_is_same_origin`; extended `__init__`; added `_handle_api`, `_api_block_response`; refactored `__call__` | +| `sapl/middleware/test_ratelimiter.py` | Extended `_make_middleware`; added 16 new tests | + +--- + ## Dynamic page caching **Goal**: Eliminate ORM queries for anonymous bot requests on list views. diff --git a/sapl/middleware/ratelimit.py b/sapl/middleware/ratelimit.py index c44738abf..219aefb89 100644 --- a/sapl/middleware/ratelimit.py +++ b/sapl/middleware/ratelimit.py @@ -2,9 +2,16 @@ RateLimitMiddleware — cross-pod rate limiting backed by shared Redis. Decision flow (per request): - 0. /api/ path AND consumer daily/weekly quota exceeded? → 429 - Anonymous /api/ (quota not exceeded): pass immediately — no IP counter, - no block key. nginx sapl_api zone (60r/m) is the burst gate. + /api/ paths — handled by _handle_api: + 0a. OPTIONS? → pass (CORS preflight must never be blocked) + 0b. Same-origin? → pass (SAPL's own browser polling) + 0c. rl:ip::blocked? → 429 (global block also covers /api/) + 0d. rl:api:ip::blocked? → 429 (API-only block) + 0e. Daily/weekly quota exceeded? → 429 + 0f. Anon + API threshold exceeded? → SET rl:api:ip::blocked, 429 + (never writes rl:ip::blocked) + 0g. Auth: falls through to _evaluate (per-user counter) + Non-/api/ paths: 1. Known bot UA? → 429 (Python list — substring match) 1b. Redis UA deny list? → 429 (runtime SET — token hash match, refreshed every 60 s) 2. Anonymous AND IP in blocked set? → 429 (authenticated users skip — have per-user limit at 3c) @@ -68,6 +75,11 @@ RL_METRICS_BLOCKED = 'rl:metrics:{ns}:{date}:blocked:{reason}' # daily counter RL_INDEX_BLOCKED_IPS = 'rl:index:blocked_ips' RL_INDEX_BLOCKED_USERS = 'rl:index:blocked_users' +# API-specific rate limit keys — scope limited to /api/, never written by non-/api/ paths. +RL_API_IP_REQUESTS = 'rl:api:ip:{ip}:reqs' +RL_API_IP_BLOCKED = 'rl:api:ip:{ip}:blocked' +RL_INDEX_API_BLOCKED_IPS = 'rl:index:api_blocked_ips' + # --------------------------------------------------------------------------- # API quota keys — per-consumer, per-day/week, tenant-scoped. # Consumer identity: authenticated users by uid, anonymous by masked IP. @@ -190,6 +202,41 @@ def smart_rate(group, request): return settings.RATE_LIMITER_RATE +def _is_same_origin(request): + """ + Return True if Origin or Referer header matches the current SAPL host. + Strips port and lowercases both sides before comparing — DNS is case-insensitive + and reverse proxies may expose a different port than the browser sees. + Checks Origin first; falls back to Referer only when Origin is absent. + Returns False when both headers are absent. + """ + from urllib.parse import urlparse + + def _normalize(host): + return host.lower().split(':', 1)[0].strip() + + try: + host = _normalize(request.get_host()) + except Exception: + return False + + origin = request.META.get('HTTP_ORIGIN', '') + if origin: + try: + return _normalize(urlparse(origin).netloc) == host + except ValueError: + return False + + referer = request.META.get('HTTP_REFERER', '') + if referer: + try: + return _normalize(urlparse(referer).netloc) == host + except ValueError: + return False + + return False + + def _is_suspicious_headers(request): """Real browsers send Accept-Language + Accept; bots frequently omit them.""" missing = sum([ @@ -265,6 +312,11 @@ class RateLimitMiddleware: self.api_quota_anon_weekly = settings.API_QUOTA_ANON_WEEKLY self.api_quota_auth_daily = settings.API_QUOTA_AUTH_DAILY self.api_quota_auth_weekly = settings.API_QUOTA_AUTH_WEEKLY + self.api_rate_limit_enabled = getattr(settings, 'API_RATE_LIMIT_ENABLED', True) + self.api_threshold = getattr(settings, 'API_RATE_LIMIT_THRESHOLD', 60) + self.api_window = getattr(settings, 'API_RATE_LIMIT_WINDOW_SECONDS', 60) + self.api_block_seconds = getattr(settings, 'API_RATE_LIMIT_BLOCK_SECONDS', 300) + self.api_same_origin_bypass = getattr(settings, 'API_RATE_LIMIT_SAME_ORIGIN_BYPASS', True) logger.info( '[RATELIMIT] anon=%s auth=%s bot=%s allowlist=%s bypass_paths=%s', settings.RATE_LIMITER_RATE, @@ -280,35 +332,18 @@ class RateLimitMiddleware: settings.API_QUOTA_AUTH_DAILY, settings.API_QUOTA_AUTH_WEEKLY, ) + logger.info( + '[API RATE LIMIT] enabled=%s threshold=%s window=%ss block=%ss same_origin_bypass=%s', + self.api_rate_limit_enabled, self.api_threshold, self.api_window, + self.api_block_seconds, self.api_same_origin_bypass, + ) def __call__(self, request): if any(p.match(request.path) for p in self._bypass_paths): return self.get_response(request) if request.path.startswith('/api/'): - exceeded = self._check_api_quota(request) - if exceeded: - ip = get_client_ip(request) - logger.warning( - 'quota_exceeded window=%s ip=%s path=%s namespace=%s', - exceeded, ip, request.path, _NAMESPACE, - extra={'ua': request.META.get('HTTP_USER_AGENT', '')}, - ) - self._inc_block_metric(f'quota_{exceeded}') - response = HttpResponse(status=429) - response['Retry-After'] = 86400 - response['X-RateLimit-Reason'] = f'quota_{exceeded}' - return response - - # Anonymous /api/ requests: quota + nginx sapl_api zone are the only - # controls. Skip _evaluate so anonymous API traffic never increments - # the global IP counter or writes a block key — a misbehaving script - # behind a NAT must not lock out the org's page requests. - # Authenticated /api/ falls through to _evaluate_authenticated normally - # (per-user counter, NAT-safe). - user = getattr(request, 'user', None) - if not (user and user.is_authenticated): - return self.get_response(request) + return self._handle_api(request) decision = self._evaluate(request) if decision['action'] == 'block': @@ -337,6 +372,99 @@ class RateLimitMiddleware: self._handle_not_found(request, decision['ip']) return response + # ------------------------------------------------------------------ + # /api/ handling + # ------------------------------------------------------------------ + + def _api_block_response(self, reason, retry_after=None): + from django.http import JsonResponse + if retry_after is None: + retry_after = self.api_block_seconds + resp = JsonResponse( + {'detail': 'API rate limit exceeded. Please reduce polling frequency.', + 'retry_after_seconds': retry_after}, + status=429, + ) + resp['Retry-After'] = retry_after + resp['X-RateLimit-Reason'] = reason + return resp + + def _handle_api(self, request): + # 1. OPTIONS preflight — CORS must never be blocked + if request.method == 'OPTIONS': + return self.get_response(request) + + # 2. Same-origin (SAPL's own polling) — no counter, no block + if self.api_same_origin_bypass and _is_same_origin(request): + return self.get_response(request) + + ip = get_client_ip(request) + + # 3. Global IP block also covers /api/ + if self._rl_cache.get(RL_IP_BLOCKED.format(ip=ip)): + logger.warning( + 'api_rate_limit_block reason=global_ip_blocked ip=%s path=%s user_agent=%s', + ip, request.path, request.META.get('HTTP_USER_AGENT', ''), + ) + self._inc_block_metric('api_global_ip_blocked') + return self._api_block_response('global_ip_blocked') + + # 4. API-specific block (blocks /api/ only, never set by non-/api/ paths) + if self._rl_cache.get(RL_API_IP_BLOCKED.format(ip=ip)): + logger.warning( + 'api_rate_limit_block reason=api_ip_blocked ip=%s path=%s user_agent=%s', + ip, request.path, request.META.get('HTTP_USER_AGENT', ''), + ) + self._inc_block_metric('api_ip_blocked') + return self._api_block_response('api_ip_blocked') + + # 5. Daily/weekly quota (existing logic, preserved) + exceeded = self._check_api_quota(request) + if exceeded: + logger.warning( + 'quota_exceeded window=%s ip=%s path=%s namespace=%s', + exceeded, ip, request.path, _NAMESPACE, + extra={'ua': request.META.get('HTTP_USER_AGENT', '')}, + ) + self._inc_block_metric(f'quota_{exceeded}') + response = HttpResponse(status=429) + response['Retry-After'] = 86400 + response['X-RateLimit-Reason'] = f'quota_{exceeded}' + return response + + user = getattr(request, 'user', None) + if not (user and user.is_authenticated): + # 6. API-specific per-minute rate limit for anonymous external callers. + # Writes rl:api:ip::blocked only — never rl:ip::blocked. + if self.api_rate_limit_enabled: + count = self._incr_with_ttl(RL_API_IP_REQUESTS.format(ip=ip), self.api_window) + if count >= self.api_threshold: + _set_block(RL_API_IP_BLOCKED.format(ip=ip), RL_INDEX_API_BLOCKED_IPS, self.api_block_seconds) + logger.warning( + 'api_rate_limit_block reason=api_threshold_exceeded ' + 'ip=%s path=%s user_agent=%s count=%s threshold=%s', + ip, request.path, request.META.get('HTTP_USER_AGENT', ''), + count, self.api_threshold, + ) + self._inc_block_metric('api_threshold_exceeded') + return self._api_block_response('api_threshold_exceeded') + return self.get_response(request) + + # 7. Authenticated /api/ — existing per-user rate limiter unchanged + decision = self._evaluate(request) + if decision['action'] == 'block': + logger.warning( + 'ratelimit_block layer=django reason=%s ip=%s path=%s namespace=%s', + decision['reason'], decision['ip'], request.path, _NAMESPACE, + extra={'ua': request.META.get('HTTP_USER_AGENT', '')}, + ) + self._inc_block_metric(decision['reason']) + response = HttpResponse(status=429) + response['Retry-After'] = self.BLOCK_TTL + response['X-RateLimit-Reason'] = decision['reason'] + return response + return self.get_response(request) + # ------------------------------------------------------------------ # Evaluation # ------------------------------------------------------------------ diff --git a/sapl/middleware/test_ratelimiter.py b/sapl/middleware/test_ratelimiter.py index 87203872f..6d4744c47 100644 --- a/sapl/middleware/test_ratelimiter.py +++ b/sapl/middleware/test_ratelimiter.py @@ -13,11 +13,14 @@ from django.test import RequestFactory from sapl.middleware.ratelimit import ( _NAMESPACE, + _is_same_origin, _is_suspicious_headers, _parse_rate, get_client_ip, make_ratelimit_cache_key, RateLimitMiddleware, + RL_API_IP_BLOCKED, + RL_API_IP_REQUESTS, RL_IP_BLOCKED, RL_USER_BLOCKED, smart_key, @@ -58,7 +61,16 @@ def _auth_req(uid=7, **kwargs): return r -def _make_middleware(allowlist=None, anon_rate='35/m', auth_rate='120/m'): +def _make_middleware( + allowlist=None, + anon_rate='35/m', + auth_rate='120/m', + api_rate_limit_enabled=True, + api_threshold=60, + api_window=60, + api_block_seconds=300, + api_same_origin_bypass=True, +): """ Return (middleware, mock_cache). @@ -79,7 +91,19 @@ def _make_middleware(allowlist=None, anon_rate='35/m', auth_rate='120/m'): mock_settings.RATE_LIMITER_RATE_AUTHENTICATED = auth_rate mock_settings.RATE_LIMITER_RATE_BOT = '5/m' mock_settings.RATE_LIMIT_ALLOWLIST_IPS = allowlist or [] + mock_settings.RATE_LIMIT_404_THRESHOLD = 20 + mock_settings.RATE_LIMIT_BYPASS_PATHS = [] mock_settings.POD_NAMESPACE = _NAMESPACE # keep module-level _NAMESPACE consistent + mock_settings.API_QUOTA_ANON_DAILY = 999999 + mock_settings.API_QUOTA_ANON_WEEKLY = 999999 + mock_settings.API_QUOTA_AUTH_DAILY = 999999 + mock_settings.API_QUOTA_AUTH_WEEKLY = 999999 + mock_settings.RATE_LIMITER_UA_BLOCKLIST_REFRESH = 60 + mock_settings.API_RATE_LIMIT_ENABLED = api_rate_limit_enabled + mock_settings.API_RATE_LIMIT_THRESHOLD = api_threshold + mock_settings.API_RATE_LIMIT_WINDOW_SECONDS = api_window + mock_settings.API_RATE_LIMIT_BLOCK_SECONDS = api_block_seconds + mock_settings.API_RATE_LIMIT_SAME_ORIGIN_BYPASS = api_same_origin_bypass with ( patch('sapl.middleware.ratelimit.caches') as mock_caches, @@ -383,3 +407,182 @@ def test_call_pass_forwards_request_to_get_response(): request = _anon_req() mw(request) mw.get_response.assert_called_once_with(request) + + +# --------------------------------------------------------------------------- +# _is_same_origin +# --------------------------------------------------------------------------- + +def test_is_same_origin_no_headers_returns_false(): + r = _factory.get('/api/materia/') + r.META['SERVER_NAME'] = 'sapl.example.com' + r.META['SERVER_PORT'] = '80' + r.META.pop('HTTP_ORIGIN', None) + r.META.pop('HTTP_REFERER', None) + assert _is_same_origin(r) is False + + +def test_is_same_origin_matching_origin(): + r = _factory.get('/api/materia/', SERVER_NAME='sapl.example.com', SERVER_PORT='80') + r.META['HTTP_ORIGIN'] = 'https://sapl.example.com' + assert _is_same_origin(r) is True + + +def test_is_same_origin_mismatched_origin(): + r = _factory.get('/api/materia/', SERVER_NAME='sapl.example.com', SERVER_PORT='80') + r.META['HTTP_ORIGIN'] = 'https://other.example.com' + assert _is_same_origin(r) is False + + +def test_is_same_origin_wrong_origin_blocks_even_if_referer_matches(): + """If Origin is present and wrong, Referer must not be consulted.""" + r = _factory.get('/api/materia/', SERVER_NAME='sapl.example.com', SERVER_PORT='80') + r.META['HTTP_ORIGIN'] = 'https://evil.com' + r.META['HTTP_REFERER'] = 'https://sapl.example.com/page/' + assert _is_same_origin(r) is False + + +def test_is_same_origin_referer_used_when_no_origin(): + r = _factory.get('/api/materia/', SERVER_NAME='sapl.example.com', SERVER_PORT='80') + r.META.pop('HTTP_ORIGIN', None) + r.META['HTTP_REFERER'] = 'https://sapl.example.com/page/?q=1' + assert _is_same_origin(r) is True + + +def test_is_same_origin_port_stripped_from_both_sides(): + """Host with port and Origin without port must match after normalization.""" + r = _factory.get('/api/materia/', SERVER_NAME='sapl.example.com', SERVER_PORT='8000') + r.META['HTTP_HOST'] = 'sapl.example.com:8000' + r.META['HTTP_ORIGIN'] = 'http://sapl.example.com' + assert _is_same_origin(r) is True + + +def test_is_same_origin_case_insensitive(): + r = _factory.get('/api/materia/', SERVER_NAME='sapl.example.com', SERVER_PORT='80') + r.META['HTTP_ORIGIN'] = 'https://SAPL.EXAMPLE.COM' + assert _is_same_origin(r) is True + + +# --------------------------------------------------------------------------- +# _handle_api — OPTIONS and same-origin bypass +# --------------------------------------------------------------------------- + +def _api_req(ip='1.2.3.4', ua='Mozilla/5.0', path='/api/materia/', method='GET', extra_meta=None): + """Anonymous /api/ request with browser headers.""" + request = _factory.generic(method, path) + request.META.update({'REMOTE_ADDR': ip, 'HTTP_USER_AGENT': ua, **_NORMAL_HEADERS}) + if extra_meta: + request.META.update(extra_meta) + request.user = MagicMock(is_authenticated=False) + return request + + +def test_api_options_passes_without_counting(): + mw, _ = _make_middleware() + mw._check_api_quota = MagicMock(return_value=None) + mw._incr_with_ttl = MagicMock() + request = _api_req(method='OPTIONS') + mw(request) + mw.get_response.assert_called_once_with(request) + mw._incr_with_ttl.assert_not_called() + + +def test_api_same_origin_passes_without_counting(): + mw, _ = _make_middleware() + mw._check_api_quota = MagicMock(return_value=None) + mw._incr_with_ttl = MagicMock() + request = _api_req(extra_meta={ + 'SERVER_NAME': 'sapl.example.com', + 'SERVER_PORT': '80', + 'HTTP_HOST': 'sapl.example.com', + 'HTTP_ORIGIN': 'https://sapl.example.com', + }) + mw(request) + mw.get_response.assert_called_once_with(request) + mw._incr_with_ttl.assert_not_called() + + +def test_api_malicious_origin_is_not_same_origin(): + mw, _ = _make_middleware(api_threshold=999) + mw._check_api_quota = MagicMock(return_value=None) + mw._incr_with_ttl = MagicMock(return_value=1) + request = _api_req(extra_meta={ + 'SERVER_NAME': 'sapl.example.com', + 'SERVER_PORT': '80', + 'HTTP_HOST': 'sapl.example.com', + 'HTTP_ORIGIN': 'https://evil.com?x=sapl.example.com', + }) + mw(request) + # Must reach the counter (not short-circuit as same-origin) + mw._incr_with_ttl.assert_called_once() + + +# --------------------------------------------------------------------------- +# _handle_api — rate limiting and block key isolation +# --------------------------------------------------------------------------- + +def test_api_external_request_increments_api_counter(): + mw, _ = _make_middleware(api_threshold=10) + mw._check_api_quota = MagicMock(return_value=None) + mw._incr_with_ttl = MagicMock(return_value=5) # under threshold + request = _api_req() + response = mw(request) + mw.get_response.assert_called_once_with(request) + call_args = mw._incr_with_ttl.call_args[0] + assert call_args[0] == RL_API_IP_REQUESTS.format(ip='1.2.3.4') + + +def test_api_threshold_exceeded_creates_api_block_not_global_block(): + mw, _ = _make_middleware(api_threshold=5) + mw._check_api_quota = MagicMock(return_value=None) + mw._incr_with_ttl = MagicMock(return_value=5) # at threshold + request = _api_req() + with patch('sapl.middleware.ratelimit._set_block') as mock_set_block: + response = mw(request) + assert response.status_code == 429 + mock_set_block.assert_called_once() + block_key = mock_set_block.call_args[0][0] + assert block_key == RL_API_IP_BLOCKED.format(ip='1.2.3.4') + assert block_key != RL_IP_BLOCKED.format(ip='1.2.3.4') + + +def test_api_global_block_also_blocks_api(): + mw, mock_cache = _make_middleware() + mw._check_api_quota = MagicMock(return_value=None) + mw._incr_with_ttl = MagicMock() + ip = '1.2.3.4' + mock_cache.get.side_effect = lambda key: 1 if key == RL_IP_BLOCKED.format(ip=ip) else None + response = mw(_api_req(ip=ip)) + assert response.status_code == 429 + assert response['X-RateLimit-Reason'] == 'global_ip_blocked' + mw._incr_with_ttl.assert_not_called() + + +def test_api_specific_block_blocks_api_only(): + mw, mock_cache = _make_middleware() + mw._check_api_quota = MagicMock(return_value=None) + mw._incr_with_ttl = MagicMock() + ip = '1.2.3.4' + mock_cache.get.side_effect = lambda key: 1 if key == RL_API_IP_BLOCKED.format(ip=ip) else None + response = mw(_api_req(ip=ip)) + assert response.status_code == 429 + assert response['X-RateLimit-Reason'] == 'api_ip_blocked' + mw._incr_with_ttl.assert_not_called() + + +def test_api_block_response_is_json_with_retry_after(): + mw, _ = _make_middleware(api_block_seconds=120) + resp = mw._api_block_response('api_threshold_exceeded') + assert resp.status_code == 429 + assert 'application/json' in resp['Content-Type'] + assert resp['Retry-After'] == '120' + assert resp['X-RateLimit-Reason'] == 'api_threshold_exceeded' + + +def test_non_api_path_uses_global_evaluate_not_api_handler(): + mw, _ = _make_middleware() + mw._handle_api = MagicMock() + mw._evaluate = MagicMock(return_value={'action': 'pass', 'ip': '1.2.3.4'}) + mw(_anon_req(path='/')) + mw._handle_api.assert_not_called() + mw._evaluate.assert_called_once() diff --git a/sapl/settings.py b/sapl/settings.py index 46a0a947d..d3ffe4a34 100644 --- a/sapl/settings.py +++ b/sapl/settings.py @@ -151,7 +151,6 @@ MIDDLEWARE = [ 'django.contrib.messages.middleware.MessageMiddleware', 'django.middleware.clickjacking.XFrameOptionsMiddleware', 'django.middleware.security.SecurityMiddleware', - 'sapl.middleware.api_emergency_block.ApiEmergencySameSiteOnlyMiddleware', # TODO: REMOVE AFTER RL WORKS! 'whitenoise.middleware.WhiteNoiseMiddleware', 'waffle.middleware.WaffleMiddleware', 'sapl.middleware.check_password.CheckWeakPasswordMiddleware', @@ -448,6 +447,14 @@ API_QUOTA_ANON_WEEKLY = config('API_QUOTA_ANON_WEEKLY', default=3500, cast=int) API_QUOTA_AUTH_DAILY = config('API_QUOTA_AUTH_DAILY', default=5000, cast=int) API_QUOTA_AUTH_WEEKLY = config('API_QUOTA_AUTH_WEEKLY', default=35000, cast=int) +# API-specific per-minute rate limit for external (non-same-origin) anonymous calls. +# Abuse writes rl:api:ip::blocked only — never rl:ip::blocked. +API_RATE_LIMIT_ENABLED = config('API_RATE_LIMIT_ENABLED', default=True, cast=bool) +API_RATE_LIMIT_THRESHOLD = config('API_RATE_LIMIT_THRESHOLD', default=60, cast=int) +API_RATE_LIMIT_WINDOW_SECONDS = config('API_RATE_LIMIT_WINDOW_SECONDS', default=60, cast=int) +API_RATE_LIMIT_BLOCK_SECONDS = config('API_RATE_LIMIT_BLOCK_SECONDS', default=300, cast=int) +API_RATE_LIMIT_SAME_ORIGIN_BYPASS = config('API_RATE_LIMIT_SAME_ORIGIN_BYPASS', default=True, cast=bool) + # Media file serving — serve_media (sapl/base/media.py) via X-Accel-Redirect. # TTL for both URL-path and storage-path access counters (DB 1). MEDIA_PATH_COUNTER_TTL = config('MEDIA_PATH_COUNTER_TTL', default=60, cast=int)