feat(llm): add configurable per-model Redis GCRA rate limiting

docker/.env.example-full

@@ -53,6 +53,16 @@ DOCUMENT_PARSER_MODEL=                     # falls back to MEMREADER_GENERAL_MOD
 IMAGE_PARSER_MODEL=                        # falls back to MEMREADER_GENERAL_MODEL when omitted
 QWEN_MODEL=qwen-flash                      # optional qwen_llm slot when QWEN_API_KEY is set
 
+## Optional per-model LLM QPS rate limiting (Redis GCRA)
+# Disabled by default. Enable explicitly and tune rules to your provider quota.
+MEMOS_LLM_RATE_LIMIT_ENABLED=false
+MEMOS_LLM_RATE_LIMIT_RULES='{"gpt-4o-mini":{"qps":5,"burst":2,"max_wait_seconds":30,"queue_capacity":16,"retry_attempts":1}}'
+# Rule keys must match actual model names; unlisted models are not limited.
+# QPS/burst are shared across workers using the same Redis/DB and model key.
+# queue_capacity is per process/model; max_wait_seconds is the permit-wait budget.
+# Reuses MEMSCHEDULER_REDIS_HOST/PORT/DB/USERNAME/PASSWORD/SSL; host is required when enabled.
+# See docs/cn/open_source/open_source_api/help/llm_qps_rate_limit.md.
+
 ## Embedding & rerank
 # embedding dim
 EMBEDDING_DIMENSION=1024
@@ -111,9 +121,9 @@ ENABLE_INTERNET=false
 # Internet search backend (bocha | tavily)
 INTERNET_SEARCH_BACKEND=bocha
 # API key for BOCHA Search
-BOCHA_API_KEY=                             # required if ENABLE_INTERNET=true and backend=bocha
+BOCHA_API_KEY=                             your-bocha-api-key and backend=bocha
 # API key for Tavily Search
-TAVILY_API_KEY=                            # required if ENABLE_INTERNET=true and backend=tavily
+TAVILY_API_KEY=                            your-bocha-api-key and backend=tavily
 # default search mode
 SEARCH_MODE=fast                          # fast | fine | mixture
 # Slow retrieval strategy configuration, rewrite is the rewrite strategy

docs/cn/open_source/open_source_api/help/llm_qps_rate_limit.md

@@ -0,0 +1,83 @@
+# LLM GCRA 限流
+
+## 环境变量
+
+限流只暴露两个环境变量，默认关闭。当前仅限制明确选中的模型。
+
+```dotenv
+MEMOS_LLM_RATE_LIMIT_ENABLED=false
+MEMOS_LLM_RATE_LIMIT_RULES='{"gpt-4o-mini":{"qps":5,"burst":2,"max_wait_seconds":30,"queue_capacity":16,"retry_attempts":1}}'
+```
+
+`RULES` 是 JSON 对象，键为实际请求的模型名。每条规则只支持以下五个参数，省略时使用代码默认值：
+
+| 参数 | 默认值 | 含义 |
+|---|---:|---|
+| qps | 5 | 全局持续放行速率；所有共享配额的 worker 合计 |
+| burst | 2 | 空闲后最多立即放行的请求总数，不是 qps 加 burst |
+| max_wait_seconds | 30 | 单次主/备用调用及其受控重试的累计许可等待预算，单位秒 |
+| queue_capacity | 16 | 每进程、每模型的等待队列上限，包含正在申请的队首 |
+| retry_attempts | 1 | 首次模型请求失败后最多重试次数；0 表示不重试 |
+
+上述示例与代码默认值及 `docker/.env.example-full` 一致，默认不启用。需要限流时显式设置 `MEMOS_LLM_RATE_LIMIT_ENABLED=true`。参数是开源部署的保守起点，不代表供应商保证的配额；应按实际配额、共享环境、worker 数、排队等待及限流错误调整。QPS 限流不等于 token 吞吐或模型在途并发限制。
+
+- 未配置 `RULES` 时，默认只选择 `gpt-4o-mini`，使用上述默认值。
+- 显式配置 `RULES` 会替换整个模型规则集合；未列出的模型不受限流影响，不隐式追加默认模型。
+- `RULES={}` 不限制任何模型；删除某个模型的条目即可取消该模型限流。
+- `ENABLED=false` 关闭整个功能。
+- 模型名不支持通配符；qps 必须为有限正数，burst 和容量为正整数，重试次数为非负整数。
+- Shell/`.env` 示例的外层单引号用于保护 JSON；在部署平台直接填写环境变量值时，不包含外层单引号。
+
+多个模型分别配置示例：
+
+```dotenv
+MEMOS_LLM_RATE_LIMIT_RULES='{"gpt-4o-mini":{"qps":5,"burst":2,"max_wait_seconds":30,"queue_capacity":16,"retry_attempts":1},"qwen-flash":{"qps":10,"burst":2,"max_wait_seconds":3,"queue_capacity":8,"retry_attempts":0}}'
+```
+
+第二个模型仅为示例，不默认启用。
+
+## Redis 与加载
+
+Redis 连接复用现有 `MEMSCHEDULER_REDIS_HOST/PORT/DB/USERNAME/PASSWORD/SSL`。缺少 host 时，第一次受控调用报配置错误；默认关闭时不创建 Redis 客户端。密码通过部署 Secret 注入。
+
+Redis key 自动按模型生成，无需 scope：
+
+```text
+memos:llm:gcra:gpt-4o-mini
+memos:llm:gcra:qwen-flash
+```
+
+同一 Redis/DB、相同前缀下，同名模型跨 worker/环境共享一个 TAT，不因 endpoint 或 API Key 不同而拆分。不同模型的 TAT 和本地队列独立；模型别名视为不同模型。各环境必须使用一致的速率和 burst。
+
+配置在创建 LLM 配置对象时加载，不逐请求读环境变量，也不自行加载 `.env`。更新部署配置后需协调重启 worker。Python 显式 `rate_limit` 配置仍可覆盖环境值，配置对象保留内部运行参数用于程序化构造和测试；这些参数不再提供环境变量入口。
+
+旧的 `MEMOS_LLM_RATE_LIMIT_*` 配置中，除 `ENABLED`、`RULES` 外均需移除，例如 MODELS、QPS、BURST、SCOPE、CONFIG_FILE、REDIS_* 和 WAIT_JITTER_SECONDS。加载时会对不支持的变量报错，避免旧配置被静默忽略。旧的模型规则中也应移除 enabled、scope、抖动及退避参数。不再支持通过环境变量指定独立 JSON 配置文件。
+
+若从旧哈希 key 或旧前缀升级，请协调所有实例切换，避免新旧 key 同时放行；新 key 初始化时会恢复一个 burst。不要在运行中随意切换前缀。
+
+## 内部行为
+
+- Lua 使用 Redis TIME，原子读取、判断和更新 TAT，拒绝不推进 TAT；Python 使用 register_script，无需本机安装 Lua。
+- 每进程、每模型只有队首申请 Redis，其他线程通过 Condition 等待。获准后立即离队发起模型调用，不等模型返回。
+- Redis 建议等待时间后附加 0～10ms 抖动。无有效 Retry-After 时使用指数退避和抖动，退避基数 1s、上限 8s。这些是内部默认值，不需要部署配置。
+- 受控调用关闭 SDK 隐藏重试。连接/超时错误及 HTTP 408、409、429、5xx 可有限重试，每次重新申请许可；Retry-After 超过内部等待上限时不提前重试。
+- 流式请求只重试建立阶段，流开始后的错误不重放。调用方提前结束时应关闭生成器。
+- 队列满、等待超时、Redis 不可用分别抛出 LLMRateLimitQueueFullError、LLMRateLimitTimeoutError、LLMRateLimitUnavailableError，不通过备用模型绕过。
+- 默认 Redis 连接和读取超时 0.5s，故障策略为 closed，即停止受控调用。网络响应迟到时不发送模型请求，也不退还已消耗或状态不确定的许可。
+- max_wait_seconds 不包含模型网络耗时和失败退避，不是整个业务请求的总超时；外层仍需业务 deadline。同步 Redis I/O 最迟要等 socket 超时才能退出。
+- 本地队列不是持久任务队列；满队列、超时和进程退出不会自动延期任务。Redis 故障切换或淘汰 TAT 也可能重置额度。
+
+## 范围与验证
+
+当前接入 OpenAILLM 及其 Qwen、DeepSeek、MiniMax 子类的 Chat Completions，包括普通调用、流式建立和备用模型。Azure、Responses API、Ollama、VLLM 等独立实现暂未接入。
+
+该版本仅控制 QPS，不控制 Token 用量、Token 增速或在途并发，不能保证解决供应商所有 429。
+
+INFO 的 `[LLM_RATE_LIMIT] sending` 记录模型、尝试序号及许可等待时间；WARNING 记录重试、队列满、等待超时和 Redis 故障。新增日志不记录请求正文或凭据。
+
+```sh
+poetry run pytest tests/configs/ tests/llms/ -q
+MEMOS_TEST_LOCAL_REDIS=1 poetry run pytest tests/llms/test_qps_rate_limit_redis.py -q
+```
+
+第二条启动隔离本地 Redis，仅 Unix socket、无 TCP、无持久化，不读取生产 Redis 配置。日志位于 pytest 管理的 `redis-gcra*` 临时目录；短路径临时 socket 退出时清理。

docs/en/open_source/open_source_api/help/llm_qps_rate_limit.md

@@ -0,0 +1,83 @@
+# LLM GCRA Rate Limiting
+
+## Environment Variables
+
+Rate limiting exposes only two environment variables and is disabled by default. Only explicitly selected models are limited.
+
+```dotenv
+MEMOS_LLM_RATE_LIMIT_ENABLED=false
+MEMOS_LLM_RATE_LIMIT_RULES='{"gpt-4o-mini":{"qps":5,"burst":2,"max_wait_seconds":30,"queue_capacity":16,"retry_attempts":1}}'
+```
+
+`RULES` is a JSON object keyed by the actual model name used in requests. Each rule accepts only the following five parameters. Omitted parameters use the code defaults:
+
+| Parameter | Default | Description |
+|---|---:|---|
+| qps | 5 | Global sustained admission rate, aggregated across all workers sharing the quota |
+| burst | 2 | Total number of requests that can be admitted immediately after an idle period; not qps plus burst |
+| max_wait_seconds | 30 | Cumulative permit-wait budget in seconds for a single primary/backup invocation and its managed retries |
+| queue_capacity | 16 | Waiting queue capacity per process and model, including the head currently requesting a permit |
+| retry_attempts | 1 | Maximum retries after the initial model request fails; 0 disables retries |
+
+The example matches the code defaults and `docker/.env.example-full`, with rate limiting disabled. Set `MEMOS_LLM_RATE_LIMIT_ENABLED=true` explicitly to enable it. These values are a conservative starting point for open-source deployments, not provider-guaranteed quotas. Adjust them based on your actual quota, shared environments, worker count, queue waits, and rate-limit errors. QPS limiting is not a token-throughput or in-flight concurrency limit.
+
+- When `RULES` is not set, only `gpt-4o-mini` is selected, using the defaults above.
+- Explicit `RULES` replace the entire model rule set. Unlisted models are unaffected; the default model is not implicitly added.
+- `RULES={}` limits no models. Remove a model entry to disable limiting for that model.
+- `ENABLED=false` disables the entire feature.
+- Model names do not support wildcards. qps must be finite and positive; burst and queue capacity must be positive integers; retry attempts must be a nonnegative integer.
+- The outer single quotes in shell/`.env` examples protect the JSON. Omit them when entering the environment variable value directly in a deployment platform.
+
+Example with separate rules for multiple models:
+
+```dotenv
+MEMOS_LLM_RATE_LIMIT_RULES='{"gpt-4o-mini":{"qps":5,"burst":2,"max_wait_seconds":30,"queue_capacity":16,"retry_attempts":1},"qwen-flash":{"qps":10,"burst":2,"max_wait_seconds":3,"queue_capacity":8,"retry_attempts":0}}'
+```
+
+The second model is illustrative and is not selected by default.
+
+## Redis and Configuration Loading
+
+The limiter reuses the existing `MEMSCHEDULER_REDIS_HOST/PORT/DB/USERNAME/PASSWORD/SSL` connection settings. A missing host causes a configuration error on the first limited invocation. No Redis client is created while the feature is disabled. Inject passwords through deployment secrets.
+
+Redis keys are generated automatically per model; no scope configuration is needed:
+
+```text
+memos:llm:gcra:gpt-4o-mini
+memos:llm:gcra:qwen-flash
+```
+
+Workers and environments using the same Redis instance, database, prefix, and model name share one theoretical arrival time (TAT). Different endpoints or API keys do not create separate quotas. Different models have independent TAT values and local queues; model aliases are treated as distinct models. All environments sharing a quota must use consistent qps and burst settings.
+
+Configuration is loaded when the LLM configuration object is created, not on every request. The limiter does not load `.env` itself. Coordinate worker restarts after changing deployment settings. Explicit Python `rate_limit` configuration can still override environment values. Configuration objects retain internal runtime parameters for programmatic construction and testing, but those parameters have no environment-variable interface.
+
+Remove all legacy `MEMOS_LLM_RATE_LIMIT_*` variables other than `ENABLED` and `RULES`, including MODELS, QPS, BURST, SCOPE, CONFIG_FILE, REDIS_*, and WAIT_JITTER_SECONDS. Unsupported variables cause a configuration-loading error rather than being silently ignored. Also remove enabled, scope, jitter, and backoff parameters from old model rules. Selecting a separate JSON configuration file through an environment variable is no longer supported.
+
+When migrating from hashed keys or an older prefix, coordinate the switch across all instances to avoid simultaneous admission through both old and new keys. A new key starts with a full burst allowance. Do not change prefixes arbitrarily while the system is running.
+
+## Internal Behavior
+
+- Lua uses Redis TIME to atomically read, check, and update TAT. Rejection does not advance TAT. Python uses register_script; a local Lua installation is not required.
+- Only the queue head requests a Redis permit for each process and model. Other threads wait on a Condition. Once admitted, a request leaves the queue immediately and starts the model call without waiting for earlier model calls to finish.
+- A random jitter of 0 to 10 ms is added to Redis's suggested wait. Without a valid Retry-After, retries use exponential backoff with jitter, a 1-second base, and an 8-second cap. These are internal defaults and need no deployment configuration.
+- Managed invocations disable hidden SDK retries. Connection/timeout errors and HTTP 408, 409, 429, and 5xx responses can be retried within the configured retry limit. Every retry must acquire a new permit. If Retry-After exceeds the internal wait limit, the request is not retried earlier than requested.
+- Streaming requests are retried only during establishment. Errors after streaming starts do not replay the stream. Callers that stop consuming early should close the generator.
+- A full queue, an expired wait budget, and Redis unavailability raise LLMRateLimitQueueFullError, LLMRateLimitTimeoutError, and LLMRateLimitUnavailableError respectively. These errors do not bypass the limiter through a backup model.
+- Redis connection and read timeouts default to 0.5 seconds. The default failure policy is closed, which stops limited invocations. A late Redis response does not cause a model request to be sent, and permits already consumed or with uncertain status are not refunded.
+- max_wait_seconds excludes model network time and failure backoff. It is not a total business-request timeout; callers still need an outer deadline. Synchronous Redis I/O may need to wait until the socket timeout before exiting.
+- The local queue is not a durable task queue. Queue overflow, wait timeouts, and process exits do not automatically defer tasks. Redis failover or eviction of TAT keys may also reset the allowance.
+
+## Scope and Verification
+
+The integration currently covers Chat Completions in OpenAILLM and its Qwen, DeepSeek, and MiniMax subclasses, including regular calls, streaming establishment, and backup models. Independent implementations such as Azure, the Responses API, Ollama, and VLLM are not integrated yet.
+
+This version controls QPS only, not token usage, token traffic growth, or in-flight concurrency. It cannot guarantee prevention of every provider-side 429 response.
+
+INFO-level `[LLM_RATE_LIMIT] sending` logs record the model, attempt number, and permit-wait duration. WARNING logs record retries, queue overflow, wait timeouts, and Redis failures. The new logs do not include request bodies or credentials.
+
+```sh
+poetry run pytest tests/configs/ tests/llms/ -q
+MEMOS_TEST_LOCAL_REDIS=1 poetry run pytest tests/llms/test_qps_rate_limit_redis.py -q
+```
+
+The second command starts an isolated local Redis instance using only a Unix socket, with no TCP listener or persistence. It does not read production Redis settings. Logs are stored in a pytest-managed `redis-gcra*` temporary directory; the short-path temporary socket is cleaned up on exit.

src/memos/configs/llm.py

@@ -3,6 +3,7 @@
 from pydantic import Field, field_validator, model_validator
 
 from memos.configs.base import BaseConfig
+from memos.configs.llm_rate_limit import LLMRateLimitConfig
 
 
 class BaseLLMConfig(BaseConfig):
@@ -23,6 +24,17 @@ class BaseLLMConfig(BaseConfig):
 
 
 class OpenAILLMConfig(BaseLLMConfig):
+    rate_limit: LLMRateLimitConfig = Field(default_factory=LLMRateLimitConfig.load)
+
+    @field_validator("rate_limit", mode="before")
+    @classmethod
+    def load_rate_limit(cls, value: Any) -> LLMRateLimitConfig:
+        if isinstance(value, LLMRateLimitConfig):
+            return value
+        if not isinstance(value, dict):
+            raise ValueError("rate_limit must be a configuration object")
+        return LLMRateLimitConfig.load(value)
+
     api_key: str = Field(..., description="API key for OpenAI")
     api_base: str = Field(
         default="https://api.openai.com/v1", description="Base URL for OpenAI API"