Chat rewrites the system message from each turn's retrieval and places history after it, so provider prefix caches miss #2524

Pre-submission checklist | 提交前检查

• I have searched existing issues and this hasn't been mentioned before
• I have read the project documentation and confirmed this issue doesn't already exist
• This issue is specific to MemOS and not a general software issue

Bug Description | 问题描述

On every chat turn, MemOS rebuilds the system message from that turn's retrieval and sends the conversation history after it. Provider prefix caches keep the longest identical prefix from the start of the request. When the first message changes, the unchanged history behind it cannot be reused.

This is how the product chat API and the library chat entry build the provider request.

What happens:

• ChatHandler.handle_chat_complete, handle_chat_stream, handle_chat_stream_playground, and handle_chat_stream_for_business_user search memories for the current query, then build one system string and place history after it.
• _build_system_prompt writes that turn's fact memories and preference text into the system string ({memories}, or an appended ## Fact Memories block). get_cloud_chat_prompt also writes the current minute into the same string, so the system message changes on the next minute even when the retrieved memories are the same.
• The playground stream uses _build_enhance_system_prompt, which writes the current time and appends PersonalMemory and OuterMemory to that same system message.
• MOS.chat in src/memos/mem_os/core.py, the chain-of-thought synthesis path in src/memos/mem_os/main.py, and SimpleMemChat use the same order: a system message that contains the memories, then the prior turns, then the current user message.
• src/memos/llms/openai.py sends that messages list to the provider unchanged.

What was expected:

The part that stays the same across turns — the fixed instructions and the prior turns — should remain a common prefix. The text that changes with this turn's retrieval should come after that prefix, so a provider prefix cache can reuse the history.

MemOSClient.chat only posts the query to the hosted /chat endpoint. It does not build the provider messages. The assembly above is in ChatHandler and in MOS.chat.

How to Reproduce | 如何重现

1. Call POST /chat/complete or POST /chat/stream twice in one conversation. On the second call, pass the history that includes the first turn's user and assistant messages. Let the two calls retrieve different memories, or call them in different minutes with the same memories.
2. Look at the messages list the handler passes to the LLM. In src/memos/api/handlers/chat_handler.py it is [system, *history[-20:], current user message]. The system content comes from _build_system_prompt or, on POST /chat/stream/playground, from _build_enhance_system_prompt. The business stream uses the same list.
3. Compare the two requests. The first message differs because it contains that turn's memories and, for the cloud prompt, the current minute. The prior turns are after that message, so they are not part of the shared prefix.
4. The same order is in MOS.chat (src/memos/mem_os/core.py): _build_system_prompt with the memories for this query, then chat_history.chat_history, then the current user message.
5. src/memos/llms/openai.py _build_request_body puts that list in the provider request as messages and does not move the system message.

A provider that caches the longest identical prefix therefore misses from the system message through the rest of the prompt. Unchanged history is not a cache hit.

Environment | 环境信息

• Python version: 3.12.15
• Operating System: macOS (Darwin 27.0.0, arm64)
• MemOS version: 2.0.33 (pip show memoryos reports Name: MemoryOS, Version: 2.0.33)

Additional Context | 其他信息

Open issues were searched for prefix cache, prompt cache, and system-prompt retrieval. None describe this message order. The chat API documentation says retrieved memories are injected into the prompt; it does not keep a stable prefix ahead of the per-turn retrieval. The open activation-memory cache report is a different problem: it is about an in-process DynamicCache growing during generation, not about the provider prompt prefix.

Willingness to Implement | 实现意愿

• I'm willing to implement this myself
• I would like someone else to implement this