[Memtensor-AI]

Description

Fix issue #2524: chat prompt rewrite busted provider prefix caches. The handler used to splice the current turn's retrieval and current minute into the system message and send that first, with chat history after it — so the first message changed every turn and the identical history behind it could not be reused.

Split the system blob into a stable prompt (byte-identical across turns in one conversation) plus a per-turn dynamic context message carrying the current time, retrieved memories, and preferences. New message layout is [stable system, ...history, dynamic context, user], so the shared cross-turn prefix now grows with the chat history. Applied in ChatHandler.handle_chat_complete, handle_chat_stream, handle_chat_stream_playground, handle_chat_stream_for_business_user, MOSCore.chat, MOS._generate_synthesis_response, and SimpleMemChat.run. Legacy callers that pass a custom base_prompt containing {memories} keep the single-system shape for backward compatibility.

Tests: added tests/api/test_chat_message_order.py (9 cases covering stable/dynamic split, cacheable assembly, cross-turn shared prefix, and legacy-base-prompt compat), updated tests/mem_os/test_memos_core.py::TestMOSChat::test_chat_with_custom_base_prompt for the new ordering, and added test_chat_places_history_before_dynamic_context_issue_2524. Full targeted run pytest tests/mem_os/ tests/api/ tests/mem_chat/ is green: 167 passed. Ruff lint + format clean on all modified files. No API schema change (no make openapi needed).

Related Issue (Required): Fixes #2524

Type of change

Please delete options that are not relevant.

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• Refactor (does not change functionality, e.g. code style improvements, linting)
• Documentation update

How Has This Been Tested?

Not run; documentation-only change.

• Unit Test
• Test Script Or Test Steps (please provide)
• Pipeline Automated API Test (please provide)

Checklist

• I have performed a self-review of my own code
• I have commented my code in hard-to-understand areas
• I have added tests that prove my fix is effective or that my feature works
• I have created related documentation issue/PR in MemOS-Docs (if applicable)
• I have linked the issue to this PR (if applicable)
• I have mentioned the person who will review this PR

@WeiminLee please review this PR.

Reviewer Checklist

• closes Chat rewrites the system message from each turn's retrieval and places history after it, so provider prefix caches miss #2524
• Made sure Checks passed
• Tests have been provided

[Memtensor-AI]

🤖 Open Code Review

Target: PR #2529
Task: e4f30089c2eb8ee3
Base: dev-v2.0.36
Head: bugfix/autodev-2524-20261021095707472
Head SHA: 6c654fc9a1bee1d2eb827e7f098848101f8d7982

🔍 OpenCodeReview found 5 issue(s) in this PR.

⚠️ 1 warning(s) occurred during review.

---

1. src/memos/mem_os/main.py (L314-L316)

Previous Finding #1 still unresolved. Both dynamic_ctx and synthesis_tail are appended as {"role": "system"} messages at non-leading positions (after chat_history.chat_history), not at index 0.

The OpenAI Chat Completions API only accepts a system message at position 0; inserting one mid-conversation triggers a 400 error on OpenAI and is silently mishandled or ignored by many other providers (Azure OpenAI, Anthropic-compatible routers, etc.).

By contrast, the sibling fix in core.py correctly wraps dynamic_ctx as a user-role framing message ("[Context for this turn]\n" + dynamic_ctx). main.py must apply the same treatment.

Suggested fix — deliver both pieces as user-role messages:

if dynamic_ctx:
    current_messages.append(
        {"role": "user", "content": "[Context for this turn]\n" + dynamic_ctx}
    )
current_messages.append(
    {"role": "user", "content": "[Instructions for this turn]\n" + synthesis_tail}
)

If keeping synthesis_tail as a system message is required for a specific LLM, it should still be placed at position 0 (prepended to stable_system) rather than inserted mid-conversation.

---

2. src/memos/api/handlers/chat_handler.py (L1294-L1300)

Previous Finding #3 partially unresolved. get_memos_prompt is called with date="N/A — see context below", which is substituted verbatim into MEMOS_PRODUCT_BASE_PROMPT as:

# Current Date: N/A — see context below

This line is present in the stable system prompt that the LLM sees on every turn. The code's own comment acknowledges the problem ("OCR round 2 finding #5 — '(see dynamic context)' still rendered as a literal…"), noting that the sentinel string merely changed from "(see dynamic context)" to "N/A — see context below"; neither value is a real date.

While the real date is correctly emitted in _build_enhance_dynamic_context, having an explicit # Current Date: N/A — see context below line in the persistent system prompt is confusing to the model and may override or conflict with the real date delivered later.

Preferred fix: strip the {date} line from MEMOS_PRODUCT_BASE_PROMPT entirely (move it to the enhance-prompt template so it only appears in the dynamic context), or pass the real current date here as well if caching is not a concern for this path.

---

3. src/memos/api/handlers/chat_handler.py (L632-L640)

Finding #6 is still unresolved. Two consecutive {"role": "user"} messages are emitted back-to-back: one carrying "[Context for this turn]\n" + dynamic_ctx and one carrying chat_req.query. The Anthropic Messages API requires strictly alternating user/assistant turns and returns a 400 error when two user messages appear in a row. Several other strict-alternation providers (Mistral, Cohere Command R, etc.) behave the same way.

A portable fix is to merge the context and the actual query into a single user message:

combined_user = "[Context for this turn]\n" + dynamic_ctx + "\n\n" + chat_req.query
current_messages = [
    {"role": "system", "content": stable_prompt},
    *history_info,
    {"role": "user", "content": combined_user},
]

Alternatively, embed the dynamic context as a <context> XML block at the top of the user turn so the query stays readable.

---

4. src/memos/api/handlers/chat_handler.py (L1264-L1275)

Finding #6 is still unresolved in _assemble_cacheable_messages as well. The same consecutive user-message pattern is produced here for all non-legacy callers: a [Context for this turn] user message followed immediately by the actual query user message. This will be rejected by strict alternating-role provider APIs (Anthropic, Mistral, Cohere, etc.).

Merge the dynamic context into the query user turn to produce a single user message:

if dynamic_ctx:
    messages.append({"role": "user", "content": "[Context for this turn]\n" + dynamic_ctx + "\n\n" + query})
else:
    messages.append({"role": "user", "content": query})
return messages

---

5. src/memos/mem_chat/simple.py (L98-L105)

Finding #9 is still unresolved. When dynamic_ctx is non-None, two consecutive {"role": "user"} messages are appended: the [Context for this turn] framing message immediately followed by the real user_input message. This violates the strict alternating user/assistant turn requirement of Anthropic's Messages API (and several other providers), which will return a 400 error.

Merge the context into the user turn instead of emitting two separate user messages:

if dynamic_ctx:
    current_messages.append({"role": "user", "content": "[Context for this turn]\n" + dynamic_ctx + "\n\n" + user_input})
else:
    current_messages.append({"role": "user", "content": user_input})

---

🧹 Filtered 1 low-confidence OCR finding(s) before posting/fix-loop (duplicate: 1).

Generated by cloud-assistant via Open Code Review.

[Memtensor-AI]

🔧 Open Code Review requested Agent fix

Open Code Review found 10 issue(s). I have resumed the development Agent to fix them.

• Task: e4f30089c2eb8ee3
• Fix attempt: 1/2
• Finding delta: 0 repeated / 10 new / 0 likely resolved

The Agent will push a new commit to this PR branch. OCR will recheck after the commit is pushed.

[Memtensor-AI]

🔧 Open Code Review requested Agent fix

Open Code Review found 3 issue(s). I have resumed the development Agent to fix them.

• Task: e4f30089c2eb8ee3
• Fix attempt: 2/2
• Finding delta: 2 repeated / 1 new / 8 likely resolved

Repeated findings are prioritized because the previous repair did not converge.
The Agent will push a new commit to this PR branch. OCR will recheck after the commit is pushed.