Skip to content

fix: 修复了代理切换或者弱网环境下的tg平台假死情况 - #9473

Open
Baneik wants to merge 2 commits into
AstrBotDevs:masterfrom
Baneik:fix/telegram-polling-watchdog
Open

fix: 修复了代理切换或者弱网环境下的tg平台假死情况#9473
Baneik wants to merge 2 commits into
AstrBotDevs:masterfrom
Baneik:fix/telegram-polling-watchdog

Conversation

@Baneik

@Baneik Baneik commented Jul 30, 2026

Copy link
Copy Markdown

问题背景

Telegram polling 偶尔会在 updater 仍显示运行中的情况下停止接收更新。该问题通常发生在网络或代理切换后:getUpdates 长轮询连接进入失效状态,但没有产生足够的异常来触发现有恢复逻辑。
在用户看来,就是消息发出后无响应,或者偶尔会展示bot输入中,但是并不会有响应,需要手动重启适配器后才能暂时修复 #8314
在最新构建中,实测并没有修复该问题。
提交的修改已在我本地实测运行多日,并没有再出现假死现象。

发生问题时通常表现为:

  • updater.running 仍为 True
  • Telegram 新消息无法继续进入 AstrBot;
  • polling 错误回调可能没有被触发;
  • Adapter 因此不会重建已经失效的 polling 客户端。

此外,直接重建整个 Telegram Application 会关闭旧的 Bot 客户端。已经创建的 TelegramPlatformEvent 仍然持有该客户端,因此恢复 polling 时可能中断正在生成或发送的回复。

问题原因

现有恢复逻辑主要依赖 getUpdates 抛出的 NetworkError。对于失效但没有及时产生异常的长轮询连接,仅依赖错误回调无法发现故障。

同时,polling 和消息发送共享同一个 Application 生命周期。重建 polling 时关闭 Application,也会关闭仍被现有事件使用的消息发送客户端。

另外,仅根据 pending_update_count 是否非零判断 polling 卡死,可能会把临时消息积压误判为故障,导致正常运行的机器人被不必要地重启。

修改内容

  • 新增每 15 秒执行一次的 polling watchdog。
  • Watchdog 通过 getWebhookInfo 检查状态。该请求使用普通 Bot API 请求池,与 getUpdates 使用的长轮询请求池相互独立。
  • 在以下情况下请求恢复 polling:
    • Watchdog 连续三次发生网络错误;
    • 连续两次检测到存在 pending update,并且没有观察到消息投递进展。
  • 以下情况会被视为 polling 仍有进展:
    • Adapter 收到了新的 Telegram 消息;
    • pending_update_count 正在下降。
  • 为普通 Bot API 和 getUpdates 请求池设置明确的 15 秒连接超时。
  • Updater.stop() 增加超时限制,避免失效的 polling 请求无限阻塞恢复流程。
  • 将 polling 和消息发送客户端的生命周期分离:
    • getUpdates 错误或 pending update 停滞时,只重建 polling 客户端;
    • 保留已经初始化且健康的 outbound client,供现有事件和进行中的消息发送继续使用;
    • 如果普通 Bot API watchdog 连续失败,则同时替换 outbound client,避免继续使用已经确认异常的客户端。
  • 如果 Application 在初始化阶段失败,则丢弃尚未成功初始化的客户端并重新创建。
  • Adapter 终止时同时关闭当前 polling Application 和被保留的 outbound Application。

恢复策略

故障信号 恢复方式
getUpdates 连续发生网络错误 重建 polling,保留健康的 outbound client
存在 pending update 且没有投递进展 重建 polling,保留健康的 outbound client
普通 Bot API watchdog 连续失败 重建 polling,并替换异常的 outbound client
Application 初始化失败 丢弃未初始化客户端并重新创建

测试覆盖

新增了以下测试:

  • Watchdog 瞬时网络错误不会触发重建;
  • Watchdog 持续失败时替换异常的 outbound client;
  • pending update 数量下降时不会误触发恢复;
  • 存在 pending update 但仍持续收到消息时不会误触发恢复;
  • 从 Adapter 的实际 run() 主循环触发 watchdog 恢复;
  • polling 恢复过程中 outbound client 仍可继续发送消息;
  • Adapter 终止时正确清理 polling 和 outbound Application;
  • 初始 Application 初始化失败时正确替换客户端;
  • Updater.stop() 卡住时能够在超时后继续清理;
  • 原有的 getUpdates 连续错误恢复逻辑保持正常。

验证方法

uv run ruff format .
uv run ruff check .
uvx pyright astrbot/core/platform/sources/telegram/tg_adapter.py
uv run pytest -q tests/test_telegram_adapter.py

## Summary by Sourcery

Improve Telegram adapter robustness against polling stalls and proxy/network switches by adding a watchdog-based health check and separating polling from outbound client lifecycle.

Bug Fixes:
- Prevent Telegram polling from silently stalling on proxy or network changes while updater still reports running.
- Ensure stalled updater.stop() calls are bounded by a timeout so shutdown and recovery do not hang.
- Avoid unnecessary polling restarts when pending updates are being processed or messages are still flowing.

Enhancements:
- Introduce a periodic polling watchdog using getWebhookInfo to detect lack of delivery progress and trigger targeted recovery.
- Separate polling application from outbound Bot API client so polling can be rebuilt without interrupting in-flight message sends.
- Add explicit connection timeouts for both regular Bot API requests and getUpdates long-polling.
- Refine recovery logging and behavior to distinguish between rebuilding polling only and replacing both polling and outbound clients.
- Ensure adapter termination cleans up both polling and outbound Telegram applications.

Tests:
- Add tests covering watchdog behavior on transient and persistent network errors, pending update detection, and integration with the main run loop.
- Add tests verifying outbound client preservation during polling recovery, proper cleanup on termination, and client replacement after initialization failures.
- Add tests ensuring stalled updater.stop() is bounded by timeout and that existing polling error recovery logic still functions as expected.

@dosubot dosubot Bot added size:L This PR changes 100-499 lines, ignoring generated files. area:platform The bug / feature is about IM platform adapter, such as QQ, Lark, Telegram, WebChat and so on. labels Jul 30, 2026
@dosubot

dosubot Bot commented Jul 30, 2026

Copy link
Copy Markdown

📄 Knowledge review

Dosu skipped reviewing this PR because your organization has used its 200 included credits for the month. Your usage will reset on 2026-08-01. To have Dosu review this PR before then, ask your organization admin to upgrade to a pro account.


Leave Feedback Ask Dosu about AstrBot Add Dosu to your team

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've left some high level feedback:

  • Consider making the polling watchdog thresholds and connect timeout (currently hardcoded to 15s and fixed counters) configurable via the Telegram adapter config so different deployment environments can tune sensitivity without code changes.
  • The outbound client lifecycle now depends on several flags (_outbound_application, _outbound_client_initialized, _replace_outbound_client_on_recovery); it would be helpful to centralize and document the state transitions to reduce the chance of subtle bugs when future changes touch polling recovery or termination.
Prompt for AI Agents
Please address the comments from this code review:

## Overall Comments
- Consider making the polling watchdog thresholds and connect timeout (currently hardcoded to 15s and fixed counters) configurable via the Telegram adapter config so different deployment environments can tune sensitivity without code changes.
- The outbound client lifecycle now depends on several flags (`_outbound_application`, `_outbound_client_initialized`, `_replace_outbound_client_on_recovery`); it would be helpful to centralize and document the state transitions to reduce the chance of subtle bugs when future changes touch polling recovery or termination.

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:platform The bug / feature is about IM platform adapter, such as QQ, Lark, Telegram, WebChat and so on. size:L This PR changes 100-499 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant