Skip to content

fix: allow running graphs inside an existing event loop (Jupyter, FastAPI) - #1117

Open
hakoke wants to merge 1 commit into
ScrapeGraphAI:mainfrom
hakoke:fix/asyncio-running-loop
Open

fix: allow running graphs inside an existing event loop (Jupyter, FastAPI)#1117
hakoke wants to merge 1 commit into
ScrapeGraphAI:mainfrom
hakoke:fix/asyncio-running-loop

Conversation

@hakoke

@hakoke hakoke commented Jul 28, 2026

Copy link
Copy Markdown

Fixes #1116.

Adds scrapegraphai/utils/event_loop.py with one helper, run_coroutine_sync: it uses asyncio.run() when no loop is running, and otherwise executes the coroutine on its own loop in a worker thread, since both asyncio.run() and run_until_complete() raise inside a running loop. The four crash sites now use it: ChromiumLoader.lazy_load, browser_base_fetch, GraphIteratorNode.execute and GenerateAnswerFromImageNode.execute. The two nodes had the check inverted, calling run_until_complete precisely when the loop was running. The worker thread follows the same idea as AbstractGraph.run_safe_async, which already offloads the blocking call to an executor. Plain scripts are unchanged: with no running loop the path is still a direct asyncio.run() on the calling thread.

Verified on Windows with Python 3.12: a real Playwright fetch through ChromiumLoader.load() from inside asyncio.run(...) now returns the page instead of raising, same for GraphIteratorNode and GenerateAnswerFromImageNode (the latter also no longer leaks a cancelled-task traceback), exceptions raised inside coroutines reach the caller unchanged, and the nine test files CI runs pass before and after (66 passed both times). Note on #1087: it refactors the scraping_fn selection a few lines above this change inside lazy_load; if it lands first I will rebase, the overlap is trivial. Happy to add a mocked unit test for the helper and register it in test-suite.yml if you want one.

@dosubot dosubot Bot added the size:M This PR changes 30-99 lines, ignoring generated files. label Jul 28, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:M This PR changes 30-99 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Graphs cannot run inside an existing event loop (Jupyter, Colab, FastAPI), still in 2.1.6

1 participant