Skip to content

refactor proof of concept - #203

Draft
raghavm243512 wants to merge 3 commits into
mainfrom
refactor/realtime_new_design
Draft

refactor proof of concept#203
raghavm243512 wants to merge 3 commits into
mainfrom
refactor/realtime_new_design

Conversation

@raghavm243512

@raghavm243512 raghavm243512 commented Aug 5, 2026

Copy link
Copy Markdown
Collaborator

This is the first of the main changes to how we organize servers and user simulators.

The key here is rather than openai_realtime_user_simulator and openai_realtime_server.py, we just have user, assistant, and openai_realtime. This means every assistant backend will work as a user, including cascade architectures, ALMs, and all the modularity that comes with those. Future contributions will be far more useful and far easier to do.

All API interaction code is shared, meaning this design scheme will mean adding support for 1 API will work in any role, no more dedicated implementations for either role in the conversation.

I started with openai realtime as it is the simplest, single part, no separate VAD and it was supported on both sides.

This PR will not be merged as is, it features temporary test gates, and the config is too restrictive for what we actually need for the user simulator, requiring a moderate size redesign for the user to really work as we want it.

There will also be deletions for old files not done here. This is solely to review, build on top of, and hopefully figure out a nice way to merge it all without the temporary clutter in the code or configs.

This has been tested with the new code path being used for both sides of the conversation and all results are identical

General shape

As mentioned, we now have user and assistant objects and these each have an instance of "backend", which can be any backend EVA connects to
This entire construction process is done in the worker like before, which simply invokes a backend factory, and gives this backend to each role
The worker never interacts with any backend, each role abstracts the API details completely from the rest of the project and each Backend class completely abstracts the actual API from the Role classes

@JosephMarinier JosephMarinier left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not now, but once all the pieces are in place, let's try to clean up the comments about historical changes. We will no longer care if the code has been refactored, moved, renamed, restored, etc. A couple of examples I saw:

Comment thread src/eva/backend/base.py Outdated
Comment on lines +3 to +4
Step 1 of the refactor (see docs/refactor-step1.md). This is the live
``Backend`` contract -- implemented by ``eva.backend.openai_realtime`` and

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Step 1 of the refactor (see docs/refactor-step1.md). This is the live
``Backend`` contract -- implemented by ``eva.backend.openai_realtime`` and
This is the live ``Backend`` contract -- implemented by ``eva.backend.openai_realtime`` and

Comment thread src/eva/assistant/pipeline/observers.py Outdated
Comment on lines +393 to +396
# Convenience wrappers over write(). Restored after commit 4bf4881 ("move and
# rename") dropped them while relocating FrameworkLogWriter from audio_bridge.py
# to this module, leaving openai_realtime_server / gemini_live_server calling
# methods that no longer existed.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
# Convenience wrappers over write(). Restored after commit 4bf4881 ("move and
# rename") dropped them while relocating FrameworkLogWriter from audio_bridge.py
# to this module, leaving openai_realtime_server / gemini_live_server calling
# methods that no longer existed.

@raghavm243512
raghavm243512 force-pushed the refactor/realtime_new_design branch 2 times, most recently from 855058f to f0ca37e Compare August 7, 2026 01:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants