Skip to content

Fix/surface pts shares audio origin #2181

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
pedroSG94 merged 3 commits into from
Aug 12, 2026
Merged

Fix/surface pts shares audio origin #2181

Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
52b7d14
Rebase surface PTS on the shared start instead of the first video frame
x270880x Aug 11, 2026
4caaf7d
fix no gl renders pts
pedroSG94 Aug 11, 2026
911fc23
code tab
pedroSG94 Aug 11, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 18 additions & 5 deletions encoder/src/main/java/com/pedro/encoder/video/VideoEncoder.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -444,6 +444,21 @@ public void formatChanged(@NonNull MediaCodec mediaCodec, @NonNull MediaFormat m
spsPpsSetted = sendSPSandPPS(mediaFormat);
}

/**
* Surface mode: rebase the timestamp to something relative. The GlInterface stamps the surface
* with the TimeUtils clock, so we share the origin with the audio encoder. A surface fed directly
* (decoder in FromFileBase, MediaProjection in DisplayBase without opengl) uses another time base
* and must be rebased against its own first frame.
* Using 1 second to difference both ways.
*/
private long rebaseSurfacePts(long pts) {
if (firstTimestamp == 0) {
boolean sharedClock = presentTimeUs > 0 && Math.abs(pts - TimeUtils.getCurrentTimeMicro()) < 1_000_000;
firstTimestamp = sharedClock ? presentTimeUs : pts;
}
return Math.max(0, pts - firstTimestamp);
}

@Override
protected boolean checkBuffer(@NonNull ByteBuffer byteBuffer, @NonNull MediaCodec.BufferInfo bufferInfo) {
if (forceKey && Build.VERSION.SDK_INT >= Build.VERSION_CODES.KITKAT) {
Expand Down Expand Up @@ -510,12 +525,10 @@ protected boolean checkBuffer(@NonNull ByteBuffer byteBuffer, @NonNull MediaCode
// Buffer mode: synthesize PTS from wall clock.
bufferInfo.presentationTimeUs = TimeUtils.getCurrentTimeMicro() - presentTimeUs;
} else {
// Surface mode: EGL timestamp is camera sensor time (nanoseconds from boot ÷ 1000).
// It has clean, jitter-free intervals — but it's a huge absolute value that breaks RTMP.
// Rebase to relative by subtracting the first frame's PTS → clean intervals, starts at 0.
if (firstTimestamp == 0) firstTimestamp = bufferInfo.presentationTimeUs;
bufferInfo.presentationTimeUs -= firstTimestamp;
bufferInfo.presentationTimeUs = rebaseSurfacePts(bufferInfo.presentationTimeUs);
}
} else if (formatVideoEncoder == FormatVideoEncoder.SURFACE) {
bufferInfo.presentationTimeUs = rebaseSurfacePts(bufferInfo.presentationTimeUs);
} else {
if (firstTimestamp == 0) firstTimestamp = bufferInfo.presentationTimeUs;
bufferInfo.presentationTimeUs -= firstTimestamp;
Expand Down
Loading