Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RTMP Playback Delayed On Tracks Without Audio #3188

Closed
tylerjroach opened this issue Aug 17, 2017 · 1 comment
Closed

RTMP Playback Delayed On Tracks Without Audio #3188

tylerjroach opened this issue Aug 17, 2017 · 1 comment
Labels

Comments

@tylerjroach
Copy link
Contributor

tylerjroach commented Aug 17, 2017

Issue description

If an rtmp stream does not have an audio track, the video will take x amount of time to display a video frame, where x is the amount of time the stream has been running.
Ex. if the rtmp stream was started 100 seconds ago, it will take Exoplayer 100 seconds to show a video frame.

The reason this occurs is because MediaCodecVideoRenderer does not extend MediaClock. With the lack of a media clock for the stream, the time source used will be from standaloneMediaClock in ExoPlayerImplInternal. standaloneMediaClock starts at a 0, so the incoming frames will be considered too early and not render until standaloneMediaClock hits the time of the frame.

if (rendererMediaClockSource != null && !rendererMediaClockSource.isEnded()) {
    rendererPositionUs = rendererMediaClock.getPositionUs();
    standaloneMediaClock.setPositionUs(rendererPositionUs);
} else {
    rendererPositionUs = standaloneMediaClock.getPositionUs();
}

Potential Solution

A quick fix is to set a start offset in FLVExtractor so that our frames start coming in at 0. I realized this is probably a hacky method and you lose the ability to know how long the stream has been running. This may not be a big deal as RTMP streams are already not seekable and do not show a duration. I'd be happy to submit a pull request if this is an acceptable solution.

Note: I'm storing the previous timestamp to re-use in the event that I get timestamps of 0 midstream. I've noticed that I have a tagTimestampUs of 0 if the rtmp server sends a header of type small. Not sure if this is an issue/bug. I didn't notice any playback issues, but it didn't feel right to pass on 0 timestamps to the readers.

/**
 * Reads the body of a tag from the provided {@link ExtractorInput}.
 *
 * @param input The {@link ExtractorInput} from which to read.
 * @return True if the data was consumed by a reader. False if it was skipped.
 * @throws IOException If an error occurred reading or parsing data from the source.
 * @throws InterruptedException If the thread was interrupted.
 */
private boolean readTagData(ExtractorInput input) throws IOException, InterruptedException {
  boolean wasConsumed = true;
  long adjustedTimestamp;
  if (tagTimestampUs > 0) {
    if (tagTimestampOffsetUs == 0 || tagTimestampUs < tagTimestampOffsetUs) {
      tagTimestampOffsetUs = tagTimestampUs;
    }
    adjustedTimestamp = Math.max(0, tagTimestampUs - tagTimestampOffsetUs);
  } else {
    adjustedTimestamp = previousAdjustedTimestamp;
  }
  previousAdjustedTimestamp = adjustedTimestamp;
  if (tagType == TAG_TYPE_AUDIO && audioReader != null) {
    audioReader.consume(prepareTagData(input), adjustedTimestamp);
  } else if (tagType == TAG_TYPE_VIDEO && videoReader != null) {
    videoReader.consume(prepareTagData(input), adjustedTimestamp);
  } else if (tagType == TAG_TYPE_SCRIPT_DATA && metadataReader != null) {
    metadataReader.consume(prepareTagData(input), adjustedTimestamp);
  } else {
    input.skipFully(tagDataSize);
    wasConsumed = false;
  }
  bytesToNextTagHeader = 4; // There's a 4 byte previous tag size before the next header.
  parserState = STATE_SKIPPING_TO_TAG_HEADER;
  return wasConsumed;
}

It looks like the better solution would be to use MediaPeriod similar to how HLS does.
I noticed that HLS sets the initial time at this line:
resetRendererPosition(readingPeriodHolder.info.startPositionUs);
I don't have a solution yet for this method.

Reproduction steps

Play any rtmp video without an audio track.

Link to test content

I was manually starting rtmp videos without audio and I can't leave it up 24x7. I can coordinate a time if anyone would like to test.

Version of ExoPlayer being used

2.5.1

Device(s) and version(s) of Android being used

All

@PanCrucian
Copy link

@tylerjroach can u please present a full code of com.google.android.exoplayer2.extractor.flv.FlvExtractor.class ?

ojw28 added a commit that referenced this issue Dec 4, 2017
Issue: #3188

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=177811487
@ojw28 ojw28 closed this as completed Dec 4, 2017
ojw28 added a commit that referenced this issue Dec 12, 2017
Issue: #3188

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=177811487
@google google locked and limited conversation to collaborators Apr 10, 2018
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Projects
None yet
Development

No branches or pull requests

3 participants