Questions about segment fetch behaviour which may return 404

[chrisfillmore]

I've been asked by our content infrastructure team to provide some insight on Shaka's behaviour when fetching segments for a live stream. They tell me that they're seeing an unusual number of requests which return 404, under the following circumstances:

1. Content is being requested by the client that is in the future (content does not exist on the origin server yet)
2. Content is being requested by the client that is in the past and outside of the availability window (this is why I was asking about Refactor to derive live edge from list of segments #999)
3. Content is being requested that is in between two valid segments (content that does not align with segment boundaries defined in the manifest)

At the moment I don't have a reproducible test case, or network logs showing the requests. As our investigation progresses I may be able to provide this. I'm hoping you can provide some insight into whether Shaka could or would make any of the above requests for segments.

We're also following up with ExoPlayer, since the issue is seen with DASH content generally. Let me know if I can provide any more information. Thanks!

[TheModMaker]

1. Content is being requested by the client that is in the future (content does not exist on the origin server yet)

Some DASH manifests can specify all future segments, so we don't have to wait for a manifest update to know the times and URLs for the media segments. This means that we could request a segment before the server generated it. This can happen if there are clock sync problems or if the server is slowing down.

This shouldn't cause a problem since we will wait a few seconds and retry the request.

2. Content is being requested by the client that is in the past and outside of the availability window (this is why I was asking about Refactor to derive live edge from list of segments #999)

This can be caused by encoder drift or clock sync issues where we choose an incorrect start time. Those may be fixed by #999.

3. Content is being requested that is in between two valid segments (content that does not align with segment boundaries defined in the manifest)

This is definitely a bug, I don't even know how this could happen. If you have a repro, please file a bug for this.

[chrisfillmore]

Thanks for the info. Regarding (3), we have an open ticket internally which alleges that you can reproduce the problem in VOD (or perhaps restarted live content) by doing a lot of scrubbing. The ticket was filed by one of our client architects who believes this is the cause of cache misses at the CDN. I've never reproduced this myself.

[joeyparrish]

@chrisfillmore, have we answered your question? Is there anything else we can do for you on this?

[chrisfillmore]

Thanks @joeyparrish, I think I have the answers I need. I'll reopen if I have to follow up again.