Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

100% CPU consumption #72

Closed
SmileLikeGod opened this issue Sep 12, 2024 · 4 comments
Closed

100% CPU consumption #72

SmileLikeGod opened this issue Sep 12, 2024 · 4 comments
Assignees
Labels
bug Something isn't working

Comments

@SmileLikeGod
Copy link

SmileLikeGod commented Sep 12, 2024

Good evening, when recording a broadcast that switches to paid streaming during the live session, fc2-live-dl-go starts consuming 100% of the CPU capacity.
Here is the log:

9:32PM ERR github.com/Darkness4/fc2-live-dl-go/fc2/fc2.go:448 > ws listen failed error="login required" channelID=58968002
9:32PM WRN github.com/Darkness4/fc2-live-dl-go/fc2/fc2_websocket.go:313 > canceled awaiting for response error="context canceled"
9:32PM WRN github.com/Darkness4/fc2-live-dl-go/utils/try/try.go:161 > canceled all tries
9:32PM ERR github.com/Darkness4/fc2-live-dl-go/fc2/fc2.go:499 > failed to fetch playlist error="context canceled" channelID=58968002
9:32PM INF github.com/Darkness4/fc2-live-dl-go/fc2/fc2.go:576 > cancelling goroutine group... channelID=58968002
9:32PM INF github.com/Darkness4/fc2-live-dl-go/fc2/fc2.go:427 > healthcheck canceled channelID=58968002

and a screenshot https://imgur.com/a/B67W1Ue
Here are the settings:

docker run -it --rm \
  -u 1000:1000 \
  -v /usr/videos/:/out \
  ghcr.io/darkness4/fc2-live-dl-go:latest download \
  --wait-for-quality-max-tries 5 \
  --no-remux \
  --poll-interval 360s\
  --format "/out/{{ .Date }} {{ .Time}}.{{ .Ext }}" 58968002

I tried waiting for 5 minutes, but it just kept using 100% of the CPU.

@Darkness4 Darkness4 added the bug Something isn't working label Sep 23, 2024
@Darkness4 Darkness4 self-assigned this Sep 23, 2024
@Darkness4
Copy link
Owner

Darkness4 commented Sep 23, 2024

Hello, it seems that FC2 has changed the login API, which no longer allows you to connect (actually you can still connect, but the logs doesn't say anything anymore).

As for why there is 100% CPU, I'm looking into it... but it will be quite difficult to find.
I will probably refactor the whole codebase to try to add new test cases. This will also permits me to easily add contract testing, which will permits me to avoid drifting from future API changes.

@Darkness4
Copy link
Owner

Darkness4 commented Sep 24, 2024

The deadlock has been found and an infinite for loop is causing 100% CPU:

fc2-live-dl-go/fc2/fc2.go

Lines 615 to 624 in 52bc0b1

for {
select {
// Received a new playlist URL
case playlist, ok := <-playlists:
if !ok {
// Playlist channel closed, meaning the stream ended.
if currentCancel != nil {
currentCancel()
}
continue

If the channel playlists is closed, the program will immediatly enter the <- playlists case with ok=false. Usual behavior is that the hls.Downloader.Read returns the err to the errChan (few lines below, in the same function), which move the program to the <-errChan case and avoid the <-playlists case. Because no err is in the errChan, the loop continues.

The continue increase the CPU cycle, causing 100% CPU. The fix is being worked on at the same time with the refactor.

@Darkness4
Copy link
Owner

Closed with merged PR #74, released on 1.8.4-rc.1. Will be released on 1.8.4 after a release candidate is considered stable (1w without bugs).

@Darkness4
Copy link
Owner

Darkness4 commented Sep 28, 2024

A regression was introduced in 1.8.4-rc.1 causing a deadlock after finishing the download. Please use 1.8.4-rc.2.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants