Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Optimized the memory usage when writing chunks #138

Merged
merged 4 commits into from
Apr 8, 2021

Conversation

hhxsv5
Copy link
Contributor

@hhxsv5 hhxsv5 commented Apr 8, 2021

Hello guys, I did a test, substr() has no memory overhead and is more efficient than str_split().

$bigStr = str_repeat('abcdefghijklmnopqrstuvwxyz0123456789', 1000000);
$memStart = memory_get_usage(true);
$timeStart = microtime(true);

// ----- substr()
$len = strlen($bigStr);
$chunkLimit = 4096;
for ($offset = 0; $offset < $len; $offset += $chunkLimit) {
    $chunk = substr($bigStr, $offset, $chunkLimit);
}

// ----- str_split()
//$chunkLimit = 4096;
//foreach (str_split($bigStr, $chunkLimit) as $chunk) {
//}

$costMem = memory_get_usage(true) - $memStart;
$costTime = microtime(true) - $timeStart;
echo "Memory: {$costMem} Time: {$costTime}\n";
// substr()
Memory: 0 Time: 0.0036599636077881

// str_split()
Memory: 2097152 Time: 0.043421983718872

reference

@driesvints driesvints changed the title Optimized the memory usage when writing chunks. Optimized the memory usage when writing chunks Apr 8, 2021
@barryvdh
Copy link
Contributor

barryvdh commented Apr 8, 2021

Why is your reference using factor 0.6 upon writing? Because of gzipping?
And why is the chunkSize 2MB instead of one there?

@hhxsv5
Copy link
Contributor Author

hhxsv5 commented Apr 8, 2021

Swoole's buffer_output_size is 2MB by default, but it sometimes fails(maybe a Swoole bug) when writing 2MB data after testing, So probably use 0.5*2MB, I think it's a safe setting.

@taylorotwell taylorotwell merged commit efc4b0a into laravel:master Apr 8, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants