Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[8.x] Document the query builder's new lazy() method #6943

Merged
merged 1 commit into from
Mar 24, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
92 changes: 66 additions & 26 deletions eloquent.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@
- [Retrieving Models](#retrieving-models)
- [Collections](#collections)
- [Chunking Results](#chunking-results)
- [Streaming Results Lazily](#streaming-results-lazily)
- [Cursors](#cursors)
- [Advanced Subqueries](#advanced-subqueries)
- [Retrieving Single Models / Aggregates](#retrieving-single-models)
Expand Down Expand Up @@ -318,9 +319,11 @@ In addition to the methods provided by Laravel's base collection class, the Eloq

Since all of Laravel's collections implement PHP's iterable interfaces, you may loop over collections as if they were an array:

foreach ($flights as $flight) {
echo $flight->name;
}
```php
foreach ($flights as $flight) {
echo $flight->name;
}
```

<a name="chunking-results"></a>
### Chunking Results
Expand All @@ -329,47 +332,84 @@ Your application may run out of memory if you attempt to load tens of thousands

The `chunk` method will retrieve a subset of Eloquent models, passing them to a closure for processing. Since only the current chunk of Eloquent models is retrieved at a time, the `chunk` method will provide significantly reduced memory usage when working with a large number of models:

use App\Models\Flight;
```php
use App\Models\Flight;

Flight::chunk(200, function ($flights) {
foreach ($flights as $flight) {
//
}
});
Flight::chunk(200, function ($flights) {
foreach ($flights as $flight) {
//
}
});
```

The first argument passed to the `chunk` method is the number of records you wish to receive per "chunk". The closure passed as the second argument will be invoked for each chunk that is retrieved from the database. A database query will be executed to retrieve each chunk of records passed to the closure.

If you are filtering the results of the `chunk` method based on a column that you will also be updating while iterating over the results, you should use the `chunkById` method. Using the `chunk` method in these scenarios could lead to unexpected and inconsistent results. Internally, the `chunkById` method will always retrieve models with an `id` column greater than the last model in the previous chunk:

Flight::where('departed', true)
->chunkById(200, function ($flights) {
$flights->each->update(['departed' => false]);
}, $column = 'id');
```php
Flight::where('departed', true)
->chunkById(200, function ($flights) {
$flights->each->update(['departed' => false]);
}, $column = 'id');
```

<a name="streaming-results-lazily"></a>
### Streaming Results Lazily

The `lazy()` method works similarly to [the `chunk` method](#chunking-results), executing the query in chunks. However, instead of passing each chunk into a callback, the `lazy()` method returns [a `LazyCollection`](/docs/{{version}}/collections#lazy-collections) of Eloquent models, which lets you interact with the results as a single stream:

```php
use App\Models\Flight;

foreach (Flight::lazy() as $flight) {
//
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@JosephSilber I'm wondering if $flight is an instance of Eloquent or a chunk set of Eloquent?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

instead of passing each chunk into a callback, the lazy() method returns [...] results as a single stream

So $flight is a single Eloquent model.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. I thought that was clear enough. Is it not?

  2. If it isn't, would this help?

    image

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ability to pass chunkCount throws me off there. Understood now

Copy link
Member Author

@JosephSilber JosephSilber Mar 25, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The $chunkCount determines how many records should be requested per chunk behind the scenes.

Bigger chunks use more memory, while smaller chunks means more queries (potentially exacerbated by eager loading relationships).

```

Once again, if you are filtering the results based on a column that you will also be updating while iterating over the results, you should use the `lazyById` method. Internally, the `lazyById` method will always retrieve models with an `id` column greater than the last model in the previous chunk:

```php
Flight::where('departed', true)
->lazyById(200, $column = 'id')
->each->update(['departed' => false]);
```

<a name="cursors"></a>
### Cursors

Similar to the `chunk` method, the `cursor` method may be used to significantly reduce your application's memory consumption when iterating through tens of thousands of Eloquent model records.
Similar to the `lazy` method, the `cursor` method may be used to significantly reduce your application's memory consumption when iterating through tens of thousands of Eloquent model records.

The `cursor` method will only execute a single database query; however, the individual Eloquent models will not be hydrated until they are actually iterated over. Therefore, only one Eloquent model is kept in memory at any given time while iterating over the cursor. Internally, the `cursor` method uses PHP [generators](https://www.php.net/manual/en/language.generators.overview.php) to implement this functionality:
The `cursor` method will only execute a single database query; however, the individual Eloquent models will not be hydrated until they are actually iterated over. Therefore, only one Eloquent model is kept in memory at any given time while iterating over the cursor.

use App\Models\Flight;
> {note} Since the `cursor` method only ever holds a single Eloquent model in memory at a time, it cannot eager load relationships. If you need to eager load models, consider using [the `lazy` method](#streaming-results-lazily) instead.

foreach (Flight::where('destination', 'Zurich')->cursor() as $flight) {
//
}
Internally, the `cursor` method uses PHP [generators](https://www.php.net/manual/en/language.generators.overview.php) to implement this functionality:

```php
use App\Models\Flight;

foreach (Flight::where('destination', 'Zurich')->cursor() as $flight) {
//
}
```

The `cursor` returns an `Illuminate\Support\LazyCollection` instance. [Lazy collections](/docs/{{version}}/collections#lazy-collections) allow you to use many of the collection methods available on typical Laravel collections while only loading a single model into memory at a time:

use App\Models\User;
```php
use App\Models\User;

$users = User::cursor()->filter(function ($user) {
return $user->id > 500;
});
$users = User::cursor()->filter(function ($user) {
return $user->id > 500;
});

foreach ($users as $user) {
echo $user->id;
}
foreach ($users as $user) {
echo $user->id;
}
```

> {note} Although the `cursor` method uses far less memory than a regular query (by only holding a single Eloquent model in memory at a time), it will still eventually run out of memory. This is [due to PHP's PDO driver internally caching all raw query results in its buffer](https://www.php.net/manual/en/mysqlinfo.concepts.buffering.php).
>
> If you're dealing with an enormous amount of data, consider using [the `lazy` method](#streaming-results-lazily) instead.

<a name="advanced-subqueries"></a>
### Advanced Subqueries
Expand Down
27 changes: 27 additions & 0 deletions queries.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@
- [Introduction](#introduction)
- [Running Database Queries](#running-database-queries)
- [Chunking Results](#chunking-results)
- [Streaming Results Lazily](#streaming-results-lazily)
- [Aggregates](#aggregates)
- [Select Statements](#select-statements)
- [Raw Expressions](#raw-expressions)
Expand Down Expand Up @@ -154,6 +155,32 @@ If you are updating database records while chunking results, your chunk results

> {note} When updating or deleting records inside the chunk callback, any changes to the primary key or foreign keys could affect the chunk query. This could potentially result in records not being included in the chunked results.

<a name="streaming-results-lazily"></a>
### Streaming Results Lazily

The `lazy()` method works similarly to [the `chunk` method](#chunking-results), executing the query in chunks. However, instead of passing each chunk into a callback, the `lazy()` method returns [a `LazyCollection`](/docs/{{version}}/collections#lazy-collections), which lets you interact with the results as a single stream:

```php
use Illuminate\Support\Facades\DB;

DB::table('users')->lazy()->each(function ($user) {
//
});
```

Once again, if you plan to update the retrieved records while iterating over them, it is best to use the `lazyById` method instead. This method will automatically paginate the results based on the record's primary key:

```php
DB::table('users')->where('active', false)
->lazyById()->each(function ($user) {
DB::table('users')
->where('id', $user->id)
->update(['active' => true]);
});
```

> {note} When updating or deleting records while iterating over them, any changes to the primary key or foreign keys could affect the chunk query. This could potentially result in records not being included in the results.

<a name="aggregates"></a>
### Aggregates

Expand Down