Skip to content

Commit

Permalink
Document the query builder's new lazy() method
Browse files Browse the repository at this point in the history
  • Loading branch information
JosephSilber committed Mar 24, 2021
1 parent f304a71 commit 4342d6d
Show file tree
Hide file tree
Showing 2 changed files with 93 additions and 26 deletions.
92 changes: 66 additions & 26 deletions eloquent.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@
- [Retrieving Models](#retrieving-models)
- [Collections](#collections)
- [Chunking Results](#chunking-results)
- [Streaming Results Lazily](#streaming-results-lazily)
- [Cursors](#cursors)
- [Advanced Subqueries](#advanced-subqueries)
- [Retrieving Single Models / Aggregates](#retrieving-single-models)
Expand Down Expand Up @@ -318,9 +319,11 @@ In addition to the methods provided by Laravel's base collection class, the Eloq

Since all of Laravel's collections implement PHP's iterable interfaces, you may loop over collections as if they were an array:

foreach ($flights as $flight) {
echo $flight->name;
}
```php
foreach ($flights as $flight) {
echo $flight->name;
}
```

<a name="chunking-results"></a>
### Chunking Results
Expand All @@ -329,47 +332,84 @@ Your application may run out of memory if you attempt to load tens of thousands

The `chunk` method will retrieve a subset of Eloquent models, passing them to a closure for processing. Since only the current chunk of Eloquent models is retrieved at a time, the `chunk` method will provide significantly reduced memory usage when working with a large number of models:

use App\Models\Flight;
```php
use App\Models\Flight;

Flight::chunk(200, function ($flights) {
foreach ($flights as $flight) {
//
}
});
Flight::chunk(200, function ($flights) {
foreach ($flights as $flight) {
//
}
});
```

The first argument passed to the `chunk` method is the number of records you wish to receive per "chunk". The closure passed as the second argument will be invoked for each chunk that is retrieved from the database. A database query will be executed to retrieve each chunk of records passed to the closure.

If you are filtering the results of the `chunk` method based on a column that you will also be updating while iterating over the results, you should use the `chunkById` method. Using the `chunk` method in these scenarios could lead to unexpected and inconsistent results. Internally, the `chunkById` method will always retrieve models with an `id` column greater than the last model in the previous chunk:

Flight::where('departed', true)
->chunkById(200, function ($flights) {
$flights->each->update(['departed' => false]);
}, $column = 'id');
```php
Flight::where('departed', true)
->chunkById(200, function ($flights) {
$flights->each->update(['departed' => false]);
}, $column = 'id');
```

<a name="streaming-results-lazily"></a>
### Streaming Results Lazily

The `lazy()` method works similarly to [the `chunk` method](#chunking-results), executing the query in chunks. However, instead of passing each chunk into a callback, the `lazy()` method returns [a `LazyCollection`](/docs/{{version}}/collections#lazy-collections) of Eloquent models, which lets you interact with the results as a single stream:

```php
use App\Models\Flight;

foreach (Flight::lazy() as $flight) {
//
}
```

Once again, if you are filtering the results based on a column that you will also be updating while iterating over the results, you should use the `lazyById` method. Internally, the `lazyById` method will always retrieve models with an `id` column greater than the last model in the previous chunk:

```php
Flight::where('departed', true)
->lazyById(200, $column = 'id')
->each->update(['departed' => false]);
```

<a name="cursors"></a>
### Cursors

Similar to the `chunk` method, the `cursor` method may be used to significantly reduce your application's memory consumption when iterating through tens of thousands of Eloquent model records.
Similar to the `lazy` method, the `cursor` method may be used to significantly reduce your application's memory consumption when iterating through tens of thousands of Eloquent model records.

The `cursor` method will only execute a single database query; however, the individual Eloquent models will not be hydrated until they are actually iterated over. Therefore, only one Eloquent model is kept in memory at any given time while iterating over the cursor. Internally, the `cursor` method uses PHP [generators](https://www.php.net/manual/en/language.generators.overview.php) to implement this functionality:
The `cursor` method will only execute a single database query; however, the individual Eloquent models will not be hydrated until they are actually iterated over. Therefore, only one Eloquent model is kept in memory at any given time while iterating over the cursor.

use App\Models\Flight;
> {note} Since the `cursor` method only ever holds a single Eloquent model in memory at a time, it cannot eager load relationships. If you need to eager load models, consider using [the `lazy` method](#streaming-results-lazily) instead.
foreach (Flight::where('destination', 'Zurich')->cursor() as $flight) {
//
}
Internally, the `cursor` method uses PHP [generators](https://www.php.net/manual/en/language.generators.overview.php) to implement this functionality:

```php
use App\Models\Flight;

foreach (Flight::where('destination', 'Zurich')->cursor() as $flight) {
//
}
```

The `cursor` returns an `Illuminate\Support\LazyCollection` instance. [Lazy collections](/docs/{{version}}/collections#lazy-collections) allow you to use many of the collection methods available on typical Laravel collections while only loading a single model into memory at a time:

use App\Models\User;
```php
use App\Models\User;

$users = User::cursor()->filter(function ($user) {
return $user->id > 500;
});
$users = User::cursor()->filter(function ($user) {
return $user->id > 500;
});

foreach ($users as $user) {
echo $user->id;
}
foreach ($users as $user) {
echo $user->id;
}
```

> {note} Although the `cursor` method uses far less memory than a regular query (by only holding a single Eloquent model in memory at a time), it will still eventually run out of memory. This is [due to PHP's PDO driver internally caching all raw query results in its buffer](https://www.php.net/manual/en/mysqlinfo.concepts.buffering.php).
>
> If you're dealing with an enormous amount of data, consider using [the `lazy` method](#streaming-results-lazily) instead.
<a name="advanced-subqueries"></a>
### Advanced Subqueries
Expand Down
27 changes: 27 additions & 0 deletions queries.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@
- [Introduction](#introduction)
- [Running Database Queries](#running-database-queries)
- [Chunking Results](#chunking-results)
- [Streaming Results Lazily](#streaming-results-lazily)
- [Aggregates](#aggregates)
- [Select Statements](#select-statements)
- [Raw Expressions](#raw-expressions)
Expand Down Expand Up @@ -154,6 +155,32 @@ If you are updating database records while chunking results, your chunk results

> {note} When updating or deleting records inside the chunk callback, any changes to the primary key or foreign keys could affect the chunk query. This could potentially result in records not being included in the chunked results.
<a name="streaming-results-lazily"></a>
### Streaming Results Lazily

The `lazy()` method works similarly to [the `chunk` method](#chunking-results), executing the query in chunks. However, instead of passing each chunk into a callback, the `lazy()` method returns [a `LazyCollection`](/docs/{{version}}/collections#lazy-collections), which lets you interact with the results as a single stream:

```php
use Illuminate\Support\Facades\DB;

DB::table('users')->lazy()->each(function ($user) {
//
});
```

Once again, if you plan to update the retrieved records while iterating over them, it is best to use the `lazyById` method instead. This method will automatically paginate the results based on the record's primary key:

```php
DB::table('users')->where('active', false)
->lazyById()->each(function ($user) {
DB::table('users')
->where('id', $user->id)
->update(['active' => true]);
});
```

> {note} When updating or deleting records while iterating over them, any changes to the primary key or foreign keys could affect the chunk query. This could potentially result in records not being included in the results.
<a name="aggregates"></a>
### Aggregates

Expand Down

0 comments on commit 4342d6d

Please sign in to comment.