Skip to content

Commit

Permalink
Update website/docs/docs/build/incremental-microbatch.md
Browse files Browse the repository at this point in the history
  • Loading branch information
nataliefiann authored Dec 5, 2024
1 parent e060242 commit 504fb91
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion website/docs/docs/build/incremental-microbatch.md
Original file line number Diff line number Diff line change
Expand Up @@ -321,7 +321,7 @@ Otherwise, if the `concurrent_batches` value isn't set _and_ `{{ this }}` isn't



### Parallel batch execution or sequential processing?
Choosing between parallel batch execution and sequential processing depends on the specific requirements of your application.

- Parallel batch execution is faster but requires logic that is independent of batch execution order. For example, if you are developing a data processing pipeline for a system that processes user transactions in batches, each batch is executed in parallel for better performance. However, the logic used to process each transaction must not depend on the order of how batches are executed or completed.
- Sequential processing isn't as performant and is slower however, it enables calculations such as cumulative metrics in microbatch models. Since cumulative metrics require data to be processed in the correct order to ensure each step builds on the previous one, sequential processing is ideal.
Expand Down

0 comments on commit 504fb91

Please sign in to comment.