Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Bulk Transaction Processing #28580

Merged
merged 26 commits into from
Feb 7, 2022

Conversation

mohammedyusufshaikh
Copy link
Contributor

@mohammedyusufshaikh mohammedyusufshaikh commented Nov 26, 2021

Before this Feature
Lets take an example to get better idea

  1. If the user had to create multiple sales invoice from multiple sales order it had to be done one by one by going into the particular sales order and then create sales invoice against it which consumed a lot of time.Similarly this was the case for many doctype's for which creating multiple docs from multiple docs was time consuming.
  2. Problem was identified from this issue Bulk Transaction Processing #28229

Tasks

  • Notification to the user when job is completed and some minor tasks.
  • Add Logger For Transactions
  • Tests
  • Increase Code Coverage for codecov/project

Following operations can be performed in bulk:

Screenshot 2021-11-26 at 1 49 25 PM

After this Feature

  1. To tackle this problem options are added into the actions menu. so when the user selects multiple invoices/orders the actions menu button appears from which user can select the bulk operation to be performed. thus saving a lot of time.
  2. Since creation of multiple invoices/orders can take a lot of time it should not block user from doing other operations. so if there are more than 10 invoices/orders it will create a background job for the same.
  3. There might be situation where in while bulk creating of invoices/orders some exceptions might occur which might halt the creation of invoices/orders. There is a mechanism to bypass failing invoices/orders and continue forward creating other invoices/orders in the queue.

bulk_transaction_processing

@codecov
Copy link

codecov bot commented Nov 26, 2021

Codecov Report

Merging #28580 (28d722e) into develop (180f06d) will decrease coverage by 0.01%.
The diff coverage is 79.56%.

❗ Current head 28d722e differs from pull request most recent head 00074bc. Consider uploading reports for the commit 00074bc to get more accurate results

@@             Coverage Diff             @@
##           develop   #28580      +/-   ##
===========================================
- Coverage    58.03%   58.02%   -0.02%     
===========================================
  Files         1091     1095       +4     
  Lines        67866    68003     +137     
===========================================
+ Hits         39388    39459      +71     
- Misses       28478    28544      +66     
Impacted Files Coverage Δ
erpnext/hooks.py 0.00% <ø> (ø)
...pnext/stock/doctype/delivery_note/delivery_note.py 65.73% <ø> (ø)
...pnext/tests/ui_test_bulk_transaction_processing.py 0.00% <0.00%> (ø)
...g/doctype/supplier_quotation/supplier_quotation.py 80.00% <50.00%> (-1.70%) ⬇️
...ctype/bulk_transaction_log/bulk_transaction_log.py 60.86% <60.86%> (ø)
erpnext/utilities/bulk_transaction.py 94.73% <94.73%> (ø)
...nsaction_log_detail/bulk_transaction_log_detail.py 100.00% <100.00%> (ø)
...tch_item_expiry_status/batch_item_expiry_status.py 69.23% <0.00%> (-25.00%) ⬇️
...wise_balance_history/batch_wise_balance_history.py 68.96% <0.00%> (-24.14%) ⬇️
...xt/stock/report/stock_analytics/stock_analytics.py 80.19% <0.00%> (-12.88%) ⬇️
... and 27 more

@ankush ankush added the needs-tests This PR needs automated unit-tests. label Nov 26, 2021
@ankush ankush added the fix-sider-issues Check and fix issues listed by Sider label Nov 26, 2021
@ankush
Copy link
Member

ankush commented Nov 26, 2021

300 sider issues 😱 Just pass new files through something like https://prettier.io/ to auto format them.

@ankush ankush marked this pull request as draft November 26, 2021 09:04
@mohammedyusufshaikh mohammedyusufshaikh self-assigned this Nov 26, 2021
@ankush
Copy link
Member

ankush commented Nov 30, 2021

reopen after resolving open issues + some basic tests.

JS code also seems to be duplicated across many different doctype.js files. Maybe create a JS util class / function somewhere?

@ankush ankush closed this Nov 30, 2021
@mohammedyusufshaikh mohammedyusufshaikh marked this pull request as ready for review December 21, 2021 09:13
@mohammedyusufshaikh mohammedyusufshaikh removed the fix-sider-issues Check and fix issues listed by Sider label Dec 22, 2021
@mohammedyusufshaikh mohammedyusufshaikh removed the needs-tests This PR needs automated unit-tests. label Dec 22, 2021
@olamide2
Copy link

olamide2 commented Jan 5, 2022

Eye on this

@mohammedyusufshaikh mohammedyusufshaikh linked an issue Jan 14, 2022 that may be closed by this pull request
.distinct()
.where(btp.retried != 1)
.where(btp.transaction_status == "Failed")
.where(btp.date == fn.CurDate())
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can only retry what failed today? Any reason for this restriction? 🤔

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is for retrying all failing transaction log from list view; IMO

  1. It should signify it will only attempt today's failed transaction.
  2. Single log should also have retry. (feature request)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the list view it shows all the log docs with their name set as current date. It doesn't show all the transactions at once. so the user can retry that specific date's failing transaction
Screenshot 2022-01-27 at 12 34 36 PM

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It doesn't show all the transactions at once. so the user can retry that specific date's failing transaction

Can they though? The query only appears to retry today's failed transactions.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No it did not but now I have made the changes to retry that specific date's transaction

erpnext/utilities/bulk_transaction.py Show resolved Hide resolved
erpnext/utilities/bulk_transaction.py Outdated Show resolved Hide resolved
erpnext/utilities/bulk_transaction.py Show resolved Hide resolved
erpnext/utilities/bulk_transaction.py Outdated Show resolved Hide resolved
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Bulk Transaction Processing
4 participants