-
Notifications
You must be signed in to change notification settings - Fork 7.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: Auto set Party in Bank Transaction #34675
Conversation
…ployee/Shareholder)
- Created Bank Party Mapper - Created class to auto match by account/iban or party name/description(fuzzy) - Automatch and set in transaction or create mapper - `rapidfuzz` introduced
…tion - Description is volatile and will keep changing - It will lead to multiple Bank Party Mapper docs for the same party that will never be referenced again - Parts of the descripton keep changing which is why it will never match a mapper record - If matched by desc, dont create mapper record.
4c13c26
to
37c1331
Compare
Codecov Report
Additional details and impacted files@@ Coverage Diff @@
## develop #34675 +/- ##
===========================================
+ Coverage 64.12% 64.17% +0.04%
===========================================
Files 810 811 +1
Lines 61899 62006 +107
===========================================
+ Hits 39695 39793 +98
- Misses 22204 22213 +9
|
- On updating bank trans.n party after submit, the corresponding mapper doc will be updated too - The mapper doc in turn will update all linked bank transactions that do not have this updated value - Added Bank Party Mapper hidden link in Bank Transaction - Rename field in BPM to `Party Name` as it does not hold description data - If a BT matches with a BPM record, link that record in the BT
- misc: Clearer naming
- Checkbox in Accounts settings "Enable Automatic Party Matching" - Check before invoking automatching methods - misc: Remove TODO comments
20fba91
to
aea4315
Compare
- A BT could have both account and iban, and a Supplier could have only IBAN set - In this case, matching by either (only account) gives no match - Match by Account OR IBAN, use `or_filters` - If matched, set both account no. and IBAN in Bank Party Mapper - Explain AutoMatchParty - Add type hints to return values - Use `set_value` to set values in BT after matching since its an after submit event
…ches - Fuzzy matching can be enabled optionally in the settings - If a query gets multiple matches with the same score, do not set a party as it is an extremely close call - misc: Add 'cancelled' status to Bank transaction - Test for skipping matching with extremely close matches
@deepeshgarg007 Docs and description has been updated. The latest changes resolve the following ambiguities:
Bank details are searched for in Bank Account now. In the case of Employee, the Bank Account is searched and then the Employee doctype as it has bank account fields in it historically.
I did brainstorm this quite a bit and discussed this with @barredterra. Smit suggested rules but that would cause users to create way too many rules to match their plethora of parties. Could not really come up with anything else that wasn't bad UX. So fuzzy matching is optional. In the docs, users are encouraged to test the feature out first. The matching might be a wee bit problematic where extremely similar names exist (e.g. Adithya Medical& Surgical vs Adithya Medical & Surgical vs Adithya Medical And Surgical) but will work great for data where there is not so much similarity. I also increased the match score cutoff to 80. Additionally fuzzy matching does not set any value if multiple matches with the same score were obtained for a query. |
- Matching by Acc No/IBAN can easily happen with Bank Accounts. It's not a tedious query - Historical lookups for Party Name/Desc match are very tricky. The user could have manually set a match and we would not know. Also this leaves the Bank Party Mapper only useful for Party Name/Desc lookups, which feels excessive. - We want to reduce the number of places the same data is stored and reduce confusion - The Party Name/Desc will optionally happen fuzzily, or not at all - There will be no Mapper lookups
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Tested matching by party name, description and IBAN. LGTM.
Todo: Update docs and PR description (to not confuse future readers).
@barredterra docs and PR description updated ✅ |
* feat: Party auto-matcher from Bank Transaction data - Created Bank Party Mapper - Created class to auto match by account/iban or party name/description(fuzzy) - Automatch and set in transaction or create mapper - `rapidfuzz` introduced * chore: Single query with or filter to search Party Mapper by name/desc * feat: Store Party bank details in party records (Customer/Supplier/Employee/Shareholder) * fix: Don't set description as key in Mapper doc if matched by description - Description is volatile and will keep changing - It will lead to multiple Bank Party Mapper docs for the same party that will never be referenced again - Parts of the descripton keep changing which is why it will never match a mapper record - If matched by desc, dont create mapper record. * feat: Manually Update/Correct Party in Bank Transaction - On updating bank trans.n party after submit, the corresponding mapper doc will be updated too - The mapper doc in turn will update all linked bank transactions that do not have this updated value - Added Bank Party Mapper hidden link in Bank Transaction - Rename field in BPM to `Party Name` as it does not hold description data - If a BT matches with a BPM record, link that record in the BT * chore: Perform automatch on submit - misc: Clearer naming * chore: Make auto matching party configurable - Checkbox in Accounts settings "Enable Automatic Party Matching" - Check before invoking automatching methods - misc: Remove TODO comments * fix: Match by both Account No and IBAN & other cleanups - A BT could have both account and iban, and a Supplier could have only IBAN set - In this case, matching by either (only account) gives no match - Match by Account OR IBAN, use `or_filters` - If matched, set both account no. and IBAN in Bank Party Mapper - Explain AutoMatchParty - Add type hints to return values - Use `set_value` to set values in BT after matching since its an after submit event * test: Match by Account No, IBAN, Party Name, Desc and match correction * fix: Remove bank details fields from Shareholder * fix: Use existing bank fields to match by bank account no/IBAN - Remove newly added fields in Party doctypes to store bank details - Use Bank Account's fields to match against account no/iban - For employee, if Bank Account does not exist, find in Employee doctype against account no/iban * fix: Tests * feat: Optional Fuzzy Matching & Skip Matches for multiple similar matches - Fuzzy matching can be enabled optionally in the settings - If a query gets multiple matches with the same score, do not set a party as it is an extremely close call - misc: Add 'cancelled' status to Bank transaction - Test for skipping matching with extremely close matches * chore: Remove Bank Party Mapper implementation - Matching by Acc No/IBAN can easily happen with Bank Accounts. It's not a tedious query - Historical lookups for Party Name/Desc match are very tricky. The user could have manually set a match and we would not know. Also this leaves the Bank Party Mapper only useful for Party Name/Desc lookups, which feels excessive. - We want to reduce the number of places the same data is stored and reduce confusion - The Party Name/Desc will optionally happen fuzzily, or not at all - There will be no Mapper lookups * chore: Remove instances of `bank_party_mapper` and use `new_doc`
# [14.28.0](v14.27.14...v14.28.0) (2023-06-28) ### Bug Fixes * asset capitalization ([#35832](#35832)) ([fb823b5](fb823b5)) * asset movement ([#35918](#35918)) ([e16c148](e16c148)) * delivery trip driver is only required on submit (backport [#35876](#35876)) ([#35893](#35893)) ([fc051d1](fc051d1)) * don't allow to make reposting entry for closing stock balance period ([e68b088](e68b088)) * filter parent warehouses not showing (backport [#35897](#35897)) ([#35899](#35899)) ([87ba196](87ba196)) * incorrect cost center error in bank reconciliation ([cacb0f6](cacb0f6)) * issue of asset value_after_depreciation field getting updated twice if workflow is enabled in Journal Entry (backport [#35821](#35821)) ([#35827](#35827)) ([20f2bef](20f2bef)) * make credit note and debit note exclusive ([#35781](#35781)) ([fafb46e](fafb46e)) * multiple Work Orders agaist same production plan ([8ddfc34](8ddfc34)) * no permission for accounts settings on payment reconciliation ([200ddbf](200ddbf)) * Payment Term must be mandatory if `Allocate Payment based on ..` is checked ([#35798](#35798)) ([3dd3935](3dd3935)) * POS Closing Entry load all invoices with one request on save ([#35819](#35819)) ([8ecca2a](8ecca2a)) * reconcile invoice against credit note. ([#35604](#35604)) ([5c388a1](5c388a1)) * Remove special treatment for P&L Accounts ([#35602](#35602)) ([b023448](b023448)) * Set Asset cost center default as PR or PI Item Cost Center while auto creating ([#35844](#35844)) ([4a7d75b](4a7d75b)) * show non-depreciable assets in fixed asset register ([#35858](#35858)) ([42d0944](42d0944)) * TDS amount calculation post LDC breach ([851b887](851b887)) * use correct fieldname for purchase receipt column in item_wise_purchase_register report ([#35828](#35828)) ([8f13b48](8f13b48)) * **ux:** PO Get Items From Open Material Requests (backport [#35894](#35894)) ([#35895](#35895)) ([2ef2057](2ef2057)) ### Features * Auto set Party in Bank Transaction ([#34675](#34675)) ([d53b197](d53b197)) * Provision to send Accounts Receivable Reports using Process SOA ([#35789](#35789)) ([21d560c](21d560c)), closes [#35707](#35707) ### Performance Improvements * improve item wise register reports ([#35908](#35908)) ([33ee011](33ee011))
Resolves part of #34477
Ref: alyf-de/banking#17
Docs: https://docs.erpnext.com/docs/v14/user/manual/en/accounts/bank-reconciliation
Issue
Enhancement
This PR adds an optional automatic Party matcher.
Enable it in Accounts Settings. Fuzzy matching can be optionally enabled
Register primary/most used bank details like Account Number & IBAN against Customer, Supplier & Employee via Bank Account or Employee DocType (for Employees). This is optional and will make the match more accurate if set by users.
Added fields to hold Counter party information in Bank Transaction.(The population of these values will depend on integrations that import transactions or the users that upload the bank statement )
If Account details are absent in the Bank Transaction, fuzzy matching is optionally done using the Party Name (Bank Statement) or Description. These are matched against Customer/Supplier/Employee names.
The matching is done and values are set on Submit if auto matching is enabled
Users can correct the Party & Party Type, if incorrect, after submission. This will only impact the edited transaction.
Working
Expand to view the working
TODO: