Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Reconcile Payments in background #34596

Merged
merged 63 commits into from
Apr 22, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
63 commits
Select commit Hold shift + click to select a range
9851358
feat: auto reconcile in background
ruthra-kumar Feb 23, 2023
67dcd44
chore: Option to enable auto reconciliation in settings
ruthra-kumar Mar 28, 2023
9e7e392
refactor: validate if feature is enabled in settings
ruthra-kumar Mar 28, 2023
05f25a6
refactor: check for running job while using reconciliation tool
ruthra-kumar Mar 27, 2023
767cda1
chore: using doc to get filter values
ruthra-kumar Mar 29, 2023
9649b6b
chore: use frappe.db.get_value in validations
ruthra-kumar Mar 29, 2023
4e945d4
chore: cleanup commented out code
ruthra-kumar Mar 29, 2023
5cd97cf
chore: replace get_list with get_all
ruthra-kumar Mar 29, 2023
a6d7eff
chore: use block scope variable
ruthra-kumar Mar 29, 2023
e92cc2d
chore: type information for functions
ruthra-kumar Mar 29, 2023
8c7a20a
refactor: flag to ignore job validation check
ruthra-kumar Mar 30, 2023
8e2c7bc
refactor: update parent doc status if all reconciled
ruthra-kumar Mar 30, 2023
01038a4
chore: create test_records file
ruthra-kumar Mar 30, 2023
a2fb9bd
test: create a bunch of vouchers for testing auto reconcile
ruthra-kumar Mar 27, 2023
c50aa38
chore: renamed auto_reconcile to process_payment_reconciliation
ruthra-kumar Mar 30, 2023
a4d3d7b
chore: another child doctype to hold payments
ruthra-kumar Mar 31, 2023
b17486d
chore: remove duplicate field
ruthra-kumar Mar 31, 2023
0c022ca
chore: add fetched payments to log
ruthra-kumar Mar 31, 2023
025ab20
chore: Popup comment message update
ruthra-kumar Apr 1, 2023
9d23c30
chore: replace get_all with get_value
ruthra-kumar Apr 1, 2023
3d9733f
chore: replace label in settings page
ruthra-kumar Apr 3, 2023
6a4f1ef
chore: remove unit test and records
ruthra-kumar Apr 3, 2023
457128c
refactor: status in reconciliation log
ruthra-kumar Apr 3, 2023
c65695c
refactor: set status in log as well
ruthra-kumar Apr 3, 2023
9afb2e8
chore: fix field name
ruthra-kumar Apr 3, 2023
898c198
chore: change triggered job name
ruthra-kumar Apr 3, 2023
da46565
chore: use status field in list view of log
ruthra-kumar Apr 3, 2023
ec2914b
chore: status while there are no allocations
ruthra-kumar Apr 3, 2023
c4f151b
refactor: split trigger function into two
ruthra-kumar Apr 3, 2023
1c21c7d
chore: adding cancelled status
ruthra-kumar Apr 3, 2023
ea74670
refactor: function trigger queued docs
ruthra-kumar Apr 3, 2023
569a1f3
chore: cron job scheduled
ruthra-kumar Apr 3, 2023
b9b4faf
chore: fixing accouts settings json file
ruthra-kumar Apr 4, 2023
1ee0c7b
chore: typos and variable scope
ruthra-kumar Apr 10, 2023
bace8f8
chore: use 'pluck' in db call
ruthra-kumar Apr 10, 2023
0aa87ee
chore: remove redundant whitelist decorator
ruthra-kumar Apr 10, 2023
187291e
chore: use single DB call to fetch values
ruthra-kumar Apr 10, 2023
32c554d
chore: replace get_all with get_value
ruthra-kumar Apr 10, 2023
676f0c1
refactor: use raw db calls to fetch reconciliation log records
ruthra-kumar Apr 10, 2023
cf2ac29
chore: update status on successful batch operation
ruthra-kumar Apr 11, 2023
c5629c2
chore: make payment table readonly
ruthra-kumar Apr 11, 2023
ded7b73
chore: ability to pause the background job
ruthra-kumar Apr 11, 2023
da13782
chore: remove isolate_each_allocation
ruthra-kumar Apr 11, 2023
385f0af
chore: more description in progress bar
ruthra-kumar Apr 11, 2023
9fb4509
refactor: partially working state
ruthra-kumar Apr 12, 2023
62a6ca4
refactor: update reconcile flag and setting hard limits for fetching
ruthra-kumar Apr 13, 2023
f23ac7a
chore: make allocation editable -- NEED TO REVERT
ruthra-kumar Apr 13, 2023
5c2f26d
chore: pause button
ruthra-kumar Apr 14, 2023
ddc699d
refactor: skip setter function in Payment Entry for better performan
ruthra-kumar Apr 14, 2023
7250772
refactor: split reconcile function and skip a setter function
ruthra-kumar Apr 17, 2023
b729714
chore: increase payment limit
ruthra-kumar Apr 17, 2023
6b8b6f8
refactor: replace frappe.db.get_all with frappe.db.get_value
ruthra-kumar Apr 17, 2023
4047494
chore: remove unwanted doctypes
ruthra-kumar Apr 18, 2023
0b8d0bd
refactor: make allocation table readonly
ruthra-kumar Apr 18, 2023
170c63b
perf: update ref_details only for newly linked invoices
ruthra-kumar Apr 19, 2023
0b930e0
chore: rename skip flag
ruthra-kumar Apr 19, 2023
dfecf49
refactor(UI): receivable_payable field should auto populate
ruthra-kumar Apr 20, 2023
0c4cd7c
refactor: no control statements in finally block
ruthra-kumar Apr 20, 2023
81e57ec
chore: cleanup section and rename checkbox
ruthra-kumar Apr 21, 2023
1d73e92
chore: update new fieldname in code
ruthra-kumar Apr 21, 2023
68f2e0d
chore: update error msg
ruthra-kumar Apr 21, 2023
b46aa3e
refactor: start and pause integrated into status
ruthra-kumar Apr 21, 2023
f99d479
refactor: added cancelled status to the log doctype
ruthra-kumar Apr 21, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,8 @@
"show_payment_schedule_in_print",
"currency_exchange_section",
"allow_stale",
"section_break_jpd0",
"auto_reconcile_payments",
"stale_days",
"invoicing_settings_tab",
"accounts_transactions_settings_section",
Expand All @@ -59,7 +61,6 @@
"acc_frozen_upto",
"column_break_25",
"frozen_accounts_modifier",
"report_settings_sb",
"tab_break_dpet",
"show_balance_in_coa"
],
Expand Down Expand Up @@ -172,11 +173,6 @@
"fieldtype": "Int",
"label": "Stale Days"
},
{
"fieldname": "report_settings_sb",
"fieldtype": "Section Break",
"label": "Report Settings"
},
{
"default": "0",
"description": "Only select this if you have set up the Cash Flow Mapper documents",
Expand Down Expand Up @@ -383,14 +379,25 @@
"fieldname": "merge_similar_account_heads",
"fieldtype": "Check",
"label": "Merge Similar Account Heads"
},
{
"fieldname": "section_break_jpd0",
"fieldtype": "Section Break",
"label": "Payment Reconciliations"
},
{
"default": "0",
"fieldname": "auto_reconcile_payments",
"fieldtype": "Check",
"label": "Auto Reconcile Payments"
}
],
"icon": "icon-cog",
"idx": 1,
"index_web_pages_for_search": 1,
"issingle": 1,
"links": [],
"modified": "2023-04-17 11:45:42.049247",
"modified": "2023-04-21 13:11:37.130743",
"modified_by": "Administrator",
"module": "Accounts",
"name": "Accounts Settings",
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -82,6 +82,32 @@ erpnext.accounts.PaymentReconciliationController = class PaymentReconciliationCo
this.frm.change_custom_button_type('Get Unreconciled Entries', null, 'default');
this.frm.change_custom_button_type('Allocate', null, 'default');
}

// check for any running reconciliation jobs
if (this.frm.doc.receivable_payable_account) {
frappe.db.get_single_value("Accounts Settings", "auto_reconcile_payments").then((enabled) => {
if(enabled) {
this.frm.call({
'method': "erpnext.accounts.doctype.process_payment_reconciliation.process_payment_reconciliation.is_any_doc_running",
"args": {
for_filter: {
company: this.frm.doc.company,
party_type: this.frm.doc.party_type,
party: this.frm.doc.party,
receivable_payable_account: this.frm.doc.receivable_payable_account
}
}
}).then(r => {
if (r.message) {
let doc_link = frappe.utils.get_form_link("Process Payment Reconciliation", r.message, true);
let msg = __("Payment Reconciliation Job: {0} is running for this party. Can't reconcile now.", [doc_link]);
this.frm.dashboard.add_comment(msg, "yellow");
}
});
}
});
}

}

company() {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -7,9 +7,12 @@
from frappe.model.document import Document
from frappe.query_builder.custom import ConstantColumn
from frappe.query_builder.functions import IfNull
from frappe.utils import flt, getdate, nowdate, today
from frappe.utils import flt, get_link_to_form, getdate, nowdate, today

import erpnext
from erpnext.accounts.doctype.process_payment_reconciliation.process_payment_reconciliation import (
is_any_doc_running,
)
from erpnext.accounts.utils import (
QueryPaymentLedger,
get_outstanding_invoices,
Expand Down Expand Up @@ -304,9 +307,7 @@ def get_allocated_entry(self, pay, inv, allocated_amount):
}
)

@frappe.whitelist()
def reconcile(self):
self.validate_allocation()
def reconcile_allocations(self, skip_ref_details_update_for_pe=False):
dr_or_cr = (
"credit_in_account_currency"
if erpnext.get_party_account_type(self.party_type) == "Receivable"
Expand All @@ -330,12 +331,35 @@ def reconcile(self):
self.make_difference_entry(payment_details)

if entry_list:
reconcile_against_document(entry_list)
reconcile_against_document(entry_list, skip_ref_details_update_for_pe)

if dr_or_cr_notes:
reconcile_dr_cr_note(dr_or_cr_notes, self.company)

@frappe.whitelist()
def reconcile(self):
if frappe.db.get_single_value("Accounts Settings", "auto_reconcile_payments"):
running_doc = is_any_doc_running(
dict(
company=self.company,
party_type=self.party_type,
party=self.party,
receivable_payable_account=self.receivable_payable_account,
)
)

if running_doc:
frappe.throw(
_("A Reconciliation Job {0} is running for the same filters. Cannot reconcile now").format(
get_link_to_form("Auto Reconcile", running_doc)
)
)
return

self.validate_allocation()
self.reconcile_allocations()
msgprint(_("Successfully Reconciled"))

self.get_unreconciled_entries()

def make_difference_entry(self, row):
Expand Down
Empty file.
Original file line number Diff line number Diff line change
@@ -0,0 +1,130 @@
// Copyright (c) 2023, Frappe Technologies Pvt. Ltd. and contributors
// For license information, please see license.txt

frappe.ui.form.on("Process Payment Reconciliation", {
onload: function(frm) {
// set queries
frm.set_query("party_type", function() {
return {
"filters": {
"name": ["in", Object.keys(frappe.boot.party_account_types)],
}
}
});
frm.set_query('receivable_payable_account', function(doc) {
return {
filters: {
"company": doc.company,
"is_group": 0,
"account_type": frappe.boot.party_account_types[doc.party_type]
}
};
});
frm.set_query('cost_center', function(doc) {
return {
filters: {
"company": doc.company,
"is_group": 0,
}
};
});
frm.set_query('bank_cash_account', function(doc) {
return {
filters:[
['Account', 'company', '=', doc.company],
['Account', 'is_group', '=', 0],
['Account', 'account_type', 'in', ['Bank', 'Cash']]
]
};
});

},
refresh: function(frm) {
if (frm.doc.docstatus==1 && ['Queued', 'Paused'].find(x => x == frm.doc.status)) {
let execute_btn = __("Start / Resume")

frm.add_custom_button(execute_btn, () => {
frm.call({
method: 'erpnext.accounts.doctype.process_payment_reconciliation.process_payment_reconciliation.trigger_job_for_doc',
args: {
docname: frm.doc.name
}
}).then(r => {
if(!r.exc) {
frappe.show_alert(__("Job Started"));
frm.reload_doc();
}
});
});
}
if (frm.doc.docstatus==1 && ['Completed', 'Running', 'Paused', 'Partially Reconciled'].find(x => x == frm.doc.status)) {
frm.call({
'method': "erpnext.accounts.doctype.process_payment_reconciliation.process_payment_reconciliation.get_reconciled_count",
args: {
"docname": frm.docname,
}
}).then(r => {
if (r.message) {
let progress = 0;
let description = "";

if (r.message.processed) {
progress = (r.message.processed/r.message.total) * 100;
description = r.message.processed + "/" + r.message.total + " processed";
} else if (r.message.total == 0 && frm.doc.status == "Completed") {
progress = 100;
}


frm.dashboard.add_progress('Reconciliation Progress', progress, description);
}
})
}
if (frm.doc.docstatus==1 && frm.doc.status == 'Running') {
let execute_btn = __("Pause")

frm.add_custom_button(execute_btn, () => {
frm.call({
'method': "erpnext.accounts.doctype.process_payment_reconciliation.process_payment_reconciliation.pause_job_for_doc",
args: {
"docname": frm.docname,
}
}).then(r => {
if (!r.exc) {
frappe.show_alert(__("Job Paused"));
frm.reload_doc()
}
});

});
}
},
company(frm) {
frm.set_value('party', '');
frm.set_value('receivable_payable_account', '');
},
party_type(frm) {
frm.set_value('party', '');
},

party(frm) {
frm.set_value('receivable_payable_account', '');
if (!frm.doc.receivable_payable_account && frm.doc.party_type && frm.doc.party) {
return frappe.call({
method: "erpnext.accounts.party.get_party_account",
args: {
company: frm.doc.company,
party_type: frm.doc.party_type,
party: frm.doc.party
},
callback: (r) => {
if (!r.exc && r.message) {
frm.set_value("receivable_payable_account", r.message);
}
frm.refresh();

}
});
}
}
});
Loading