Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

✨ NEW: Add Backend bulk methods #5171

Merged
merged 17 commits into from
Oct 16, 2021

Conversation

chrisjsewell
Copy link
Member

@chrisjsewell chrisjsewell commented Oct 8, 2021

Partially extracted from (and required by) #5145 and also work towards #5154

Adds Backend.bulk_insert and Backend.bulk_update methods, and moves the delete_nodes_and_connections function to a method on the Backend.
This removes the need for using backend specific code outside of the backend, for these use cases (still plenty more to fix)

delete_nodes_and_connections performs multiple operations and so should only be called within a transaction, thus it raises an AssertionError if this is not the case.

(this addresses, but maybe not fully, #3382 and #2496)

@codecov
Copy link

codecov bot commented Oct 8, 2021

Codecov Report

Merging #5171 (346a6b8) into develop (0b7db7b) will increase coverage by 0.04%.
The diff coverage is 93.07%.

Impacted file tree graph

@@             Coverage Diff             @@
##           develop    #5171      +/-   ##
===========================================
+ Coverage    80.93%   80.96%   +0.04%     
===========================================
  Files          536      535       -1     
  Lines        37056    37185     +129     
===========================================
+ Hits         29986    30104     +118     
- Misses        7070     7081      +11     
Flag Coverage Δ
django 75.75% <64.74%> (-0.03%) ⬇️
sqlalchemy 74.86% <58.39%> (-0.07%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
aiida/backends/sqlalchemy/utils.py 95.00% <ø> (-1.77%) ⬇️
aiida/backends/utils.py 100.00% <ø> (+4.77%) ⬆️
aiida/orm/implementation/django/backend.py 94.83% <90.91%> (-5.17%) ⬇️
aiida/orm/implementation/sqlalchemy/backend.py 94.74% <92.86%> (-2.27%) ⬇️
aiida/orm/implementation/backends.py 93.75% <94.12%> (-0.53%) ⬇️
aiida/backends/sqlalchemy/models/group.py 93.55% <100.00%> (+0.45%) ⬆️
aiida/orm/entities.py 98.33% <100.00%> (+0.11%) ⬆️
aiida/orm/implementation/sql/backends.py 100.00% <100.00%> (ø)
aiida/tools/graph/deletions.py 76.75% <100.00%> (+1.14%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 0b7db7b...346a6b8. Read the comment docs.

@chrisjsewell chrisjsewell changed the title ✨ NEW: Add Backend.bulk_insert & Backend.bulk_update ✨ NEW: Add Backend bulk methods Oct 10, 2021
@@ -78,46 +94,115 @@ def transaction(self):
if session.in_transaction():
with session.begin_nested():
yield session
session.commit()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems an important change - was it wrong before?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes it was a regression error from moving to the SQLAlchemy v2 API, that I realised in #4534, i.e. it was not actually committing the transaction when exiting the transaction context for archive imports

giovannipizzi
giovannipizzi previously approved these changes Oct 14, 2021
Copy link
Member

@giovannipizzi giovannipizzi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi, I'm approving, but great if you could double check my comment above.

Also, one more general question, now many PRs have at the bottom a number of "errors" under the title "Unchanged files with check annotations". What are these? Should they be ignored?

@chrisjsewell
Copy link
Member Author

Also, one more general question, now many PRs have at the bottom a number of "errors" under the title "Unchanged files with check annotations". What are these? Should they be ignored?

Yeh I don't know either, its showing errors when it encounters an exception raise, which seems nonsensical for a static analyser. May be related to actions/toolkit#457

@chrisjsewell chrisjsewell merged commit 8fb1457 into aiidateam:develop Oct 16, 2021
@chrisjsewell chrisjsewell deleted the bulk-methods branch October 16, 2021 02:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants