#44 Sampling tasks from user data #115
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
yk
reviewed
Dec 28, 2022
| @@ -31,3 +31,5 @@ class Post(SQLModel, table=True): | |||
| ) | |||
| payload_type: str = Field(nullable=False, max_length=200) | |||
| payload: PayloadContainer = Field(sa_column=sa.Column(payload_column_type(PayloadContainer), nullable=True)) | |||
| depth: int = Field(sa_column=sa.Column(sa.Integer, default=0, server_default=sa.text("0"), nullable=False)) | |||
There was a problem hiding this comment.
do we really need all these attributes?
it seems like sqlmodel might do the right thing with depth: int = 0 without any field definition at all, no?
There was a problem hiding this comment.
There was an issue adding default boolean value (ack, done), the value did not propagate properly to existing rows. Solution was to specify server_default, so I did it the same way to other columns.
|
LGTM, but I'll leave the review to @andreaskoepf bc more familiar with this code |
andreaskoepf
approved these changes
Dec 28, 2022
There was a problem hiding this comment.
My thoughts:
- depth and children_count are incrementally calculated, e.g. we must from now on not directly delete elements from post without going through python code that updates the parent element.
- regarding sampling &
order by random()I have read different reports about its performance online (some say it "performs a full table scan and also ordering. Therefore this method is not preferred for tables with large number of rows because of performance reasons." ..) Postgres also supports TABLESAMPLE, which is probably faster.
In general we will need very soon a more sophisticated sampling function that takes a weights into account.
|
linking #44 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Tasks are now sampled randomly from user-uploaded data.
resolves #44