Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[DAR-2044][External] Bundle tags bound for the same item when using CSV importer #840

Merged
merged 2 commits into from
May 13, 2024

Conversation

JBWilkie
Copy link
Collaborator

@JBWilkie JBWilkie commented May 12, 2024

Problem

When importing CSV tags, the importer currently sends one request per tag per item. If importing multiple tags to the same item, this can be sped up by bundling tags bound for the same item in the same request

Solution

This PR groups tags bound for the same item into the same request payload, and adds unit tests. I only had to implement this for csv_tags, since it turns out this was already implemented for csv_tags_video

I thought about splitting up annotation payloads if they grow beyond a certain size, but I tested up to 100000 tags in a single payload without issue

Changelog

Improved speed of CSV tag importer when importing multiple tags to the same dataset item

Copy link

linear bot commented May 12, 2024

@JBWilkie JBWilkie merged commit 384f1ef into master May 13, 2024
16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants