Add multi-label support for label-studio integration

[tataganesh]

What changes are proposed in this pull request?

Resolves #2946.
This pull request adds multi-label classification support to FiftyOne's label studio integration.

How is this patch tested? If it is not, please explain why.

A subset of the Fashion-MNIST dataset was used to generate a small multi-label classification dataset. The code for creating the dataset and launching label studio for annotation -

import fiftyone as fo
import fiftyone.zoo as foz
from fiftyone import ViewField as F

dataset = foz.load_zoo_dataset(
    "fashion-mnist", dataset_name="fmnist-multilabel-annotation", split="test"
)
dataset.persistent = True

# Add new classes for assigning multiple labels
classes = dataset.default_classes + ["UpperBody", "LowerBody", "FootWear"]

shirt_sneaker_view = (
    dataset
    .filter_labels("ground_truth", F("label").is_in(["Shirt", "Sneaker"]))
    .take(5, seed = 42)
    )

anno_key = "labelstudio_multilabel_recipe"
if anno_key in dataset.list_annotation_runs():
    print(f"Annotation run with key {anno_key} already exists. Deleting...")
    dataset.delete_annotation_run(anno_key)


label_schema = {
    "modified_ground_truth": {
        "type": "classifications",
        "classes": classes
    },
}

shirt_sneaker_view.annotate(
    anno_key,
    backend="labelstudio",
    label_schema=label_schema,
    launch_editor=True,
)

After annotating the 5 samples with multiple labels (jn this case, annotating shirt images with ("Shirt", "UpperBody") and sneaker images with ("Sneaker", "Footwear"), view the annotations on the Fiftyone app.

import fiftyone as fo
import fiftyone.zoo as foz

anno_key = "labelstudio_multilabel_recipe"

dataset = foz.load_zoo_dataset(
    "fashion-mnist", dataset_name="fmnist-multilabel-annotation", split="test"
)
dataset.load_annotations(anno_key)

# Load the view that was annotated in the App
view = dataset.load_annotation_view(anno_key)
session = fo.launch_app(view=view)
session.wait()

Screenshot of app with multi-label annotations

Release Notes

Is this a user-facing change that should be mentioned in the release notes?

• No. You can skip the rest of this section.
• Yes. Give a description of this change to be included in the release
  notes for FiftyOne users.

Label Studio integration in FiftyOne now support multi-label classification. For more info, checkout the description
of #2946

What areas of FiftyOne does this PR affect?

• App: FiftyOne application changes
• Build: Build and test infrastructure changes
• Core: Core fiftyone Python library changes
• Documentation: FiftyOne documentation changes
• Other

Summary by CodeRabbit

• New Features

  • Introduced support for a new label type, "classifications", enhancing flexibility in the labeling system.

• Improvements

  • Enhanced handling of multi-label classifications for better organization and structure in label management.
  • Updated return type for classifications to ensure consistency across the module.

• Bug Fixes

  • Improved label verification by adding conditions to ensure valid classification objects.

• Documentation

  • Updated tests to reflect changes in classification handling, increasing robustness and code cleanliness.

[coderabbitai]

Walkthrough

The changes involve enhancing the FiftyOne label handling system, specifically supporting multi-label classifications through the integration with Label Studio. New functionalities include adding "classifications" as a supported label type, refining the processing of annotations, and restructuring how classifications are represented. These updates aim to improve label management and facilitate the integration experience between FiftyOne and Label Studio.

Changes

Files	Change Summary
fiftyone/utils/labelstudio.py	Added "classifications" to supported label types; refactored annotation handling for better multi-label support; changed _from_choices to return a fol.Classifications object.
tests/intensive/labelstudio_tests.py	Revised the fiftyone key structure in label_mappings to use fo.Classifications; minor code cleanup in test_import_labels; enhanced label verification in _assert_labels_equal.

Sequence Diagram(s)

sequenceDiagram
    participant User
    participant LabelStudio
    participant FiftyOne

    User->>LabelStudio: Initiates labeling process
    LabelStudio->>FiftyOne: Sends labeling data
    FiftyOne->>FiftyOne: Processes labels
    Note over FiftyOne: Checks for classifications type
    FiftyOne->>FiftyOne: Maps classifications to their IDs
    FiftyOne->>LabelStudio: Returns processed annotations

Loading

Assessment against linked issues

Objective	Addressed	Explanation
Support for multi-label classifications with Classifications (##2946)	✅
Enhance integration experience between FiftyOne and Label Studio (##2946)	✅

Poem

In a field of labels bright,
A rabbit hops with pure delight.
Multi-class now takes the stage,
With FiftyOne, we turn the page.
Classifications dance and play,
Sharing tags in a joyful way! 🐇✨

---

Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

Share

• X
• Mastodon
• Reddit
• LinkedIn

Tips

Chat

There are 3 ways to chat with CodeRabbit:

• Review comments: Directly reply to a review comment made by CodeRabbit. Example:
  • I pushed a fix in commit <commit_id>.
  • Generate unit testing code for this file.
  • Open a follow-up GitHub issue for this discussion.
• Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
  • @coderabbitai generate unit testing code for this file.
  • @coderabbitai modularize this function.
• PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
  • @coderabbitai generate interesting stats about this repository and render them as a table.
  • @coderabbitai show all the console.log statements in this repository.
  • @coderabbitai read src/utils.ts and generate unit testing code.
  • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
  • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (invoked as PR comments)

• @coderabbitai pause to pause the reviews on a PR.
• @coderabbitai resume to resume the paused reviews.
• @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
• @coderabbitai full review to do a full review from scratch and review all the files again.
• @coderabbitai summary to regenerate the summary of the PR.
• @coderabbitai resolve resolve all the CodeRabbit review comments.
• @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
• @coderabbitai help to get help.

Additionally, you can add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.

Documentation and Community

• Visit our Documentation for detailed information on how to use CodeRabbit.
• Join our Discord Community to get help, request features, and share feedback.
• Follow us on X/Twitter for updates and announcements.

[coderabbitai]

Actionable comments posted: 0

Review details

Configuration used: .coderabbit.yaml
Review profile: CHILL

Commits

Files that changed from the base of the PR and between 76cdadf and e61d137.

Files selected for processing (2)

• fiftyone/utils/labelstudio.py (3 hunks)
• tests/intensive/labelstudio_tests.py (3 hunks)

Additional context used

Ruff

fiftyone/utils/labelstudio.py

421-421: Ambiguous variable name: l

(E741)

---

903-903: Ambiguous variable name: l

(E741)

Additional comments not posted (5)

tests/intensive/labelstudio_tests.py (2)

268-273: Refactor to use fo.Classifications for multi-label support.

The change from individual fo.Classification instances to a single fo.Classifications instance is a good practice for handling multi-label scenarios. This aligns with the PR's objective of enhancing multi-label classification support.

---

677-678: Ensure all classifications have valid labels.

Adding a condition to check if all classifications have a valid label is a good practice for data integrity and error handling.

fiftyone/utils/labelstudio.py (3)

103-103: Add support for "classifications" label type.

Adding "classifications" to the list of supported label types is crucial for enabling multi-label classification functionality. This change is necessary for the integration to handle more complex labeling scenarios.

---

420-427: Refactor _import_annotations to handle Classifications.

The modification to handle instances of fol.Classifications in the _import_annotations method is essential for supporting multi-label classifications. This change ensures that the system can process and map multiple classifications correctly.

Tools

Ruff

421-421: Ambiguous variable name: l

(E741)

---

902-904: Change return type to fol.Classifications in _from_choices.

Adjusting the return type to fol.Classifications in the _from_choices function aligns with the new structure introduced in other parts of the module. This consistency is crucial for handling multi-label classifications effectively.

Tools

Ruff

903-903: Ambiguous variable name: l

(E741)

[ehofesmann]

Thanks for the contribution @tataganesh !!

From my testing, this is working great for creating new multilabel classification annotations with label studio.

There are some issues I'm running into when working with existing classifications fields that already exist in FiftyOne when the annotation task is being created, but I'm seeing the same thing when working with Classification fields even outside of this PR, so it is likely a preexisting bug. (For future reference: #4779)