One of the biggest pain points when handling large, machine learning datasets is being able to easily slice and manipulate labels in various ways. Machine Learning engineers and researchers often need to select a subset of the dataset (i.e. those that contain a particular set of object classes) and tag them to revisit them later or share with colleagues (i.e. "Used to train model v1").
A big benefit of using the Suite is exactly this — the Suite provides features like tag, filter, and search to give users (both ML engineers and Labeling teams) the versatility to manipulate labels easily and boost their productivity.
A Label Tag is a customizable marker that can be used to leave reminders or other vital information to oneself or other users.
For example, an engineer may notify another engineer that the tagged image data has been used on an AI prototype and does not want the tagged image(s) to be re-used by another engineer viewing the same dataset. A manager may choose to mark a set of labels that must be reviewed for quality assurance purposes.
Create a Label Tag
1. In the Labels tab, select the labels that you would like to add Tags to.
2. Click Edit Tags in the upper-right toolbar.
An empty checkbox(□) means none of the selected labels have the particular tag.
A checked box(☑) means all of the selected labels have the particular tag.
A checkbox with a dash ([-]) means some of the selected labels have the particular tag.
You may either choose from the list of existing tags, or create a new tag by writing out the new tag and then pressing the Enter key.
Delete a Label Tag
To delete a Label Tag filter, find tagged labels first and manually remove the tag from them. We will be adding the function to track which labels the tag was removed from after deleting.
Filter and Search Labels
In the Labels tab, apply filters and search query to find labels that match your desired criteria.
Data Key (Name) Search
Filter labels based on the data key (original data file name). Searches for partial matches.
Filter labels by the custom Label Tags
Filter labels based on the annotated object class
Filter labels based on whether or not they have been reviewed (see: Manual Review)
Filter labels by the name of the user that reviewed them (see: Manual Review)
Filter labels by the name of the user assigned to the labeling task
Filter labels by the type of Quality Assurance method used (Consensus Labels, Qualified Labels, None-QA Labels)
Filter labels by the Consistency Score (for Consensus Labeling QA method)
Filter labels by Issue threads (whether an issue thread exists; the user who created the thread, etc.)
Filter labels by the date the raw data was added to the project
Filter labels by the last updated timestamp. Any changes to the label, such as annotations, status changes, tagging, assignee assignments, etc. are considered updates
A project may have data that is stored in different Datasets. Filter labels by the associated dataset.
Filter labels based on whether a pre-label exists. Auto-labeling populates each label with a Pre-label.
Filter labels based on the status of Auto-label. If the Auto-label is getting processed, the status will show up as "is processing". If the request failed, the status will show up as "has failed". request fails, you can check it with the 'has failed' filter.
For labels applied with auto-labeling, you can filter based on auto-labeling difficulty (easy, medium, hard)
Filter labels by a specific Label ID. You can find the Label ID for each label in the downloaded Zip file after an export.