train/test split ID models

[ralphpeterson]

Hi all,

@Niegil-Francis and I are experimenting with 3D tracking using the new multi-view GUI + sleap-anipose and are getting the following error trying to train ID models (same error for both bottom-up and top-down). Any idea what could be going on here? Seems to be an issue in determining the train/test split.

Thanks!

Ralph

INFO:sleap.nn.training:Creating training and validation splits from validation fraction: 0.1
Traceback (most recent call last):
  File "C:\Users\daq2\anaconda3\envs\sleap_mv\Scripts\sleap-train-script.py", line 33, in <module>
    sys.exit(load_entry_point('sleap', 'console_scripts', 'sleap-train')())
  File "c:\users\daq2\niegil_codes\sleap\sleap\nn\training.py", line 2013, in main
    trainer = create_trainer_using_cli(args=args)
  File "c:\users\daq2\niegil_codes\sleap\sleap\nn\training.py", line 2005, in create_trainer_using_cli
    video_search_paths=args.video_paths,
  File "c:\users\daq2\niegil_codes\sleap\sleap\nn\training.py", line 673, in from_config
    with_track_only=is_id_model,
  File "c:\users\daq2\niegil_codes\sleap\sleap\nn\training.py", line 150, in from_config
    with_track_only=with_track_only,
  File "c:\users\daq2\niegil_codes\sleap\sleap\nn\training.py", line 218, in from_labels
    validation,
  File "c:\users\daq2\niegil_codes\sleap\sleap\nn\data\training.py", line 49, in split_labels_train_val
    idx_train, idx_val = train_test_split(list(range(len(labels))), test_size=n_val)
  File "C:\Users\daq2\anaconda3\envs\sleap_mv\lib\site-packages\sklearn\model_selection\_split.py", line 2423, in train_test_split
    n_samples, test_size, train_size, default_test_size=0.25
  File "C:\Users\daq2\anaconda3\envs\sleap_mv\lib\site-packages\sklearn\model_selection\_split.py", line 2046, in _validate_shuffle_split
    "(0, 1) range".format(test_size, n_samples)
ValueError: test_size=1 should be either positive and smaller than the number of samples 0 or a float in the (0, 1) range

[talmo]

Hey @ralphpeterson,

Do you have a screenshot from the main GUI window? It looks like it's not finding the labeled frames in the project.

Thanks!

Talmo

[ralphpeterson]

Thanks for the quick response! Here's a screenshot. It's worth noting that we do not see this error when training non-ID models.

[talmo]

Ahh, perfect contextual note!

Yes, for supervised ID models, the animals need to be assigned to a track and have that track be named consistently for all annotations of that same identity (double click the track name in the Instances tab if you want to rename it).

Since supervised ID models require a track to assign instances to classes, instances without a track are filtered out -- hence the issue with generating splits when you have 0 frames.