od-tc

Ordering Dependency & Task Consistency

Here is provided a convenient implementation to apply Task Consistency or Ordering Dependency refinement on the SSN output and print the evaluation result.

File Description

• action_sequence_statistics.py - generates a mat file containing
  • Markov transfer matrix
  • Distribution of the first step in a video
• perform_refine.py - perform our methods on the SSN result.

Usage

action_sequence_statistics.py

python3 action_sequence_statistics.py <json_dataset> <output_mat>

Here <json_dataset> is the JSON annotation file with the same structure as our COIN dataset provides while <output_mat> is a MATLAB/SciPy matrix file which comprises four matrices:

• init_dist - non-normalized distribution of the first step in a video with shape like (1, nb_step).
• normalized_init_dist - the normalized version of init_dist.
• frequency_mat - non-normalized transfer matrix which shape like (nb_step, nb_step), in which t[i][j] denotes the statistical frequency of transfering from step i to step j.
• normalized_frequency_mat - the normalized version of frequency_mat.

perform_refine.py

[1] Just evaluation.

python3 perform_refine.py --matrix <consistency_matrix> --groundtruth <json_dataset> --scores <SSN_scores> [--weights <weights>]

<consistency_matrix> is the consistency matrix mentioned in tc-ssn which is generated by some program gen_matrix.py. <json_dataset is the aforementioned JSON annotation. <SSN_scores> is the SSN output described in tc-ssn. --weights is used to customize the fusion weights if there are multiple score files specified.

[2] Perform TC.

python3 perform_refine.py --matrix <...> --groundtruth <...> --scores <...> --refinement TC [--attenuation_coefficient <ac>]

[3] Perform OD.

python3 perform_refine.py --matrix <...> --groundtruth <...> --scores <...> --refinement OD [--refinement-weights w1 w2]

--refinement-weights indicate lambda_1 and lambda_2 in our paper.

[4] Perform OD & TC sequentially.

python3 perform_refine.py --matrix <...> --groundtruth <...> --scores <...> --refinement OD TC [--attenuation_coefficient <...>] [--refinement-weights w1 w1]

[5] Perform TC & OD sequentially.

python3 perform_refine.py --matrix <...> --groundtruth <...> --scores <...> --refinement TC OD [--attenuation_coefficient <...>] [--refinement-weights w1 w1]

You may apply TC and OD in any order for any times if you like simply by appending TC or OD behind option --refinement.