RuntimeException: inferenceWithScores not supported for ViterbiLayer

[hriaz17]

I am trying to get the scores for the predictions of the LSTM-CRF model.

In line 327 of Metal.scala, replacing: Layers.predict(model, taskId, sentence, modHeadPairsOpt, constEmbeddings) function with Layers.predictWithScores(model, taskId, sentence, modHeadPairsOpt, constEmbeddings, true) causes the exception to be raised.

In line 295 of Layers.scala, predictWithScores makes a call to inferenceWithScores in ViterbiForwardLayer.scala which is not implemented.

Should I replace the call to inferenceWithScores with just inference, which is being used by the default Layers.predict?

[kwalcock]

I'm finally looking. Sorry for the delay.

[kwalcock]

I wonder whether the scores are really necessary given that access isn't provided. So the hierarchy is

FinalLayer
  <- ForwardLayer
    <- GreedyForwardLayer
    <- ViterbiForwardLayer

and Greedy has it while Viterbi doesn't. It seems like there needs to be a Utils.viterbiWithScores for ViterbiForwardLayer.inferenceWithScores in the same way inference calls Utils.viterbi. That could probably be made by keeping track of backScores along with backPointers.

That probably doesn't answer your question. If you replace inferenceWithScore with just inference won't you lose the scores you are trying to locate? Is it loss() that you need instead?

[MihaiSurdeanu]

This one is on me. I implemented it for greedy inference, and did not implement it yet for Viterbi.
Haris, for now please change the inference type to greedy in the final layer, in the config file. It should work afterwards.

[hriaz17]

I guess what I really need to return from the viterbi() method in Utils.scala is pathScore along with bestPath. That would involve minor modifications to viterbi() and inference() but this is a short term solution that won't return an IndexedSeq[IndexedSeq[(String, Float)] nested array as specified in the inferenceWithScores() method.

[hriaz17]

This one is on me. I implemented it for greedy inference, and did not implement it yet for Viterbi. Haris, for now please change the inference type to greedy in the final layer, in the config file. It should work afterwards.

ok.

[kwalcock]

@harisriaz17, keep in mind there is one viterbi in org.clulab.dynet.Utils. Perhaps in the end one version can be used for both the existing and the new case.

[kwalcock]

Will this for sure be needed eventually, or is it contingent on things that might not happen? I.e., will implementing it now be potentially wasted effort?