the result of baseline profilememorynet of task convai2 is so bad

[zhaoguangxiang]

prediction: i love to do , but i am a nurse START START START START START START START START START START START START START START START START START START START START START START START START START START START START START START START START START START START START START START START START START START START START START START START START START START START
test:{'hits@10': 0.525, 'total': 7801, 'loss': 4.539, 'hits@1': 0.0495, 'f1': 0.0324, 'accuracy': 0, 'hits@5': 0.267, 'hits@100': 1.0, 'ppl': 93.59}

when i run the train.py under the profilememory folder

[zhaoguangxiang]

None of the other parameters have changed, except that the batch is set to 64

[emilydinan]

It's possible that you need to train for longer. It looks like in that training script it is set to validate every 3 seconds which is something it definitely should not do. I will put out a PR to fix this.

If you look at this script here you can interact with the profile memory model trained on convai2 that seems (at least qualitatively) better and was trained for much longer.

[alexholdenmiller]

closing for now, feel free to reopen if you have more questions