Gensim 4.0 loading Phraser trained from Gensim 3.x

[canmingh]

Problem description

Gensim 4.0 cannot load Phraser model from Gensim 3.x

Steps/code/corpus to reproduce

gemsim.model.load("saved_phraser.pkl") 

Will not be able to load phraser trained in gensim 3.x due to a bug in source code. See below.

Versions

This line of code from 4.0.1 (and also current development branch) need to be fixed:
https://github.com/RaRe-Technologies/gensim/blob/4.0.1/gensim/models/phrases.py#L367

model.phrasegrams = {
  str(model.delimiter.join(component), encoding='utf8'): score
  for key, val in phrasegrams.items()
}

The existing code will only keep the first phrase. To upgrade and load all phrases, it should be replaced with:

model.phrasegrams = {
  str(model.delimiter.join(key), encoding='utf8'): val
  for key, val in phrasegrams.items()
}

[piskvorky]

You're right, thanks. Can you open a PR?

[canmingh]

Just created a pull request. See above. Thanks.