Skip to content

Sudachi version 0.7.4

Compare
Choose a tag to compare
@github-actions github-actions released this 02 Jul 07:27

Highlights

  • Add Tokenizer.lazyTokenizeSentences(SplitMode mode, Readable input), that performs analysis lazily and saves memory usage (#231)
    • Tokenizer.tokenizeSentences(SplitMode mode, Reader input) is marked as deprecated.

Fixed

  • Do not segfault on tokenizing with closed dictionary (#217)
  • The default config sudachi.json sets non-existent property joinKanjiNumeric in JoinNumericPlugin (#221)
  • fix incorrect size calculation when expand (#227)
  • Update tutorial.md (#226)