Section: User Contributed Perl Documentation (1)Updated: 2010-09-21Local indexUp
mmseg - maximum matching segment Chinese text.
mmseg -d dict_file [option]... [corpus_file]...
mmseg is a tool for segmenting Chinese text into words using maximum matching algorithm. mmseg segments corpus_file, or standard input if no filename is specified, and write the segmented result to standard output.
Use dict_file as lexicon. A default lexicon can be found at /usr/share/sunpinyin-slm/dict.utf8.
Output Format, can be 'text' or 'bin'. default 'bin'.
Normally, in text mode, word text are output, while in binary mode,
binary short integer of the word-ids are written to stdout.
Sentence token id. Default 10.
It will be written to output in binary mode after every sentence.
Show Id info. Under text output format mode, attach id after known words.
If under binary mode, print id(s) in text.
Ambiguious means ABC => ABC or ABC. If specified (AMBI-ID != 0),
The sequence ABC will not be segmented, in binary mode, the AMBI-ID is written out; in text mode, "<ambi>ABC</ambi>" will be output. Default is 0.
Under binary mode, consecutive id of 0 are merged into one 0.
Under text mode, no space are inserted between unknown-words.