Go 語言高效分詞,支援英文、中文、日文等。
該版本主要增加:AlphaNum 拉丁字母和數字分詞設定,RemoveToken,gse.New(),分詞 ToLower 設定;Tirm 清除標點符號和空格,TextFreq 設定預設載入 frenquency,cutDAGNoHMM 和最短路徑加動態規劃配置,支援 ", " 分割字典檔案等功能。
增加更多示例、測試和 benchmark 程式碼, 修復 bug。
專案地址:https://github.com/go-ego/gse
add alphanum split words support
use variable to control default frequency
add gse.New() function and update test code,
add New() function AlphaNum set support
add ToLower set support and update code
add remove token support and test code,
add addToken and removeToken benchmark test,
update benchmark, example.go and godoc
add more test code
add English example code,
Add English token test code and Update README.md
update segment test code
update segment test code, add more test and remove unless code
add Trim punct and space function support,
add Trim test code,
add Trim benchmark test code,
add Trim example code
update Trim function not exclude "include other char string"
update DictPaths() allow use ", " to separated dictionary files
add cutDAGNoHMM support and test code,
add Cut() cutDAGNoHMM option support,
add test and benchmark code
add python and node binding
update gse code
export splitTextToWords function
export lookupTokens function
update example code
update data re base code
add Find token example code
update English example code
update example code use mixed text
update dictionary example code
update code fixed example print
update hmm test and example code
update gse server code
update server code add hmm support,
update jquery.min.js to v3.5.0
update server.go listen log print
update code and README.md
Update README.md, add binding...
update godoc
update load model godoc
update godoc and var name
update go mod pkg and version
remove unless go.sum hash
update go mod and remove vendor
add github workflow support
update go.yml
update and fmt circle.yml, travis.yml
add CI go1.13x support remove go1.11.x
rename utils.go to seg_utils.go
add more benchmark support
add hmm benchmark test support
add segmenter benchmark test code
add idf and pos files
update issue and pull request template
update segment test code use expect
update alpha num test code
[admin
]