首页

Javascript

Html
Css
Node.js
Electron
移动开发
小程序
工具类
服务端
浏览器相关
前端收藏
其他
关于
公司注册

nodejieba "结巴"中文分词安装使用

2019年01月16日 发布 阅读(6228) 作者：Jerman

NodeJieba是”结巴”中文分词的 Node.js 版本实现， 由CppJieba提供底层分词算法实现， 是兼具高性能和易用性两者的 Node.js 中文分词组件。

安装

  1. npm install nodejieba --save

安装报错：

  1. D:\project\imqianduan>npm install nodejieba --save
  3. > nodejieba@2.3.0 install D:\project\imqianduan\node_modules\nodejieba
  4. > node-gyp rebuild
  7. D:\project\imqianduan\node_modules\nodejieba>if not defined npm_config_node_gyp (node "C:\nvm\v10.15.0\node_modules\npm\node_modules\npm-lifecycle\node-gyp-bin\\..\..\node_modules\node-gyp\bin\node-gyp.js" rebuild )  else (node "C:\nvm\v10.15.0\node_modules\npm\node_modules\node-gyp\bin\node-gyp.js" rebuild )
  8. 在此解决方案中一次生成一个项目。若要启用并行生成，请添加“/m”开关。
  9.   index.cpp
  10.   nodejieba.cpp
  11. ..\lib\nodejieba.cpp : warning C4819: The file contains a character that cannot be represented in the current code page (936). Save the file in Unicode format to prevent data loss [D:\project\imqianduan
  12. \node_modules\nodejieba\build\nodejieba.vcxproj]
  13.   win_delay_load_hook.cc
  14. c:\users\administrator\.node-gyp\10.15.0\include\node\v8.h(2570): error C2144: syntax error : 'int' should be preceded by ';' (..\lib\index.cpp) [D:\project\imqianduan\node_modules\nodejieba\build\nodejieba.
  15. vcxproj]
  16. c:\users\administrator\.node-gyp\10.15.0\include\node\v8.h(2570): error C4430: missing type specifier - int assumed. Note: C++ does not support default-int (..\lib\index.cpp) [D:\project\imqianduan\node_modu
  17. les\nodejieba\build\nodejieba.vcxproj]
  18. c:\users\administrator\.node-gyp\10.15.0\include\node\v8.h(2570): error C2144: syntax error : 'int' should be preceded by ';' (..\lib\nodejieba.cpp) [D:\project\imqianduan\node_modules\nodejieba\build\nodeji
  19. eba.vcxproj]
  20. c:\users\administrator\.node-gyp\10.15.0\include\node\v8.h(2570): error C4430: missing type specifier - int assumed. Note: C++ does not support default-int (..\lib\nodejieba.cpp) [D:\project\imqianduan\node_
  21. modules\nodejieba\build\nodejieba.vcxproj]
  22. c:\users\administrator\.node-gyp\10.15.0\include\node\v8.h(4273): error C2610: 'v8::WasmCompiledModule::TransferrableModule::TransferrableModule(v8::WasmCompiledModule::TransferrableModule &&)' : is not a special member fun
  23. ction which can be defaulted (..\lib\nodejieba.cpp) [D:\project\imqianduan\node_modules\nodejieba\build\nodejieba.vcxproj]
  24. c:\users\administrator\.node-gyp\10.15.0\include\node\v8.h(4273): error C2610: 'v8::WasmCompiledModule::TransferrableModule::TransferrableModule(v8::WasmCompiledModule::TransferrableModule &&)' : is not a special member fun
  25. ction which can be defaulted (..\lib\index.cpp) [D:\project\imqianduan\node_modules\nodejieba\build\nodejieba.vcxproj]
  26. c:\users\administrator\.node-gyp\10.15.0\include\node\v8.h(4276): error C2610: 'v8::WasmCompiledModule::TransferrableModule &v8::WasmCompiledModule::TransferrableModule::operator =(v8::WasmCompiledModule::TransferrableModul
  27. e &&)' : is not a special member function which can be defaulted (..\lib\nodejieba.cpp) [D:\project\imqianduan\node_modules\nodejieba\build\nodejieba.vcxproj]
  28. c:\users\administrator\.node-gyp\10.15.0\include\node\v8.h(4276): error C2610: 'v8::WasmCompiledModule::TransferrableModule &v8::WasmCompiledModule::TransferrableModule::operator =(v8::WasmCompiledModule::TransferrableModul
  29. e &&)' : is not a special member function which can be defaulted (..\lib\index.cpp) [D:\project\imqianduan\node_modules\nodejieba\build\nodejieba.vcxproj]
  30. c:\users\administrator\.node-gyp\10.15.0\include\node\v8.h(4362): error C2610: 'v8::WasmModuleObjectBuilderStreaming::WasmModuleObjectBuilderStreaming(v8::WasmModuleObjectBuilderStreaming &&)' : is not a special member func
  31. tion which can be defaulted (..\lib\nodejieba.cpp) [D:\project\imqianduan\node_modules\nodejieba\build\nodejieba.vcxproj]
  32. c:\users\administrator\.node-gyp\10.15.0\include\node\v8.h(4367): error C2610: 'v8::WasmModuleObjectBuilderStreaming &v8::WasmModuleObjectBuilderStreaming::operator =(v8::WasmModuleObjectBuilderStreaming &&)' : is not a spe
  33. cial member function which can be defaulted (..\lib\nodejieba.cpp) [D:\project\imqianduan\node_modules\nodejieba\build\nodejieba.vcxproj]
  34. c:\users\administrator\.node-gyp\10.15.0\include\node\v8.h(4362): error C2610: 'v8::WasmModuleObjectBuilderStreaming::WasmModuleObjectBuilderStreaming(v8::WasmModuleObjectBuilderStreaming &&)' : is not a special member func
  35. tion which can be defaulted (..\lib\index.cpp) [D:\project\imqianduan\node_modules\nodejieba\build\nodejieba.vcxproj]
  36. c:\users\administrator\.node-gyp\10.15.0\include\node\v8.h(4632): error C2146: syntax error : missing ';' before identifier 'size_t' (..\lib\nodejieba.cpp) [D:\project\imqianduan\node_modules\nodejieba\build
  37. \nodejieba.vcxproj]
  38. c:\users\administrator\.node-gyp\10.15.0\include\node\v8.h(4367): error C2610: 'v8::WasmModuleObjectBuilderStreaming &v8::WasmModuleObjectBuilderStreaming::operator =(v8::WasmModuleObjectBuilderStreaming &&)' : is not a spe
  39. cial member function which can be defaulted (..\lib\index.cpp) [D:\project\imqianduan\node_modules\nodejieba\build\nodejieba.vcxproj]
  40. c:\users\administrator\.node-gyp\10.15.0\include\node\v8.h(4632): error C4430: missing type specifier - int assumed. Note: C++ does not support default-int (..\lib\nodejieba.cpp) [D:\project\imqianduan\node_
  41. modules\nodejieba\build\nodejieba.vcxproj]
  42. c:\users\administrator\.node-gyp\10.15.0\include\node\v8.h(4632): error C2146: syntax error : missing ';' before identifier 'size_t' (..\lib\index.cpp) [D:\project\imqianduan\node_modules\nodejieba\build\nod
  43. ejieba.vcxproj]
  44. c:\users\administrator\.node-gyp\10.15.0\include\node\v8.h(4632): error C4430: missing type specifier - int assumed. Note: C++ does not support default-int (..\lib\index.cpp) [D:\project\imqianduan\node_modu
  45. les\nodejieba\build\nodejieba.vcxproj]
  46. C:\Users\administrator\.node-gyp\10.15.0\include\node\node_buffer.h(36): error C2597: illegal reference to non-static member 'v8::TypedArray::kMaxLength' (..\lib\index.cpp) [D:\project\imqianduan\node_module
  47. s\nodejieba\build\nodejieba.vcxproj]
  48. C:\Users\administrator\.node-gyp\10.15.0\include\node\node_buffer.h(36): error C2597: illegal reference to non-static member 'v8::TypedArray::kMaxLength' (..\lib\nodejieba.cpp) [D:\project\imqianduan\node_mo
  49. dules\nodejieba\build\nodejieba.vcxproj]
  50. C:\Users\administrator\.node-gyp\10.15.0\include\node\node_buffer.h(36): error C2440: 'initializing' : cannot convert from 'unknown' to 'const unsigned int' (..\lib\index.cpp) [D:\project\imqianduan\node_mod
  51. ules\nodejieba\build\nodejieba.vcxproj]
  52.           There is no context in which this conversion is possible
  53. C:\Users\administrator\.node-gyp\10.15.0\include\node\node_buffer.h(36): error C2440: 'initializing' : cannot convert from 'unknown' to 'const unsigned int' (..\lib\nodejieba.cpp) [D:\project\imqianduan\node
  54. _modules\nodejieba\build\nodejieba.vcxproj]
  55.           There is no context in which this conversion is possible
  56. d:\project\imqianduan\node_modules\nodejieba\deps\cppjieba\Utils.hpp : warning C4819: The file contains a character that cannot be represented in the current code page (936). Save the file in Unicode fo
  57. rmat to prevent data loss (..\lib\nodejieba.cpp) [D:\project\imqianduan\node_modules\nodejieba\build\nodejieba.vcxproj]
  58. D:\project\imqianduan\node_modules\nodejieba\deps\cppjieba/TextRankExtractor.hpp : warning C4819: The file contains a character that cannot be represented in the current code page (936). Save the file i
  59. n Unicode format to prevent data loss (..\lib\nodejieba.cpp) [D:\project\imqianduan\node_modules\nodejieba\build\nodejieba.vcxproj]
  60. gyp ERR! build error

或者报下面错误：

  1. ```
  2. gyp ERR! configure error
  3. gyp ERR! stack Error: Can't find Python executable "python", you can set the PYTHON env variable.
  4. gyp ERR! stack     at PythonFinder.failNoPython (D:\Program Files\node_modules\npm\node_modules\node-gyp\lib\configure.js:483:19)
  5. gyp ERR! stack     at PythonFinder.<anonymous> (D:\Program Files\node_modules\npm\node_modules\node-gyp\lib\configure.js:508:16)
  6. gyp ERR! stack     at D:\Program Files\node_modules\npm\node_modules\graceful-fs\polyfills.js:284:29
  7. gyp ERR! stack     at FSReqWrap.oncomplete (fs.js:154:21)
  8. gyp ERR! System Windows_NT 6.1.7601
  9. gyp ERR! command "C:\\Program Files\\nodejs\\node.exe" "D:\\Program Files\\node_modules\\npm\\node_modules\\node-gyp\\bin\\node-gyp.js" "rebuild"
  10. gyp ERR! cwd G:\works\my_project\imqianduan.com\node_modules\nodejieba
  11. gyp ERR! node -v v10.15.0
  12. gyp ERR! node-gyp -v v3.6.2
  13. gyp ERR! not ok
  14. npm WARN @ant-design/icons-react@1.1.2 requires a peer of react@16.x but none is installed. You must install peer dependencies yourself.
  15. npm WARN acorn-jsx@5.0.1 requires a peer of acorn@^6.0.0 but none is installed. You must install peer dependencies yourself.
  16. npm WARN antd@3.11.6 requires a peer of react@>=16.0.0 but none is installed. You must install peer dependencies yourself.
  17. npm WARN antd@3.11.6 requires a peer of react-dom@>=16.0.0 but none is installed. You must install peer dependencies yourself.
  18. npm WARN co-mocha@1.2.2 requires a peer of mocha@>=1.18 <6 but none is installed. You must install peer dependencies yourself.
  19. npm WARN create-react-context@0.2.2 requires a peer of react@^0.14.0 || ^15.0.0 || ^16.0.0 but none is installed. You must install peer dependencies yourself.
  20. npm WARN draft-js@0.10.5 requires a peer of react@^0.14.0 || ^15.0.0-rc || ^16.0.0-rc || ^16.0.0 but none is installed. You must install peer dependencies yourself.
  21. npm WARN draft-js@0.10.5 requires a peer of react-dom@^0.14.0 || ^15.0.0-rc || ^16.0.0-rc || ^16.0.0 but none is installed. You must install peer dependencies yourself.
  22. npm WARN eslint-import-resolver-webpack@0.10.1 requires a peer of webpack@>=1.11.0 but none is installed. You must install peer dependencies yourself.
  23. npm WARN espower-typescript@9.0.1 requires a peer of typescript@>= 2.4.2 but none is installed. You must install peer dependencies yourself.
  24. npm WARN rc-editor-mention@1.1.12 requires a peer of react@>=15.x but none is installed. You must install peer dependencies yourself.
  25. npm WARN rc-editor-mention@1.1.12 requires a peer of react-dom@>=15.x but none is installed. You must install peer dependencies yourself.
  26. npm WARN rc-hammerjs@0.6.9 requires a peer of react@^0.14.3 || ^15.0.0 || ^16.0.0 but none is installed. You must install peer dependencies yourself.
  27. npm WARN rc-switch@1.8.0 requires a peer of react@^16.0.0 but none is installed. You must install peer dependencies yourself.
  28. npm WARN rc-switch@1.8.0 requires a peer of react-dom@^16.0.0 but none is installed. You must install peer dependencies yourself.
  29. npm WARN react-lazy-load@3.0.13 requires a peer of react@^0.14.0 || ^15.0.0-0 || ^16.0.0 but none is installed. You must install peer dependencies yourself.
  30. npm WARN react-lazy-load@3.0.13 requires a peer of react-dom@^0.14.0 || ^15.0.0-0 || ^16.0.0 but none is installed. You must install peer dependencies yourself.
  31. npm WARN react-slick@0.23.2 requires a peer of react@^0.14.0 || ^15.0.1 || ^16.0.0 but none is installed. You must install peer dependencies yourself.
  32. npm WARN react-slick@0.23.2 requires a peer of react-dom@^0.14.0 || ^15.0.1 || ^16.0.0 but none is installed. You must install peer dependencies yourself.
  33. npm WARN The package moment is included as both a dev and production dependency.
  34. npm WARN optional SKIPPING OPTIONAL DEPENDENCY: fsevents@1.2.4 (node_modules\fsevents):
  35. npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for fsevents@1.2.4: wanted {"os":"darwin","arch":"any"} (current: {"os":"win32","arch":"x64"})
  37. npm ERR! code ELIFECYCLE
  38. npm ERR! errno 1
  39. npm ERR! nodejieba@2.3.0 install: `node-gyp rebuild`
  40. npm ERR! Exit status 1
  41. npm ERR!
  42. npm ERR! Failed at the nodejieba@2.3.0 install script.
  43. npm ERR! This is probably not a problem with npm. There is likely additional logging output above.
  45. npm ERR! A complete log of this run can be found in:
  46. npm ERR!     C:\Users\Administrator\AppData\Roaming\npm-cache\_logs\2019-03-15T23_28_57_609Z-debug.log

直接安装会各种报错的，因为底层是C++实现的，所以必须有C++编译环境，必须安装node-gyp,从上面报错的信息也可以看出，它是依赖node-gyp的

  1. npm install -g node-gyp

node-gyp是需要依赖python2.7和微软的vc++构建工具来进行编译,这在linux系统上没问题，因为linux上都默认安装了，但是windows操作系统在默认情况下不会安装python2.7和vc++构建工具。

windows下安装编译环境（linux忽略)

  1. npm install --global --production windows-build-tools

安装好node-gyp和windows-build-tools后，就可以直接使用npm install nodejieba --save安装了

windows-build-tools安装失败怎么办？
1.它依赖.NET Framework ，下载： https://dotnet.microsoft.com/download/dotnet-framework-runtime/net472
2.手动安装Visual Studio 2017 https://visualstudio.microsoft.com/zh-hans/downloads/
3.切换npm源，我使用nrm切换过cnpm,taobao都不行，最后切换到npm才OK
源不行的报错：

  1. ---------- Visual Studio Build Tools ----------
  2. 2019-03-15T23:11:12 : Verbose : Spawning uninstall stub
  3. 2019-03-15T23:11:13 : Verbose : [InstallerImpl]: Rpc connection was cl
  4. 2019-03-15T23:11:13 : Verbose : [InstallerImpl]: Stream was closed
  5. 2019-03-15T23:11:13 : Verbose : [SetupUpdaterImpl]: Rpc connection was
  6. 2019-03-15T23:11:13 : Verbose : [SetupUpdaterImpl]: Stream was closed

4.安装windows-build-tools需要耐心等待

最后安装成功是这样：

  1. > windows-build-tools@5.1.0 postinstall C:\Users\Administrator\AppData\Roaming\npm\node_modules\windows-build-tools
  2. > node ./dist/index.js
  6. Downloading python-2.7.15.amd64.msi
  7. [>                                            ] 0.0% (0 B/s)
  8. Downloaded python-2.7.15.amd64.msi. Saved to C:\Users\Administrator\.windows-build-tools\python-2.7.15.amd64.msi.
  9. Downloading vs_BuildTools.exe
  10. [>                                            ] 0.0% (0 B/s)
  11. Downloaded vs_BuildTools.exe. Saved to C:\Users\Administrator\.windows-build-tools\vs_BuildTools.exe.
  13. Starting installation...
  14. Launched installers, now waiting for them to finish.
  15. This will likely take some time - please be patient!
  17. Status from the installers:
  18. ---------- Visual Studio Build Tools ----------
  19. Successfully installed Visual Studio Build Tools.
  20. ------------------- Python --------------------
  21. Successfully installed Python 2.7
  23. Now configuring the Visual Studio Build Tools and Python...
  25. All done!
  27. + windows-build-tools@5.1.0
  28. updated 53 packages in 492.995s

使用

使用比较简单，可以参考官方说明https://github.com/yanyiwu/nodejieba ，或后面参考的文章

到于效果，这个文章最后面的也许你还感兴趣的相关文章推荐列表，就是通过它分词后，匹配出来的。

参考

https://www.jianshu.com/p/2b831714bbff
https://75team.com/post/segment-with-nodejs.html

版权声明：本站文章除特别声明外，均采用署名-非商业性使用-禁止演绎 4.0 国际 许可协议，如需转载，请注明出处

© 2018 IMQIANDUAN.COM 粤ICP备18143223号