首页
Javascript
Html
Css
Node.js
Electron
移动开发
工具类
服务端
浏览器相关
前端收藏
其他
关于

nodejieba "结巴"中文分词安装使用

2019年01月16日 发布 阅读(1066) 作者:懒炖

NodeJieba是”结巴”中文分词的 Node.js 版本实现, 由CppJieba提供底层分词算法实现, 是兼具高性能和易用性两者的 Node.js 中文分词组件。

安装

  1. npm install nodejieba --save

安装报错:

  1. D:\project\imqianduan>npm install nodejieba --save
  2. > nodejieba@2.3.0 install D:\project\imqianduan\node_modules\nodejieba
  3. > node-gyp rebuild
  4. D:\project\imqianduan\node_modules\nodejieba>if not defined npm_config_node_gyp (node "C:\nvm\v10.15.0\node_modules\npm\node_modules\npm-lifecycle\node-gyp-bin\\..\..\node_modules\node-gyp\bin\node-gyp.js" rebuild ) else (node "C:\nvm\v10.15.0\node_modules\npm\node_modules\node-gyp\bin\node-gyp.js" rebuild )
  5. 在此解决方案中一次生成一个项目。若要启用并行生成,请添加“/m”开关。
  6. index.cpp
  7. nodejieba.cpp
  8. ..\lib\nodejieba.cpp : warning C4819: The file contains a character that cannot be represented in the current code page (936). Save the file in Unicode format to prevent data loss [D:\project\imqianduan
  9. \node_modules\nodejieba\build\nodejieba.vcxproj]
  10. win_delay_load_hook.cc
  11. c:\users\administrator\.node-gyp\10.15.0\include\node\v8.h(2570): error C2144: syntax error : 'int' should be preceded by ';' (..\lib\index.cpp) [D:\project\imqianduan\node_modules\nodejieba\build\nodejieba.
  12. vcxproj]
  13. c:\users\administrator\.node-gyp\10.15.0\include\node\v8.h(2570): error C4430: missing type specifier - int assumed. Note: C++ does not support default-int (..\lib\index.cpp) [D:\project\imqianduan\node_modu
  14. les\nodejieba\build\nodejieba.vcxproj]
  15. c:\users\administrator\.node-gyp\10.15.0\include\node\v8.h(2570): error C2144: syntax error : 'int' should be preceded by ';' (..\lib\nodejieba.cpp) [D:\project\imqianduan\node_modules\nodejieba\build\nodeji
  16. eba.vcxproj]
  17. c:\users\administrator\.node-gyp\10.15.0\include\node\v8.h(2570): error C4430: missing type specifier - int assumed. Note: C++ does not support default-int (..\lib\nodejieba.cpp) [D:\project\imqianduan\node_
  18. modules\nodejieba\build\nodejieba.vcxproj]
  19. c:\users\administrator\.node-gyp\10.15.0\include\node\v8.h(4273): error C2610: 'v8::WasmCompiledModule::TransferrableModule::TransferrableModule(v8::WasmCompiledModule::TransferrableModule &&)' : is not a special member fun
  20. ction which can be defaulted (..\lib\nodejieba.cpp) [D:\project\imqianduan\node_modules\nodejieba\build\nodejieba.vcxproj]
  21. c:\users\administrator\.node-gyp\10.15.0\include\node\v8.h(4273): error C2610: 'v8::WasmCompiledModule::TransferrableModule::TransferrableModule(v8::WasmCompiledModule::TransferrableModule &&)' : is not a special member fun
  22. ction which can be defaulted (..\lib\index.cpp) [D:\project\imqianduan\node_modules\nodejieba\build\nodejieba.vcxproj]
  23. c:\users\administrator\.node-gyp\10.15.0\include\node\v8.h(4276): error C2610: 'v8::WasmCompiledModule::TransferrableModule &v8::WasmCompiledModule::TransferrableModule::operator =(v8::WasmCompiledModule::TransferrableModul
  24. e &&)' : is not a special member function which can be defaulted (..\lib\nodejieba.cpp) [D:\project\imqianduan\node_modules\nodejieba\build\nodejieba.vcxproj]
  25. c:\users\administrator\.node-gyp\10.15.0\include\node\v8.h(4276): error C2610: 'v8::WasmCompiledModule::TransferrableModule &v8::WasmCompiledModule::TransferrableModule::operator =(v8::WasmCompiledModule::TransferrableModul
  26. e &&)' : is not a special member function which can be defaulted (..\lib\index.cpp) [D:\project\imqianduan\node_modules\nodejieba\build\nodejieba.vcxproj]
  27. c:\users\administrator\.node-gyp\10.15.0\include\node\v8.h(4362): error C2610: 'v8::WasmModuleObjectBuilderStreaming::WasmModuleObjectBuilderStreaming(v8::WasmModuleObjectBuilderStreaming &&)' : is not a special member func
  28. tion which can be defaulted (..\lib\nodejieba.cpp) [D:\project\imqianduan\node_modules\nodejieba\build\nodejieba.vcxproj]
  29. c:\users\administrator\.node-gyp\10.15.0\include\node\v8.h(4367): error C2610: 'v8::WasmModuleObjectBuilderStreaming &v8::WasmModuleObjectBuilderStreaming::operator =(v8::WasmModuleObjectBuilderStreaming &&)' : is not a spe
  30. cial member function which can be defaulted (..\lib\nodejieba.cpp) [D:\project\imqianduan\node_modules\nodejieba\build\nodejieba.vcxproj]
  31. c:\users\administrator\.node-gyp\10.15.0\include\node\v8.h(4362): error C2610: 'v8::WasmModuleObjectBuilderStreaming::WasmModuleObjectBuilderStreaming(v8::WasmModuleObjectBuilderStreaming &&)' : is not a special member func
  32. tion which can be defaulted (..\lib\index.cpp) [D:\project\imqianduan\node_modules\nodejieba\build\nodejieba.vcxproj]
  33. c:\users\administrator\.node-gyp\10.15.0\include\node\v8.h(4632): error C2146: syntax error : missing ';' before identifier 'size_t' (..\lib\nodejieba.cpp) [D:\project\imqianduan\node_modules\nodejieba\build
  34. \nodejieba.vcxproj]
  35. c:\users\administrator\.node-gyp\10.15.0\include\node\v8.h(4367): error C2610: 'v8::WasmModuleObjectBuilderStreaming &v8::WasmModuleObjectBuilderStreaming::operator =(v8::WasmModuleObjectBuilderStreaming &&)' : is not a spe
  36. cial member function which can be defaulted (..\lib\index.cpp) [D:\project\imqianduan\node_modules\nodejieba\build\nodejieba.vcxproj]
  37. c:\users\administrator\.node-gyp\10.15.0\include\node\v8.h(4632): error C4430: missing type specifier - int assumed. Note: C++ does not support default-int (..\lib\nodejieba.cpp) [D:\project\imqianduan\node_
  38. modules\nodejieba\build\nodejieba.vcxproj]
  39. c:\users\administrator\.node-gyp\10.15.0\include\node\v8.h(4632): error C2146: syntax error : missing ';' before identifier 'size_t' (..\lib\index.cpp) [D:\project\imqianduan\node_modules\nodejieba\build\nod
  40. ejieba.vcxproj]
  41. c:\users\administrator\.node-gyp\10.15.0\include\node\v8.h(4632): error C4430: missing type specifier - int assumed. Note: C++ does not support default-int (..\lib\index.cpp) [D:\project\imqianduan\node_modu
  42. les\nodejieba\build\nodejieba.vcxproj]
  43. C:\Users\administrator\.node-gyp\10.15.0\include\node\node_buffer.h(36): error C2597: illegal reference to non-static member 'v8::TypedArray::kMaxLength' (..\lib\index.cpp) [D:\project\imqianduan\node_module
  44. s\nodejieba\build\nodejieba.vcxproj]
  45. C:\Users\administrator\.node-gyp\10.15.0\include\node\node_buffer.h(36): error C2597: illegal reference to non-static member 'v8::TypedArray::kMaxLength' (..\lib\nodejieba.cpp) [D:\project\imqianduan\node_mo
  46. dules\nodejieba\build\nodejieba.vcxproj]
  47. C:\Users\administrator\.node-gyp\10.15.0\include\node\node_buffer.h(36): error C2440: 'initializing' : cannot convert from 'unknown' to 'const unsigned int' (..\lib\index.cpp) [D:\project\imqianduan\node_mod
  48. ules\nodejieba\build\nodejieba.vcxproj]
  49. There is no context in which this conversion is possible
  50. C:\Users\administrator\.node-gyp\10.15.0\include\node\node_buffer.h(36): error C2440: 'initializing' : cannot convert from 'unknown' to 'const unsigned int' (..\lib\nodejieba.cpp) [D:\project\imqianduan\node
  51. _modules\nodejieba\build\nodejieba.vcxproj]
  52. There is no context in which this conversion is possible
  53. d:\project\imqianduan\node_modules\nodejieba\deps\cppjieba\Utils.hpp : warning C4819: The file contains a character that cannot be represented in the current code page (936). Save the file in Unicode fo
  54. rmat to prevent data loss (..\lib\nodejieba.cpp) [D:\project\imqianduan\node_modules\nodejieba\build\nodejieba.vcxproj]
  55. D:\project\imqianduan\node_modules\nodejieba\deps\cppjieba/TextRankExtractor.hpp : warning C4819: The file contains a character that cannot be represented in the current code page (936). Save the file i
  56. n Unicode format to prevent data loss (..\lib\nodejieba.cpp) [D:\project\imqianduan\node_modules\nodejieba\build\nodejieba.vcxproj]
  57. gyp ERR! build error

或者报下面错误:

  1. ```
  2. gyp ERR! configure error
  3. gyp ERR! stack Error: Can't find Python executable "python", you can set the PYTHON env variable.
  4. gyp ERR! stack at PythonFinder.failNoPython (D:\Program Files\node_modules\npm\node_modules\node-gyp\lib\configure.js:483:19)
  5. gyp ERR! stack at PythonFinder.<anonymous> (D:\Program Files\node_modules\npm\node_modules\node-gyp\lib\configure.js:508:16)
  6. gyp ERR! stack at D:\Program Files\node_modules\npm\node_modules\graceful-fs\polyfills.js:284:29
  7. gyp ERR! stack at FSReqWrap.oncomplete (fs.js:154:21)
  8. gyp ERR! System Windows_NT 6.1.7601
  9. gyp ERR! command "C:\\Program Files\\nodejs\\node.exe" "D:\\Program Files\\node_modules\\npm\\node_modules\\node-gyp\\bin\\node-gyp.js" "rebuild"
  10. gyp ERR! cwd G:\works\my_project\imqianduan.com\node_modules\nodejieba
  11. gyp ERR! node -v v10.15.0
  12. gyp ERR! node-gyp -v v3.6.2
  13. gyp ERR! not ok
  14. npm WARN @ant-design/icons-react@1.1.2 requires a peer of react@16.x but none is installed. You must install peer dependencies yourself.
  15. npm WARN acorn-jsx@5.0.1 requires a peer of acorn@^6.0.0 but none is installed. You must install peer dependencies yourself.
  16. npm WARN antd@3.11.6 requires a peer of react@>=16.0.0 but none is installed. You must install peer dependencies yourself.
  17. npm WARN antd@3.11.6 requires a peer of react-dom@>=16.0.0 but none is installed. You must install peer dependencies yourself.
  18. npm WARN co-mocha@1.2.2 requires a peer of mocha@>=1.18 <6 but none is installed. You must install peer dependencies yourself.
  19. npm WARN create-react-context@0.2.2 requires a peer of react@^0.14.0 || ^15.0.0 || ^16.0.0 but none is installed. You must install peer dependencies yourself.
  20. npm WARN draft-js@0.10.5 requires a peer of react@^0.14.0 || ^15.0.0-rc || ^16.0.0-rc || ^16.0.0 but none is installed. You must install peer dependencies yourself.
  21. npm WARN draft-js@0.10.5 requires a peer of react-dom@^0.14.0 || ^15.0.0-rc || ^16.0.0-rc || ^16.0.0 but none is installed. You must install peer dependencies yourself.
  22. npm WARN eslint-import-resolver-webpack@0.10.1 requires a peer of webpack@>=1.11.0 but none is installed. You must install peer dependencies yourself.
  23. npm WARN espower-typescript@9.0.1 requires a peer of typescript@>= 2.4.2 but none is installed. You must install peer dependencies yourself.
  24. npm WARN rc-editor-mention@1.1.12 requires a peer of react@>=15.x but none is installed. You must install peer dependencies yourself.
  25. npm WARN rc-editor-mention@1.1.12 requires a peer of react-dom@>=15.x but none is installed. You must install peer dependencies yourself.
  26. npm WARN rc-hammerjs@0.6.9 requires a peer of react@^0.14.3 || ^15.0.0 || ^16.0.0 but none is installed. You must install peer dependencies yourself.
  27. npm WARN rc-switch@1.8.0 requires a peer of react@^16.0.0 but none is installed. You must install peer dependencies yourself.
  28. npm WARN rc-switch@1.8.0 requires a peer of react-dom@^16.0.0 but none is installed. You must install peer dependencies yourself.
  29. npm WARN react-lazy-load@3.0.13 requires a peer of react@^0.14.0 || ^15.0.0-0 || ^16.0.0 but none is installed. You must install peer dependencies yourself.
  30. npm WARN react-lazy-load@3.0.13 requires a peer of react-dom@^0.14.0 || ^15.0.0-0 || ^16.0.0 but none is installed. You must install peer dependencies yourself.
  31. npm WARN react-slick@0.23.2 requires a peer of react@^0.14.0 || ^15.0.1 || ^16.0.0 but none is installed. You must install peer dependencies yourself.
  32. npm WARN react-slick@0.23.2 requires a peer of react-dom@^0.14.0 || ^15.0.1 || ^16.0.0 but none is installed. You must install peer dependencies yourself.
  33. npm WARN The package moment is included as both a dev and production dependency.
  34. npm WARN optional SKIPPING OPTIONAL DEPENDENCY: fsevents@1.2.4 (node_modules\fsevents):
  35. npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for fsevents@1.2.4: wanted {"os":"darwin","arch":"any"} (current: {"os":"win32","arch":"x64"})
  36. npm ERR! code ELIFECYCLE
  37. npm ERR! errno 1
  38. npm ERR! nodejieba@2.3.0 install: `node-gyp rebuild`
  39. npm ERR! Exit status 1
  40. npm ERR!
  41. npm ERR! Failed at the nodejieba@2.3.0 install script.
  42. npm ERR! This is probably not a problem with npm. There is likely additional logging output above.
  43. npm ERR! A complete log of this run can be found in:
  44. npm ERR! C:\Users\Administrator\AppData\Roaming\npm-cache\_logs\2019-03-15T23_28_57_609Z-debug.log

直接安装会各种报错的,因为底层是C++实现的,所以必须有C++编译环境,必须安装node-gyp,从上面报错的信息也可以看出,它是依赖node-gyp

  1. npm install -g node-gyp

node-gyp是需要依赖python2.7和微软的vc++构建工具来进行编译,这在linux系统上没问题,因为linux上都默认安装了,但是windows操作系统在默认情况下不会安装python2.7和vc++构建工具。

windows下安装编译环境(linux忽略)

  1. npm install --global --production windows-build-tools

安装好node-gypwindows-build-tools后,就可以直接使用npm install nodejieba --save安装了

windows-build-tools安装失败怎么办?
1.它依赖.NET Framework ,下载: https://dotnet.microsoft.com/download/dotnet-framework-runtime/net472
2.手动安装Visual Studio 2017 https://visualstudio.microsoft.com/zh-hans/downloads/
3.切换npm源,我使用nrm切换过cnpm,taobao都不行,最后切换到npm才OK
源不行的报错:

  1. ---------- Visual Studio Build Tools ----------
  2. 2019-03-15T23:11:12 : Verbose : Spawning uninstall stub
  3. 2019-03-15T23:11:13 : Verbose : [InstallerImpl]: Rpc connection was cl
  4. 2019-03-15T23:11:13 : Verbose : [InstallerImpl]: Stream was closed
  5. 2019-03-15T23:11:13 : Verbose : [SetupUpdaterImpl]: Rpc connection was
  6. 2019-03-15T23:11:13 : Verbose : [SetupUpdaterImpl]: Stream was closed

4.安装windows-build-tools需要耐心等待

最后安装成功是这样:

  1. > windows-build-tools@5.1.0 postinstall C:\Users\Administrator\AppData\Roaming\npm\node_modules\windows-build-tools
  2. > node ./dist/index.js
  3. Downloading python-2.7.15.amd64.msi
  4. [> ] 0.0% (0 B/s)
  5. Downloaded python-2.7.15.amd64.msi. Saved to C:\Users\Administrator\.windows-build-tools\python-2.7.15.amd64.msi.
  6. Downloading vs_BuildTools.exe
  7. [> ] 0.0% (0 B/s)
  8. Downloaded vs_BuildTools.exe. Saved to C:\Users\Administrator\.windows-build-tools\vs_BuildTools.exe.
  9. Starting installation...
  10. Launched installers, now waiting for them to finish.
  11. This will likely take some time - please be patient!
  12. Status from the installers:
  13. ---------- Visual Studio Build Tools ----------
  14. Successfully installed Visual Studio Build Tools.
  15. ------------------- Python --------------------
  16. Successfully installed Python 2.7
  17. Now configuring the Visual Studio Build Tools and Python...
  18. All done!
  19. + windows-build-tools@5.1.0
  20. updated 53 packages in 492.995s

使用

使用比较简单,可以参考官方说明https://github.com/yanyiwu/nodejieba ,或后面参考的文章

到于效果,这个文章最后面的也许你还感兴趣的相关文章推荐列表,就是通过它分词后,匹配出来的。

参考

https://www.jianshu.com/p/2b831714bbff
https://75team.com/post/segment-with-nodejs.html

版权声明:本站文章除特别声明外,均采用署名-非商业性使用-禁止演绎 4.0 国际 许可协议,如需转载,请注明出处

评论

  •