I hope you are doing well! I have a question about my kibana development environment. I have developed kibana from source on one workstation and copied it from there and pasted it on another work station. it isn't working at another place and I can not develop it there because of security restrictions. Is there any other way? should it work there?
Can you share a little more info? What security restrictions are in place? What error logs are you seeing?
I'm trying to think what we'd see, but without logs it's hard to debug. Network restrictions would cause issues. We'd probably see fs errors under selinux.
My appologies, these are just proxy setups, nothing else. but after moving whole development environment when I run yarn es snapshot, it simply gives the following error. This is not supposed to happen because the same setup works on another workstation.
$ yarn es snapshot
yarn run v1.22.5
$ node scripts/es snapshot
Error: Cannot find module '@kbn/es'
Require stack:
- C:\TEMP\dev-kibana7133\kibana\scripts\es.js
at Function.Module._resolveFilename (internal/modules/cjs/loader.js:902:15)
at Module.Hook._require.Module.require (C:\TEMP\dev-kibana7133\kibana\node_m
odules\require-in-the-middle\index.js:61:29)
at require (internal/modules/cjs/helpers.js:92:18)
at Object.<anonymous> (C:\TEMP\dev-kibana7133\kibana\scripts\es.js:13:13)
at Module._compile (internal/modules/cjs/loader.js:1085:14)
at Object.Module._extensions..js (internal/modules/cjs/loader.js:1114:10)
at Module.load (internal/modules/cjs/loader.js:950:32)
at Function.Module._load (internal/modules/cjs/loader.js:790:14)
at Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js
:76:12)
at internal/main/run_main_module.js:17:47
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this comm
and.
$ yarn kbn bootstrap
yarn run v1.22.5
$ node scripts/kbn bootstrap
info [bazel_tools] installing Bazel tools
warn failed to reach ci-stats service [reason=no response], retrying in 1 seconds
warn failed to reach ci-stats service [reason=no response], retrying in 2 seconds
warn failed to reach ci-stats service [reason=no response], retrying in 3 seconds
warn failed to reach ci-stats service [reason=no response], retrying in 4 seconds
warn unable to report 1 timing, failed to reach ci-stats service too many times
ERROR [bootstrap] failed:
ERROR Error: Command failed with exit code 1: npm install --global @bazel/bazelisk@1.7.5
npm ERR! code ENOTFOUND
npm ERR! errno ENOTFOUND
npm ERR! network request to https://registry.yarnpkg.com/@bazel%2fbazelisk failed, reason: getaddrinfo ENOTFOUND registry.yarnpkg.com
npm ERR! network This is a problem related to network connectivity.
npm ERR! network In most cases you are behind a proxy or have bad network settings.
npm ERR! network
npm ERR! network If you are behind a proxy, please make sure that the
npm ERR! network 'proxy' config is set properly. See: 'npm help config'
npm ERR! A complete log of this run can be found in:
npm ERR! C:\Users\MMS5822\AppData\Roaming\npm-cache\_logs\2021-07-12T17_23_30_528Z-debug.log
at makeError (C:\TEMP\dev-kibana7133\kibana\packages\kbn-pm\dist\index.js:35184:11)
at handlePromise (C:\TEMP\dev-kibana7133\kibana\packages\kbn-pm\dist\index.js:34095:26)
at processTicksAndRejections (internal/process/task_queues.js:95:5)
at async installBazelTools (C:\TEMP\dev-kibana7133\kibana\packages\kbn-pm\dist\index.js:48420:5)
at async Object.run (C:\TEMP\dev-kibana7133\kibana\packages\kbn-pm\dist\index.js:8937:5)
at async runCommand (C:\TEMP\dev-kibana7133\kibana\packages\kbn-pm\dist\index.js:59803:5)
at async Module.run (C:\TEMP\dev-kibana7133\kibana\packages\kbn-pm\dist\index.js:242:3)
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
I tried copying the development environment again but it does not copy contents of node_modules. Is that supposed to happen? It does not even give me any errors. Please help!
I can not remove proxy on my other workstation where my plugin is. So I can not run yarn kbn bootstrap there.
as per this article, development is not possible when behind corporate proxy.
So, as I said I am not able to create kibana development envrionment on my workstation because of corporate proxy.
on other workstation, I downloaded Kibana 7.13.3 from source from github and ran yarn kbn bootstrap there and created a custom plugin. after that I was able to run elasticsearch and kibana.
So I can created the whole kibana folder into USB drive then I found out it copies all the folders inside @kbn directory but not the contents. I hope this makes sense.
in addition to that, I copied everything manually in USB drive and running it again on my workstation. I am getting following errors.
$ yarn es snapshot
yarn run v1.22.5
$ node scripts/es snapshot
info Installing from snapshot
│ info version: 7.13.3
│ info install path: C:\TEMP\dev-kibana7133\kibana\.es\7.13.3
│ info license: basic
│ info Downloading snapshot manifest from https://storage.googleapis.com/kibana-ci-es-snapshots-daily/7.13.3/manifest-latest-verified.json
│ warn ...failure, retrying in 5 seconds: request to https://storage.googleapis.com/kibana-ci-es-snapshots-daily/7.13.3/manifest-latest-verified.json failed, reason: get
addrinfo ENOTFOUND storage.googleapis.com
│ info ...retrying
│ warn ...failure, retrying in 5 seconds: request to https://storage.googleapis.com/kibana-ci-es-snapshots-daily/7.13.3/manifest-latest-verified.json failed, reason: get
addrinfo ENOTFOUND storage.googleapis.com
│ info ...retrying
│ warn ...failure, retrying in 5 seconds: request to https://storage.googleapis.com/kibana-ci-es-snapshots-daily/7.13.3/manifest-latest-verified.json failed, reason: get
addrinfo ENOTFOUND storage.googleapis.com
│ info ...retrying
│ warn ...failure, retrying in 5 seconds: request to https://storage.googleapis.com/kibana-ci-es-snapshots-daily/7.13.3/manifest-latest-verified.json failed, reason: get
addrinfo ENOTFOUND storage.googleapis.com
│ info ...retrying
│ERROR Unhandled error
│ERROR FetchError: request to https://storage.googleapis.com/kibana-ci-es-snapshots-daily/7.13.3/manifest-latest-verified.json failed, reason: getaddrinfo ENOTFOUND stor
age.googleapis.com
│ at ClientRequest.<anonymous> (C:\TEMP\dev-kibana7133\kibana\node_modules\node-fetch\lib\index.js:1461:11)
│ at ClientRequest.emit (events.js:375:28)
│ at TLSSocket.socketErrorListener (_http_client.js:475:9)
│ at TLSSocket.emit (events.js:375:28)
│ at emitErrorNT (internal/streams/destroy.js:106:8)
│ at emitErrorCloseNT (internal/streams/destroy.js:74:3)
│ at processTicksAndRejections (internal/process/task_queues.js:82:21)
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
Have you tried setting the proxy for yarn and npm?
yarn config set proxy http://username:password@host:port
yarn config set https-proxy http://username:password@host:port
In case you use a self-signed CA in your environment, you may have to add the CA in your trust store for npm or disable strict ssl. This would be the best option to bootstrap the env on your machine.
npm config set proxy http://username:password@host:port
npm config set https-proxy http://username:password@host:port
npm set strict-ssl false
I think the proxy settings are working now. but now I am getting these errors.
$ yarn kbn bootstrap
yarn run v1.22.5
$ node scripts/kbn bootstrap
info [bazel_tools] installing Bazel tools
succ [bazel_tools] all bazel tools are correctly installed
info [bazel] FATAL: AcquireLock(c:\temp\kibana-development\kibana\unc\sh12cffp0001\users-d\ms582\_bazel_ms582\bvg2zzma\lock): CreateFile failed: (error: 3): The system c
annot find the path specified.
info [bazel]
warn failed to reach ci-stats service [reason=no response], retrying in 1 seconds
warn failed to reach ci-stats service [reason=no response], retrying in 2 seconds
warn failed to reach ci-stats service [reason=no response], retrying in 3 seconds
warn failed to reach ci-stats service [reason=no response], retrying in 4 seconds
warn unable to report 1 timing, failed to reach ci-stats service too many times
ERROR [bootstrap] failed:
ERROR The bazel command that was running failed to complete.
error Command failed with exit code 1.
Did you do a clean install or on a copied directory with your plugin files? A general approach should be you copy your plugin code and setup a new dev env using the kibana plugin guide.
right now I am not copying the plugin code. As I was able to bypass proxy, I downloaded kibana 7.13.3 from the source and trying to setup a dev env again.
while doing that, I am getting the error above.
I am using gitbash. As per this article, it says that it is recommended using cmd or powershell for bazel but kibana development is recommended using gitbash.
but you are right because in powershell and cmd the errors are somewhat different.
PS C:\TEMP\kibana-development\kibana> yarn kbn bootstrap
yarn run v1.21.1
$ node scripts/kbn bootstrap
info [bazel_tools] installing Bazel tools
succ [bazel_tools] all bazel tools are correctly installed
info [bazel] INFO: Invocation ID: 40268a01-6f12-41ce-987e-8b0c1c077899
info [bazel] INFO: Repository build_bazel_rules_nodejs instantiated at:
info [bazel] C:/temp/kibana-development/kibana/WORKSPACE.bazel:11:13: in <toplevel>
info [bazel] Repository rule http_archive defined at:
info [bazel] C:/users/ms82/_bazel_ms82/bvg2zzma/external/bazel_tools/tools/build_defs/repo/http.bzl:336:31: in <toplevel>
info [bazel] WARNING: Download from https://github.com/bazelbuild/rules_nodejs/releases/download/3.2.3/rules_nodejs-3.2.3.tar.gz failed: class com.google.devtools.build.lib.bazel.repository.downloader.UnrecoverableHttpException Unknown host: github.com
info [bazel] ERROR: An error occurred during the fetch of repository 'build_bazel_rules_nodejs':
info [bazel] Traceback (most recent call last):
info [bazel] File "C:/users/ms82/_bazel_ms82/bvg2zzma/external/bazel_tools/tools/build_defs/repo/http.bzl", line 111, column 45, in _http_archive_impl
info [bazel] download_info = ctx.download_and_extract(
info [bazel] Error in download_and_extract: java.io.IOException: Error downloading [https://github.com/bazelbuild/rules_nodejs/releases/download/3.2.3/rules_nodejs-3.2.3.tar.gz] to C:/users/ms82/_bazel_ms82/bvg2zzma/external/build_bazel_rules_nodejs/temp5743307366720225877/rules_nodejs-3.2.3.tar.gz: Unknown host: github.com
info [bazel] ERROR: Error fetching repository: Traceback (most recent call last):
info [bazel] File "C:/users/ms82/_bazel_ms82/bvg2zzma/external/bazel_tools/tools/build_defs/repo/http.bzl", line 111, column 45, in _http_archive_impl
info [bazel] download_info = ctx.download_and_extract(
info [bazel] Error in download_and_extract: java.io.IOException: Error downloading [https://github.com/bazelbuild/rules_nodejs/releases/download/3.2.3/rules_nodejs-3.2.3.tar.gz] to C:/users/ms82/_bazel_ms52/bvg2zzma/external/build_bazel_rules_nodejs/temp5743307366720225877/rules_nodejs-3.2.3.tar.gz: Unknown host: github.com
info [bazel] ERROR: no such package '@build_bazel_rules_nodejs//': java.io.IOException: Error downloading [https://github.com/bazelbuild/rules_nodejs/releases/download/3.2.3/rules_nodejs-3.2.3.tar.gz] to C:/users/ms82/_bazel_ms82/bvg2zzma/external/build_bazel_rules_nodejs/temp5743307366720225877/rules_nodejs-3.2.3.tar.gz: Unknown host: github.com
info [bazel] INFO: Elapsed time: 1.475s
info [bazel] INFO: 0 processes.
info [bazel]
warn failed to reach ci-stats service [reason=no response], retrying in 1 seconds
warn failed to reach ci-stats service [reason=no response], retrying in 2 seconds
warn failed to reach ci-stats service [reason=no response], retrying in 3 seconds
warn failed to reach ci-stats service [reason=no response], retrying in 4 seconds
warn unable to report 1 timing, failed to reach ci-stats service too many times
ERROR [bootstrap] failed:
ERROR The bazel command that was running failed to complete.
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
Thanks for helping me man! it worked. I deleted proxy settings, added them again.
copied developed environment again properly. I am still getting the following error but it works.
warn failed to reach ci-stats service [reason=no response], retrying in 1 seconds
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.