I'am using Logstash 7.3 installed locally on server with:
sudo apt-get update && sudo apt-get install logstash
The goal is use filebeat to send logs to Elastic from Logstash but when I run Logstash, they starts on 127.0.0.1 and its not accessible by external machine..
When I run: $ /usr/share/logstash/bin/logstash -e "" --log.level=debugI see: [DEBUG] 2019-10-15 18:23:14.272 [LogStash::Runner] runner - http.host: "127.0.0.1" [DEBUG] 2019-10-15 18:23:14.273 [LogStash::Runner] runner - http.port: 9600..9700 [DEBUG] 2019-10-15 18:23:14.273 [LogStash::Runner] runner - http.environment: "production"
And when I run netstat: $ netstat -na |grep 9600I get: tcp 0 0 127.0.0.1:9600 0.0.0.0:* LISTEN
On my /etc/logstash/logstash.yml I have:
http.host: "10.198.9.233"http.port: 9600-9700
I already tried to put any address on http.host like the real machine IP, 0.0.0.0, with no space indentation, 1 space, 2 spaces.. with tab but its not working.. I'am thinking witch Logstash is not applying my configs on /etc/logstash/logstash.yml
Anybody can help me? Thx
Badger
October 15, 2019, 11:02pm
2
I think that is the monitoring interface, which means it is not directly relavent for ingestion.
Does your logstash configuration contain a beats input?
From the documentation here
Use the given string as the configuration data. Same syntax as the config file. If no input is specified, then the following is used as the default input: input { stdin { type => stdin } } and if no output is specified, then the following is used as the default output: output { stdout { codec => rubydebug } } . If you wish to use both defaults, please use the empty string for the -e flag. The default is nil.
The way I read this is that as there is no external input or output, that it would make sense for logstash to start only using a loopback interface (I can replicate this behavior in my test environment)
Have you tried loading it by specifying the configuration file?
@Badger and @TechGeekNZ
Yes, I have the pipeline configured on:/etc/logstash/conf.d/
And yes, I already tried to run logstash by specifying the configuration file, its works fine but Logstash continues starts locally on "127.0.0.1"
Everything works fine, all pipeline files are working very well and sending data to Elastc. The only problem is that logstash is starting locally on 127.0.0.1 instead of 0.0.0.0 or 10.198.9.233.
Thx!
I see, that puts things into better perspective
Is it possible to paste the full debugging logs. Have a look through the logs and see if you can find any errors around loading the logstash.yml file. Also, what are the permissions on the file?
Oh, sorry for the delay,
No, I haven't see any errors analysing the logs. And the file permissions is ok..
$ /usr/share/logstash/bin/logstash --log.level=debugDebug logs: Thread.exclusive is deprecated, use Thread::Mutex [DEBUG] 2019-10-16 16:17:21.751 [main] scaffold - Found module {:module_name=>"fb_apache", :directory=>"/usr/share/logstash/modules/fb_apache/configuration"} [DEBUG] 2019-10-16 16:17:21.757 [main] registry - Adding plugin to the registry {:name=>"fb_apache", :type=>:modules, :class=>#<LogStash::Modules::Scaffold:0x440dc4f7 @directory="/usr/share/logstash/modules/fb_apache/configuration", @module_name="fb_apache", @kibana_version_parts=["6", "0", "0"]>} [DEBUG] 2019-10-16 16:17:21.759 [main] scaffold - Found module {:module_name=>"netflow", :directory=>"/usr/share/logstash/modules/netflow/configuration"} [DEBUG] 2019-10-16 16:17:21.759 [main] registry - Adding plugin to the registry {:name=>"netflow", :type=>:modules, :class=>#<LogStash::Modules::Scaffold:0x5a342baf @directory="/usr/share/logstash/modules/netflow/configuration", @module_name="netflow", @kibana_version_parts=["6", "0", "0"]>} [DEBUG] 2019-10-16 16:17:22.205 [LogStash::Runner] runner - -------- Logstash Settings (* means modified) --------- [DEBUG] 2019-10-16 16:17:22.205 [LogStash::Runner] runner - node.name: "elasticsearch.intranet.explora" [DEBUG] 2019-10-16 16:17:22.205 [LogStash::Runner] runner - path.data: "/usr/share/logstash/data" [DEBUG] 2019-10-16 16:17:22.205 [LogStash::Runner] runner - modules.cli: [] [DEBUG] 2019-10-16 16:17:22.205 [LogStash::Runner] runner - modules: [] [DEBUG] 2019-10-16 16:17:22.205 [LogStash::Runner] runner - modules_list: [] [DEBUG] 2019-10-16 16:17:22.206 [LogStash::Runner] runner - modules_variable_list: [] [DEBUG] 2019-10-16 16:17:22.206 [LogStash::Runner] runner - modules_setup: false [DEBUG] 2019-10-16 16:17:22.206 [LogStash::Runner] runner - config.test_and_exit: false [DEBUG] 2019-10-16 16:17:22.206 [LogStash::Runner] runner - config.reload.automatic: false [DEBUG] 2019-10-16 16:17:22.206 [LogStash::Runner] runner - config.reload.interval: 3000000000 [DEBUG] 2019-10-16 16:17:22.206 [LogStash::Runner] runner - config.support_escapes: false [DEBUG] 2019-10-16 16:17:22.206 [LogStash::Runner] runner - config.field_reference.parser: "STRICT" [DEBUG] 2019-10-16 16:17:22.206 [LogStash::Runner] runner - metric.collect: true [DEBUG] 2019-10-16 16:17:22.206 [LogStash::Runner] runner - pipeline.id: "main" [DEBUG] 2019-10-16 16:17:22.206 [LogStash::Runner] runner - pipeline.system: false [DEBUG] 2019-10-16 16:17:22.206 [LogStash::Runner] runner - pipeline.workers: 4 [DEBUG] 2019-10-16 16:17:22.206 [LogStash::Runner] runner - pipeline.batch.size: 125 [DEBUG] 2019-10-16 16:17:22.206 [LogStash::Runner] runner - pipeline.batch.delay: 50 [DEBUG] 2019-10-16 16:17:22.207 [LogStash::Runner] runner - pipeline.unsafe_shutdown: false [DEBUG] 2019-10-16 16:17:22.207 [LogStash::Runner] runner - pipeline.java_execution: true [DEBUG] 2019-10-16 16:17:22.207 [LogStash::Runner] runner - pipeline.reloadable: true [DEBUG] 2019-10-16 16:17:22.207 [LogStash::Runner] runner - pipeline.plugin_classloaders: false [DEBUG] 2019-10-16 16:17:22.207 [LogStash::Runner] runner - path.plugins: [] [DEBUG] 2019-10-16 16:17:22.207 [LogStash::Runner] runner - config.debug: false [DEBUG] 2019-10-16 16:17:22.207 [LogStash::Runner] runner - *log.level: "debug" (default: "info") [DEBUG] 2019-10-16 16:17:22.207 [LogStash::Runner] runner - version: false [DEBUG] 2019-10-16 16:17:22.207 [LogStash::Runner] runner - help: false [DEBUG] 2019-10-16 16:17:22.207 [LogStash::Runner] runner - log.format: "plain" [DEBUG] 2019-10-16 16:17:22.207 [LogStash::Runner] runner - http.host: "127.0.0.1" [DEBUG] 2019-10-16 16:17:22.207 [LogStash::Runner] runner - http.port: 9600..9700 [DEBUG] 2019-10-16 16:17:22.207 [LogStash::Runner] runner - http.environment: "production" [DEBUG] 2019-10-16 16:17:22.207 [LogStash::Runner] runner - queue.type: "memory" [DEBUG] 2019-10-16 16:17:22.207 [LogStash::Runner] runner - queue.drain: false [DEBUG] 2019-10-16 16:17:22.207 [LogStash::Runner] runner - queue.page_capacity: 67108864 [DEBUG] 2019-10-16 16:17:22.207 [LogStash::Runner] runner - queue.max_bytes: 1073741824 [DEBUG] 2019-10-16 16:17:22.207 [LogStash::Runner] runner - queue.max_events: 0 [DEBUG] 2019-10-16 16:17:22.208 [LogStash::Runner] runner - queue.checkpoint.acks: 1024 [DEBUG] 2019-10-16 16:17:22.208 [LogStash::Runner] runner - queue.checkpoint.writes: 1024 [DEBUG] 2019-10-16 16:17:22.208 [LogStash::Runner] runner - queue.checkpoint.interval: 1000 [DEBUG] 2019-10-16 16:17:22.208 [LogStash::Runner] runner - queue.checkpoint.retry: false [DEBUG] 2019-10-16 16:17:22.208 [LogStash::Runner] runner - dead_letter_queue.enable: false [DEBUG] 2019-10-16 16:17:22.208 [LogStash::Runner] runner - dead_letter_queue.max_bytes: 1073741824 [DEBUG] 2019-10-16 16:17:22.208 [LogStash::Runner] runner - slowlog.threshold.warn: -1 [DEBUG] 2019-10-16 16:17:22.208 [LogStash::Runner] runner - slowlog.threshold.info: -1 [DEBUG] 2019-10-16 16:17:22.208 [LogStash::Runner] runner - slowlog.threshold.debug: -1 [DEBUG] 2019-10-16 16:17:22.208 [LogStash::Runner] runner - slowlog.threshold.trace: -1 [DEBUG] 2019-10-16 16:17:22.208 [LogStash::Runner] runner - keystore.classname: "org.logstash.secret.store.backend.JavaKeyStore" [DEBUG] 2019-10-16 16:17:22.208 [LogStash::Runner] runner - keystore.file: "/usr/share/logstash/config/logstash.keystore" [DEBUG] 2019-10-16 16:17:22.208 [LogStash::Runner] runner - path.queue: "/usr/share/logstash/data/queue" [DEBUG] 2019-10-16 16:17:22.208 [LogStash::Runner] runner - path.dead_letter_queue: "/usr/share/logstash/data/dead_letter_queue" [DEBUG] 2019-10-16 16:17:22.208 [LogStash::Runner] runner - path.settings: "/usr/share/logstash/config" [DEBUG] 2019-10-16 16:17:22.208 [LogStash::Runner] runner - path.logs: "/usr/share/logstash/logs" [DEBUG] 2019-10-16 16:17:22.208 [LogStash::Runner] runner - xpack.management.enabled: false [DEBUG] 2019-10-16 16:17:22.209 [LogStash::Runner] runner - xpack.management.logstash.poll_interval: 5000000000 [DEBUG] 2019-10-16 16:17:22.209 [LogStash::Runner] runner - xpack.management.pipeline.id: ["main"] [DEBUG] 2019-10-16 16:17:22.209 [LogStash::Runner] runner - xpack.management.elasticsearch.username: "logstash_system" [DEBUG] 2019-10-16 16:17:22.209 [LogStash::Runner] runner - xpack.management.elasticsearch.hosts: ["https://localhost:9200"] [DEBUG] 2019-10-16 16:17:22.209 [LogStash::Runner] runner - xpack.management.elasticsearch.ssl.verification_mode: "certificate" [DEBUG] 2019-10-16 16:17:22.209 [LogStash::Runner] runner - xpack.management.elasticsearch.sniffing: false [DEBUG] 2019-10-16 16:17:22.209 [LogStash::Runner] runner - xpack.monitoring.enabled: false [DEBUG] 2019-10-16 16:17:22.209 [LogStash::Runner] runner - xpack.monitoring.elasticsearch.hosts: ["http://localhost:9200"]
Badger
October 16, 2019, 9:33pm
7
I don't see any indication you have configured a beats input.
From the debug, the following line;
[DEBUG] 2019-10-16 16:17:22.208 [LogStash::Runner] runner - path.settings: "/usr/share/logstash/config"
It's looking in that directory for the logstash.yml file. You mentioned earlier that this file is located in /etc/logstash/logstash.yml
This would indicate that the configuration is not being picked up. You can specify the location of logstash.yml with the command line option --path.settings
Give that a go with the debug and see if it then picks up the settings file. When it's picking up the correct configuration file, expect to see something like this in the debug log;
[2019-10-17T10:23:34,395][DEBUG][logstash.runner ] *http.host: "172.16.40.72" (default: "127.0.0.1")
1 Like
system
November 13, 2019, 9:45pm
9
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
