Logstash Output dosent publishes logs while using systemctl

Elastic Stack Logstash
1

  1. If I start the logstash service using service or systemctl, it starts there are no errors started successfully but logs dosen't get published there in kibana.

  2. If i use " bin/logstash -f /etc/logstash/conf.d/logstash.conf " then logs are publishing to kibana.

  3. I tried with this solution provided here Logstash service doesn't output logs, command line does . Then also no luck.

Any help would be appreciated

2

The content of logstash.yml, pipelines.yml and logstash.service as well as the logs of Logstash might help to find the problem.

3

HI Jenni,
Kindly find out the details below

  1. logstash.yml
path.data: /var/lib/logstash
pipeline.ordered: auto
path.logs: /var/log/logstash
  1. pipelines.yml
path.config: "/etc/logstash/conf.d/*.conf"
  1. logstash.service
[Unit]
Description=logstash
[Service]
Type=simple
User=logstash
Group=logstash
EnvironmentFile=-/etc/default/logstash
EnvironmentFile=-/etc/sysconfig/logstash
ExecStart=/usr/share/logstash/bin/logstash "--path.settings" "/usr/share/logstash/config"
Restart=always
WorkingDirectory=/
Nice=19
LimitNOFILE=16384
TimeoutStopSec=infinity
[Install]
WantedBy=multi-user.target
  1. Logs of logstash:
● logstash.service - logstash
   Loaded: loaded (/etc/systemd/system/logstash.service; disabled; vendor preset: enabled)
   Active: active (running) since Thu 2020-07-09 11:51:32 UTC; 51min ago
 Main PID: 20795 (java)
    Tasks: 49 (limit: 4915)
   CGroup: /system.slice/logstash.service
           └─20795 /usr/bin/java -Xms1g -Xmx1g -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -Djava.awt.headless=true -Dfile.encoding=UTF-8 -Djruby.compile.invokedynamic=true -Djruby.jit

Jul 09 11:51:56 ip-10-0-0-123 logstash[20795]: [2020-07-09T11:51:56,406][WARN ][logstash.outputs.amazonelasticsearch][main] Detected a 6.x and above cluster: the `type` event field won't be used to determine the document _type {:es_versi
Jul 09 11:51:56 ip-10-0-0-123 logstash[20795]: [2020-07-09T11:51:56,431][INFO ][logstash.outputs.amazonelasticsearch][main] New Elasticsearch output {:class=>"LogStash::Outputs::AmazonElasticSearch", :hosts=>["https://vpc-logerror-vilj5v
Jul 09 11:51:56 ip-10-0-0-123 logstash[20795]: [2020-07-09T11:51:56,457][INFO ][logstash.outputs.amazonelasticsearch][main] Using mapping template from {:path=>nil}
Jul 09 11:51:56 ip-10-0-0-123 logstash[20795]: [2020-07-09T11:51:56,497][INFO ][logstash.outputs.amazonelasticsearch][main] Attempting to install template {:manage_template=>{"template"=>"logstash-*", "version"=>60002, "settings"=>{"inde
Jul 09 11:51:56 ip-10-0-0-123 logstash[20795]: [2020-07-09T11:51:56,548][INFO ][logstash.javapipeline    ][main] Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>8, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50, "pip
Jul 09 11:51:58 ip-10-0-0-123 logstash[20795]: [2020-07-09T11:51:58,041][INFO ][logstash.inputs.file     ][main] No sincedb_path set, generating one based on the "path" setting {:sincedb_path=>"/var/lib/logstash/plugins/inputs/file/.sinc
Jul 09 11:51:58 ip-10-0-0-123 logstash[20795]: [2020-07-09T11:51:58,086][INFO ][logstash.javapipeline    ][main] Pipeline started {"pipeline.id"=>"main"}
Jul 09 11:51:58 ip-10-0-0-123 logstash[20795]: [2020-07-09T11:51:58,244][INFO ][filewatch.observingtail  ][main][08f9a378b558a96db1555616c42d9ed88ef9b26655c2be176bc35c226b2e5572] START, creating Discoverer, Watch with file and sincedb co
Jul 09 11:51:58 ip-10-0-0-123 logstash[20795]: [2020-07-09T11:51:58,248][INFO ][logstash.agent           ] Pipelines running {:count=>1, :running_pipelines=>[:main], :non_running_pipelines=>[]}
Jul 09 11:51:58 ip-10-0-0-123 logstash[20795]: [2020-07-09T11:51:58,718][INFO ][logstash.agent           ] Successfully started Logstash API endpoint {:port=>9600}
4

Also the files inside /etc/logstash/conf.d/
It should be input, filter and output parts in the config..

5

actually i did that also giving the /etc/logstash/conf.d/ in alogstash.service path setting and also in pipeline.yml but nothing changed

6

Your path.settings should be /etc/logstash if your logstash.yml is in that folder. If it doesn't work, please post the full log.

7

Hi jenni ,
As you suggested i changed and now

  1. This is my logstash.service
[Unit]
Description=logstash

[Service]
Type=simple
User=logstash
Group=logstash
# Load env vars from /etc/default/ and /etc/sysconfig/ if they exist.
# Prefixing the path with '-' makes it try to load, but if the file doesn't
# exist, it continues onward.
EnvironmentFile=-/etc/default/logstash
EnvironmentFile=-/etc/sysconfig/logstash
ExecStart=/usr/share/logstash/bin/logstash "--path.settings" "/etc/logstash"
Restart=always
WorkingDirectory=/
Nice=19
LimitNOFILE=16384

# When stopping, how long to wait before giving up and sending SIGKILL?
# Keep in mind that SIGKILL on a process can cause data loss.
TimeoutStopSec=infinity

[Install]
WantedBy=multi-user.target

  1. And my logstash.yml is in the path /etc/logstash

  2. logstash execution logs is as follows

logstash.service - logstash
   Loaded: loaded (/etc/systemd/system/logstash.service; disabled; vendor preset: enabled)
   Active: active (running) since Thu 2020-07-09 11:51:32 UTC; 27s ago
 Main PID: 20795 (java)
    Tasks: 50 (limit: 4915)
   CGroup: /system.slice/logstash.service
           └─20795 /usr/bin/java -Xms1g -Xmx1g -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -Djava.awt.headless=true -Dfile.encoding=UTF-8 -Djruby.compile.invokedynamic=true -Djruby.jit

Jul 09 11:51:56 ip-10-0-0-123 logstash[20795]: [2020-07-09T11:51:56,406][WARN ][logstash.outputs.amazonelasticsearch][main] Detected a 6.x and above cluster: the `type` event field won't be used to determine the document _type {:es_versi
Jul 09 11:51:56 ip-10-0-0-123 logstash[20795]: [2020-07-09T11:51:56,431][INFO ][logstash.outputs.amazonelasticsearch][main] New Elasticsearch output {:class=>"LogStash::Outputs::AmazonElasticSearch", :hosts=>["https://vpc-logerror-vilj5v
Jul 09 11:51:56 ip-10-0-0-123 logstash[20795]: [2020-07-09T11:51:56,457][INFO ][logstash.outputs.amazonelasticsearch][main] Using mapping template from {:path=>nil}
Jul 09 11:51:56 ip-10-0-0-123 logstash[20795]: [2020-07-09T11:51:56,497][INFO ][logstash.outputs.amazonelasticsearch][main] Attempting to install template {:manage_template=>{"template"=>"logstash-*", "version"=>60002, "settings"=>{"inde
Jul 09 11:51:56 ip-10-0-0-123 logstash[20795]: [2020-07-09T11:51:56,548][INFO ][logstash.javapipeline    ][main] Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>8, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50, "pip
Jul 09 11:51:58 ip-10-0-0-123 logstash[20795]: [2020-07-09T11:51:58,041][INFO ][logstash.inputs.file     ][main] No sincedb_path set, generating one based on the "path" setting {:sincedb_path=>"/var/lib/logstash/plugins/inputs/file/.sinc
Jul 09 11:51:58 ip-10-0-0-123 logstash[20795]: [2020-07-09T11:51:58,086][INFO ][logstash.javapipeline    ][main] Pipeline started {"pipeline.id"=>"main"}
Jul 09 11:51:58 ip-10-0-0-123 logstash[20795]: [2020-07-09T11:51:58,244][INFO ][filewatch.observingtail  ][main][08f9a378b558a96db1555616c42d9ed88ef9b26655c2be176bc35c226b2e5572] START, creating Discoverer, Watch with file and sincedb co
Jul 09 11:51:58 ip-10-0-0-123 logstash[20795]: [2020-07-09T11:51:58,248][INFO ][logstash.agent           ] Pipelines running {:count=>1, :running_pipelines=>[:main], :non_running_pipelines=>[]}
Jul 09 11:51:58 ip-10-0-0-123 logstash[20795]: [2020-07-09T11:51:58,718][INFO ][logstash.agent           ] Successfully started Logstash API endpoint {:port=>9600}
lines 1-18/18 (END)

  1. Eventhough im not successful in publishing the logs to kibana.

  2. But if i use this command bin/logstash -f /etc/logstash/conf.d/logstash.conf yes im able to push the logs but as a service not able to.

8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.