What is the meaning of this log message

Elastic Stack Logstash
1

In my /var/log/logstash/logstash.log.1 has only the following and nothing is getting published from logstash.
Please help me to fix this,

{:timestamp=>"2017-11-29T06:07:47.853000+0000", :message=>"SIGTERM received. Shutting down the pipeline.", :level=>:warn}
{:timestamp=>"2017-11-29T06:07:47.853000+0000", :message=>"SIGTERM received. Shutting down the pipeline.", :level=>:warn}

2

HI @Kasun_Siyambalapitiy

can you provide more information about this?

which version of logstash are you using?
did you change something? (upgrade to 6.0? )
that's all inside your logs?
what's your setup? (beats ships logs to logstash? or is logstash using the file input?)

i mean, how should someone help you with this? more information, please

3

Hi lueneburger,

Pardon me for not filling up details, I am using logstash 2.1.3on a server running Ubuntu 14.04 and use a log file (which logs always gets appended) as the input and outputs using http [htpps]. This is the config file I am using

input {
  file {
    path => "/opt/nginx/logs/products.log"
    type => "product_log"
  }
}

filter {
  if [type] == "product_log" {
	  mutate { replace => { "type" => "product_log" } }
  grok {
    match => { "message" => "%{IPORHOST:http_host} %{IPORHOST:clientip} \[%{HTTPDATE:timestamp}\] \"(?:%{WORD:verb} %{NOTSPACE:request}(?: HTTP/%{NUMBER:httpversion})?|%{DATA:rawrequest})\" %{NUMBER:response} (?:%{NUMBER:bytes}|-) %{QS:referrer} %{QS:agent} %{NUMBER:request_time:float} %{EMAILADDRESS:email_addr} %{QS:wsid} %{QS:product_name} %{QS:product_version} %{QS:product_file}" }
  }
  date {
    match => [ "timestamp" , "dd/MMM/YYYY:HH:mm:ss Z" ]
  }
  geoip {
    source => "clientip"
  }
  }
}

output {
  if [type] == "product_log" {
	  http  {
    url => "https://blabla.com/endpoints/nginx_product_receiver"
    http_method => "post"
    format => "json"
    headers => [
      'Authorization', 'Basic blablablablablablablabla'
    ]
  }
  }
}

Initially Logstash was correctly publishing contents of the above file to some endpoint and I changed the endpoint to a new one, by changing the URL to new one and updating Authorization details accordingly.
Then I ran sudo service logstash configtest and it gave me Configuration OK. As all are OK I ran sudo service logstash restart to apply the changes done on above configs and this was the output

Killing logstash (pid 24859) with SIGTERM
Waiting logstash (pid 24859) to die...
Waiting logstash (pid 24859) to die...
logstash stopped.
logstash started.

Once it is restarted no data is been received from new endpoint. This is the last log being published by logstash under /var/log/logstash/logstash.log.1

{:timestamp=>"2017-11-29T06:07:47.853000+0000", :message=>"SIGTERM received. Shutting down the pipeline.", :level=>:warn}
{:timestamp=>"2017-11-29T06:07:47.853000+0000", :message=>"SIGTERM received. Shutting down the pipeline.", :level=>:warn}

Have I missed anything on the above steps? are there any special commands to be run when changing and existing conf? and is there any way to check from the server side (which runs logstash) to see whether it published data? please help me to get this fixed, thanks in advance

4

Hi @Kasun_Siyambalapitiy ,

thanks for the additional information, think also other people could help :slight_smile:

did you tested what happens if you replace the current output with stdout? link
test if it starts then, without your http output.

if this not helps, keep the stdout output and remove the filters.... etc
think that would be a good way to find the current issue.

Also what happens if you change back to the old output endpoint?

Cheers

5

Hi @lueneburger,
Thank you for the reply, I changed back to the old endpoint and to the wonder it is also not receiving any data. There is an another host(machine) with the same setup in operation and publishing to the old endpoint, so I checked for inconsistencies of the two config files ( that is after reverting the current to the previous endpoint) and both of them are 100% identical. What may be the cause for the the other not to publish to the endpoint which it published earlier. Your help is highly appreciated.

Thanks.

6

hello, @Kasun_Siyambalapitiy

are your nginx server generate new rows in log file?

7

Hi @nugusbayevkk ,

Yeah, it generates logs at a minimum rate of 1 log for second. But they are not getting published to the endpoint specified in the config file. One more thing, I also added the following line segment to the config in order to receive output to console,

output {
  stdout { codec => json }
}

so the new config file looks like below,

input {
  file {
    path => "/opt/nginx/logs/products.log"
    type => "product_log"
  }
}

filter {
  if [type] == "product_log" {
	  mutate { replace => { "type" => "product_log" } }
  grok {
    match => { "message" => "%{IPORHOST:http_host} %{IPORHOST:clientip} \[%{HTTPDATE:timestamp}\] \"(?:%{WORD:verb} %{NOTSPACE:request}(?: HTTP/%{NUMBER:httpversion})?|%{DATA:rawrequest})\" %{NUMBER:response} (?:%{NUMBER:bytes}|-) %{QS:referrer} %{QS:agent} %{NUMBER:request_time:float} %{EMAILADDRESS:email_addr} %{QS:wsid} %{QS:product_name} %{QS:product_version} %{QS:product_file}" }
  }
  date {
    match => [ "timestamp" , "dd/MMM/YYYY:HH:mm:ss Z" ]
  }
  geoip {
    source => "clientip"
  }
  }
}

output {
  if [type] == "product_log" {
	  http  {
    url => "https://blabla.com/endpoints/nginx_product_receiver"
    http_method => "post"
    format => "json"
    headers => [
      'Authorization', 'Basic blablablablablablablabla'
    ]
  }
  }
}
output {
  stdout { codec => json }
}

and restarted logstash after checking its configuration are ok. logstash restarted successfully but console logs are also not printed, how do I get the output in console?

8

@Kasun_Siyambalapitiy
are your user have access to read from products.log*?
are you try to delete temporary filter part of configuration?
try this.. and check what happened

do you have opportunity to output your data to elasticsearch?
try this..

or if it possible, share some rows of nginx logs

9

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.