"Successfully start Logstash API endpoint {:port =>9600}" Stuck

Elastic Stack Logstash
1

Hi everyone, currently stuck trying to setup Logstash to accept iis logs. I'm on a windows 10 workstation with the logs being copied from external servers. At the moment when logstash is ran, it starts, then stops on the successfully start message. It can't seem to find the log files at all. I've looked through a lot of other post and checked file extension, sincedb = null etc. But none of these previous fixes work. Anyone have an ideas what to change?

The command to start logstash used is
C:\elk\logstash-6.6.0\bin\logstash.bat --debug -f logstash.conf

My filter.conf looks like this

    input {
	file {
	type=>"iis"
	path => "C:/elk/LOGS/u_ex160905.log"
	start_position => "beginning"
	sincedb_path => "NUL"
	ignore_older => 0
	}
}
	
filter{
  grok {
    match => ["message", "%{TIMESTAMP_ISO8601} %{IPORHOST:s-ip} %{WORD:cs-method} %{NOTSPACE:cs-uri-query} %{NOTSPACE:cs-uri-stem} %{NOTSPACE:cs-port} %{NOTSPACE:cs-username} %{IPORHOST:c-ip} %{NOTSPACE:cs(User-Agent)} %{NOTSPACE:cs(Referer)} %{NUMBER:sc-status} %{NUMBER:sc-substatus} %{NUMBER:c-win32-status} %{NUMBER:time-taken}"]
  }
}

output {
	elasticsearch {
		hosts => ["localhost:9200"]
		index => "Test"
}
	stdout {
		codec => rubydebug
		}
		
  }

This is the pipeline.

- pipeline.id: iis
  queue.type: persisted
  path.config: ["C:/elk/logstash-6.6.0/config/*.conf"]

Heres an example of the logs I'm trying to process

2016-09-05 01:12:46 192.168.10.10 GET /viewer-webtier/pcc.ashx/Document/q/0-0/Text DocumentID=u9_2hDfuotQuIiLbutGJ9Bb6PA6glSm_a1S6wdFniuCdFZIWnqNt6efbE-f9gb_wRBZzo0ZiTuhxXoW5PUOZcCw 443 smcintosh 117.120.18.136 Mozilla/5.0+(Windows+NT+10.0;+WOW64;+Trident/7.0;+rv:11.0)+like+Gecko https://testwebsite.com 200 0 0 62

After running debug this is what it spits out

[2019-02-28T10:09:11,717][INFO ][logstash.pipeline        ] Pipeline started successfully {:pipeline_id=>"main", :thread=>"#<Thread:0x4e824a1d sleep>"}
[2019-02-28T10:09:11,759][INFO ][logstash.agent           ] Pipelines running {:count=>1, :running_pipelines=>[:main], :non_running_pipelines=>[]}
[2019-02-28T10:09:11,764][INFO ][filewatch.observingtail  ] START, creating Discoverer, Watch with file and sincedb collections
[2019-02-28T10:09:11,785][DEBUG][logstash.agent           ] Starting puma
[2019-02-28T10:09:11,799][DEBUG][logstash.agent           ] Trying to start WebServer {:port=>9600}
[2019-02-28T10:09:11,850][DEBUG][logstash.api.service     ] [api-service] start
[2019-02-28T10:09:12,033][INFO ][logstash.agent           ] Successfully started Logstash API endpoint {:port=>9600}
[2019-02-28T10:09:15,574][DEBUG][logstash.instrument.periodicpoller.cgroup] One or more required cgroup files or directories not found: /proc/self/cgroup, /sys/fs/cgroup/cpuacct, /sys/fs/cgroup/cpu
[2019-02-28T10:09:15,583][DEBUG][logstash.instrument.periodicpoller.jvm] collector name {:name=>"ParNew"}
[2019-02-28T10:09:15,584][DEBUG][logstash.instrument.periodicpoller.jvm] collector name {:name=>"ConcurrentMarkSweep"}
[2019-02-28T10:09:16,721][DEBUG][logstash.pipeline        ] Pushing flush onto pipeline {:pipeline_id=>"main", :thread=>"#<Thread:0x4e824a1d sleep>"}
[2019-02-28T10:09:20,576][DEBUG][logstash.instrument.periodicpoller.cgroup] One or more required cgroup files or directories not found: /proc/self/cgroup, /sys/fs/cgroup/cpuacct, /sys/fs/cgroup/cpu
2

The only pipeline it is running is "main", and your pipeline is called "iis". But you are running logstash with -f, so it will ignore pipelines.yml anyway. Do you want to use -f or pipelines.yml. You have to pick one or the other, you cannot use both in the same logstash instance.

If you have DEBUG enabled with a file input you would be seeing messages such as

[INFO ][filewatch.observingtail  ] START, creating Discoverer, Watch with file and sincedb collections

so I suspect you are not running the configuration you are trying to run.

3

Hi, thanks for the reply.

Which one would be better suited or easier to setup? Running logstash with -f or using the pipelines.yml
And how would you go about both methods.

When I run with debug enabled there is that specific file, just after its compiled the Grok

[2019-02-28T11:56:35,705][INFO ][logstash.pipeline        ] Pipeline started successfully {:pipeline_id=>"main", :thread=>"#<Thread:0x543acce1 run>"}
[2019-02-28T11:56:35,753][INFO ][logstash.agent           ] Pipelines running {:count=>1, :running_pipelines=>[:main], :non_running_pipelines=>[]}
[2019-02-28T11:56:35,761][INFO ][filewatch.observingtail  ] START, creating Discoverer, Watch with file and sincedb collections_
4

Using the provided config file on Windows I get exactly the same result, BUT if I then add additional lines to the log files, they get detected and parsed correctly by logstash.

For some reason, it appears though start_position and ignore_older aren't working. I am definitely using the right .conf file. I've even tried "mode => read" but the behaviour doesn't change: logstash still only tails the files.

Can anyone else verify this on Windows 10?

5

I HAVE FOUND THE SOLUTION!
Hopefully anyone searching for why file input isn't working will find this helpful.

The way in which ignore_older is used has CHANGED between 6.3.2 and 6.4 of logstash.
If you have "ignore_older => 0", this now means "ignore anything more than 0 seconds old".

Remove this line and instead use "close_older => "1 hour" and "stat_interval => "1 second".
Full credit goes tothis stack overflow post that helped me fix it.

How something that changes the way this option works SO FUNDAMENTALLY has not been widely publicized is beyond me. I've lost many hours trying to work this out (plus all of the hundreds of previous examples of config files online are now completely wrong).

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.