Regarding conf file configuration

Elastic Stack Logstash
1

i want to read live logs from server what are the configuration i need to set in logstash conf file

2

Use the file input plugin. See https://www.elastic.co/guide/en/logstash/current/config-examples.html#_processing_apache_logs for an example.

3

i have installed apache server on my local machine and i want read its log what configuration file i should use. I went through the above link but i am not able to configure the conf file in logstash. I am getting following error:-

[2017-11-02T17:33:53,061][WARN ][logstash.outputs.elasticsearch] Attempted to re
surrect connection to dead ES instance, but got an error. {:url=>"http://localho
st:9200/", :error_type=>LogStash::Outputs::ElasticSearch::HttpClient::Pool::BadR
esponseCodeError, :error=>"Got response code '401' contacting Elasticsearch at U
RL 'http://localhost:9200/'"}

the following conf file i have run.
input {
file {
path => "C:\Program Files\Apache Software Foundation\Tomcat 8.5\logs"
}
}

filter {
if [path] =~ "access" {
mutate { replace => { type => "apache_access" } }
grok {
match => { "message" => "%{COMBINEDAPACHELOG}" }
}
date {
match => [ "timestamp" , "dd/MMM/yyyy:HH:mm:ss Z" ]
}
} else if [path] =~ "error" {
mutate { replace => { type => "apache_error" } }
} else {
mutate { replace => { type => "random_logs" } }
}
}

output {
elasticsearch { hosts => ["localhost:9200"] }
stdout { codec => rubydebug }
}

4

Your ES instance required authentication and you haven't configured that in your elasticsearch output.

5

so what i have to right for that in output.

6

i have started xamp apche server on 81 port how i can authenticate with that server and what conf file i should write in logstash.

7

I'm confused. What does Apache and port 81 have to do with Elasticsearch?

8

i provided the authentication required for the ES instance, there is no error and logstash pipeline is started successfully but there is no further processing. I think that it is unable to read the path provided for the logs. can you help ?

9

path => "C:\Program Files\Apache Software Foundation\Tomcat 8.5\logs"

The path setting should contain a filename or a filename pattern, not a directory. Also, you may have to use forward slashes instead of backslashes.

10

thank you.
With your help i am able to run the config file and the pipeline is successfully started. but i am unable to process further and i am stuck in between by following the link (https://www.elastic.co/guide/en/logstash/current/config-examples.html#_processing_apache_logs) i can't understand what the below command means.

127.0.0.1 - - [11/Dec/2013:00:01:45 -0800] "GET /xampp/status.php HTTP/1.1" 200 3891 "http://cadenza/xampp/navi.php" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:25.0) Gecko/20100101 Firefox/25.0"

can you help me with it.
thanks.

11

I don't understand the question. What command are you talking about? The line you pasted above (beginning with "127.0.0.1") is an HTTP log entry.