Require Help in understanding logstash

Hi
I am new to logstash . I gone through some documentation about logstash and feel good to learn more about it.

The requirement which i am trying is .
To read the log file and mathc the keyword Access in the log file and display the lines

I have tried one sample conf file to read the log file and write the data which i search as output .

My conf file follows as like this

input
{
file {
path => ["/lee/logs/server.log"]
}
}
filter {
grok {
match => [ "message", "%{WORD:custom_name}" ]
}
}
output {
stdout {
codec => rubydebug
}

}

I verified by config it says OK

[lee@server1 logstash-1.5.0]$ bin/logstash -f test2.conf --configtest
Configuration OK
[lee@server1 logstash-1.5.0]$ bin/logstash -f test2.conf
Logstash startup completed
Access

I gave the input Access aftert this am not seeing any other thing.

Please help me if i am missing something to check this .

I Dont understand what is happening where i need to check as well after this

Thanks in Advance

Regards
Lee

Has /lee/logs/server.log changed since you ran Logstash the last time?

Hi Magnus

Thank you for your reply .

I checked the file and could see there is no change to my server.log

Then there's nothing for Logstash to do. Logstash reads from the end of the file. If you want to reprocess old files you have to set start_position => beginning and either set sincedb_path => "/dev/null" or delete the sincedb file.

When playing around with Logstash I recommend that you use the stdin input instead and feed Logstash data via stdin.

Hi Again

Correct me if i am wrong.

Stdin input is the input which will be passed in runtime . If i want to use the file how can i use it .
Could you refer me some paths about how to use stdin input to get the data from file

thanks in advance

With the stdin input plugin you can pass data to Logstash in any of the following two ways:

/path/to/logstash ... < /path/to/logfile
cat /path/to/logfile | /path/to/logstash ...

This is obviously not specific to Logstash but how Unix-like shells work in general.

But again, this is rarely used for real work but it's quite useful if you want to play around without dealing with sincedb files etc.

Hi Magnus

now my logstash config is reading the data of my log file and showing in the output . But with that i am getting the following information in my test as well
A plugin had an unrecoverable error. Will restart this plugin.
Plugin: <LogStash::Inputs::File path=>["/lee/logs/server.log"], start_position=>"beginning", sincedb_path=>"/logs/", delimiter=>"\n">
Error: No such file or directory - /logs/.9788.32761.156686 {:level=>:error}

Could you please help me on this. where i am wrong what needs to look to fix this error as well
input
{
file {
path => ["/lee/logs/server.log"]
start_position => "beginning"
sincedb_path => "/logs/"
}
}
filter {
grok {
match => [ "message", "%{WORD:custom_name}" ]
}
}
output {
stdout {
codec => line
}

}

Thanks in Advance

I found it My sincedb path is incorrect . I rectified it now and it is working.

Thank you for helpfule hints

Hi..Less_ps
I am getting the same error while using sincedb..could you please tell how i can correct or set my sincedb path.

Hi There,

Hope it will solve your problem
add sincedb_path to the path where you have read and write access for the file.
Example => "/home/user/Downloads/dbflile"
it will create one file with name dbfile and this file keeps track of the current position of monitored log files.

Exmple
input {
file {
path => "/home/user/Downloads/sample.csv"
type => "Sample"
start_position => "beginning"
sincedb_path => "/home/user/Downloads/dbflile"
}
}

//Lee

Hi Lee_PS,

Thanks for the replay.
Actually i am using Windows OS so it will be different or it will be same.

Thanks,
Renu

Hi,

Could you please check..this code
if ([message] =~ /(XAPMErrorEnrichmentXB :: Order line value is)/) {
grok {
match => {"message" => "(?(XAPMErrorEnrichmentXB :: Order line
value is ).{9})"}
add_field =>{ "orderId" => "%{result}" }
}
}

is it correct or not?

Thanks,
Renu