New Log file not pull up on kibana

Elastic Stack Logstash
1

Hello dears,
I have 2 types of logs, pfSense which works perfectly , and I want to add other logs (saplog) in a file.
I already create a filter (sapfilter) for these new logs, I try it with logstash-f and that's work. When I refresh my index, 22 new fields of my sapfilter has been added.
The problem is that the log file don't pull up on kibana. My input ran with logstash-f ... There is my input

#saprouter log files
input {
file {
path => "/var/log/saprouterlog/saprouterlog.txt_a_20170509_090647-20170509_130347"
start_position => "beginning"
sincedb_path => "/dev/null"
type => "saplog"
}
}

2

Help my friends :slight_smile:

3

Does the logstash user have permission to read those log files? The file itself needs to be readable and all directories leading up to the directory containing the file must be executable.

4

Actually, it works but it so long (1-2hour for one file)
Can we do this more quickly ?

5

up up

6

up uuuuuuuuuuuuuuuuuuuuup

7

up uuuuuuuuuuuuuuuuuuuuup

If you're trying to annoy the people that can help you you're certainly on the right track.

Actually, it works but it so long (1-2hour for one file)
Can we do this more quickly ?

Perhaps. How big are the files? What does the Logstash configuration look like? What possible bottlenecks are there?

8

Oups sorry for annoying :confused:
One file is 490Ko
Logstash configuration is fat (3pages).

9

490Ko

490 kilobytes?

10

yes :slight_smile:

11

Such a small file should be processed within seconds. The Logstash logs might contain clues about what's taking so long.

1 Like
12

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.