Help needed for reading Logs

Elastic Stack Logstash
1

Hi there,
Could someone guide me on how to use pipeline(|) as separator for the Logs.
I tried gsub. But everytime I use gsub, it says non ascii character found in the log.
Any help would be appreciated..

Regards,
Hari

2

Please show an example of the kind of log you want to process. Please show us what you've tried so far. It's not clear why you'd want to use gsub. It sounds like a simple csv filter would work.

3

Hi Magnus,

This is my sample log :

21674|glsServer.ec|ServerInit|351|2016-10-23 11:08:02|LOG|lCnt=0, service_name=ApplPref
21674|glsServer.ec|ServerInit|351|2016-10-23 11:08:02|LOG|lCnt=1, service_name=ApplList

And am trying this is filter:
filter {
mutate {
gsub => [
#Replace pipeline
"message","|"," "
]
}
grok {
match => { "message" => "%{NUMBER:col1} %{WORD:col2} %{WORD:col3} %{NUMBER:col4} %{WORD:col5} %{WORD:col6} %{WORD:col7}" }
}

Is there any direct way to make it accept the pipeline?

4

you dont have to change do pipe(|).

you can write the pipe in the grok pattern like this: |

so you would have something like %{NUMBER:col1}|%{WORD:col2}

5

Indeed, escape the | or just use the csv filter instead of the grok filter.

6

Hi,

Thanks much. It works. Now that I have followed the instructions and avoided the errors, I am still not able to see the logs in Kibana UI. I do not get any errors when pushing the Logs in. Any idea what could have went wrong? I tried making changes in the Log file once logstashed in. No changes in the window however.

Thanks in advance,
Hari

7

What does your configuration look like? Exactly how did you change the file?

8

Hi there,

Sorry for my delay response. Was little busy in catching up other things. The things worked perfectly. Thanks for all your inputs. Although I am with a new query here.

I want to use URL search using current timestamp. To be precise, I want to take logs that have fallen in the last 15 mins of time.

Am looking for something like:

http://localhost:9200/_search?q="sql" AND @timestamp:"2016-11-16T17:33:19"

Can someone help what should i use in the @timestamp parameter to get the specified result

9

Please post Elasticsearch questions in the Elasticsearch category.

10

Sure thanks

11

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.