How to get the date object substring in logstash filter

Elastic Stack Logstash
1

In Logstash filter i want to use yyyy-mm .
I should get yyyy-mm from date object using substring function.
How to get the substring of date object in logstash filter
Thanks in advance.

2

I invite you to try this logstash configuration :

filter {
  grok {
    match => { "@timestamp" => "(?<month>[0-9]{4}\-[0-9]{2})"}
  }
}

This will store "yyyy-mm" into "month" field

1 Like
3

Thank you very much.
It is working.

4

Please suggest how to send NULL date value to elasticsearch using logstash.
Now null date value going as "null" string.
But it should be NULL date.
Thank in advance.

5

I think the only way to that is :

ruby {
  code => "event['@timestamp'] = nil"
}
6

Thank you very much.Its working.

7

in csv filter quote_char default value is """ .
In my file fields are not enclosed with "(doublequote).
But one of the field having "(doublequote).

Logstash failed with csv parse failure exception.
logstash confisidering "(doublequote) as default value and failing with csv parse failure exception.

Please suggest how to avoid this.
Thank in advance.

8

To avoid csv filter interprets " character, I invite you to set a very special char as "quote_char" option.
For example :

csv {
  quote_char => "`"
}
9

Above quote used in the mysql queries to escape mysql predefined variables. same query written to log. if i use that quote also csv failure exception displayed.
Please suggest another.

10

Ok.
Maybe you could use another special char like $ or ; or ¥ :slight_smile:

If you want another solution, tell me.

11

Please tell me another solution.

12

OK, so the other solution aims to transform your log line to a standard csv line.
These are the logstash filters to set before csv filter :

mutate {
  gsub => [ 'message', '"',  '""' ]
}
mutate {
  gsub => [ 'message', ',',  '","' ]
}
mutate {
  replace => { 'message' => '"%{message}"' }
}
14

It is a really different subject, so I invite you to open a new topic for that.

16

One more time, that's a different subject, so I invite you to open a new topic.