Date Filtering from a Text File

Elastic Stack Logstash
1

Hi,

My text file reads something like -

03 Aug 2016 05:55:38,759|INFO |snjcprddex36|DataElt|||||03 Aug 2016 05:55:38,759|INFO |snjcprddex36|DataElt|||||OUT_OF_SCOPE

Is there any way I can manually filter the date and time and create a separate field?

Thanks

2

HI,
yes, in logstash you can use this collection of filters to get what you aim for:

  1. Use the split filter to break you line in fields, your delimiter here is clearly the | character, isn't?
  2. After that you have an event with multiple fields, one per each bucket result of the splitting.
  3. Now you can use the date filter with the datetime field and parse them.

For exact documentation I've link each filter documentation for you.

Hope it helps,

  • purbon
3

It's not entirely clear to me if the example above is a single line that should be split into multiple events, a single line with a single event, or two lines that should result in two events.

Use the split filter to break you line in fields, your delimiter here is clearly the | character, isn't?

The split filter splits a field into multiple events. You probably meant the mutate filter's split option, but I think the csv filter would be a better match.

4

@magnusbaeck you're right, the event line is a bit confusing, I agree using CSV filter here will work, apologies if the message has understood as misleading.