Convert string field into Date field

[Ahmed_Rezgui]

Hello Everyone
I'm trying to index emails into elasticsearch with logstash, the problem is that two fields of the outputs are parsed as String fields but they are supposed to be "date" fields.

The format of the fields is as below :
"x-dbworld-deadline" => "31-Jul-2019"
"x-dbworld-start-date" => "18-Nov-2019"

How can I convert these two fields into date fields ?

You will find attached my config file

[tritri]

To add datetime values to your messages, use the Datetime plugin.
To your filter section, add:

date {
  match => ["x-dbworld-deadline", "dd-MMM-yyyy"]
  match => ["x-dbworld-start-date", "dd-MMM-yyyy"]
}

This will search for the fields you defined in your grok pattern, and create a date field.

As a sidenote, you may have to install the datetime plugin first:
https://www.elastic.co/guide/en/logstash/current/working-with-plugins.html

[Ahmed_Rezgui]

Thanks for your response.

When I tried to execute my config file, I got this error :

Here is my config file after adding the datetime values

Can someone help me please to solve this issue ?

Thanks

[ITIC (Jordi)]

Hi

I don't think you can have duplicate parameters in one single filter. I'd reccomend you split your date{} filter in two, one for each field.

Hope this helps.

[Ahmed_Rezgui]

I tried what you recommended, the config file is now as below :

Now I have another problem, this keeps running non-stop :

Any idea please ?

[ITIC (Jordi)]

Hi

Never used imap{} input, so you'll have to check its parameters, but you seem to be getting an "invalid credentials" error, and logstash just keeps trying.

Besides, you have a check_interval => 10 parameter, so I guess logstash will check your imap account every 10 seconds (when you get your credentials right).

Other than that, I don't know why it keeps running non-stop.

[Ahmed_Rezgui]

Even when I tried to take off the parameter check_interval, it show the same errors

Can someone help me with this please ?

[ITIC (Jordi)]

Hi

All I see in your snippet of the logs is that your IMAP is not responding due to invalid credentials, and logstash simply keeps trying.

[Ahmed_Rezgui]

I resolved the problem

I just had to allow less secure apps to access my Gmail account like this :

Thanks