Date Format error

Elastic Stack Elasticsearch
1

Hi there,

I'm sorry, there is plenty of similar topic but none match my problem.

I'm trying to set un an ELK system, i'm running with a ubuntu14.04 LT, a postgres database and logstash.
In my databse i got this :

this is my ES index :

curl -X PUT http://localhost:9200/ligneserac/_mapping/serac -d '
{
  "serac":{
    "date_detection": false,
    "properties": {
      "id":{
        "index" : "not_analyzed",
        "type" : "string"
      },
      "scan_date":{
        "index" : "not_analyzed",
        "type" : "date",
	"format" : "yyyy-MM-dd'T'hh:mm:ssZZ" 
      },
      "scan_cab":{
        "index" : "not_analyzed",
        "type" : "string"
      },
      "line_id":{
        "index" : "not_analyzed",
        "type" : "string"
      },
      "nom_product":{
        "index" : "not_analyzed",
        "type" : "string"
      },
      "num_lot":{
        "index" : "not_analyzed",
        "type" : "string"
      }
    }
  }
}'

The problem is :
when i don't add a date format - ma data seem to not be transfered to ES
when i use this format : "yyyy-MM-dd'T'HH:mm:ssZZ" which match my database date format, i have this error :

erreur index.JPG940×76 17.5 KB

When i use the same format but without the 'T' : "yyyy-MM-dd HH:mm:ssZZ" or "yyyy/MM/dd HH:mm:ssZZ" or whatever do not creat an index error - i got this error on each entry of my database :

erreur entre.JPG942×167 59.7 KB

the only solution i find to put my entire database to ES is to not use the time base event option in kibana :confused:

Thanks for reading,

Josselin.

2

Hi,

If you want same format as in db in your elasticsearch remove the format code and use the following mapping it should work. please try this.

  "scan_date": {
              "type": "date",
              **"format": "dateOptionalTime"**
           },

So it will adjust the same format using in database in es too. hope it helps

Thanks
phani

3

Hi Josselin,

Can you format your code please (don't use citation but </> icon which is made for that)?
Please avoid pasting images and prefer pasting lines of code of logs or whatever.

Also, judging on the image filenames, please note that you can also ask in french for questions in https://discuss.elastic.co/c/in-your-native-tongue/discussions-en-francais. Not saying that you should not ask here. Don't get me wrong! :stuck_out_tongue:

You mentioned Logstash but we don't see your logstash config here.
I'd first try to simply use the jdbc input and only use stdout output plugin to print the event which is received by Logstash.