Facing issue when there is non uniform data


(Amit) #1

Hi Guys,

I am using ElasticSearch with Python. In my txt file DOB is non uniform like 1991-12-12 , 1991/12/12, 1991-Dec-12 something like that.Using same formate date like YYYY-MM-DD, I can import txt file but not able to import file due to that non uniform data. Manually it't time consuming and can't edit data which is in GB. Kindly suggest me solution...


(David Pilato) #2

If you know all date formats in advance you can define them in the mapping.


(Amit) #3

Hi,

You mean in python script?

If you have any sample or example please share it.


(David Pilato) #4

No I meant "elasticsearch mapping".

https://www.elastic.co/guide/en/elasticsearch/reference/2.0/date.html


(Amit) #5

Hi David,

Thanks for solution. Still I am facing when date is like 1987/09/09.

One more concern I have, I want to generate auto unique id for each row. How can I do that?


(David Pilato) #6

Don't provide any id and it will be automatically generated.


(Amit) #7

It would auto gerenate id which would be random but if I want to ID in uniform like 1,2 3...

CSV file is big so cant manually enter ID.

Still facing problem with date data type. I have to manually edit date in YYYY-MM-DD form.


(David Pilato) #8

If you are injecting with Python you could probably add a counter in your script.


(system) #9