Hi.
I got a project where we insert bunch of data through spark streaming. Everything was fine until a colleague got a strange behaviour I never had.
A date is mapped as "yyyy-MM-dd' 'HH:mm:ss", and only on his computer, some (about 1%) of the date, and randoms ones when we delete and replay the same datas, are inserted with the date of the insert instead of the real date.
I tried a lots of thing, like adding a duplicate of the fields, and it was fine.
So, maybe it's because we made an alias, and the alias contains other index (we use index_yy_ww) which have older mapping (I first tried mapping date with "yyyy-MM-dd HH:mm:ss"). But that's weird, and I would like to know why it's happening.
Or maybe I get the wrong format? the docs is not really well documented. If I read it correctly, I should use "yyyy'-'MM'-'dd' 'HH':'mm':'ss" but didn't find any example of this online, thus using the one I found most on stackoverflow ("yyyy-MM-dd' 'HH:mm:ss").
Thanks for the help.