Hi, I am trying to inject csv file into elasticsearch. I have a row named "starttime". In this columns some rows are missing. I have converted this to datetime format using pandas to_datetime. Now when I inject this data into elastic I got the error that empty date cannot parse empty date'.
I found that at that particular row my value is empty string i.e. ''
How to manage this missing data.
I have tried putting NaT, NaN but that doesn't work.
For now I have put interger 0 there and it is working but I am trying to find the best way to mange it.
Thanks
I would try to not set this field at all, so it does not get any value assigned and is considered missing.
@spinscale Hi, thanks for your response.
Actually while saving the raw data into csv, I am using df.to_csv() and parameter "na_rep" its default value i.e. ''.
So how to make missing data assigned to no value in pandas and therefore considered missing.
I set it to None and tried but that failed and error log is:
{'type': 'mapper_parsing_exception', 'reason': 'failed to parse', 'caused_by': {'type': 'json_parse_exception', 'reason': "Non-standard token 'NaN': enable JsonParser.Feature.ALLOW_NON_NUMERIC_NUMBERS
Hi, I used ignore_malformed in mapping and it's kind solve my problem. But still I am looking for other ways to do it.
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.