Ignore malformed for date

Elastic Stack Elasticsearch
1

In Elastic

for field of type date , even if the ignore malformed attribute is set to true , it indexes the malformed data into this field.

Ideally this field value should be empty in case the incoming data is malformed.

2

Can you send a script which reproduce your issue? Including mapping creation...

3

Index Creation as follows :

curl -XPUT "http://localhost:9200/testindex?pretty" -d'
{
"mappings": {
"testmapping" : {
"properties": {
"date": {"format": "strict_date_optional_time", "ignore_malformed": true, "type": "date" }
}
}
}
}
'

insert data as follows :

curl -XPOST localhost:9200/testindex/testmapping -d '{
"date" : "8.1/10"
}'

As a result of above data gets indexed , ideally the field value should be empty as it's malformed.

4

But I'm sure you can't search for that field.

The field actually has not been indexed. You can still see it in the _source field because we don't modify what you are sending but it does not mean that the field has been indexed.

5

I can search the document using

http://localhost:9200/testindex/_search?pretty=true&q=:

  1. When you say the field has not been indexed , can you please elaborate?
  2. Does ignore_malformed to true, does that not mean data will not be stored into that field if it's malformed ?
6

The document is indexed. All fields but this one are indexed. If you want to reject the whole document don't set the ignore malformed setting

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.