Force all the JSON values as String type - Error - Merging dynamic updates triggered a conflict

Paco_B · March 3, 2016, 12:23pm

Hello,

We are trying to store a lot of JSON but ElasticSearch throws the following exception:

Merging dynamic updates triggered a conflict: mapper [our_type] of different type, current_type [string], merged_type [long].

Could we force ElasticSearch to store with the same type String for all the values from the JSON?

Many thanks.

Regards,
Paco.

jimczi · March 3, 2016, 2:21pm

You can use dynamic templates: https://www.elastic.co/guide/en/elasticsearch/reference/current/dynamic-templates.html

I think that something like the following should work:

{
  "mappings": {
    "_default_": {
      "dynamic_templates": [
        {
          "rule1": {
            "mapping": {
              "type": "string"
            },
            "match": "*",
            "match_mapping_type": "*"
          }
        }
      ]
    }
  }
}
```

Paco_B · March 3, 2016, 2:25pm

Many thanks, I will try it

Could it affect to the performance?

Thanks again

Regards,
Paco.

jimczi · March 3, 2016, 2:27pm

Could it affect to the performance?

Nothing you should worried about

Paco_B · March 3, 2016, 2:28pm

Many thanks!

Paco_B · March 3, 2016, 2:51pm

Hello again,

After creating a Java method

	    CreateIndexRequestBuilder createIndexRequestBuilder = client.admin().indices().prepareCreate(INDEX_NAME);
		// CREATE SETTINGS
		String settings_json = new String(Files.readAllBytes((Paths.get("C:\\mapping.json")))) ;
		createIndexRequestBuilder.setSettings(settings_json);
		// CREATE MAPPING
		String mapping_json = new String(Files.readAllBytes((Paths.get("C:\\mapping.json")))) ;
		createIndexRequestBuilder.addMapping(INDEX_NAME, mapping_json);
		CreateIndexResponse indexResponse = createIndexRequestBuilder.execute().actionGet();

I got the following exception:

Caused by: MapperParsingException[Root mapping definition has unsupported parameters:  [mappings : {_default_={dynamic_templates=[{rule1={mapping={type=string}, match_mapping_type=*, match=*}}]}}]]
	at org.elasticsearch.index.mapper.DocumentMapperParser.checkNoRemainingFields(DocumentMapperParser.java:192)
	at org.elasticsearch.index.mapper.DocumentMapperParser.parse(DocumentMapperParser.java:180)
	at org.elasticsearch.index.mapper.DocumentMapperParser.parseCompressed(DocumentMapperParser.java:121)
	at org.elasticsearch.index.mapper.MapperService.parse(MapperService.java:391)
	at org.elasticsearch.index.mapper.MapperService.merge(MapperService.java:265)
	at org.elasticsearch.cluster.metadata.MetaDataCreateIndexService$2.execute(MetaDataCreateIndexService.java:378)
	... 6 more

I am using Elastic Search 2.1.1.

Many thanks again.

Regards,
Paco.

Paco_B · March 3, 2016, 4:31pm

Very very strange...because of after using the "curl" command and the following script I could do it (but it was impossible using the Java API):

curl -XPOST "http://localhost:9200/my_index" -d "{\"mappings\": {\"_default_\": {\"dynamic_templates\": [{\"rule1\": {\"mapping\": {\"type\": \"string\"}, \"match\": \"*\", \"match_mapping_type\": \"*\"}}]}}}"

On the other hand, after configuring the index with the previous script, I launched my application but I receive the following exception from Elastic Search:

Exception 'failed to parse [tag_a]' while indexing the following content: {"tag_a":{"tag_b":{"tag_c": ... }}
Exception in thread "main" MapperParsingException[failed to parse [tag_a]]; nested: IllegalArgumentException[unknown property [tag_b]];
	at org.elasticsearch.index.mapper.FieldMapper.parse(FieldMapper.java:339)
	at org.elasticsearch.index.mapper.DocumentParser.parseObjectOrField(DocumentParser.java:314)
	at org.elasticsearch.index.mapper.DocumentParser.parseAndMergeUpdate(DocumentParser.java:762)
	at org.elasticsearch.index.mapper.DocumentParser.parseObject(DocumentParser.java:357)
	at org.elasticsearch.index.mapper.DocumentParser.parseObject(DocumentParser.java:257)
	at org.elasticsearch.index.mapper.DocumentParser.innerParseDocument(DocumentParser.java:127)
	at org.elasticsearch.index.mapper.DocumentParser.parseDocument(DocumentParser.java:79)
	at org.elasticsearch.index.mapper.DocumentMapper.parse(DocumentMapper.java:304)
	at org.elasticsearch.index.shard.IndexShard.prepareIndex(IndexShard.java:551)
	at org.elasticsearch.index.shard.IndexShard.prepareIndex(IndexShard.java:542)
	at org.elasticsearch.action.support.replication.TransportReplicationAction.prepareIndexOperationOnPrimary(TransportReplicationAction.java:1049)
	at org.elasticsearch.action.support.replication.TransportReplicationAction.executeIndexRequestOnPrimary(TransportReplicationAction.java:1060)
	at org.elasticsearch.action.index.TransportIndexAction.shardOperationOnPrimary(TransportIndexAction.java:170)
	at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryPhase.performOnPrimary(TransportReplicationAction.java:579)
	at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryPhase$1.doRun(TransportReplicationAction.java:452)
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: unknown property [tag_b]
	at org.elasticsearch.index.mapper.core.StringFieldMapper.parseCreateFieldForString(StringFieldMapper.java:348)
	at org.elasticsearch.index.mapper.core.StringFieldMapper.parseCreateField(StringFieldMapper.java:294)
	at org.elasticsearch.index.mapper.FieldMapper.parse(FieldMapper.java:331)
	... 18 more

I can confirm to you it is because after configuring the default mapping, ES thinks everything is a String, and in this case it is a subdocument. Is there another way to specify that all the values different than a subdocument or arrays, i.e. simple types, are Strings, or do I have to specify one by one all the types that I want to define as Strings? This last choice it cannot be an option because I have thousand of types.

Many thanks.

Regards,
Paco.

jimczi · March 3, 2016, 5:11pm

Is there another way to specify that all the values different than a subdocument or arrays, i.e. simple types, are Strings, or do I have to specify one by one all the types that I want to define as Strings? This last choice it cannot be an option because I have thousand of types.

Only these types can be automatically detected: boolean, date, double, long, object, string, so you'll have to define a rule for boolean, date, double, long and string only.
You can check this page for further documentation:

Paco_B · March 4, 2016, 9:07am

Many thanks

I think that it could be the expected configuration:

{
  "mappings": { "my_type": {
      "dynamic_templates": [
        {
          "booleans": {
            "match_mapping_type": "boolean",
            "mapping": {
              "type": "string"
            }
          }
        },
		{
          "dates": {
            "match_mapping_type": "date",
            "mapping": {
              "type": "string"
            }
          }
        },
		{
          "doubles": {
            "match_mapping_type": "double",
            "mapping": {
              "type": "string"
            }
          }
        },
		{
          "longs": {
            "match_mapping_type": "long",
            "mapping": {
              "type": "string"
            }
          }
        }	
      ]
    }
  }
}

But now, I have another problem when the tag could be a simple type or an object, for example:

<a>
  <b>1234</b>
  <b myAttr="attrValue">crash!</b>
</a>

If you see, the first "a.b" tag, its simple value is 1234 but the mapper will use the type String.
The problem is in the second tag "a.b", because the content is a "object" due to the internal attribute. Then, when I launch the application I see the following exception:

MapperParsingException[Merging dynamic updates triggered a conflict: mapper [a.b] of different type, current_type [string], merged_type [ObjectMapper]]

If I redefine the previous configuration and I add the "object" as "String":

{
 "objects": {
 "match_mapping_type": "object",
 "mapping": {
   "type": "string"
   }
}

I will get the following exception:

MapperParsingException[failed to parse [a]]; nested: IllegalArgumentException[unknown property [b]];

When I use a simple XML like this:

<a><b>hello</b></a>

Any other idea?

Many thanks.

Regards,
Paco.

Paco_B · March 5, 2016, 1:46pm

Could anyone help to me in this topic?

Thanks a lot!

JvonRudno · April 19, 2016, 9:17am

Hi Paco_B!!
Did you solve your problem, I have a similar situation and I can not find the solution. I will appreciate if you share with me your knowledge.

Paco_B · April 19, 2016, 10:00am

Nobody helped to me

I used "Spark" technology to solve the issue.

Good luck!