Upgrading to 2.x: convert "index_name" to "copy_to"

Elastic Stack Elasticsearch
1

Hi, we are upgrading our elasticsearch from 1.7.5 to 2.3. There is one case in our mapping seems having problem to reproduce the same query behavior in 1.7:

mapping

{
    "dynamic": False,
    "properties": {
		"aaa": {
			"properties": {
				"b1": {
					"type": "string",
                                        "copy_to": "aaa"
				},
				"b2": {
					"type": "string",
                                        "copy_to": "aaa"
				}
			}
		}
	}
}

Previously, we allow users to query fields on "aaa.b1", "aaa.b2", and just "aaa" field for either b1 or b2 by combining "path": "just_name" and "index_name": "aaa" in the mapping. However, now with copy_to, we cannot index values from aaa.b1, and aaa.b2 collectively to just "aaa" as "aaa" is now defined as an object field.

Any solution for that?

Thanks!

2

w00t! That sounds like a hack to me!

Funny...

You can may be try to copy to aaa.aaa which might work but I'm super unsure...

3

For example a use case might be:

{“response”: {“src1”: “value1”},{“src2”:“value2”}}

I would like our users can query on “response” field for either value1 or value2. I know I can do response.all as the copy_to field, but query on “response” field is more natural to our users, as it follows an ontology we are using, and users have an expectation to query on response field. I also cannot construct the data as

{“response”: [“value1”, “value2”]}

As it loses the src info.

4

Did you try what I wrote?

5

Hi David, just tried. It only makes query "aaa.aaa:v1" work, but not "aaa:v1".

The mapping looks like this:

m1 = {
    "dynamic": False,
    "properties": {
		"aaa": {
			"properties": {
				"b1": {
					"type": "string",
                                        "copy_to": "aaa.aaa"
				},
				"b2": {
					"type": "string",
                                        "copy_to": "aaa.aaa"
				},
                "aaa" : {"type": "string"}
			}
		}
	}
}

and the test doc is:

{"aaa":{"b1": "v1", "b2":"v2"}}
6

Note: please use </> when posting code. I updated your answer.

Well. Yes... So I don't see any other solution than reworking your json content.
Not sure if in your ingestion pipeline you can mutate some fields and for example instead of indexing:

{
  "foo": {
     "a": "bar",
     "b": "baz"
}

index

{
  "fake_foo": {
     "a": "bar",
     "b": "baz"
}

And then use copy_to to copy fake_foo.a and fake_foo.b to foo?
Potentially use the same copy_to feature to copy fake_foo.a to foo.a and fake_foo.b to foo.b?
Then don't index fake_*.

Can't see another way to achieve the same.

I hope this could help.

7

Thanks! For now I think I can ask users to query as "aaa.*:value1", instead of previous "aaa:value1". Changing the json structure is not ideal, as this structure is relatively fixed.