Copy _id field to another field for prefix query


(Troy Collinsworth) #1

I have an index that stores versioned schema files where the filename is entity~date~hash.schema.json and forms the unique id that is stored in the _id field.

I want to use a copy_to to copy the _id field to an altId field so I don't have to rewrite other application code that creates and indexes the data. I can't figure out how to create an ES mapping that does this. Is it possible?

On 2.4 I used a prefix query with a groovy script to get the latest schema, see query A below. Finally moved up to ES 5.0 so that I could take advantage of deleteByQuery, waitfor, and refresh to have reliable tests without waiting for commit/refresh cycle. So had to rewrite queries for scripts and change field to stored_fields - no biggie, see Query B. However, the real problem is that the prefix queries on _id are no longer supported in 5.0.

Query A worked with 2.4:

{
  index: 'uts_msg_body_json_schema',
  type: 'definition',
  body: {
    from: 0, size: 1,
    query: {
      prefix: { _id: "REPLACE_ME" }
    },
    sort: {
      _script: {
        script: "Double.parseDouble(doc[\"_uid\"].value.split(\"~\")[1].split(\"[.schema]\")[0].replace('v',''))",
        lang: "groovy",
        type: "number",
        order: "desc"
      }
    }
  }
}

Query B not working with 5.0 due to prefix query on _id no longer supported,
Need to figure out how to copy _id to altid field:

{
  index: 'uts_msg_body_json_schema',
  type: 'definition',
  body: {
    from: 0, size: 1,
    query: {
      prefix: { _id: "REPLACE_ME" }
    },
    sort: {
      _script: {
        type: "number",
        script: {
          lang: 'groovy',
          inline: "Double.parseDouble(doc[\"_uid\"].value.split(\"~\")[1].split(\"[.schema]\")[0].replace('v',''))"              
        },
        order: "desc"
      }
    }
  }
}

(Troy Collinsworth) #2

Bit the bullet and added an altId field and updated the indexing to populate it. Couldn't get the copy_to to work because it complained that _id was already defined.


(system) #3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.