Changing the _type of all entries

Elastic Stack Elasticsearch
1

Hey guys,

Would really appreciate if someone could tell me how I can change all of my _type entries in a index to be the same.

Right now some are 'log' and some are 'doc' when really they should all be doc.
I hear that reindexing would work however I want to keep the index name, furthermore doesn't reindexing cause issues with mapping? Since I want the mapping to remain the same.

The only reason this is needed, is because Elastic's python API seems to limit search queries based on the _type which is causing major headache. I'd like to keep all _type's the same otherwise not all the entries get returned. More info here: https://github.com/elastic/elasticsearch-dsl-py/issues/761#issuecomment-340278067

Thank you.

2

You need to reindex. If you do that into the same index it will not change the mapping, because it will use the existing template.

You will have to run a delete by query after you are done, to remove the documents with the old type.

3

To do so I am trying to reindex like so. I have setup the mapping for the new index.
However I keep getting the error
{ "error" : { "root_cause" : [ { "type" : "null_pointer_exception", "reason" : "Can't reindex without a destination type!" } ], "type" : "null_pointer_exception", "reason" : "Can't reindex without a destination type!" }, "status" : 500 }

This is my query:

curl -XPOST -d '{
  "source": {
    "index": "logs",
    "type": ["log", "doc"]
  },
  "dest": {
    "index": "new_logs",
    "type": "doc"
  },
  "script": {
    "source": "ctx._type = 'doc';"
  }
}' 'localhost:9200/_reindex?pretty'

This should take log and doc types and add them to the new index all under the doc type. However I can't see to solve the error I keep getting about how the destination needs a type. I thought I specified that properly?

4

Why not just reindex the current log types to be doc and in the existing index? They won't conflict and you reduce your reindex load by half (of the types).

5

I'm confused I thought you just said that wasn't possible to reindex in an existing index? How could I change all the log types to doc then?

And thanks for taking the time to help me out Mark, I appreciate it

6

Nope?

Something like what you have, eg;

curl -XPOST -d '{
  "source": {
    "index": "logs",
    "type": "log"
  },
  "dest": {
    "index": "logs",
    "type": "doc"
  },
  "script": {
    "source": "ctx._type = 'doc';"
  }
}' 'localhost:9200/_reindex?pretty'
7

Haha this is why I was getting confused about your replies, because _reindex does not allow source and dest to be the same.

{
  "error" : {
    "root_cause" : [
      {
        "type" : "action_request_validation_exception",
        "reason" : "Validation Failed: 1: reindex cannot write into an index its reading from [logs];"
      }
    ],
    "type" : "action_request_validation_exception",
    "reason" : "Validation Failed: 1: reindex cannot write into an index its reading from [logs];"
  },
  "status" : 400
}
8

Oh, my mistake then :frowning:

1 Like
9

Solved thanks :slight_smile:

10

How'd you end up solving it? It might be useful for others in future :slight_smile:

11

Removing the script completely, specifying the type to import to fixed the issue in the new index! It combined the log and doc types from the logs index into new_logs.

curl -XPOST -d '{
  "source": {
    "index": "logs",
  },
  "dest": {
    "index": "new_logs",
    "type": "doc"
  }
}' 'localhost:9200/_reindex?pretty'
1 Like
12

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.