Not able to reindex data


(Kavya Sabu) #1

I wanted to rename a field in my logs in an existing index. So i created an alias for the existing index and pointed it to a new index. It doesn't show any documents in the new index. When i use the scan and scroll approach with the scroll ID it returns an error "unit is missing or unrecognized". The logs are not getting pushed into new index. How do i solve this?


(David Pilato) #2

You probably did not give a unit to the scroll time.


(Kavya Sabu) #3

Yes i did: GET /_search/scroll?scroll=1m followed by scroll id


(David Pilato) #4

May be a full example would help. I'm clueless here.


(Kavya Sabu) #5

This is a part of my logstash.conf file for renaming the fields:

mutate{
rename => { "NoOfViews" => "TotalViews" }
}
mutate{
rename => { "NoOfDownloads" => "TotalDownloads" }
}
output {
stdout {codec => rubydebug}
elasticsearch {
hosts => "localhost"
index => "movie_trial_indexer"

}

I added the above "rename" filter lines to the conf file after the index movie_trial_indexerwas created. In order to reindex the data i tried creating an alias for movie_trial_indexer and link the alias to a new index movie1_trial_indexeras follows:

PUT /movie_trial_indexer/_alias/ind1
POST /_aliases
{
"actions": [
{ "remove": {
"alias": "ind1",
"index": "movie_trial_indexer"
}
}

}

POST /_aliases
{
"actions": [
{ "add": {
"alias": "ind1",
"index": "movie1_trial_indexer"
}
}
]
}

Then to reindex i used the scan and scroll approach:

GET /movie_trial_indexer/_search?search_type=scan&scroll=1m
{
"query": { "match_all": {}},
"size": 1000
}

It returned a scroll ID. Then i executed the following to retrieve results:

GET /_search/scroll?scroll=1m c2Nhbjs1OzU5MjE3Ok9GcFk1ZUZTVGt1VFhxcHYwZW5VVUE7NTkyMTU6T0ZwWTVlRlNUa3VUWHFwdjBlblVVQTs1OTIxODpPRnBZNWVGU1RrdVRYcXB2MGVuVVVBOzU5MjE2Ok9GcFk1ZUZTVGt1VFhxcHYwZW5VVUE7NTkyMTk6T0ZwWTVlRlNUa3VUWHFwdjBlblVVQTsxO3RvdGFsX2hpdHM6Nzs=

It was showing an error like this:

"type": "parse_exception",
"reason": "Failed to parse setting [scroll] with value [1m scroll_id] as a time value: unit is missing or unrecognized"

If i execute the following using the new index :

GET /movie1_trial_indexer/movie/_search

I get:

"took": 1,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"failed": 0
},
"hits": {
"total": 0,
"max_score": null,
"hits": []
}

But there are no logs in the new index. How do i push the logs from old index to new index with changes updated?


(system) #6