I wanted to rename a field in my logs in an existing index. So i created an alias for the existing index and pointed it to a new index. It doesn't show any documents in the new index. When i use the scan and scroll approach with the scroll ID it returns an error "unit is missing or unrecognized". The logs are not getting pushed into new index. How do i solve this?
You probably did not give a unit to the scroll time.
Yes i did: GET /_search/scroll?scroll=1m followed by scroll id
May be a full example would help. I'm clueless here.
This is a part of my logstash.conf file for renaming the fields:
mutate{
rename => { "NoOfViews" => "TotalViews" }
}
mutate{
rename => { "NoOfDownloads" => "TotalDownloads" }
}
output {
stdout {codec => rubydebug}
elasticsearch {
hosts => "localhost"
index =>"movie_trial_indexer"
}
I added the above "rename" filter lines to the conf file after the index movie_trial_indexer
was created. In order to reindex the data i tried creating an alias for movie_trial_indexer
and link the alias to a new index movie1_trial_indexer
as follows:
PUT /movie_trial_indexer/_alias/ind1
POST /_aliases
{
"actions": [
{ "remove": {
"alias": "ind1",
"index": "movie_trial_indexer"
}
}}
POST /_aliases
{
"actions": [
{ "add": {
"alias": "ind1",
"index": "movie1_trial_indexer"
}
}
]
}
Then to reindex i used the scan and scroll approach:
GET /movie_trial_indexer/_search?search_type=scan&scroll=1m
{
"query": { "match_all": {}},
"size": 1000
}
It returned a scroll ID. Then i executed the following to retrieve results:
GET /_search/scroll?scroll=1m c2Nhbjs1OzU5MjE3Ok9GcFk1ZUZTVGt1VFhxcHYwZW5VVUE7NTkyMTU6T0ZwWTVlRlNUa3VUWHFwdjBlblVVQTs1OTIxODpPRnBZNWVGU1RrdVRYcXB2MGVuVVVBOzU5MjE2Ok9GcFk1ZUZTVGt1VFhxcHYwZW5VVUE7NTkyMTk6T0ZwWTVlRlNUa3VUWHFwdjBlblVVQTsxO3RvdGFsX2hpdHM6Nzs=
It was showing an error like this:
"type": "parse_exception",
"reason": "Failed to parse setting [scroll] with value [1m scroll_id] as a time value: unit is missing or unrecognized"
If i execute the following using the new index :
GET /movie1_trial_indexer/movie/_search
I get:
"took": 1,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"failed": 0
},
"hits": {
"total": 0,
"max_score": null,
"hits":
}
But there are no logs in the new index. How do i push the logs from old index to new index with changes updated?