Hello,
This may sound like a backwards way of doing things but unfortunately my hands are very much tied.
There is a remote ElasticSearch server which is being populated with log data. I need to mirror the content of that server onto my local ElasticSearch server. I only have read access to this server and have been using the Scroll API to gather all log data within a certain time frame. Obtaining this has been straightforward enough, but when it comes to uploading it to my local server I am having some issues.
Is it possible for the response data I have to be read into my ElasticSearch server and have it properly indexed etc? The current response data looks like so:
{
"_index" : "A_TEST",
"_type" : "010",
"_id" : "6122049e1be05b23138fd7df2a1905a20cfcd39b",
"_score" : 1.0,
"_source" : {
"hostname" : "00000000d38406e7",
"date_timestamp" : "2016-11-24T12:08:27.174",
"tx-time" : "2017-06-13T07:51:33+02:00",
"message" : "1479989307 174000000 -16.1811089 6.42675947 -3.0798585 6.61311561 1.63192656
"data_time" : 1479989307174,
"data" : {
"AngleA" : 2.0694754,
"AngleB" : -1.70794374,
"AngleC" : 3.2744498,
"AngleD" : 277.993374,
"AngleE" : 1.01377407
}
}
}
I want to be able to have ElasticSearch/LogStash read this and set up the correct indices and field names as on the other server.
If anyone can recommend a different way of doing this or explain how it can be done then I'd be very grateful. I would have thought it could be read into ElasticSearch pretty easily but I am inexperienced with it.
Many thanks,
Peter