Duplicating ElasticSearch Content from Scroll Data


This may sound like a backwards way of doing things but unfortunately my hands are very much tied.

There is a remote ElasticSearch server which is being populated with log data. I need to mirror the content of that server onto my local ElasticSearch server. I only have read access to this server and have been using the Scroll API to gather all log data within a certain time frame. Obtaining this has been straightforward enough, but when it comes to uploading it to my local server I am having some issues.

Is it possible for the response data I have to be read into my ElasticSearch server and have it properly indexed etc? The current response data looks like so:

	"_index" : "A_TEST",
"_type" : "010",
"_id" : "6122049e1be05b23138fd7df2a1905a20cfcd39b",
"_score" : 1.0,
"_source" : {
  "hostname" : "00000000d38406e7",
  "date_timestamp" : "2016-11-24T12:08:27.174",
  "tx-time" : "2017-06-13T07:51:33+02:00",
  "message" : "1479989307   174000000 -16.1811089 6.42675947 -3.0798585 6.61311561 1.63192656 
  "data_time" : 1479989307174,
  "data" : {
	"AngleA" : 2.0694754,
	"AngleB" : -1.70794374,
	"AngleC" : 3.2744498,
	"AngleD" : 277.993374,
	"AngleE" : 1.01377407

I want to be able to have ElasticSearch/LogStash read this and set up the correct indices and field names as on the other server.

If anyone can recommend a different way of doing this or explain how it can be done then I'd be very grateful. I would have thought it could be read into ElasticSearch pretty easily but I am inexperienced with it.

Many thanks,

See https://www.elastic.co/guide/en/elasticsearch/reference/master/docs-reindex.html#reindex-from-remote

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.