# Transfer log events as files between Logstash instances

**URL:** <https://discuss.elastic.co/t/transfer-log-events-as-files-between-logstash-instances/353503>\
**Category:** Logstash\
**Created:** [February 17, 2024, 9:16am UTC](https://discuss.elastic.co/t/transfer-log-events-as-files-between-logstash-instances/353503 "2024-02-17T09:16:47Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![olavur](https://avatars.discourse-cdn.com/v4/letter/o/87869e/32.png) [@olavur](https://discuss.elastic.co/u/olavur)\
**Post date:** [February 17, 2024, 9:16am UTC](https://discuss.elastic.co/t/transfer-log-events-as-files-between-logstash-instances/353503/1 "2024-02-17T09:16:47Z")

</div>

Hi,

I have two logstash instances. One instance A receives events from elastic agents. The second, instance B, inputs the events and outputs them further downstream.

But my only option is to transfer files from instance A to instance B.

I hoped that the output and input plugins could be configured with a codec which would handled this. But I am unable to find a working configuration.

For example, it seems that the metadata information is not included in the json\_lines output.

Basically I would like, for example, the output file plugin on instance A to write everything about the log events to files. Transfer the files to server B. Let logstash on server B have a input plugin configured which reads the events and process them further downstream.

Any suggestions ?

---

<div class="post-metadata">

**Author:** ![strawgate](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/strawgate/32/131008_2.png) [@strawgate](https://discuss.elastic.co/u/strawgate)\
**Post date:** [February 17, 2024, 12:09pm UTC](https://discuss.elastic.co/t/transfer-log-events-as-files-between-logstash-instances/353503/2 "2024-02-17T12:09:51Z")

</div>

> [@olavur](#):
>
> For example, it seems that the metadata information is not included in the json\_lines output.

Simply copy the @metadata Field using a log stash mutate copy filter. Copy it to a field called metadata\_backup. Export to a file and then when you import the file on the other logstash simply use a mutate copy filter to copy from metadata\_backup to @metadata.

---

<div class="post-metadata">

**Author:** ![olavur](https://avatars.discourse-cdn.com/v4/letter/o/87869e/32.png) [@olavur](https://discuss.elastic.co/u/olavur)\
**Post date:** [February 17, 2024, 9:37pm UTC](https://discuss.elastic.co/t/transfer-log-events-as-files-between-logstash-instances/353503/3 "2024-02-17T21:37:51Z")

</div>

Thank you for the recommendation.

And do you suggest that I use the json\_lines plugins for output and input ?

---

<div class="post-metadata">

**Author:** ![strawgate](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/strawgate/32/131008_2.png) [@strawgate](https://discuss.elastic.co/u/strawgate)\
**Post date:** [February 17, 2024, 10:21pm UTC](https://discuss.elastic.co/t/transfer-log-events-as-files-between-logstash-instances/353503/4 "2024-02-17T22:21:03Z")

</div>

I'm not sure why you need to save to a file to transfer so I'm not sure what the best might be but json lines seems like a good option to me

---

<div class="post-metadata">

**Author:** ![olavur](https://avatars.discourse-cdn.com/v4/letter/o/87869e/32.png) [@olavur](https://discuss.elastic.co/u/olavur)\
**Post date:** [February 19, 2024, 11:34am UTC](https://discuss.elastic.co/t/transfer-log-events-as-files-between-logstash-instances/353503/5 "2024-02-19T11:34:15Z")

</div>

Thank you @strawgate. On instance A I configured the logstash with

```auto
filter {
  mutate {
    copy => { "@metadata" => "metadata_backup" }
  }
}

```

And then used the file output plugin with the json\_lines codec.

One the receiving instance B I used the json codec on the file input plugin and the following filter configuration

```auto
filter {
  mutate {
    copy => { "metadata_backup" => "@metadata" }
  }
}

```

Works.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [March 18, 2024, 11:35am UTC](https://discuss.elastic.co/t/transfer-log-events-as-files-between-logstash-instances/353503/6 "2024-03-18T11:35:09Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.