# Logstash to Logstash (Lumberjack) host details **URL:** \ **Category:** Logstash\ **Created:** [October 27, 2020, 6:34am UTC](https://discuss.elastic.co/t/logstash-to-logstash-lumberjack-host-details/253390 "2020-10-27T06:34:49Z")\ **Posts on this page:** 5\ **Page:** 1
**Author:** ![cliff.wakefield](https://avatars.discourse-cdn.com/v4/letter/c/96bed5/32.png) [@cliff.wakefield](https://discuss.elastic.co/u/cliff.wakefield)\ **Post date:** [October 27, 2020, 6:34am UTC](https://discuss.elastic.co/t/logstash-to-logstash-lumberjack-host-details/253390/1 "2020-10-27T06:34:49Z")
I am setting up a two-tier Logstash setup for our customers. On-premises servers running syslog and beats will connect to 1 or more on-premises Logstash servers, which will be running syslog and beats pipelines configured using the lumberjack output plugin to forward the events to cloud-based Logstash servers hosted in AWS. I have this working and both syslog and beat events are arriving at the AWS hosted Logstash servers and being written out to S3 as desired. However, I would like to add fields to the events as they pass through the on-premises Logstash servers to record the details of the Logstash server they pass through, the equivalent of an X-Forwarded-For header a HTTP proxy server adds. I am unable to find any variables/fields that contain details of the Logstash server itself. The environment variable $HOSTNAME contains the hostname of the Logstash server but it would be good to get its FQDN and IP address included also. Anyone done anything like this before? Anyone have any ideas? ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [October 27, 2020, 2:44pm UTC](https://discuss.elastic.co/t/logstash-to-logstash-lumberjack-host-details/253390/2 "2020-10-27T14:44:41Z")
> [@cliff.wakefield](#): > > the equivalent of an X-Forwarded-For header a HTTP proxy server adds X-Forwarded-For? The beats will add their hostname to the messages they forward, and the syslog input should be adding a [host] field that contains the ip address the message was received from. If you want the name of the server running logstash you could try something like ``` ruby { init => "require 'socket'" code => "event.set('some field', Socket.gethostname)" } ``` ---
**Author:** ![cliff.wakefield](https://avatars.discourse-cdn.com/v4/letter/c/96bed5/32.png) [@cliff.wakefield](https://discuss.elastic.co/u/cliff.wakefield)\ **Post date:** [October 27, 2020, 8:28pm UTC](https://discuss.elastic.co/t/logstash-to-logstash-lumberjack-host-details/253390/3 "2020-10-27T20:28:15Z")
My current solution is to use environment variables but its rather inelegant. Logstash is actually running inside a **Docker** container, so through **Docker Compose** I am able to pass environment variables from a `.env` file. This `.env` is generated by a Pre-script run by `systemd` before the Docker Compose is started. In the Pre-script I have ```auto echo "HOST_HOSTNAME `hostname`" >> .env echo "HOST_FQDN `hostname -f`" >> .env echo "HOST_IP_ADDRESSES `hostname -I`" >> .env ``` **NOTE:** The Pre-script already existed as its brings in some secrets from AWS Systems Manager Parameter Store. **NOTE:** There is also a Post-script to remove `.env`. Then I can access those environment variables in a series of mutates, including a split() on $HOST\_IP\_ADDRESSES. ---
**Author:** ![cliff.wakefield](https://avatars.discourse-cdn.com/v4/letter/c/96bed5/32.png) [@cliff.wakefield](https://discuss.elastic.co/u/cliff.wakefield)\ **Post date:** [October 27, 2020, 8:36pm UTC](https://discuss.elastic.co/t/logstash-to-logstash-lumberjack-host-details/253390/4 "2020-10-27T20:36:30Z")
So the contents of `.env` ```auto HOST_HOSTNAME=logstash-relay HOST_FQDN=logstash-relay.shared HOST_IP_ADDRESSES=10.211.55.42 172.17.0.1 fdb2:2c26:f4e4:0:21c:42ff:feed:db4a ``` Passed through Docker Compose ```auto environment: - "HOST_HOSTNAME=${HOST_HOSTNAME}" - "HOST_FQDN=${HOST_FQDN}" - "HOST_IP_ADDRESSES=${HOST_IP_ADDRESSES}" ``` Then mutated in my `syslog` pipeline ```auto filter { mutate { add_field => { "[forwarder][hostname]" => "${HOST_HOSTNAME}" } add_field => { "[forwarder][fqdn]" => "${HOST_FQDN}" } add_field => { "[forwarder][ip]" => "${HOST_IP_ADDRESSES}" } } mutate { split => { "[forwarder][ip]" => " " } } } ``` Results in events now looking like this from the `syslog` pipeline on the forwarder. ```auto { "severity" => 6, "agent" => { "type" => "syslog" }, "pid" => "661", "program" => "snapd", "logsource" => "zenlinux.shared", "message" => "storehelpers.go:551: cannot refresh: snap has no updates available: \"core18\", \"lxd\", \"snapd\"", "priority" => 30, "@timestamp" => 2020-10-27T20:33:37.000Z, "forwarder" => { "hostname" => "logstash-relay", "fqdn" => "logstash-relay.shared", "ip" => [ [0] "10.211.55.42", [1] "172.17.0.1", [2] "fdb2:2c26:f4e4:0:21c:42ff:feed:db4a" ] }, "host" => "10.211.55.39", "@version" => "1", "facility" => 3, "severity_label" => "Informational", "facility_label" => "system", "timestamp" => "Oct 27 20:33:37" } ``` The `host` field is left untouched as it contains the IP address of the `syslog` client machine. I will also be adding this to `beats` but need to read up on where to put it in the ECS scheme. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [November 24, 2020, 8:36pm UTC](https://discuss.elastic.co/t/logstash-to-logstash-lumberjack-host-details/253390/5 "2020-11-24T20:36:42Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.