# Remove header information added by logstash

**URL:** https://discuss.elastic.co/t/remove-header-information-added-by-logstash/108892
**Category:** Logstash
**Created:** [November 23, 2017, 1:13pm UTC](https://discuss.elastic.co/t/remove-header-information-added-by-logstash/108892 "2017-11-23T13:13:01Z")
**Posts on this page:** 6
**Page:** 1

<div class="post-metadata">

### Author: ![vishal0708gautam](https://avatars.discourse-cdn.com/v4/letter/v/ee59a6/32.png) [@vishal0708gautam](https://discuss.elastic.co/u/vishal0708gautam)
#### Post date: [November 23, 2017, 1:13pm UTC](https://discuss.elastic.co/t/remove-header-information-added-by-logstash/108892/1 "2017-11-23T13:13:01Z")

</div>

Hello experts,

We are using logstash as syslog event receiver and forwarder using UDP & TCP and it is doing as expected but on forwarding syslog events to output, logstash is adding its own header, we want to remove this header from syslog. Please find below expected syslog and the syslog received from logstash

## Expected syslog

Nov 23 10:11:39 127.0.0.1 token1: expired, successful,Info,0000,00002,Linux,Test

## Syslog received from logstash

**Nov 23 10:01:40 10.140.190.105 LOGSTASH[-]:** \<13\>Nov 23 10:01:40 127.0.0.11 token1: expired, successful,Info,0000,00002,Linux,Test

As you can see extra logstash header info is getting appended in the syslog. Kindly suggest how to remove this header.  
I have seen other threads with similar problems as well but it didnt solve the purpose.

---

<div class="post-metadata">

### Author: ![guyboertje](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/guyboertje/32/31592_2.png) [@guyboertje](https://discuss.elastic.co/u/guyboertje)
#### Post date: [November 23, 2017, 2:56pm UTC](https://discuss.elastic.co/t/remove-header-information-added-by-logstash/108892/2 "2017-11-23T14:56:28Z")

</div>

Please post your config here using triple backticks ``` above and below the config text.

---

<div class="post-metadata">

### Author: ![vishal0708gautam](https://avatars.discourse-cdn.com/v4/letter/v/ee59a6/32.png) [@vishal0708gautam](https://discuss.elastic.co/u/vishal0708gautam)
#### Post date: [November 24, 2017, 4:54am UTC](https://discuss.elastic.co/t/remove-header-information-added-by-logstash/108892/3 "2017-11-24T04:54:32Z")

</div>

Please find below the config I am using

```auto
input {

  syslog {
    port => 1468
  }
  udp {
    port => 514
    type => syslog
  }
}

output {
  syslog{
   host => "10.140.190.105"
   port => 1468
   protocol => tcp
  }
}

```

---

<div class="post-metadata">

### Author: ![vishal0708gautam](https://avatars.discourse-cdn.com/v4/letter/v/ee59a6/32.png) [@vishal0708gautam](https://discuss.elastic.co/u/vishal0708gautam)
#### Post date: [November 24, 2017, 11:42am UTC](https://discuss.elastic.co/t/remove-header-information-added-by-logstash/108892/4 "2017-11-24T11:42:30Z")

</div>

Solved

I got the solution to the above problem

Used TCP output plugin & changed config file

## Please find the below config file

```auto
input {

  syslog {
    port => 1468
  }
  udp {
    port => 514
    type => syslog
  }

}

output {

  tcp{
   host => "10.140.190.105"
   port => 1468

   codec => line {
   format => "%{message}"
   }

 }

}

```

* * *

But now facing a new issue i.e. while sending syslog event output some times TCP hangs and then outputs the syslog event. The major problem here is that if any event comes in between for output, because TCP output is hanged, the event drop or not sent.

## Please find below log of TCP while it hangs

```auto
[2017-11-24T08:10:34,608][WARN][logstash.outputs.tcp] tcp output exception {:host=>"10.140.190.105", :port=>1468, :exception=>#<EOFError: End of file reached>, :backtrace=>["org/jruby/RubyIO.java:3030:in `sysread'", "/usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-tcp-4.0.2/lib/logstash/outputs/tcp.rb:162:in `register'", "org/jruby/RubyProc.java:281:in `call'", "/usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-codec-line-3.0.4/lib/logstash/codecs/line.rb:54:in `encode'", "/usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-tcp-4.0.2/lib/logstash/outputs/tcp.rb:207:in `receive'", "/usr/share/logstash/logstash-core/lib/logstash/outputs/base.rb:92:in `multi_receive'", "org/jruby/RubyArray.java:1613:in `each'", "/usr/share/logstash/logstash-core/lib/logstash/outputs/base.rb:92:in `multi_receive'", "/usr/share/logstash/logstash-core/lib/logstash/output_delegator_strategies/single.rb:15:in `multi_receive'", "org/jruby/ext/thread/Mutex.java:149:in `synchronize'", "/usr/share/logstash/logstash-core/lib/logstash/output_delegator_strategies/single.rb:14:in `multi_receive'", "/usr/share/logstash/logstash-core/lib/logstash/output_delegator.rb:49:in `multi_receive'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:434:in `output_batch'", "org/jruby/RubyHash.java:1342:in `each'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:433:in `output_batch'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:381:in `worker_loop'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:342:in `start_workers'"]}

```

Please suggest the solution for this problem

---

<div class="post-metadata">

### Author: ![guyboertje](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/guyboertje/32/31592_2.png) [@guyboertje](https://discuss.elastic.co/u/guyboertje)
#### Post date: [November 24, 2017, 12:08pm UTC](https://discuss.elastic.co/t/remove-header-information-added-by-logstash/108892/5 "2017-11-24T12:08:26Z")

</div>

Here is the ruby code that that **warning** log message is coming from:

```ruby
        begin
          client_socket = connect unless client_socket
          r,w,e = IO.select([client_socket], [client_socket], [client_socket], nil)
          # don't expect any reads, but a readable socket might
          # mean the remote end closed, so read it and throw it away.
          # we'll get an EOFError if it happens.
          client_socket.sysread(16384) if r.any?

          # Now send the payload
          client_socket.syswrite(payload) if w.any?
        rescue => e
          @logger.warn("tcp output exception", :host => @host, :port => @port,
                       :exception => e, :backtrace => e.backtrace)
          client_socket.close rescue nil
          client_socket = nil
          sleep @reconnect_interval
          retry
        end

```

This seems to mean that the remote end closed early and we sleep and retry at the default `reconnect_interval` of 10 seconds.  
I suggest that you investigate the network activity, correct the problem and reduce the `reconnect_interval` to 1 second.

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [December 22, 2017, 12:08pm UTC](https://discuss.elastic.co/t/remove-header-information-added-by-logstash/108892/6 "2017-12-22T12:08:27Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
