# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=826 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 827 --- ## [Kibana Visualization (data table) export w/ Logstash](https://discuss.elastic.co/t/kibana-visualization-data-table-export-w-logstash/118965)
**Author:** [@alexus](https://discuss.elastic.co/u/alexus)\ **Replies:** 0\ **Last updated:** [February 8, 2018, 3:59am UTC](https://discuss.elastic.co/t/kibana-visualization-data-table-export-w-logstash/118965 "2018-02-08T03:59:55Z")
Is there a way to export Kibana's Visualization Ddata table (with (sub) buckets)) results with Logstash? --- ## [Logstash-tcp-input not parsing proxy protocol header](https://discuss.elastic.co/t/logstash-tcp-input-not-parsing-proxy-protocol-header/118957)
**Author:** [@brendan](https://discuss.elastic.co/u/brendan)\ **Replies:** 0\ **Last updated:** [February 8, 2018, 2:26am UTC](https://discuss.elastic.co/t/logstash-tcp-input-not-parsing-proxy-protocol-header/118957 "2018-02-08T02:26:32Z")
I have an NGINX loadbalancer configured to use proxy protocol when sending to logstash instances on the backend. Here is my input config input { tcp { port =\> 5002 proxy\_protocol =\> true type =\> "cisco-asa" } } t… --- ## [Docker logstash terminated during first run](https://discuss.elastic.co/t/docker-logstash-terminated-during-first-run/118923)
**Author:** [@Stephen\_Im](https://discuss.elastic.co/u/Stephen_Im)\ **Replies:** 0\ **Last updated:** [February 7, 2018, 8:39pm UTC](https://discuss.elastic.co/t/docker-logstash-terminated-during-first-run/118923 "2018-02-07T20:39:32Z")
Hi, I am installing logstash on docker. I installed elasticsearch and kibana, but failed to run logstash. Below is the message I am getting. I created logstash.conf file under "/usr/share/logstash/pipelilne", but bel… --- ## [LogstashSocketAppender include environment variable in output](https://discuss.elastic.co/t/logstashsocketappender-include-environment-variable-in-output/118940)
**Author:** [@dtanner](https://discuss.elastic.co/u/dtanner)\ **Replies:** 0\ **Last updated:** [February 7, 2018, 10:56pm UTC](https://discuss.elastic.co/t/logstashsocketappender-include-environment-variable-in-output/118940 "2018-02-07T22:56:15Z")
Can you include the value of an environment variable in the json output when using the LogstashSocketAppender? e.g. How would I include the value of an environment variable named FOO with every log message? Would it lo… --- ## [Logstash filter based on beat.name - not type possible?](https://discuss.elastic.co/t/logstash-filter-based-on-beat-name-not-type-possible/118879)
**Author:** [@ssi](https://discuss.elastic.co/u/ssi)\ **Replies:** 3\ **Last updated:** [February 7, 2018, 9:32pm UTC](https://discuss.elastic.co/t/logstash-filter-based-on-beat-name-not-type-possible/118879 "2018-02-07T21:32:26Z")
Hi im trying to get at logstash filter to fire based on one of the fields or tag instead of type. the reason being that i have multible types of beats coming in (winlogbeat and filebeat) i thought it would be easy peasy … --- ## [Extract IP from multiple lines](https://discuss.elastic.co/t/extract-ip-from-multiple-lines/118910)
**Author:** [@dvi\_469](https://discuss.elastic.co/u/dvi_469)\ **Replies:** 4\ **Last updated:** [February 7, 2018, 9:15pm UTC](https://discuss.elastic.co/t/extract-ip-from-multiple-lines/118910 "2018-02-07T21:15:28Z")
Using ELK 5.6 Filebeat ships a log to Logstash that looks like this: 1.1.1.1 2.2.2.2 3.3.3.3 etc. What would be the best way to extract each IP address and put it into its own field, like peer1, peer2, peer3, etc..… --- ## [Logstash sincedb\_path is not working](https://discuss.elastic.co/t/logstash-sincedb-path-is-not-working/118918)
**Author:** [@databeata](https://discuss.elastic.co/u/databeata)\ **Replies:** 3\ **Last updated:** [February 7, 2018, 9:14pm UTC](https://discuss.elastic.co/t/logstash-sincedb-path-is-not-working/118918 "2018-02-07T21:14:59Z")
This is not working. There is no output. I want to use sincedb\_path =\> "/mnt/elk/logstash/scripts/sincedb/from\_nifi\_dev\_logs\_nifi\_6 sudo rm -rf /mnt/elk/logstash/scripts/sincedb/from\_nifi\_dev\_logs\_nifi\_6 sudo /usr/shar… --- ## [Logstash issue with stomp](https://discuss.elastic.co/t/logstash-issue-with-stomp/118883)
**Author:** [@moriol](https://discuss.elastic.co/u/moriol)\ **Replies:** 0\ **Last updated:** [February 7, 2018, 4:36pm UTC](https://discuss.elastic.co/t/logstash-issue-with-stomp/118883 "2018-02-07T16:36:54Z")
I am getting the following error when trying to listen to ActiveMQ with stomp: \[ERROR\] 2018-02-07 17:24:33.003 \[\[main\]\ **Author:** [@jclose](https://discuss.elastic.co/u/jclose)\ **Replies:** 2\ **Last updated:** [February 7, 2018, 3:59pm UTC](https://discuss.elastic.co/t/trouble-getting-logstash-to-pick-up-an-environment-variable/118871 "2018-02-07T15:59:10Z") I am consistently getting an error with trying to pick up the environment variable within my input config file. Here is what I have: input { redis { data\_type =\> "list" key =\> "syslog" host … --- ## [Bootstrap broker disconnected after upgrading logstash 2.x to 5.x](https://discuss.elastic.co/t/bootstrap-broker-disconnected-after-upgrading-logstash-2-x-to-5-x/118829)
**Author:** [@chitraj8](https://discuss.elastic.co/u/chitraj8)\ **Replies:** 0\ **Last updated:** [February 7, 2018, 11:39am UTC](https://discuss.elastic.co/t/bootstrap-broker-disconnected-after-upgrading-logstash-2-x-to-5-x/118829 "2018-02-07T11:39:20Z")
Unable to connect brokers though the params are correct. API endpoint was set to connect elastic search. Consumer config values been read after that the error about disconnect appears: \[2018-02-07T06:33:11,399\]\[INFO … --- ## [Docker.elastic.co/logstash/logstash version 5.6.1 does not send logs to ElasticSearch on a different server](https://discuss.elastic.co/t/docker-elastic-co-logstash-logstash-version-5-6-1-does-not-send-logs-to-elasticsearch-on-a-different-server/118717)
**Author:** [@ajstark123](https://discuss.elastic.co/u/ajstark123)\ **Replies:** 6\ **Last updated:** [February 7, 2018, 2:08pm UTC](https://discuss.elastic.co/t/docker-elastic-co-logstash-logstash-version-5-6-1-does-not-send-logs-to-elasticsearch-on-a-different-server/118717 "2018-02-07T14:08:58Z")
The docker.elastic.co/logstash/logstash:5.6.1 image is not sending logs to an ElasticSearch cluster that is running on different servers. We are running this docket image in docker toolbox on a windows 7.0 PC. The Ela… --- ## [LogStash filter not works with FileBeat input](https://discuss.elastic.co/t/logstash-filter-not-works-with-filebeat-input/118836)
**Author:** [@Uwais\_Ibrahim](https://discuss.elastic.co/u/Uwais_Ibrahim)\ **Replies:** 0\ **Last updated:** [February 7, 2018, 12:01pm UTC](https://discuss.elastic.co/t/logstash-filter-not-works-with-filebeat-input/118836 "2018-02-07T12:01:06Z")
Hi, I have an ELK stack running on my server. Everything looks good except when I tried to filter out the timestamp from the message part of catalina.out log file. The below is the message. 2018-01-23 12:49:17,944 \[ht… --- ## [Logstash-output-mongodb depends on older version](https://discuss.elastic.co/t/logstash-output-mongodb-depends-on-older-version/118616)
**Author:** [@sana](https://discuss.elastic.co/u/sana)\ **Replies:** 1\ **Last updated:** [February 7, 2018, 1:29pm UTC](https://discuss.elastic.co/t/logstash-output-mongodb-depends-on-older-version/118616 "2018-02-07T13:29:16Z")
hello, everytime i try to install the mongo output plugin, it always show this error: Plugin version conflict, aborting ERROR: Installation Aborted, message: Bundler could not find compatible versions for gem "mongo": … --- ## [Problem Kibana Timestamp - log date](https://discuss.elastic.co/t/problem-kibana-timestamp-log-date/118811)
**Author:** [@samsam](https://discuss.elastic.co/u/samsam)\ **Replies:** 1\ **Last updated:** [February 7, 2018, 1:28pm UTC](https://discuss.elastic.co/t/problem-kibana-timestamp-log-date/118811 "2018-02-07T13:28:20Z")
Hi, I'm in internship and i have to do configure ELK. The problem is that I can not get the date of the logs in the timestamp of kibana. I think it's not difficult but i'm not arrived. My filter is : filter { if "k… --- ## [Correlating two different feilds from two different indxes](https://discuss.elastic.co/t/correlating-two-different-feilds-from-two-different-indxes/118846)
**Author:** [@Raj\_Kumar](https://discuss.elastic.co/u/Raj_Kumar)\ **Replies:** 0\ **Last updated:** [February 7, 2018, 1:05pm UTC](https://discuss.elastic.co/t/correlating-two-different-feilds-from-two-different-indxes/118846 "2018-02-07T13:05:53Z")
Hi All, Is it possible to correlate two different fields from two different indexes using logstash using mutate or translate filter? Like filter { if \[netflow\]\[ipv4\_src\_addr\] == \[entity\_threat\] { mutate { add\_… --- ## [JDBC connection to DB2 failed](https://discuss.elastic.co/t/jdbc-connection-to-db2-failed/118617)
**Author:** [@Farhad\_Yousefi](https://discuss.elastic.co/u/Farhad_Yousefi)\ **Replies:** 5\ **Last updated:** [February 7, 2018, 12:41pm UTC](https://discuss.elastic.co/t/jdbc-connection-to-db2-failed/118617 "2018-02-07T12:41:25Z")
I want to check one of our DB2 database's table via logstash but I got this exception. \[2018-02-06T13:34:34,175\]\[ERROR\]\[logstash.agent \] Pipeline aborted due to error {:exception=\>#\ **Author:** [@YvorL](https://discuss.elastic.co/u/YvorL)\ **Replies:** 0\ **Last updated:** [February 7, 2018, 12:23pm UTC](https://discuss.elastic.co/t/cant-follow-tutorial/118840 "2018-02-07T12:23:05Z") Hi! I started the logstash tutorial. I installed elasticsearch, kibana, logstash (as a service) and filebeat (6.2). When I get to the point where I should run: bin/logstash -f first-pipeline.conf --config.reload.automa… --- ## [Can I read input file's metadata (file creation time) through logstash?](https://discuss.elastic.co/t/can-i-read-input-files-metadata-file-creation-time-through-logstash/118707)
**Author:** [@Dimitry\_Khan](https://discuss.elastic.co/u/Dimitry_Khan)\ **Replies:** 2\ **Last updated:** [February 7, 2018, 12:00pm UTC](https://discuss.elastic.co/t/can-i-read-input-files-metadata-file-creation-time-through-logstash/118707 "2018-02-07T12:00:59Z")
Hi, can logstash read the metadata of the input file; say, the file creation time? --- ## [Ruby Filter: Getting more info for logstash.filters.ruby errors](https://discuss.elastic.co/t/ruby-filter-getting-more-info-for-logstash-filters-ruby-errors/118831)
**Author:** [@vicpav](https://discuss.elastic.co/u/vicpav)\ **Replies:** 0\ **Last updated:** [February 7, 2018, 11:43am UTC](https://discuss.elastic.co/t/ruby-filter-getting-more-info-for-logstash-filters-ruby-errors/118831 "2018-02-07T11:43:57Z")
The Logstash 5.6.7 logging pipeline in use has about 50 conditionals inside the filter, most of them resulting in data manipulations through Ruby codes in a form if \ { ruby { code =\> ” \ **Author:** [@Anuar\_Mukatov](https://discuss.elastic.co/u/Anuar_Mukatov)\ **Replies:** 24\ **Last updated:** [February 7, 2018, 11:24am UTC](https://discuss.elastic.co/t/datetime-from-sql-to-timestamp-logstash/117594 "2018-02-07T11:24:02Z") Hello everybody! Help me please with @timestamp, i have jdbc input with mssql server, and in my output i have variable - datetime, which include time of created table. How i can use it instead @timestamp? this is my co… --- ## [Logstasch eventlog event fieds is written to kafka in chinese](https://discuss.elastic.co/t/logstasch-eventlog-event-fieds-is-written-to-kafka-in-chinese/118818)
**Author:** [@gititc](https://discuss.elastic.co/u/gititc)\ **Replies:** 0\ **Last updated:** [February 7, 2018, 11:08am UTC](https://discuss.elastic.co/t/logstasch-eventlog-event-fieds-is-written-to-kafka-in-chinese/118818 "2018-02-07T11:08:36Z")
I'm trying to write event log messages from specific source to kafka. The problem is that all textual event fields are written to kafka in chinese. Because of that: filter is not working (the if statement for specifi… --- ## [Config error](https://discuss.elastic.co/t/config-error/118721)
**Author:** [@zozo6015](https://discuss.elastic.co/u/zozo6015)\ **Replies:** 2\ **Last updated:** [February 7, 2018, 10:59am UTC](https://discuss.elastic.co/t/config-error/118721 "2018-02-07T10:59:39Z")
Hello, I am trying to parse a java log file with grok but the config test is telling me that: The given configuration is invalid. Reason: no implicit conversion of Array into Hash. Putting the config into debug mode I g… --- ## [Queue persistence - which is the most suitable solution](https://discuss.elastic.co/t/queue-persistence-which-is-the-most-suitable-solution/118808)
**Author:** [@pacojones](https://discuss.elastic.co/u/pacojones)\ **Replies:** 0\ **Last updated:** [February 7, 2018, 10:32am UTC](https://discuss.elastic.co/t/queue-persistence-which-is-the-most-suitable-solution/118808 "2018-02-07T10:32:50Z")
Hi all. I'm new at Elastic stack, my team is exploring its possibilities in order to develop a centralised logging solution, and one feature we'd like to have by definition in our system is to ensure that messages that … --- ## [Custom log file Filtering](https://discuss.elastic.co/t/custom-log-file-filtering/118584)
**Author:** [@RickJC](https://discuss.elastic.co/u/RickJC)\ **Replies:** 3\ **Last updated:** [February 7, 2018, 10:26am UTC](https://discuss.elastic.co/t/custom-log-file-filtering/118584 "2018-02-07T10:26:22Z")
Hi All, I'm a newbie so sorry if my question is silly. I've just started out with a ELK stack which works great (6.1) and using filebeat to ship logs. My question is, I want to filter out the "message" part of the log s… --- ## [Problem configuring netflow in elk stack](https://discuss.elastic.co/t/problem-configuring-netflow-in-elk-stack/116661)
**Author:** [@BoffinPanda](https://discuss.elastic.co/u/BoffinPanda)\ **Replies:** 3\ **Last updated:** [February 7, 2018, 9:16am UTC](https://discuss.elastic.co/t/problem-configuring-netflow-in-elk-stack/116661 "2018-02-07T09:16:28Z")
Hello. I've configured the elk stack and now I want to add netflow to logstash. I've made the necessary configuration files and even edited the modules in logstash.yml to enable netflow. However, I continue to get Time… --- ## [Relating two fields - using translate or mutate](https://discuss.elastic.co/t/relating-two-fields-using-translate-or-mutate/115734)
**Author:** [@Raj\_Kumar](https://discuss.elastic.co/u/Raj_Kumar)\ **Replies:** 4\ **Last updated:** [February 7, 2018, 9:02am UTC](https://discuss.elastic.co/t/relating-two-fields-using-translate-or-mutate/115734 "2018-02-07T09:02:53Z")
Hi All, I have a index for netflow data and recently I was trying to use the combine threat intelligence to see if there is any communication from any black listed ip's. I have two tags one is netflow and other one is t… --- ## [Experiences with Logstash Redis spiped](https://discuss.elastic.co/t/experiences-with-logstash-redis-spiped/118227)
**Author:** [@itspecialistsimon](https://discuss.elastic.co/u/itspecialistsimon)\ **Replies:** 4\ **Last updated:** [February 7, 2018, 8:58am UTC](https://discuss.elastic.co/t/experiences-with-logstash-redis-spiped/118227 "2018-02-07T08:58:42Z")
Hi everyone, does anybody have some knowledge regarding the abovementioned subject? My setup is the following: Filebeat/Syslog --\> Logstash(A) --\> RabbitMQ --\> |Firewall| --\> Logstash(B) --\> Elasticsearch \<-- Kibana I… --- ## [Grok pattern failure](https://discuss.elastic.co/t/grok-pattern-failure/118614)
**Author:** [@cilzzz](https://discuss.elastic.co/u/cilzzz)\ **Replies:** 1\ **Last updated:** [February 7, 2018, 7:20am UTC](https://discuss.elastic.co/t/grok-pattern-failure/118614 "2018-02-07T07:20:26Z")
Hey everyone i am trying to extract two patterns from this log file . i want the server name (LSB4....) and the Status \> Nov 1 07:01:39 Task Completed: LSB4-35vls-M2-DESERT158 The following task completed:#012#012 609… --- ## [Conversion of field from string to int64](https://discuss.elastic.co/t/conversion-of-field-from-string-to-int64/118611)
**Author:** [@moriol](https://discuss.elastic.co/u/moriol)\ **Replies:** 1\ **Last updated:** [February 7, 2018, 7:10am UTC](https://discuss.elastic.co/t/conversion-of-field-from-string-to-int64/118611 "2018-02-07T07:10:12Z")
I have some data that is being ingested via Logstash and then being passed to Kafka as an output. From Kafka the data is then being passed to InfluxDB. I have a field which is ingested as a string, an example is as this… --- ## [Kafka partition keep rebalancing among the logstash instances](https://discuss.elastic.co/t/kafka-partition-keep-rebalancing-among-the-logstash-instances/118758)
**Author:** [@shjdwxy](https://discuss.elastic.co/u/shjdwxy)\ **Replies:** 0\ **Last updated:** [February 7, 2018, 6:24am UTC](https://discuss.elastic.co/t/kafka-partition-keep-rebalancing-among-the-logstash-instances/118758 "2018-02-07T06:24:35Z")
I use logstash to consume logs from kafka and insert them to es. The topic has 80 partitions so I have 80 logstash instances to consume the topic. Kafka input of each logstash is configured with one thread and all of l… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=825) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=827)