# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1038 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1039 --- ## [Mapping conflict2!](https://discuss.elastic.co/t/mapping-conflict2/74642)
**Author:** [@jonasv](https://discuss.elastic.co/u/jonasv)\ **Replies:** 2\ **Last updated:** [February 10, 2017, 10:41pm UTC](https://discuss.elastic.co/t/mapping-conflict2/74642 "2017-02-10T22:41:29Z")
Hey Guys, I'm trying to make tile map working in Kibana, but currently it doesn't work. When I add index to visualize it in Kibana I get this: Mapping conflict! A field is defined as several types (string, integer, etc… --- ## [Help developing a ruby plugin](https://discuss.elastic.co/t/help-developing-a-ruby-plugin/73451)
**Author:** [@gmcgilli](https://discuss.elastic.co/u/gmcgilli)\ **Replies:** 3\ **Last updated:** [February 10, 2017, 7:37pm UTC](https://discuss.elastic.co/t/help-developing-a-ruby-plugin/73451 "2017-02-10T19:37:07Z")
Can anyone guide me towards a really good sample output plugin with the latest version? Everything online I've seen appears old with a different structure. Likely a newbie mistake but appreciate the help thanks in adv… --- ## [Bug in 5.2.0: --logstash.data command line option ignored](https://discuss.elastic.co/t/bug-in-5-2-0-logstash-data-command-line-option-ignored/74662)
**Author:** [@LaszloHont](https://discuss.elastic.co/u/LaszloHont)\ **Replies:** 4\ **Last updated:** [February 10, 2017, 7:16pm UTC](https://discuss.elastic.co/t/bug-in-5-2-0-logstash-data-command-line-option-ignored/74662 "2017-02-10T19:16:11Z")
logstash isn't parsing the --path.data option on the command line. Bug? -bash-4.1$ /opt/logstash-LATEST/bin/logstash --path.data /opt/logstash-data/data/ ERROR StatusLogger No log4j2 configuration file found. Using de… --- ## [Adding a type field dynamically for logs](https://discuss.elastic.co/t/adding-a-type-field-dynamically-for-logs/73883)
**Author:** [@Maria\_Delarosa](https://discuss.elastic.co/u/Maria_Delarosa)\ **Replies:** 11\ **Last updated:** [February 10, 2017, 5:59pm UTC](https://discuss.elastic.co/t/adding-a-type-field-dynamically-for-logs/73883 "2017-02-10T17:59:43Z")
Logstash is gathering logs from multiple directories. Each directory represents logs from an specific application. I am using the type to indicate the type of logs. However, I have many applications and adding each one w… --- ## [Best way to send complex logs from NodeJS](https://discuss.elastic.co/t/best-way-to-send-complex-logs-from-nodejs/74670)
**Author:** [@Simon\_Bruneaud](https://discuss.elastic.co/u/Simon_Bruneaud)\ **Replies:** 0\ **Last updated:** [February 10, 2017, 2:35pm UTC](https://discuss.elastic.co/t/best-way-to-send-complex-logs-from-nodejs/74670 "2017-02-10T14:35:28Z")
Hey, I have a NodeJS application, and I would like to send logs to our ELK server (in a docker-container). I read a lot about Logstash pipeline, and a lot of articles about how to send logs. But I can't figure out wha… --- ## [Version error with both Java 7 and 8](https://discuss.elastic.co/t/version-error-with-both-java-7-and-8/72775)
**Author:** [@bence.takacs](https://discuss.elastic.co/u/bence.takacs)\ **Replies:** 2\ **Last updated:** [January 30, 2017, 8:44am UTC](https://discuss.elastic.co/t/version-error-with-both-java-7-and-8/72775 "2017-01-30T08:44:14Z")
Unsupported major.minor version 52.0 after installing logstash: yum install logstash (from repo 'https://artifacts.elastic.co/GPG-KEY-elasticsearch') installing java 8 edit startup options: JAVACMD="${JAVA8\_HOME}/bin… --- ## [Logstash configuration to receive from winlogbeat and filebeat](https://discuss.elastic.co/t/logstash-configuration-to-receive-from-winlogbeat-and-filebeat/74636)
**Author:** [@ArjunSISA](https://discuss.elastic.co/u/ArjunSISA)\ **Replies:** 2\ **Last updated:** [February 10, 2017, 1:19pm UTC](https://discuss.elastic.co/t/logstash-configuration-to-receive-from-winlogbeat-and-filebeat/74636 "2017-02-10T13:19:35Z")
hi everyone, I have recently installed filebeat and winlogbeat to forward the logs to logstash. can u tel me how to configure logstash to receive logs from both filebeat and winlogbeat ? --- ## [Csv output failing in logstash 5.1.2](https://discuss.elastic.co/t/csv-output-failing-in-logstash-5-1-2/73751)
**Author:** [@data](https://discuss.elastic.co/u/data)\ **Replies:** 2\ **Last updated:** [February 10, 2017, 12:44pm UTC](https://discuss.elastic.co/t/csv-output-failing-in-logstash-5-1-2/73751 "2017-02-10T12:44:41Z")
Facing issues with csv output in logstash 5.1.2 output { csv { fields =\> \["time", "level", "fun", "data", "thread"\] path =\> "/data/outputcsv.csv" } } generates output as sequence of strings - %{host} %{message}%{ho… --- ## [Date pattern wrong, grok parse failure](https://discuss.elastic.co/t/date-pattern-wrong-grok-parse-failure/74622)
**Author:** [@elasticheart](https://discuss.elastic.co/u/elasticheart)\ **Replies:** 1\ **Last updated:** [February 10, 2017, 12:08pm UTC](https://discuss.elastic.co/t/date-pattern-wrong-grok-parse-failure/74622 "2017-02-10T12:08:47Z")
Hi, I have a log shipped through Filebeat with date like Fri Nov 11 11:00:39 GMT 2016. I would like to add this to a timestamp variable in Logstash using match filter like; (?\%{DAY} %{MONTH} %{MONTHDAY} %{… --- ## [Multiple logstash filters with grok and mutate](https://discuss.elastic.co/t/multiple-logstash-filters-with-grok-and-mutate/74643)
**Author:** [@jstar](https://discuss.elastic.co/u/jstar)\ **Replies:** 0\ **Last updated:** [February 10, 2017, 11:28am UTC](https://discuss.elastic.co/t/multiple-logstash-filters-with-grok-and-mutate/74643 "2017-02-10T11:28:31Z")
Hi everybody, I working on a logstash filter that contains both grok and mutate. The idea is to create two fields field1 and field2 with grok and remove some information from a field message Here is my configuration g… --- ## [SQL connection exception continuing in Logstash](https://discuss.elastic.co/t/sql-connection-exception-continuing-in-logstash/74641)
**Author:** [@elasticheart](https://discuss.elastic.co/u/elasticheart)\ **Replies:** 0\ **Last updated:** [February 10, 2017, 11:12am UTC](https://discuss.elastic.co/t/sql-connection-exception-continuing-in-logstash/74641 "2017-02-10T11:12:24Z")
Hi, I am using jdbc input in Logstash, to fetch log entries from two different postgres tables. The queries are scheduled o run every hour, which fetched new nows only. Today, I am getting no entries in my elasticsearc… --- ## [How can use Avro serialize codec when output message to Kafka?](https://discuss.elastic.co/t/how-can-use-avro-serialize-codec-when-output-message-to-kafka/74018)
**Author:** [@elkan1788](https://discuss.elastic.co/u/elkan1788)\ **Replies:** 2\ **Last updated:** [February 10, 2017, 9:59am UTC](https://discuss.elastic.co/t/how-can-use-avro-serialize-codec-when-output-message-to-kafka/74018 "2017-02-10T09:59:40Z")
Hi guys. There had meet a problem when use Avro framework serialize codec for output message to Kafka. The Logstash configure file setting like below: input { file { path =\> "/data/tmp/\*.log" start… --- ## [Logstash-input-google\_pubsub install (version conflict) on logstash 5.2](https://discuss.elastic.co/t/logstash-input-google-pubsub-install-version-conflict-on-logstash-5-2/74625)
**Author:** [@mostafa\_marji](https://discuss.elastic.co/u/mostafa_marji)\ **Replies:** 0\ **Last updated:** [February 10, 2017, 9:14am UTC](https://discuss.elastic.co/t/logstash-input-google-pubsub-install-version-conflict-on-logstash-5-2/74625 "2017-02-10T09:14:50Z")
Validating logstash-input-google\_pubsub Installing logstash-input-google\_pubsub Plugin version conflict, aborting ERROR: Installation Aborted, message: Bundler could not find compatible versions for gem "logstash-core": … --- ## [String to json](https://discuss.elastic.co/t/string-to-json/74517)
**Author:** [@Oleksii\_Mazilov](https://discuss.elastic.co/u/Oleksii_Mazilov)\ **Replies:** 1\ **Last updated:** [February 10, 2017, 8:47am UTC](https://discuss.elastic.co/t/string-to-json/74517 "2017-02-10T08:47:06Z")
Guys, need your help. I have field in message: "additional\_parameters":"error:Null,status:success" How to convert this field to json: "additional\_parameters":"{"error": "Null","status": "success"}" --- ## [Grok parse failure in Logstash](https://discuss.elastic.co/t/grok-parse-failure-in-logstash/74601)
**Author:** [@elasticheart](https://discuss.elastic.co/u/elasticheart)\ **Replies:** 1\ **Last updated:** [February 10, 2017, 8:22am UTC](https://discuss.elastic.co/t/grok-parse-failure-in-logstash/74601 "2017-02-10T08:22:00Z")
I am using filebeat to push files to kafka and then consume it using Logstash. Below is the filebeat output. { "@timestamp": "2017-02-10T05:07:06.895Z", "beat": { "hostname": "mypc", "name": "mybeat", "… --- ## [Logstash CSV output giving wrong format](https://discuss.elastic.co/t/logstash-csv-output-giving-wrong-format/74413)
**Author:** [@Mielzus](https://discuss.elastic.co/u/Mielzus)\ **Replies:** 1\ **Last updated:** [February 10, 2017, 5:15am UTC](https://discuss.elastic.co/t/logstash-csv-output-giving-wrong-format/74413 "2017-02-10T05:15:17Z")
I am using Logstash to parse a file containing single line JSON data and output it in a CSV formatted file. Instead of outputting the data as nice separated values it is giving me single line data using timestamp, host,… --- ## [Log4j2 plugin installation error](https://discuss.elastic.co/t/log4j2-plugin-installation-error/74574)
**Author:** [@akhilesh.tomar05](https://discuss.elastic.co/u/akhilesh.tomar05)\ **Replies:** 0\ **Last updated:** [February 9, 2017, 9:57pm UTC](https://discuss.elastic.co/t/log4j2-plugin-installation-error/74574 "2017-02-09T21:57:27Z")
I get this error when I try to install log4j2 plugin. --- ## [Best way to prevent add\_fields with %{} when field does not exist](https://discuss.elastic.co/t/best-way-to-prevent-add-fields-with-when-field-does-not-exist/74444)
**Author:** [@djhart](https://discuss.elastic.co/u/djhart)\ **Replies:** 2\ **Last updated:** [February 9, 2017, 9:33pm UTC](https://discuss.elastic.co/t/best-way-to-prevent-add-fields-with-when-field-does-not-exist/74444 "2017-02-09T21:33:42Z")
I have a lot of csv fields that i'm using a csv filter to parse and sometimes the fields are blank. Right after parsing them, I assign the results to new fields using mutate's add\_field. Here is a sample csv { sepa… --- ## [Netflow w/GeoIP always results in \_grokparsefailure](https://discuss.elastic.co/t/netflow-w-geoip-always-results-in--grokparsefailure/74559)
**Author:** [@AJSchroeder\_DS](https://discuss.elastic.co/u/AJSchroeder_DS)\ **Replies:** 0\ **Last updated:** [February 9, 2017, 8:15pm UTC](https://discuss.elastic.co/t/netflow-w-geoip-always-results-in--grokparsefailure/74559 "2017-02-09T20:15:18Z")
Hello, I'm pretty new to ELK, but I'm trying to get ELK stood up to collect Netflow data from our network devices. I'm also using the GeoIP location abilities and no matter what the messages that show up in elasticsear… --- ## [How can i convert text column in sql to array type field in elasticsearch using Logstash?](https://discuss.elastic.co/t/how-can-i-convert-text-column-in-sql-to-array-type-field-in-elasticsearch-using-logstash/74505)
**Author:** [@Adinarayana\_Murthy](https://discuss.elastic.co/u/Adinarayana_Murthy)\ **Replies:** 2\ **Last updated:** [February 9, 2017, 8:26pm UTC](https://discuss.elastic.co/t/how-can-i-convert-text-column-in-sql-to-array-type-field-in-elasticsearch-using-logstash/74505 "2017-02-09T20:26:20Z")
Hi I want to convert the SQL column that contains text data to array of strings before storing into the elastics server. eg: Data in SQL with column name as "tags" tags : "Monday Today lastday… --- ## [Anyone using the logstash-output-iothub plugin?](https://discuss.elastic.co/t/anyone-using-the-logstash-output-iothub-plugin/74535)
**Author:** [@Kenfucius](https://discuss.elastic.co/u/Kenfucius)\ **Replies:** 1\ **Last updated:** [February 9, 2017, 7:19pm UTC](https://discuss.elastic.co/t/anyone-using-the-logstash-output-iothub-plugin/74535 "2017-02-09T19:19:55Z")
I started using this plug in: https://github.com/tac0x2a/logstash-output-iothub/blob/master/README.md The documentation says this: "This plugin provide sending messages to Microsoft Azure IoTHub." I've gone as f… --- ## [No data being discovered in Kibana front end](https://discuss.elastic.co/t/no-data-being-discovered-in-kibana-front-end/74131)
**Author:** [@gperez234](https://discuss.elastic.co/u/gperez234)\ **Replies:** 1\ **Last updated:** [February 9, 2017, 4:48pm UTC](https://discuss.elastic.co/t/no-data-being-discovered-in-kibana-front-end/74131 "2017-02-09T16:48:28Z")
I am installing the entire ELK stack and have been following your getting started guide. My issue is that I am not seeing logs in Kibana, so I tried combing through logs to find out why. I seem to be getting errors bet… --- ## [Filter not working when something has a period in it's name](https://discuss.elastic.co/t/filter-not-working-when-something-has-a-period-in-its-name/74523)
**Author:** [@Ben\_Hoffman](https://discuss.elastic.co/u/Ben_Hoffman)\ **Replies:** 3\ **Last updated:** [February 9, 2017, 4:14pm UTC](https://discuss.elastic.co/t/filter-not-working-when-something-has-a-period-in-its-name/74523 "2017-02-09T16:14:28Z")
So I need to write a filter that changes all the periods in field names to underscores. I amusing mutate, and I can do some things and not other things. For reference here is my current output in Kibana. See those fie… --- ## [How to have LS\_CONF\_DIR to have regular express](https://discuss.elastic.co/t/how-to-have-ls-conf-dir-to-have-regular-express/74309)
**Author:** [@macymin](https://discuss.elastic.co/u/macymin)\ **Replies:** 2\ **Last updated:** [February 9, 2017, 3:42pm UTC](https://discuss.elastic.co/t/how-to-have-ls-conf-dir-to-have-regular-express/74309 "2017-02-09T15:42:08Z")
hi, I Have 2 different logstash conf file need to be loaded together in one logstash startup file. 1st conf is located in /etc/logstash/automation/active/conf.d/indexer\_test.conf another is located in /etc/logstash/au… --- ## [Sending logstash data to multiple elastic instances/clusters](https://discuss.elastic.co/t/sending-logstash-data-to-multiple-elastic-instances-clusters/54482)
**Author:** [@zoplex](https://discuss.elastic.co/u/zoplex)\ **Replies:** 11\ **Last updated:** [February 9, 2017, 3:03pm UTC](https://discuss.elastic.co/t/sending-logstash-data-to-multiple-elastic-instances-clusters/54482 "2017-02-09T15:03:43Z")
Is it possible to send same data records/events from Logstash output section to two different elastic targets - one 2-node cluster and another separate elastic instance ? Second target would be for testing - so that same… --- ## [Centralised logging for Apache](https://discuss.elastic.co/t/centralised-logging-for-apache/73825)
**Author:** [@vineets928](https://discuss.elastic.co/u/vineets928)\ **Replies:** 9\ **Last updated:** [February 9, 2017, 2:36pm UTC](https://discuss.elastic.co/t/centralised-logging-for-apache/73825 "2017-02-09T14:36:52Z")
Hi, I am new in elk stack. Please someone help me. We have 4 application running on apache server. Client don't want to use filebeat to push log individually to logstash as this will affect the performance of the appli… --- ## [Mutate remove\_field not working for JSON field](https://discuss.elastic.co/t/mutate-remove-field-not-working-for-json-field/52482)
**Author:** [@Mick\_Mahoney](https://discuss.elastic.co/u/Mick_Mahoney)\ **Replies:** 6\ **Last updated:** [February 9, 2017, 2:02pm UTC](https://discuss.elastic.co/t/mutate-remove-field-not-working-for-json-field/52482 "2017-02-09T14:02:49Z")
Appreciate it if someone can spot the problem with this. Trying to remove the \[beat\]\[name\] field. I'm not getting any errors but the field is not getting removed. Can anyone spot why please ? JSON input { "@time… --- ## [S3 Input plugin fails to start](https://discuss.elastic.co/t/s3-input-plugin-fails-to-start/74512)
**Author:** [@Tonni\_Hult](https://discuss.elastic.co/u/Tonni_Hult)\ **Replies:** 0\ **Last updated:** [February 9, 2017, 1:51pm UTC](https://discuss.elastic.co/t/s3-input-plugin-fails-to-start/74512 "2017-02-09T13:51:49Z")
Hi I'm trying to get the S3 input plugin to work but it just fails saying it has an unrecoverable error. Hope that any of you recognize the error and can point me in the right direction because my googling is coming up s… --- ## [Logstash plugin installed but not found](https://discuss.elastic.co/t/logstash-plugin-installed-but-not-found/24369)
**Author:** [@whyapenny](https://discuss.elastic.co/u/whyapenny)\ **Replies:** 14\ **Last updated:** [February 9, 2017, 1:46pm UTC](https://discuss.elastic.co/t/logstash-plugin-installed-but-not-found/24369 "2017-02-09T13:46:35Z")
I have installed a logstash plugin logstash-input-jms. I build the .gem file and used the bin/plugin install command. It shows it installed successfully, yet when i run bin/plugin list, it is not listed. I tried with … --- ## [10-20% message loss when using Stomp Output](https://discuss.elastic.co/t/10-20-message-loss-when-using-stomp-output/74501)
**Author:** [@Mr.EoniX](https://discuss.elastic.co/u/Mr.EoniX)\ **Replies:** 0\ **Last updated:** [February 9, 2017, 1:02pm UTC](https://discuss.elastic.co/t/10-20-message-loss-when-using-stomp-output/74501 "2017-02-09T13:02:32Z")
Hi there, We are experiencing 10-20% message drop when sending messages from Logstash 5.2 to a Universal Messaging Queue with the STOMP protocol. (Also verified with Logstash 2.4) Config to reproduce: input { … [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1037) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1039)