# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=886 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 887 --- ## [Replacing escape characters](https://discuss.elastic.co/t/replacing-escape-characters/105294)
**Author:** [@GambitK](https://discuss.elastic.co/u/GambitK)\ **Replies:** 0\ **Last updated:** [October 25, 2017, 8:35pm UTC](https://discuss.elastic.co/t/replacing-escape-characters/105294 "2017-10-25T20:35:35Z")
Hello, I'm trying to replace escape characters. I have log lines where the source is escaping all '' and also '"', so it becomes '\\' and '"' when it comes to logstash. I've tried gsub =\> \[ "message", "\\"","""\], gsub =\> … --- ## [Logstash UDP plugin](https://discuss.elastic.co/t/logstash-udp-plugin/105273)
**Author:** [@maddie0726](https://discuss.elastic.co/u/maddie0726)\ **Replies:** 11\ **Last updated:** [October 25, 2017, 8:19pm UTC](https://discuss.elastic.co/t/logstash-udp-plugin/105273 "2017-10-25T20:19:53Z")
I have Logstash, Kibana and Elasticsearch version 5.6.2 running on Windows 2012 R2. I am trying to read in live data over a udp port and I am getting no data. Here is my input in my config file: input { udp { port =\> … --- ## [Json to grok pattern](https://discuss.elastic.co/t/json-to-grok-pattern/105242)
**Author:** [@mbvelo](https://discuss.elastic.co/u/mbvelo)\ **Replies:** 4\ **Last updated:** [October 25, 2017, 8:17pm UTC](https://discuss.elastic.co/t/json-to-grok-pattern/105242 "2017-10-25T20:17:56Z")
Please asisst, how do i convert a json object to Grok Pattern --- ## [File input to grok filter to elasticsearch output](https://discuss.elastic.co/t/file-input-to-grok-filter-to-elasticsearch-output/105126)
**Author:** [@cm47](https://discuss.elastic.co/u/cm47)\ **Replies:** 3\ **Last updated:** [October 25, 2017, 7:18pm UTC](https://discuss.elastic.co/t/file-input-to-grok-filter-to-elasticsearch-output/105126 "2017-10-25T19:18:06Z")
Hi, I am trying to configure logstash to read lines from a custom file. Logstash configuration file: input { file { path =\> "/home/centos/customlog/testfile.log" codec =\> line start\_position =\> "beginning" ignore\_… --- ## [How to merge JSON documents when ingesting?](https://discuss.elastic.co/t/how-to-merge-json-documents-when-ingesting/105096)
**Author:** [@colinphillips](https://discuss.elastic.co/u/colinphillips)\ **Replies:** 2\ **Last updated:** [October 25, 2017, 6:33pm UTC](https://discuss.elastic.co/t/how-to-merge-json-documents-when-ingesting/105096 "2017-10-25T18:33:12Z")
I'm new to logstash / beats and I'm wondering how best to go about transforming / merging two JSON documents. Here's an example of what I'm trying to do: Document 1: { "id": 1234, "things": \[ { "id": "1234", "lat… --- ## [Logs going to the wrong index](https://discuss.elastic.co/t/logs-going-to-the-wrong-index/104955)
**Author:** [@shroh](https://discuss.elastic.co/u/shroh)\ **Replies:** 2\ **Last updated:** [October 25, 2017, 3:20pm UTC](https://discuss.elastic.co/t/logs-going-to-the-wrong-index/104955 "2017-10-25T15:20:51Z")
Hi , I have three logstash config files 00\_inputs.conf, 01\_app1.conf, 02\_app2.conf. 01\_inputs.conf just defines the beats input and thats it, nothing more. 02\_app1.conf filter { if \[fields\]\[index\] == "app\_fischer" { … --- ## [Create column dynamically](https://discuss.elastic.co/t/create-column-dynamically/105238)
**Author:** [@uzncn](https://discuss.elastic.co/u/uzncn)\ **Replies:** 0\ **Last updated:** [October 25, 2017, 11:59am UTC](https://discuss.elastic.co/t/create-column-dynamically/105238 "2017-10-25T11:59:08Z")
Hi all, Is there any way to create column name inside column value alsa dynamically. --- ## [Error: Cannot create pipeline](https://discuss.elastic.co/t/error-cannot-create-pipeline/105255)
**Author:** [@kaochan](https://discuss.elastic.co/u/kaochan)\ **Replies:** 2\ **Last updated:** [October 25, 2017, 2:24pm UTC](https://discuss.elastic.co/t/error-cannot-create-pipeline/105255 "2017-10-25T14:24:00Z")
Hi, I'm new in ELK and i'm having a problem with logstash trying to filter my application log. This is my config file: input { file { path =\> "/var/log/application/application.log" start\_position =\> "beginning" ty… --- ## [Exception when trying to use mutate to convert an integer to boolean](https://discuss.elastic.co/t/exception-when-trying-to-use-mutate-to-convert-an-integer-to-boolean/105253)
**Author:** [@jdmcalee](https://discuss.elastic.co/u/jdmcalee)\ **Replies:** 1\ **Last updated:** [October 25, 2017, 2:08pm UTC](https://discuss.elastic.co/t/exception-when-trying-to-use-mutate-to-convert-an-integer-to-boolean/105253 "2017-10-25T14:08:40Z")
First off, I am using 6.0.0-rc1 in docker. I'm using the jdbc input plugin to fetch some data out of mysql, and I need to convert an integer (tinyint in the schema) to boolean to suit my desired index type. The values in… --- ## [Date can not parse date formated field](https://discuss.elastic.co/t/date-can-not-parse-date-formated-field/105218)
**Author:** [@stevie](https://discuss.elastic.co/u/stevie)\ **Replies:** 3\ **Last updated:** [October 25, 2017, 1:59pm UTC](https://discuss.elastic.co/t/date-can-not-parse-date-formated-field/105218 "2017-10-25T13:59:10Z")
Version: logstash 5.6.3 We have 2 date filters in our logstash config. The first parses the timestamp field we matched in our logs. We want to have always the same timestamp format so we use date to do this. Second da… --- ## [Different Log Types from different Beat Apps on different Servers into Kibana](https://discuss.elastic.co/t/different-log-types-from-different-beat-apps-on-different-servers-into-kibana/105225)
**Author:** [@Misanthrop](https://discuss.elastic.co/u/Misanthrop)\ **Replies:** 1\ **Last updated:** [October 25, 2017, 12:09pm UTC](https://discuss.elastic.co/t/different-log-types-from-different-beat-apps-on-different-servers-into-kibana/105225 "2017-10-25T12:09:36Z")
Hey all, I am really new and we are trying to implement the full stack onto our Server-Infrastructure. However I cannot find any example via google regarding the following request: How do you process multiple different… --- ## [How we can send updated mysql record to elastic using logstash](https://discuss.elastic.co/t/how-we-can-send-updated-mysql-record-to-elastic-using-logstash/105224)
**Author:** [@Shekhar\_Pandey](https://discuss.elastic.co/u/Shekhar_Pandey)\ **Replies:** 1\ **Last updated:** [October 25, 2017, 12:07pm UTC](https://discuss.elastic.co/t/how-we-can-send-updated-mysql-record-to-elastic-using-logstash/105224 "2017-10-25T12:07:39Z")
How we can chnage logstash config, so that when any updated record found in mysql database, logstash fetch only that. Is it possible. my config. file is like this... input { jdbc { jdbc\_driver\_library =\> "/root/mysql… --- ## [How to read XML logs](https://discuss.elastic.co/t/how-to-read-xml-logs/104664)
**Author:** [@abhisek](https://discuss.elastic.co/u/abhisek)\ **Replies:** 9\ **Last updated:** [October 25, 2017, 12:05pm UTC](https://discuss.elastic.co/t/how-to-read-xml-logs/104664 "2017-10-25T12:05:53Z")
Hi team, My log file is "|" separated but two fields contains XML data. please let me know how can i read this. --- ## [Problem with gzip\_lines codec](https://discuss.elastic.co/t/problem-with-gzip-lines-codec/105235)
**Author:** [@sbampa](https://discuss.elastic.co/u/sbampa)\ **Replies:** 0\ **Last updated:** [October 25, 2017, 11:49am UTC](https://discuss.elastic.co/t/problem-with-gzip-lines-codec/105235 "2017-10-25T11:49:33Z")
Hi all, i'm using the following configuration in logstash: file { type =\> "gzip" path =\> "/export/home/elastic/logstash-5.6.1/list.txt" start\_position =\> "beginning" codec =\> gzip\_lines add\_field =\> … --- ## [Python SocketHandler charset](https://discuss.elastic.co/t/python-sockethandler-charset/105197)
**Author:** [@a8j8i8t8](https://discuss.elastic.co/u/a8j8i8t8)\ **Replies:** 3\ **Last updated:** [October 25, 2017, 9:44am UTC](https://discuss.elastic.co/t/python-sockethandler-charset/105197 "2017-10-25T09:44:40Z")
Hello, I'm using Python SocketHandler to send logs over UDP port to Logstash. My Logstash input config is like below: - input { udp { port =\> 33333 type =\> "test" } } And my python logging config is as be… --- ## [Logstash timeout sending messagges to ES cluster](https://discuss.elastic.co/t/logstash-timeout-sending-messagges-to-es-cluster/105188)
**Author:** [@rschirin](https://discuss.elastic.co/u/rschirin)\ **Replies:** 3\ **Last updated:** [October 25, 2017, 9:12am UTC](https://discuss.elastic.co/t/logstash-timeout-sending-messagges-to-es-cluster/105188 "2017-10-25T09:12:42Z")
Hi, currently I have: 1 LS server 3 ES nodes (components of a cluster) I know that I can configure LS to send messages to different ES hosts (by config file). I also undestood that LS sends messages using round robin… --- ## [All output fail if one of many outputs fails](https://discuss.elastic.co/t/all-output-fail-if-one-of-many-outputs-fails/105199)
**Author:** [@fluency03](https://discuss.elastic.co/u/fluency03)\ **Replies:** 0\ **Last updated:** [October 25, 2017, 8:26am UTC](https://discuss.elastic.co/t/all-output-fail-if-one-of-many-outputs-fails/105199 "2017-10-25T08:26:51Z")
Logstash -\> TCP: tcp output exception, EOFError: End of file reached According to: https://github.com/logstash-plugins/logstash-output-tcp/issues/9, and https://github.com/logstash-plugins/logstash-output-tcp/issues/… --- ## [Using logstash to import csv files into elasticsearch](https://discuss.elastic.co/t/using-logstash-to-import-csv-files-into-elasticsearch/104114)
**Author:** [@alexus](https://discuss.elastic.co/u/alexus)\ **Replies:** 7\ **Last updated:** [October 25, 2017, 5:18am UTC](https://discuss.elastic.co/t/using-logstash-to-import-csv-files-into-elasticsearch/104114 "2017-10-25T05:18:04Z")
I'm using Running Logstash on Docker | Logstash Reference \[5.6\] | Elastic to import csv file into elasticsearch. following is content from my ./pipeline directory: input { file { path =\> "/csv/in/\*.csv" … --- ## [Detect logstash conf files inside a subdirectory](https://discuss.elastic.co/t/detect-logstash-conf-files-inside-a-subdirectory/105009)
**Author:** [@uLan](https://discuss.elastic.co/u/uLan)\ **Replies:** 3\ **Last updated:** [October 25, 2017, 4:12am UTC](https://discuss.elastic.co/t/detect-logstash-conf-files-inside-a-subdirectory/105009 "2017-10-25T04:12:30Z")
Is it possible for logstash to detect conf files in a subdirectory? For example it's configure right now to look in /etc/logstash/conf.d but my conf files are separated into sub directories e.g. /etc/logstash/conf.d/foo … --- ## [Logstash trying to replace geoip.country\_name](https://discuss.elastic.co/t/logstash-trying-to-replace-geoip-country-name/104812)
**Author:** [@jojo58](https://discuss.elastic.co/u/jojo58)\ **Replies:** 1\ **Last updated:** [October 25, 2017, 3:44am UTC](https://discuss.elastic.co/t/logstash-trying-to-replace-geoip-country-name/104812 "2017-10-25T03:44:00Z")
I noticed that the naming conventions between the kibana region map and the logstash geoip plugin aren't the same. And when i try to visualize my data i get errors from the region map telling me that it can't find "Repub… --- ## [What are the different ways that we can send log messages from log stash server to Graylog server](https://discuss.elastic.co/t/what-are-the-different-ways-that-we-can-send-log-messages-from-log-stash-server-to-graylog-server/104970)
**Author:** [@Umesh\_Dc](https://discuss.elastic.co/u/Umesh_Dc)\ **Replies:** 1\ **Last updated:** [October 25, 2017, 3:11am UTC](https://discuss.elastic.co/t/what-are-the-different-ways-that-we-can-send-log-messages-from-log-stash-server-to-graylog-server/104970 "2017-10-25T03:11:19Z")
Hi All, I am trying to send log messages on one of my application using logstash to Graylog server. I Found that we can use gelf output in order to send messages to Graylog server over UPD protocol. I want to send pl… --- ## [STOMP + SSL support?](https://discuss.elastic.co/t/stomp-ssl-support/105134)
**Author:** [@peterneorr](https://discuss.elastic.co/u/peterneorr)\ **Replies:** 0\ **Last updated:** [October 24, 2017, 10:48pm UTC](https://discuss.elastic.co/t/stomp-ssl-support/105134 "2017-10-24T22:48:47Z")
Hello all, I'm trying to find out if there's a way to use the STOMP output plugin with SSL. I see there is a quite old pull request on the github repo, but the docs don't mention anything about ssl options. Regards, … --- ## [Logstash Kafka Input Plugin Throws Errors missing class name (\`org.apache.kafka.clients.consumer.ConsumerConfig')](https://discuss.elastic.co/t/logstash-kafka-input-plugin-throws-errors-missing-class-name-org-apache-kafka-clients-consumer-consumerconfig/105128)
**Author:** [@Randall\_Svancara](https://discuss.elastic.co/u/Randall_Svancara)\ **Replies:** 3\ **Last updated:** [October 24, 2017, 10:22pm UTC](https://discuss.elastic.co/t/logstash-kafka-input-plugin-throws-errors-missing-class-name-org-apache-kafka-clients-consumer-consumerconfig/105128 "2017-10-24T22:22:27Z")
I would really like to use this input plug-in but in the time I have spent futzing around with it I could of written something in GoLang.....The problem is that no matter what I do the plugin does not load: Version: … --- ## [Getting Logstash X-Pack To Work](https://discuss.elastic.co/t/getting-logstash-x-pack-to-work/103095)
**Author:** [@hmmxp](https://discuss.elastic.co/u/hmmxp)\ **Replies:** 1\ **Last updated:** [October 24, 2017, 8:40pm UTC](https://discuss.elastic.co/t/getting-logstash-x-pack-to-work/103095 "2017-10-24T20:40:13Z")
Dear All, I am trying to get X-Pack Monitoring To Work or Logstash But Not Seeing Logstash Under Monitoring Tab In Kibana X-Pack Is Running Basic License I Already Disabled X-Pack Security Through xpack.security.enabl… --- ## [Using wildcard to update/remove fields inside logstash filter](https://discuss.elastic.co/t/using-wildcard-to-update-remove-fields-inside-logstash-filter/104460)
**Author:** [@bishplz](https://discuss.elastic.co/u/bishplz)\ **Replies:** 2\ **Last updated:** [October 20, 2017, 5:50pm UTC](https://discuss.elastic.co/t/using-wildcard-to-update-remove-fields-inside-logstash-filter/104460 "2017-10-20T17:50:18Z")
Hi I have a data structure like this. This is an stdout after parsing with XML filter. { "offset" =\> 6859148, "input\_type" =\> "log", "source" =\> "requests.log.6", "type" =\> "log", "datestamp" =\> "08-09-17 03:03:07… --- ## [Two GeoIP sources and target locations](https://discuss.elastic.co/t/two-geoip-sources-and-target-locations/104184)
**Author:** [@mwelninski](https://discuss.elastic.co/u/mwelninski)\ **Replies:** 0\ **Last updated:** [October 17, 2017, 6:08am UTC](https://discuss.elastic.co/t/two-geoip-sources-and-target-locations/104184 "2017-10-17T06:08:53Z")
Hello, let me first say that I appreciate any help that can be provided around this issue I have been fighting with. Background: I am loading in firewall logs into a 5.6 ELK Stack running on Windows. The information is… --- ## [I have the same problem](https://discuss.elastic.co/t/i-have-the-same-problem/105080)
**Author:** [@Anderson\_Sinche](https://discuss.elastic.co/u/Anderson_Sinche)\ **Replies:** 1\ **Last updated:** [October 24, 2017, 3:47pm UTC](https://discuss.elastic.co/t/i-have-the-same-problem/105080 "2017-10-24T15:47:34Z")
Continuing the discussion from Logstash: Error: mongodb.jdbc.MongoDriver not loaded. Are you sure you've included the correct jdbc driver in :jdbc\_driver\_library?: --- ## [Logstash.conf Not Working (Cannot Create Pipeline)](https://discuss.elastic.co/t/logstash-conf-not-working-cannot-create-pipeline/105078)
**Author:** [@Anshuman\_Kirty](https://discuss.elastic.co/u/Anshuman_Kirty)\ **Replies:** 1\ **Last updated:** [October 24, 2017, 2:34pm UTC](https://discuss.elastic.co/t/logstash-conf-not-working-cannot-create-pipeline/105078 "2017-10-24T14:34:51Z")
Logstash.conf input { jdbc { jdbc\_connection\_string =\> "jdbc:mysql://localhost:3306/testdb" jdbc\_user =\> "root" jdbc\_password ="explorer" jdbc\_dr… --- ## [Logstash is importing more data than supposed](https://discuss.elastic.co/t/logstash-is-importing-more-data-than-supposed/104920)
**Author:** [@afonso.ribeiro](https://discuss.elastic.co/u/afonso.ribeiro)\ **Replies:** 2\ **Last updated:** [October 24, 2017, 1:59pm UTC](https://discuss.elastic.co/t/logstash-is-importing-more-data-than-supposed/104920 "2017-10-24T13:59:18Z")
Hello guys, I'm currently importing data from csv through logstash to elasticsearch. I'm importing 26595 lines but logstash seems to be importing or more than 26595 or less lines. Is there any explanation for this? T… --- ## [Error while inserting bulk data in big size via logstash to elasticsearch](https://discuss.elastic.co/t/error-while-inserting-bulk-data-in-big-size-via-logstash-to-elasticsearch/105031)
**Author:** [@PinkiMondal](https://discuss.elastic.co/u/PinkiMondal)\ **Replies:** 1\ **Last updated:** [October 24, 2017, 1:43pm UTC](https://discuss.elastic.co/t/error-while-inserting-bulk-data-in-big-size-via-logstash-to-elasticsearch/105031 "2017-10-24T13:43:23Z")
Hi All I am using elasticsearch version 5.4.1 and I have 8 Million records combined all the indices. Now I have one index which contains 21 lakhs records and I want to insert more records in that Index First bulk reque… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=885) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=887)