# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1144 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1145 --- ## [Logstash reading new lines issues](https://discuss.elastic.co/t/logstash-reading-new-lines-issues/52163)
**Author:** [@Kennedy\_Kan1](https://discuss.elastic.co/u/Kennedy_Kan1)\ **Replies:** 4\ **Last updated:** [June 9, 2016, 5:34am UTC](https://discuss.elastic.co/t/logstash-reading-new-lines-issues/52163 "2016-06-09T05:34:22Z")
I have written a conf file in logstash to read the jms log and the problems are I could not break the records into newlines fore filtering. Here is the raw data ####\ \<\> \<1442678412960\> \<809000\>… --- ## [Logstash Kafka input failed to parse JSON](https://discuss.elastic.co/t/logstash-kafka-input-failed-to-parse-json/51620)
**Author:** [@Hongwei\_Liu](https://discuss.elastic.co/u/Hongwei_Liu)\ **Replies:** 1\ **Last updated:** [June 9, 2016, 3:32am UTC](https://discuss.elastic.co/t/logstash-kafka-input-failed-to-parse-json/51620 "2016-06-09T03:32:28Z")
hi experts, i am using logstash to read data from kafka cluster and output to elasticsearch. but i encountered this error - in the message it contains hex characters, so failed to parse the json. {:timestamp=\>"2016-0… --- ## [Logstash not creating correct index for filebeat and packet beat](https://discuss.elastic.co/t/logstash-not-creating-correct-index-for-filebeat-and-packet-beat/52147)
**Author:** [@shubham\_deodia](https://discuss.elastic.co/u/shubham_deodia)\ **Replies:** 1\ **Last updated:** [June 9, 2016, 2:32am UTC](https://discuss.elastic.co/t/logstash-not-creating-correct-index-for-filebeat-and-packet-beat/52147 "2016-06-09T02:32:15Z")
I have set up my elk stack like this. I am trying to ship logs and top data through topbeat and filebeat with custom index name. Although, Logstash is not creating any index for the data i am passing with custom index… --- ## [JDBC input scheduling question](https://discuss.elastic.co/t/jdbc-input-scheduling-question/52268)
**Author:** [@daq](https://discuss.elastic.co/u/daq)\ **Replies:** 0\ **Last updated:** [June 9, 2016, 2:25am UTC](https://discuss.elastic.co/t/jdbc-input-scheduling-question/52268 "2016-06-09T02:25:26Z")
I added a jdbc input to my config files: input { jdbc { jdbc\_driver\_library =\> "/usr/share/java/postgresql94-jdbc.jar" jdbc\_driver\_class =\> "org.postgresql.Driver" jdbc\_connection\_string =\> "jdbc:postgres… --- ## [Grok patterns](https://discuss.elastic.co/t/grok-patterns/52199)
**Author:** [@Piyush\_Paritosh](https://discuss.elastic.co/u/Piyush_Paritosh)\ **Replies:** 1\ **Last updated:** [June 9, 2016, 12:25am UTC](https://discuss.elastic.co/t/grok-patterns/52199 "2016-06-09T00:25:58Z")
This is my line from from the logs: 12345|xyz|123456789||19-SEP-11 11.59.55.000000 PM|10-AUG-12 05.07.21.000000 AM|xyzabc|blueberry|Address|{"fields"=\>{"address1"=\>"pinkdor", "address2"=\>"bluedor", "address3"=\>"", "cit… --- ## [How to get client ip address](https://discuss.elastic.co/t/how-to-get-client-ip-address/52111)
**Author:** [@Sai\_Birada](https://discuss.elastic.co/u/Sai_Birada)\ **Replies:** 1\ **Last updated:** [June 9, 2016, 12:07am UTC](https://discuss.elastic.co/t/how-to-get-client-ip-address/52111 "2016-06-09T00:07:05Z")
I had configured filebeat on client machines and logstash on central server. I want to log client ip as a field, so that i can filter the logs based on thier host ips, filebeat is logging hostname, but i need to log host… --- ## [Logstash + Get DateTime from filename](https://discuss.elastic.co/t/logstash-get-datetime-from-filename/49808)
**Author:** [@sarthak2016](https://discuss.elastic.co/u/sarthak2016)\ **Replies:** 5\ **Last updated:** [June 8, 2016, 10:08pm UTC](https://discuss.elastic.co/t/logstash-get-datetime-from-filename/49808 "2016-06-08T22:08:39Z")
Hello All, I am facing a issue to fetch the DateTime from filename. Input File name : ServerInfo\_04022016 1204.csv Expected output to Fetch DateTime : 04/02/2016 12:04 input { file { path =\> "C:\\logstash-2.3.1\\CS… --- ## [Logstash not sending data to elasticsearch when run as daemon](https://discuss.elastic.co/t/logstash-not-sending-data-to-elasticsearch-when-run-as-daemon/52245)
**Author:** [@djerkin](https://discuss.elastic.co/u/djerkin)\ **Replies:** 2\ **Last updated:** [June 8, 2016, 9:12pm UTC](https://discuss.elastic.co/t/logstash-not-sending-data-to-elasticsearch-when-run-as-daemon/52245 "2016-06-08T21:12:27Z")
Hi! Running into an issue where if I run ./logstash -f manually everything is fine. Reads a log file, logs show up in ES. When starting the service, however, it starts normally and no errors are seen, but no data is p… --- ## [JDBC and tracking\_column(s) questions](https://discuss.elastic.co/t/jdbc-and-tracking-column-s-questions/51997)
**Author:** [@ron.west](https://discuss.elastic.co/u/ron.west)\ **Replies:** 1\ **Last updated:** [June 8, 2016, 6:49pm UTC](https://discuss.elastic.co/t/jdbc-and-tracking-column-s-questions/51997 "2016-06-08T18:49:09Z")
I have 2 questions about the jdbc and tracking columns that my google foo isn't coming up with a good answer for. https://www.elastic.co/guide/en/logstash/current/plugins-inputs-jdbc.html (my source server is a sql s… --- ## [Logstash filter not able to split](https://discuss.elastic.co/t/logstash-filter-not-able-to-split/52205)
**Author:** [@jagan](https://discuss.elastic.co/u/jagan)\ **Replies:** 3\ **Last updated:** [June 8, 2016, 6:33pm UTC](https://discuss.elastic.co/t/logstash-filter-not-able-to-split/52205 "2016-06-08T18:33:33Z")
Hi Gurus, please help me out . I am trying to split the nested fields in log files but not able to do .Could you please help me out . He is the sample file: Mon May 30 00:30:26 PDT 2016 kthr memory page disk faults c… --- ## [JSON and syslog in LogStash — how to handle them both?](https://discuss.elastic.co/t/json-and-syslog-in-logstash-how-to-handle-them-both/52172)
**Author:** [@jojo](https://discuss.elastic.co/u/jojo)\ **Replies:** 1\ **Last updated:** [June 8, 2016, 6:32pm UTC](https://discuss.elastic.co/t/json-and-syslog-in-logstash-how-to-handle-them-both/52172 "2016-06-08T18:32:17Z")
I send 2 different types of data to Logstash from filebeat: json and syslog. Previously it was only json and it worked fine but now I've added syslog also. I want to display them both in Kibana in separate dashboards… --- ## [How i can use logstash to convert json data into a csv file?](https://discuss.elastic.co/t/how-i-can-use-logstash-to-convert-json-data-into-a-csv-file/52226)
**Author:** [@abrkreddy](https://discuss.elastic.co/u/abrkreddy)\ **Replies:** 1\ **Last updated:** [June 8, 2016, 6:26pm UTC](https://discuss.elastic.co/t/how-i-can-use-logstash-to-convert-json-data-into-a-csv-file/52226 "2016-06-08T18:26:33Z")
strong text --- ## [Logstash performance degradation after upgrade](https://discuss.elastic.co/t/logstash-performance-degradation-after-upgrade/51817)
**Author:** [@dkota](https://discuss.elastic.co/u/dkota)\ **Replies:** 2\ **Last updated:** [June 8, 2016, 6:09pm UTC](https://discuss.elastic.co/t/logstash-performance-degradation-after-upgrade/51817 "2016-06-08T18:09:03Z")
we were getting 60-80K indexing rate in our production cluster, after upgrading the cluster (from v 2.2.2) we had this go down to 12-14K. Elasticsearch 2.3.1 Logstash 2.3.1 Marvel 2.3.1 Shield … --- ## [Optional fields in grok pattern are not working](https://discuss.elastic.co/t/optional-fields-in-grok-pattern-are-not-working/51854)
**Author:** [@punu31090](https://discuss.elastic.co/u/punu31090)\ **Replies:** 1\ **Last updated:** [June 8, 2016, 6:07pm UTC](https://discuss.elastic.co/t/optional-fields-in-grok-pattern-are-not-working/51854 "2016-06-08T18:07:28Z")
Following are my log lines: Timestamp: 27-05-2016 13:25:17 Title:Customer.GetCustomer() Message: Searched Customers for number : 123 InnerException: An error has occurred StackTrace: An error has occurred Machine: PQR … --- ## [Webhdfs error : Plugin not defined in namespace, checking for plugin file](https://discuss.elastic.co/t/webhdfs-error-plugin-not-defined-in-namespace-checking-for-plugin-file/52236)
**Author:** [@jay224](https://discuss.elastic.co/u/jay224)\ **Replies:** 0\ **Last updated:** [June 8, 2016, 4:47pm UTC](https://discuss.elastic.co/t/webhdfs-error-plugin-not-defined-in-namespace-checking-for-plugin-file/52236 "2016-06-08T16:47:07Z")
I am trying to output to webhdfs my output config is: output { webhdfs { path =\> 'filepath' single\_file\_per\_thread =\> true host =\> 'hostname' port =\> '50070' co… --- ## [Logstash Map Paarsing error status 400](https://discuss.elastic.co/t/logstash-map-paarsing-error-status-400/52045)
**Author:** [@Kennedy\_Kan1](https://discuss.elastic.co/u/Kennedy_Kan1)\ **Replies:** 3\ **Last updated:** [June 8, 2016, 1:20pm UTC](https://discuss.elastic.co/t/logstash-map-paarsing-error-status-400/52045 "2016-06-08T13:20:15Z")
When I am trying to use logstash to read through a configuration file, I come up with map parsing error. :response=\>{"index"=\>{"\_index"=\>"logstash-2016.06.07", "\_type"=\>"txt", "\_id"=\>nil, "status"=\>400, "error"=\>{"ty… --- ## [Running logstash as a deamon](https://discuss.elastic.co/t/running-logstash-as-a-deamon/51754)
**Author:** [@mke](https://discuss.elastic.co/u/mke)\ **Replies:** 8\ **Last updated:** [June 8, 2016, 1:48pm UTC](https://discuss.elastic.co/t/running-logstash-as-a-deamon/51754 "2016-06-08T13:48:05Z")
I try to start logstash as a deamon but for some reason it does not seem to read my logstash.conf which I placed in /etc/logstash/conf.d/ What could be the reason? When I run it like sudo bin/logstash -f ./logstash.co… --- ## [Logstash 2.2.3 randomly stops forwarding logs to Elastic Search Node](https://discuss.elastic.co/t/logstash-2-2-3-randomly-stops-forwarding-logs-to-elastic-search-node/52078)
**Author:** [@Jamin\_Becker](https://discuss.elastic.co/u/Jamin_Becker)\ **Replies:** 3\ **Last updated:** [June 8, 2016, 1:19pm UTC](https://discuss.elastic.co/t/logstash-2-2-3-randomly-stops-forwarding-logs-to-elastic-search-node/52078 "2016-06-08T13:19:14Z")
Hi All, I have set up an cluster with 1 logstash collector and 1 elastic search node. I have my logstash listening on TCP/UDP syslog port 514, and forwarding all logs to my elastic-search node. Currently, my elastic-se… --- ## [Adding ingestion time in logs upto nanoseconds for more precision](https://discuss.elastic.co/t/adding-ingestion-time-in-logs-upto-nanoseconds-for-more-precision/52048)
**Author:** [@lucifer](https://discuss.elastic.co/u/lucifer)\ **Replies:** 2\ **Last updated:** [June 8, 2016, 12:26pm UTC](https://discuss.elastic.co/t/adding-ingestion-time-in-logs-upto-nanoseconds-for-more-precision/52048 "2016-06-08T12:26:10Z")
Can I Add ingestion time in logs upto nanoseconds for more precision by specifying it somewhere in the configuration file??? --- ## [Logstash import failed](https://discuss.elastic.co/t/logstash-import-failed/51569)
**Author:** [@danivu](https://discuss.elastic.co/u/danivu)\ **Replies:** 13\ **Last updated:** [June 8, 2016, 10:54am UTC](https://discuss.elastic.co/t/logstash-import-failed/51569 "2016-06-08T10:54:37Z")
Hi, I made a successful import data into ELK, logstash, elastic search kibana, after several days i rerun import i get error: io/console not supported; tty will not be manipulated Install the multipart-post gem. Loa… --- ## [Separate part of log in Logstash](https://discuss.elastic.co/t/separate-part-of-log-in-logstash/52064)
**Author:** [@John\_Newman](https://discuss.elastic.co/u/John_Newman)\ **Replies:** 2\ **Last updated:** [June 8, 2016, 10:02am UTC](https://discuss.elastic.co/t/separate-part-of-log-in-logstash/52064 "2016-06-08T10:02:41Z")
Hello, I have problem with config grok plugin. I'm beginner and I don't know yet how it all works. This is part of logstash conf: filter { grok{ match =\> { "message" =\> "%{TIMESTAMP\_ISO8601:formattedDate}.\* … --- ## [How to tune Logstash to Elasticsearch shipping](https://discuss.elastic.co/t/how-to-tune-logstash-to-elasticsearch-shipping/51333)
**Author:** [@danygor](https://discuss.elastic.co/u/danygor)\ **Replies:** 4\ **Last updated:** [June 8, 2016, 9:39am UTC](https://discuss.elastic.co/t/how-to-tune-logstash-to-elasticsearch-shipping/51333 "2016-06-08T09:39:24Z")
Hi all, I am writing because I need your help in tuning my LS to ES communication. What I've done: I set up a testbed to measure my Logstash performance, the input consists of 4 large log files and my LS instance can… --- ## [\[Solved\] Requirement to Install Logstash](https://discuss.elastic.co/t/solved-requirement-to-install-logstash/52181)
**Author:** [@Anuprita\_Wayangankar](https://discuss.elastic.co/u/Anuprita_Wayangankar)\ **Replies:** 4\ **Last updated:** [June 8, 2016, 9:12am UTC](https://discuss.elastic.co/t/solved-requirement-to-install-logstash/52181 "2016-06-08T09:12:35Z")
Hello, I want to work with Logstash. I have Windows 7 and also .Net 2012. Can anybody guide me about the installation ? Thanks & Regards. --- ## [Parsing .db file in Logstash](https://discuss.elastic.co/t/parsing-db-file-in-logstash/52087)
**Author:** [@anonymus123](https://discuss.elastic.co/u/anonymus123)\ **Replies:** 3\ **Last updated:** [June 8, 2016, 6:39am UTC](https://discuss.elastic.co/t/parsing-db-file-in-logstash/52087 "2016-06-08T06:39:25Z")
Hello, Im looking for a way in which i can forward .db file to logstash and parse the events in this file. What is the best way for doing so? The logs are aggregated in one .db file on a windows workstation by some appli… --- ## [How to replace %5C with backslash](https://discuss.elastic.co/t/how-to-replace-5c-with-backslash/51866)
**Author:** [@eprst](https://discuss.elastic.co/u/eprst)\ **Replies:** 1\ **Last updated:** [June 8, 2016, 5:52am UTC](https://discuss.elastic.co/t/how-to-replace-5c-with-backslash/51866 "2016-06-08T05:52:14Z")
i need to replace %5C in field some fields with backslash, the problem is that i can't do it with mutate gsub. i've tried replacement string : '\\','\\\\','\\\\\\','\\\\\\\\','\\\\\\\\\\' nothing is working. is there any solution for … --- ## [Couple of issues with logstash input file :(](https://discuss.elastic.co/t/couple-of-issues-with-logstash-input-file/49585)
**Author:** [@jamesl](https://discuss.elastic.co/u/jamesl)\ **Replies:** 9\ **Last updated:** [June 8, 2016, 5:00am UTC](https://discuss.elastic.co/t/couple-of-issues-with-logstash-input-file/49585 "2016-06-08T05:00:22Z")
Hello. Got a couple of questions relating to how the input file really works. I would like to brand (change type) for certain inputs, but the system doesn't seem to be picking up my 'type' comment. I also am having troub… --- ## [Beats input: The circuit breaker has detected a slowdown or stall in the pipeline](https://discuss.elastic.co/t/beats-input-the-circuit-breaker-has-detected-a-slowdown-or-stall-in-the-pipeline/52062)
**Author:** [@Sumantha\_Kannantha](https://discuss.elastic.co/u/Sumantha_Kannantha)\ **Replies:** 1\ **Last updated:** [June 8, 2016, 1:54am UTC](https://discuss.elastic.co/t/beats-input-the-circuit-breaker-has-detected-a-slowdown-or-stall-in-the-pipeline/52062 "2016-06-08T01:54:50Z")
we are getting below error in logstash log log: Beats input: The circuit breaker has detected a slowdown or stall in the pipeline, the input is closing the current connection and rejecting new connection until the pi… --- ## [Geoip filter logstash](https://discuss.elastic.co/t/geoip-filter-logstash/52128)
**Author:** [@bishaka](https://discuss.elastic.co/u/bishaka)\ **Replies:** 11\ **Last updated:** [June 7, 2016, 11:41pm UTC](https://discuss.elastic.co/t/geoip-filter-logstash/52128 "2016-06-07T23:41:06Z")
Hi, I am trying to show location of users in the Kibana Tile Map. How do i do it using the Geoip filter? Can someone point me to some good resource? I have ELK stack setup. --- ## [Can't change timezone when convert time from unixtime](https://discuss.elastic.co/t/cant-change-timezone-when-convert-time-from-unixtime/52068)
**Author:** [@neolithik](https://discuss.elastic.co/u/neolithik)\ **Replies:** 3\ **Last updated:** [June 7, 2016, 11:22pm UTC](https://discuss.elastic.co/t/cant-change-timezone-when-convert-time-from-unixtime/52068 "2016-06-07T23:22:12Z")
hi i have same string in log : 1465301456.359 127.0.0.1 127.0.0.1 0.036 GET /cashier/manipulation/searchDBy as you see date in hear in unix format so i'm use date filter date { match =\> \["msec", "YYYY.MM.dd HH:mm:ss"… --- ## [Snmptrap input](https://discuss.elastic.co/t/snmptrap-input/51578)
**Author:** [@Virkel](https://discuss.elastic.co/u/Virkel)\ **Replies:** 4\ **Last updated:** [June 7, 2016, 10:20pm UTC](https://discuss.elastic.co/t/snmptrap-input/51578 "2016-06-07T22:20:08Z")
Hello, I am currently attempting to configure snmptrap input for my elk server on a VM, I'm new to elk and only been working with it for about two weeks now. My configuration is below: input { snmptrap { port =\> 162 … [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1143) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1145)