# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1118 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1119 --- ## [Invalid config logstash file](https://discuss.elastic.co/t/invalid-config-logstash-file/57339)
**Author:** [@Fred\_Dionisio](https://discuss.elastic.co/u/Fred_Dionisio)\ **Replies:** 1\ **Last updated:** [August 6, 2016, 8:54am UTC](https://discuss.elastic.co/t/invalid-config-logstash-file/57339 "2016-08-06T08:54:51Z")
I'm following a basic tutorial since i'm new in this, and i've the following config file: input { file { path =\> "D:\\elastic\\logstash\\data\\data.csv" start\_position =\> "beginning" } } filter { csv … --- ## [Elastic search not ingesting data](https://discuss.elastic.co/t/elastic-search-not-ingesting-data/57197)
**Author:** [@James\_Kazie](https://discuss.elastic.co/u/James_Kazie)\ **Replies:** 3\ **Last updated:** [August 5, 2016, 10:36pm UTC](https://discuss.elastic.co/t/elastic-search-not-ingesting-data/57197 "2016-08-05T22:36:08Z")
I suspect i am doing something blatantly wrong. Logstash appears to be going through my data properly and showing the 5000 data points being sent to elastic search. However when i look up the index in elastic search it s… --- ## [Using the Split filter with a dynamic field](https://discuss.elastic.co/t/using-the-split-filter-with-a-dynamic-field/57247)
**Author:** [@3ygun](https://discuss.elastic.co/u/3ygun)\ **Replies:** 0\ **Last updated:** [August 4, 2016, 4:04pm UTC](https://discuss.elastic.co/t/using-the-split-filter-with-a-dynamic-field/57247 "2016-08-04T16:04:35Z")
Question Can you use the split filter with a dynamic field ("%{some\_field}")? Problem The documentation doesn't say (Split filter) and the intuitive split { field =\> "%{test}" } doesn't seem to work. Test case below: T… --- ## [Translate plugin doesn't work](https://discuss.elastic.co/t/translate-plugin-doesnt-work/54034)
**Author:** [@Alexey\_Khudyakov](https://discuss.elastic.co/u/Alexey_Khudyakov)\ **Replies:** 5\ **Last updated:** [August 5, 2016, 7:06pm UTC](https://discuss.elastic.co/t/translate-plugin-doesnt-work/54034 "2016-08-05T19:06:17Z")
Dear community, I tried to add descriptions of network protocols but unfortunately it doesn't work :frowning: Here is the part of logstash's config file #Protocol friendly naming translate { field =\> "\[… --- ## [5.0alpha logstash-output-email](https://discuss.elastic.co/t/5-0alpha-logstash-output-email/57013)
**Author:** [@DreadPirateRob](https://discuss.elastic.co/u/DreadPirateRob)\ **Replies:** 12\ **Last updated:** [August 5, 2016, 6:38pm UTC](https://discuss.elastic.co/t/5-0alpha-logstash-output-email/57013 "2016-08-05T18:38:32Z")
Trying to install the logstash-output-email plugin, but not working. Can I get a lil help with trying to figure out what's up? \[root@logstash logstash\]# bin/logstash-plugin install logstash-output-email Validatin… --- ## [Inserting huge xml into elasticsearch](https://discuss.elastic.co/t/inserting-huge-xml-into-elasticsearch/57383)
**Author:** [@Thanos](https://discuss.elastic.co/u/Thanos)\ **Replies:** 0\ **Last updated:** [August 5, 2016, 6:11pm UTC](https://discuss.elastic.co/t/inserting-huge-xml-into-elasticsearch/57383 "2016-08-05T18:11:23Z")
Hello, i'm trying to process the .xml file provided by the MSR conference 2015 (http://2015.msrconf.org/challenge\_data/). Being new to elasticsearch/logstash i really could use some advice about setting up the .conf fil… --- ## [Logstash location](https://discuss.elastic.co/t/logstash-location/56925)
**Author:** [@wldbs0508](https://discuss.elastic.co/u/wldbs0508)\ **Replies:** 4\ **Last updated:** [August 5, 2016, 5:42pm UTC](https://discuss.elastic.co/t/logstash-location/56925 "2016-08-05T17:42:47Z")
Hi, I would like to set up our Elasticsearch Cluster on some hosts and it consists of 4~5 hosts. I also want to use Logstash to put data into Elasticsearch cluster. My website will run Logstash when customer requests i… --- ## [Http\_poller install](https://discuss.elastic.co/t/http-poller-install/57162)
**Author:** [@bishaka](https://discuss.elastic.co/u/bishaka)\ **Replies:** 2\ **Last updated:** [August 5, 2016, 4:48pm UTC](https://discuss.elastic.co/t/http-poller-install/57162 "2016-08-05T16:48:09Z")
In order to install and use the http\_poller plugin on logstash..is there anything else I need to do other than the following? bin/plugin install logstash-input-http\_poller Because after I install it and try to see w… --- ## [Unable to extract fields from Message Using GROK Filter](https://discuss.elastic.co/t/unable-to-extract-fields-from-message-using-grok-filter/57366)
**Author:** [@Usman\_Ismail](https://discuss.elastic.co/u/Usman_Ismail)\ **Replies:** 1\ **Last updated:** [August 5, 2016, 4:44pm UTC](https://discuss.elastic.co/t/unable-to-extract-fields-from-message-using-grok-filter/57366 "2016-08-05T16:44:08Z")
I have a log file collected using a FileBeat that looks like: 10:21:38.014 DEBUG org.apache.zookeeper.ClientCnxn \[main-SendThread(zookeeper3.piranhakik.com:2181)\] - Got ping response for sessionid: 0x355bca8e39f0c51 aft… --- ## [Unexpected \_grokparsefailure while parsing square brackets with custom pattern](https://discuss.elastic.co/t/unexpected--grokparsefailure-while-parsing-square-brackets-with-custom-pattern/57375)
**Author:** [@apelsina](https://discuss.elastic.co/u/apelsina)\ **Replies:** 0\ **Last updated:** [August 5, 2016, 4:40pm UTC](https://discuss.elastic.co/t/unexpected--grokparsefailure-while-parsing-square-brackets-with-custom-pattern/57375 "2016-08-05T16:40:50Z")
I try to parse some simple log and get \_grokparsefailure which I can't figure out. Here's the log record example: \[DEBUG\]\[444\]\[Main\] It contains loglevel, pid and thread name. I configured rather simple pattern for t… --- ## [Logstash consuming a lot of CPU?](https://discuss.elastic.co/t/logstash-consuming-a-lot-of-cpu/57372)
**Author:** [@niraj\_kumar](https://discuss.elastic.co/u/niraj_kumar)\ **Replies:** 0\ **Last updated:** [August 5, 2016, 4:36pm UTC](https://discuss.elastic.co/t/logstash-consuming-a-lot-of-cpu/57372 "2016-08-05T16:36:19Z")
We are using logstash version 1.5.6 with s3 input & elasticsearch output plugin been used. The CPU is spiking up to 40%. Is this normal for logstash? \`top - 16:34:09 up 16:27, 1 user, load average: 2.66, 2.41, 3.08 T… --- ## [Logstash 5.0.0-alpha4 installation and configuration problem](https://discuss.elastic.co/t/logstash-5-0-0-alpha4-installation-and-configuration-problem/56865)
**Author:** [@abu.sayeed](https://discuss.elastic.co/u/abu.sayeed)\ **Replies:** 6\ **Last updated:** [August 5, 2016, 4:19pm UTC](https://discuss.elastic.co/t/logstash-5-0-0-alpha4-installation-and-configuration-problem/56865 "2016-08-05T16:19:51Z")
System OS: Ubuntu 14.04.4 LTS java -version: java version "1.8.0\_101" Java(TM) SE Runtime Environment (build 1.8.0\_101-b13) Java HotSpot(TM) 64-Bit Server VM (build 25.101-b13, mixed mode) I install Logstash just f… --- ## [How to filter the json files which are in combined and coming to logstash unable to read](https://discuss.elastic.co/t/how-to-filter-the-json-files-which-are-in-combined-and-coming-to-logstash-unable-to-read/57365)
**Author:** [@Vivek\_Raj](https://discuss.elastic.co/u/Vivek_Raj)\ **Replies:** 1\ **Last updated:** [August 5, 2016, 3:51pm UTC](https://discuss.elastic.co/t/how-to-filter-the-json-files-which-are-in-combined-and-coming-to-logstash-unable-to-read/57365 "2016-08-05T15:51:29Z")
Hi everyone, We are trying to pull the data from kafka queue using logstash the data in Json formate. Its pulling the data but all json file are combine and coming it looks so clumsy. How to fix that issue. Any idea wh… --- ## [Logstash blocs backend application server](https://discuss.elastic.co/t/logstash-blocs-backend-application-server/57244)
**Author:** [@yevgenl](https://discuss.elastic.co/u/yevgenl)\ **Replies:** 4\ **Last updated:** [August 5, 2016, 2:02pm UTC](https://discuss.elastic.co/t/logstash-blocs-backend-application-server/57244 "2016-08-05T14:02:03Z")
Server configuration: Glassfish -\> Log4j socket appender -\> Logstash log4j input -\> Logstash elasticsearch output Everything works as expected but if elasticsearch goes down my application server become blocked. Nobo… --- ## [Rsyslog and Logstash production setup](https://discuss.elastic.co/t/rsyslog-and-logstash-production-setup/57205)
**Author:** [@davideagle](https://discuss.elastic.co/u/davideagle)\ **Replies:** 3\ **Last updated:** [August 5, 2016, 12:00pm UTC](https://discuss.elastic.co/t/rsyslog-and-logstash-production-setup/57205 "2016-08-05T12:00:58Z")
I am new to ELK stack and I was hoping someone could answer a short question regarding architecture. I'm exploring the possibility of shipping syslog from multiple servers (to start with in the tens) to elasticsearch. wo… --- ## [Logstash Filter: How To Add Tag For “If X Event Happens More Than Y Times In Z Seconds”](https://discuss.elastic.co/t/logstash-filter-how-to-add-tag-for-if-x-event-happens-more-than-y-times-in-z-seconds/57286)
**Author:** [@joshuasmith](https://discuss.elastic.co/u/joshuasmith)\ **Replies:** 2\ **Last updated:** [August 5, 2016, 10:44am UTC](https://discuss.elastic.co/t/logstash-filter-how-to-add-tag-for-if-x-event-happens-more-than-y-times-in-z-seconds/57286 "2016-08-05T10:44:41Z")
I am new to Logstash and am trying to write a filter that will add a tag when event X is seen more than Y times in Z seconds. For instance, in the following example log, I want to apply a tag to traffic from the IP add… --- ## [Unable to find plugins when unit testing using rspec on windows 10](https://discuss.elastic.co/t/unable-to-find-plugins-when-unit-testing-using-rspec-on-windows-10/57230)
**Author:** [@cwerge](https://discuss.elastic.co/u/cwerge)\ **Replies:** 1\ **Last updated:** [August 5, 2016, 10:40am UTC](https://discuss.elastic.co/t/unable-to-find-plugins-when-unit-testing-using-rspec-on-windows-10/57230 "2016-08-05T10:40:34Z")
Hi Using logstash 2.3.1, after having run the "logstash-plugin install --development", the gemfile is updated and i can find the plugins manually. The top of the ruby file containing the test contain the following l… --- ## [Elasticsearch stop indexing with custom timestamp](https://discuss.elastic.co/t/elasticsearch-stop-indexing-with-custom-timestamp/56710)
**Author:** [@fantaghost](https://discuss.elastic.co/u/fantaghost)\ **Replies:** 8\ **Last updated:** [August 5, 2016, 10:26am UTC](https://discuss.elastic.co/t/elasticsearch-stop-indexing-with-custom-timestamp/56710 "2016-08-05T10:26:18Z")
Hi All, I'm processing two different log type using elasticsearch (each log type is stored in different index). I have a problem processing the timestamp field only with one log type. I would like to create @timestamp st… --- ## [Unable to import from logstash to ES after upgrading to alpha release of xpack](https://discuss.elastic.co/t/unable-to-import-from-logstash-to-es-after-upgrading-to-alpha-release-of-xpack/57198)
**Author:** [@geethaRam](https://discuss.elastic.co/u/geethaRam)\ **Replies:** 3\ **Last updated:** [August 5, 2016, 7:19am UTC](https://discuss.elastic.co/t/unable-to-import-from-logstash-to-es-after-upgrading-to-alpha-release-of-xpack/57198 "2016-08-05T07:19:30Z")
I had standalone ES - 2.3.4 , Logstash l;atest and Kibana - 4.5.3. - All of this worked with no issues in my local. I upgraded to Alpha version 5 for ES and Kibana and also installed xpack, After this, Iam not able to … --- ## [Multiple metrics in Logstash filter](https://discuss.elastic.co/t/multiple-metrics-in-logstash-filter/57307)
**Author:** [@vishwanathh](https://discuss.elastic.co/u/vishwanathh)\ **Replies:** 1\ **Last updated:** [August 5, 2016, 6:33am UTC](https://discuss.elastic.co/t/multiple-metrics-in-logstash-filter/57307 "2016-08-05T06:33:09Z")
Hi, It it possible to provide different clear intervals for two different metrics. For example, I want to do sum of "Number" columns every 5 seconds and max of "Number" columns after 60 seconds only? Thanks, Vishwa… --- ## [Define Parent-Child Relationship in Logstash](https://discuss.elastic.co/t/define-parent-child-relationship-in-logstash/51467)
**Author:** [@Alaska](https://discuss.elastic.co/u/Alaska)\ **Replies:** 2\ **Last updated:** [August 5, 2016, 4:24am UTC](https://discuss.elastic.co/t/define-parent-child-relationship-in-logstash/51467 "2016-08-05T04:24:00Z")
Hi, According to https://github.com/logstash-plugins/logstash-output-elasticsearch/issues/129, this seems possible to do now. I'm trying to index two different sources (2 sql tables), and then correlate a parent-chil… --- ## [Logstash Elasticsearch output host with no port](https://discuss.elastic.co/t/logstash-elasticsearch-output-host-with-no-port/57294)
**Author:** [@wldbs0508](https://discuss.elastic.co/u/wldbs0508)\ **Replies:** 0\ **Last updated:** [August 5, 2016, 1:43am UTC](https://discuss.elastic.co/t/logstash-elasticsearch-output-host-with-no-port/57294 "2016-08-05T01:43:35Z")
Hi, I would like to use Logstash to index data into Elasticsearch. I have to use VIP to connect to Elasticsearch cluster. My VIP already contains default port number. For example, my host name is "myhost.com" and I gen… --- ## [How to automatically start logstash](https://discuss.elastic.co/t/how-to-automatically-start-logstash/57276)
**Author:** [@kartikvelastic](https://discuss.elastic.co/u/kartikvelastic)\ **Replies:** 1\ **Last updated:** [August 5, 2016, 12:42am UTC](https://discuss.elastic.co/t/how-to-automatically-start-logstash/57276 "2016-08-05T00:42:01Z")
/opt/logstash/bin/logstash -f logstash-apache.conf is not equivalent to systemctl start logstash even though the only file in /etc/logstash/conf.d is logstash-apache.conf Please advise. --- ## [Having issue on running a simple config file](https://discuss.elastic.co/t/having-issue-on-running-a-simple-config-file/57238)
**Author:** [@Reghuraman\_S](https://discuss.elastic.co/u/Reghuraman_S)\ **Replies:** 1\ **Last updated:** [August 5, 2016, 12:39am UTC](https://discuss.elastic.co/t/having-issue-on-running-a-simple-config-file/57238 "2016-08-05T00:39:20Z")
In Windows 7, aommand I used to run logstash; ..\\logstash-2.3.2\\bin\\logstash -f "new-apache.conf" --debug Config File: new-apache.conf; input { file { path =\> "server.log" start\_position =\> "beginni… --- ## [Automatic fields for a JSON input?](https://discuss.elastic.co/t/automatic-fields-for-a-json-input/57126)
**Author:** [@Zesix](https://discuss.elastic.co/u/Zesix)\ **Replies:** 1\ **Last updated:** [August 5, 2016, 12:22am UTC](https://discuss.elastic.co/t/automatic-fields-for-a-json-input/57126 "2016-08-05T00:22:25Z")
I'm feeding JSON files to logstash from filebeat. Logstash is set to output JSON to ElasticSearch. However, my JSON key-value pairs all show up as a giant clump within the "message" field. I believe ElasticSearch is de… --- ## [How to find the day of the week from timestamp](https://discuss.elastic.co/t/how-to-find-the-day-of-the-week-from-timestamp/57105)
**Author:** [@rajkamalkool6](https://discuss.elastic.co/u/rajkamalkool6)\ **Replies:** 1\ **Last updated:** [August 5, 2016, 12:18am UTC](https://discuss.elastic.co/t/how-to-find-the-day-of-the-week-from-timestamp/57105 "2016-08-05T00:18:04Z")
Hi, I have a timestamp 2015-11-01 21:45:25,296 like I mentioned above. is it possible to extract the the day of the week(Mon, Tue,etc) using any regular expression or grok pattern. Thanks in advance --- ## [Building a Syslog Infrastructure](https://discuss.elastic.co/t/building-a-syslog-infrastructure/57089)
**Author:** [@DanMa](https://discuss.elastic.co/u/DanMa)\ **Replies:** 1\ **Last updated:** [August 5, 2016, 12:17am UTC](https://discuss.elastic.co/t/building-a-syslog-infrastructure/57089 "2016-08-05T00:17:06Z")
Hello together, we want to build a scalable Syslog Infrastructure with the ELK Stack. We have some branch Offices and one head Office. We would like only collect the syslog data in the branch offices and send them… --- ## [Rename logstash sincedb file](https://discuss.elastic.co/t/rename-logstash-sincedb-file/57233)
**Author:** [@nabbaraju](https://discuss.elastic.co/u/nabbaraju)\ **Replies:** 1\ **Last updated:** [August 5, 2016, 12:14am UTC](https://discuss.elastic.co/t/rename-logstash-sincedb-file/57233 "2016-08-05T00:14:44Z")
I have a logstash that runs with sincedb\_path set to "sincedb\_test" and I want to rename the file as "test"; will it cause all logs to be re-processed again? If I rename the sincedb and run logstash again will it ignore… --- ## [Convert into IP adrress](https://discuss.elastic.co/t/convert-into-ip-adrress/57284)
**Author:** [@bishaka](https://discuss.elastic.co/u/bishaka)\ **Replies:** 1\ **Last updated:** [August 5, 2016, 12:11am UTC](https://discuss.elastic.co/t/convert-into-ip-adrress/57284 "2016-08-05T00:11:48Z")
Hi, Does anyone know how I can convert the following into an IP address in logstash? This is basically the address of a caller. I am trying to display the callers location on Kibana Map. --- ## [Logstash is pulling the records from DB multiple times](https://discuss.elastic.co/t/logstash-is-pulling-the-records-from-db-multiple-times/56779)
**Author:** [@sivapuvvada](https://discuss.elastic.co/u/sivapuvvada)\ **Replies:** 3\ **Last updated:** [August 4, 2016, 8:39pm UTC](https://discuss.elastic.co/t/logstash-is-pulling-the-records-from-db-multiple-times/56779 "2016-08-04T20:39:52Z")
I am using logstash to pull the records from DB2 . but one record is pulling multilpe times .. How can i debug the issue .. It is pulling the same record with fraction of milli seconds difference [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1117) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1119)