# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1003 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1004 --- ## [Logstash Watch Folder for files changes](https://discuss.elastic.co/t/logstash-watch-folder-for-files-changes/81770)
**Author:** [@kab](https://discuss.elastic.co/u/kab)\ **Replies:** 2\ **Last updated:** [April 10, 2017, 9:14am UTC](https://discuss.elastic.co/t/logstash-watch-folder-for-files-changes/81770 "2017-04-10T09:14:38Z")
Hi, Logstash v1.5.4 With this release version of logstash , any ideas of best way to scan folder to see if any files exists and push the file names to elastic search ? the selection pattern should be based on the c… --- ## [Handling elastic mapping errors when using logstash elastic output plugin](https://discuss.elastic.co/t/handling-elastic-mapping-errors-when-using-logstash-elastic-output-plugin/81777)
**Author:** [@Sharbel\_Cherian\_Konn](https://discuss.elastic.co/u/Sharbel_Cherian_Konn)\ **Replies:** 0\ **Last updated:** [April 10, 2017, 8:55am UTC](https://discuss.elastic.co/t/handling-elastic-mapping-errors-when-using-logstash-elastic-output-plugin/81777 "2017-04-10T08:55:42Z")
Hi We have built and deployed an ELK Solution for one of our partners suing Elastic Cloud and a Logstash Cluster. The logs are generated from many different apps developed by different teams and there have been instanc… --- ## [\[logstash 5.3\]\[plugin output elasticsearch\] Using env variable in conf file](https://discuss.elastic.co/t/logstash-5-3-plugin-output-elasticsearch-using-env-variable-in-conf-file/81472)
**Author:** [@fares](https://discuss.elastic.co/u/fares)\ **Replies:** 4\ **Last updated:** [April 10, 2017, 8:42am UTC](https://discuss.elastic.co/t/logstash-5-3-plugin-output-elasticsearch-using-env-variable-in-conf-file/81472 "2017-04-10T08:42:41Z")
Hello, I'm trying to use env var in my conf file elasticsearch{ ... hosts =\> ${ES\_HOST} } My var contains host:port type value, ie localhost:9200 Got this error ERROR logstash.agent - Cannot load an invalid config… --- ## [How to measure performance of Logstash](https://discuss.elastic.co/t/how-to-measure-performance-of-logstash/81327)
**Author:** [@parrot](https://discuss.elastic.co/u/parrot)\ **Replies:** 2\ **Last updated:** [April 10, 2017, 7:03am UTC](https://discuss.elastic.co/t/how-to-measure-performance-of-logstash/81327 "2017-04-10T07:03:09Z")
Hi, i need a tool to measure the performance of Logstash - i want to try different configurations and choose the one with the best performance (minimum time until i get the output). What is the best way to check it? --- ## [NameError: SSLConnectionSocket Factory not found in packages](https://discuss.elastic.co/t/nameerror-sslconnectionsocket-factory-not-found-in-packages/81748)
**Author:** [@Girish\_K\_T](https://discuss.elastic.co/u/Girish_K_T)\ **Replies:** 0\ **Last updated:** [April 10, 2017, 6:42am UTC](https://discuss.elastic.co/t/nameerror-sslconnectionsocket-factory-not-found-in-packages/81748 "2017-04-10T06:42:08Z")
Hi, I am new to Logstash. I was trying the command logstash -f logstash.conf and I'm getting the following error... Settings: Default pipeline workers: 4 ←\[31mPipeline aborted due to error {:exception=\>#\ **Author:** [@jbl](https://discuss.elastic.co/u/jbl)\ **Replies:** 1\ **Last updated:** [April 10, 2017, 5:31am UTC](https://discuss.elastic.co/t/exec-plugin/81717 "2017-04-10T05:31:22Z") Hello, \*\*\* goal is to process logstash data in realtime \*\*\* I am new to logstash, and i tried the exec plugin in order to process the output with php script : output { exec { command =\> "php /opt/admin/s… --- ## [Multiline filter/Codec question](https://discuss.elastic.co/t/multiline-filter-codec-question/81549)
**Author:** [@maxw2016](https://discuss.elastic.co/u/maxw2016)\ **Replies:** 5\ **Last updated:** [April 10, 2017, 5:26am UTC](https://discuss.elastic.co/t/multiline-filter-codec-question/81549 "2017-04-10T05:26:27Z")
Hello, I'm working on exim email logs and I'm stuck with this codec. here is an example from the log: 2017-04-03 02:19:58 H=(localhost) \[117.0.54.236\] F=test@earatt.net rejected RCPT smityrd@example.com: Rejected messa… --- ## [Log stash xml parse](https://discuss.elastic.co/t/log-stash-xml-parse/81649)
**Author:** [@anshul\_pandey](https://discuss.elastic.co/u/anshul_pandey)\ **Replies:** 1\ **Last updated:** [April 10, 2017, 5:18am UTC](https://discuss.elastic.co/t/log-stash-xml-parse/81649 "2017-04-10T05:18:51Z")
I would like some recommendation on how to parse a given xml document at logstash if i am looking for value of specific field which can appear in different fields in multiple xmls doc , for ex product ,sales, inventory,x… --- ## [Mismatch in custom date throws \_\_dateparsefailure logstash](https://discuss.elastic.co/t/mismatch-in-custom-date-throws---dateparsefailure-logstash/81733)
**Author:** [@ankitc7](https://discuss.elastic.co/u/ankitc7)\ **Replies:** 0\ **Last updated:** [April 10, 2017, 2:01am UTC](https://discuss.elastic.co/t/mismatch-in-custom-date-throws---dateparsefailure-logstash/81733 "2017-04-10T02:01:21Z")
The input is comma separated values: "2010-08-19","09:12:55","56095675" I created the custom date\_time field which appears to right format 2010-08-19;09:12:55 but not matching. filter { grok { match =\> { "messag… --- ## [Filebeat vs logstash for handling multiline](https://discuss.elastic.co/t/filebeat-vs-logstash-for-handling-multiline/81713)
**Author:** [@santhosh244](https://discuss.elastic.co/u/santhosh244)\ **Replies:** 6\ **Last updated:** [April 9, 2017, 5:18pm UTC](https://discuss.elastic.co/t/filebeat-vs-logstash-for-handling-multiline/81713 "2017-04-09T17:18:56Z")
Hi, We have 40 instances sending logs parallely to centralized logstash via filebeat. We have tried multiline codec with logstash and filebeat both as well. Both seems to be working. But we see few issues with logs… --- ## [Zimbra logstash index pattern](https://discuss.elastic.co/t/zimbra-logstash-index-pattern/81667)
**Author:** [@mhamed](https://discuss.elastic.co/u/mhamed)\ **Replies:** 3\ **Last updated:** [April 9, 2017, 12:33am UTC](https://discuss.elastic.co/t/zimbra-logstash-index-pattern/81667 "2017-04-09T00:33:38Z")
Hi.. i need an index pattern for kibana to filter zimbra's log (mailbox.log ,audit.log, zimbra.log...) ..Thank you ^\_^ --- ## [Kibana 5.1.1 tile map (no results found)](https://discuss.elastic.co/t/kibana-5-1-1-tile-map-no-results-found/81377)
**Author:** [@bishaka](https://discuss.elastic.co/u/bishaka)\ **Replies:** 1\ **Last updated:** [April 8, 2017, 11:35pm UTC](https://discuss.elastic.co/t/kibana-5-1-1-tile-map-no-results-found/81377 "2017-04-08T23:35:54Z")
Logstash: 5.1.1 Kibana : 5.1.1 Elasticsearch: 5.1.1 Hi, I have created a sample file with some lat and longs to visualize in Kibana 5. input { file { path =\> "sample.log" type =\> "ips" start\_position =\> "beginn… --- ## [How disable logstash Monitoring APIs](https://discuss.elastic.co/t/how-disable-logstash-monitoring-apis/81635)
**Author:** [@s.tkachenko](https://discuss.elastic.co/u/s.tkachenko)\ **Replies:** 1\ **Last updated:** [April 8, 2017, 11:13pm UTC](https://discuss.elastic.co/t/how-disable-logstash-monitoring-apis/81635 "2017-04-08T23:13:38Z")
Hello. How disable logstash Monitoring APIs? And disable started binding http.port (9600)? --- ## [Unable to output to BigQuery](https://discuss.elastic.co/t/unable-to-output-to-bigquery/81694)
**Author:** [@Filipe\_Ferminiano\_Ro](https://discuss.elastic.co/u/Filipe_Ferminiano_Ro)\ **Replies:** 0\ **Last updated:** [April 8, 2017, 11:13pm UTC](https://discuss.elastic.co/t/unable-to-output-to-bigquery/81694 "2017-04-08T23:13:05Z")
I'm trying to send logstash data to BigQuery with this plugin: https://www.elastic.co/guide/en/logstash/current/plugins-outputs-google\_bigquery.html#plugins-outputs-google\_bigquery-csv\_schema The files are being writ… --- ## [Event type for class variable](https://discuss.elastic.co/t/event-type-for-class-variable/81674)
**Author:** [@jonyang](https://discuss.elastic.co/u/jonyang)\ **Replies:** 0\ **Last updated:** [April 8, 2017, 8:40am UTC](https://discuss.elastic.co/t/event-type-for-class-variable/81674 "2017-04-08T08:40:35Z")
my log file data looks something like this: line 1: date time field; server name + other fields subsequent lines: ( details ) i used the above suggestions successfully saved class variables @@ldate, @@server from the fi… --- ## [Lost connection in logstash forwarder when addind patterns and grok filters](https://discuss.elastic.co/t/lost-connection-in-logstash-forwarder-when-addind-patterns-and-grok-filters/81657)
**Author:** [@aziz\_007](https://discuss.elastic.co/u/aziz_007)\ **Replies:** 2\ **Last updated:** [April 8, 2017, 12:14am UTC](https://discuss.elastic.co/t/lost-connection-in-logstash-forwarder-when-addind-patterns-and-grok-filters/81657 "2017-04-08T00:14:55Z")
i work with zimbra i have created a grok filter for mailbox.log and a pattern file i have put the filter file called zimbra\_mailbox.conf in /etc/logstash/conf.d/ and i have created a directory called patterns in /etc/l… --- ## [Keeping global variables in LS?!](https://discuss.elastic.co/t/keeping-global-variables-in-ls/39908)
**Author:** [@jjdepaul](https://discuss.elastic.co/u/jjdepaul)\ **Replies:** 22\ **Last updated:** [April 7, 2017, 11:48pm UTC](https://discuss.elastic.co/t/keeping-global-variables-in-ls/39908 "2017-04-07T23:48:50Z")
Is there a way to keep variables at a global level?! Here is an excerpt from the log file that we are processing. As you can see there is data about CustomerAccountNumber, and ContractNumber. The trouble is that … --- ## [Target month name to @timestamp](https://discuss.elastic.co/t/target-month-name-to-timestamp/81621)
**Author:** [@Emilien](https://discuss.elastic.co/u/Emilien)\ **Replies:** 4\ **Last updated:** [April 7, 2017, 3:30pm UTC](https://discuss.elastic.co/t/target-month-name-to-timestamp/81621 "2017-04-07T15:30:22Z")
Hi everyone, I'm using logstash to manage my catalina logs. The issue I encounter is the fact that I have to type of timestamp in my logs. 2017-04-04 10:16:54,297 04-Apr-2017 10:16:54.443 Here my filter configuratio… --- ## [Logstash JMX sizing](https://discuss.elastic.co/t/logstash-jmx-sizing/54381)
**Author:** [@domisan](https://discuss.elastic.co/u/domisan)\ **Replies:** 5\ **Last updated:** [April 7, 2017, 3:27pm UTC](https://discuss.elastic.co/t/logstash-jmx-sizing/54381 "2017-04-07T15:27:26Z")
Hi, I've been using the Logstash JMX input succesfully for some time to monitor up to 10 Java JMX enabled processes with 1 Logstash process and nb\_thread set to 4 (default). However, I've recently configured a Logstash… --- ## [Logstash jmx input slow](https://discuss.elastic.co/t/logstash-jmx-input-slow/81531)
**Author:** [@k1ng87](https://discuss.elastic.co/u/k1ng87)\ **Replies:** 2\ **Last updated:** [April 7, 2017, 3:26pm UTC](https://discuss.elastic.co/t/logstash-jmx-input-slow/81531 "2017-04-07T15:26:23Z")
I'm using a the logstash jmx input pluggin to poll a specific mbean every second (return about 4000 messages every second). What I'm seeing right now, is that its not able to poll every second as it seems that there is s… --- ## [Cannot convert String into Integer](https://discuss.elastic.co/t/cannot-convert-string-into-integer/56588)
**Author:** [@hassanfarid](https://discuss.elastic.co/u/hassanfarid)\ **Replies:** 3\ **Last updated:** [April 7, 2017, 3:17pm UTC](https://discuss.elastic.co/t/cannot-convert-string-into-integer/56588 "2017-04-07T15:17:48Z")
I am using logstash 2.3.4 - the configuration worked for previous inputs in file, some input is now crashing it. How to find the error log line? Exception in pipelineworker, the pipeline stopped processing new events, … --- ## [Error registering Logstash plugin using Docker image](https://discuss.elastic.co/t/error-registering-logstash-plugin-using-docker-image/81633)
**Author:** [@Kevin\_Thorpe](https://discuss.elastic.co/u/Kevin_Thorpe)\ **Replies:** 0\ **Last updated:** [April 7, 2017, 3:16pm UTC](https://discuss.elastic.co/t/error-registering-logstash-plugin-using-docker-image/81633 "2017-04-07T15:16:32Z")
I'm trying to set up elk under Rancher/Docker and am having problems with logstash. Elasticsearch and Kibana are up and running from your images. I'm trying to run two logstash instances, a collector to stuff messages in… --- ## [Logstash is not create indexes](https://discuss.elastic.co/t/logstash-is-not-create-indexes/81568)
**Author:** [@igormarqs](https://discuss.elastic.co/u/igormarqs)\ **Replies:** 5\ **Last updated:** [April 7, 2017, 1:24pm UTC](https://discuss.elastic.co/t/logstash-is-not-create-indexes/81568 "2017-04-07T13:24:51Z")
hi, i have started with ELK but i have some error. when i start kibana and look for some index name or pattern, can't find any logstash-\* if i use option "Use event times to create index names \[DEPRECATED\]" with name "… --- ## [Logstash Kafka output: adds extra topics](https://discuss.elastic.co/t/logstash-kafka-output-adds-extra-topics/81618)
**Author:** [@JoaquinS](https://discuss.elastic.co/u/JoaquinS)\ **Replies:** 0\ **Last updated:** [April 7, 2017, 2:00pm UTC](https://discuss.elastic.co/t/logstash-kafka-output-adds-extra-topics/81618 "2017-04-07T14:00:51Z")
Hello, I have a config file with Kafka output like this: topic\_id =\> "%{product}-%{event\_type}" It work fine and index the log to the corresponding topic, but additionaly it creates a new topic called "%{product}" … --- ## [Not able to pull arabic data by using JDBC input](https://discuss.elastic.co/t/not-able-to-pull-arabic-data-by-using-jdbc-input/80902)
**Author:** [@Dan](https://discuss.elastic.co/u/Dan)\ **Replies:** 1\ **Last updated:** [April 7, 2017, 1:37pm UTC](https://discuss.elastic.co/t/not-able-to-pull-arabic-data-by-using-jdbc-input/80902 "2017-04-07T13:37:00Z")
I have stored arabic language in my my sql DB. I am using JDBC input and elasticsearch out put in logstash. But my jdbc input not bringing arabic language, Insted its pulling ????????(question Marks). Please help me … --- ## [Offline plugin installation problem](https://discuss.elastic.co/t/offline-plugin-installation-problem/81099)
**Author:** [@mevbas](https://discuss.elastic.co/u/mevbas)\ **Replies:** 1\ **Last updated:** [April 7, 2017, 1:16pm UTC](https://discuss.elastic.co/t/offline-plugin-installation-problem/81099 "2017-04-07T13:16:59Z")
Hi, I created logstash-offline-plugins zip file for email output plugin with Logstash 5.3.0 Zip file contains -logstash folder --dependencies ---mail-2.6.4.gem ---mime-types-2.6.2.gem --logstash-output-email-4.0.3.gem… --- ## [Convert escaped-json to json](https://discuss.elastic.co/t/convert-escaped-json-to-json/81387)
**Author:** [@ikirani](https://discuss.elastic.co/u/ikirani)\ **Replies:** 2\ **Last updated:** [April 7, 2017, 10:32am UTC](https://discuss.elastic.co/t/convert-escaped-json-to-json/81387 "2017-04-07T10:32:19Z")
Hi, One field in an incoming message is escaped-json-string. I need to convert it to JSON in logstash. What's the best way to do this? I've tried the JSON filter as below: json { source =\> … --- ## [Logstash csv output plugins giving problem](https://discuss.elastic.co/t/logstash-csv-output-plugins-giving-problem/80969)
**Author:** [@Nikparab](https://discuss.elastic.co/u/Nikparab)\ **Replies:** 11\ **Last updated:** [April 7, 2017, 10:31am UTC](https://discuss.elastic.co/t/logstash-csv-output-plugins-giving-problem/80969 "2017-04-07T10:31:12Z")
I write output-csv.conf file and run it under bin folder of logstash. input { elasticsearch { hosts =\> \["localhost:9200"\] index =\> "mongo3" query =\> '{ "query": { "bool":{ "must":{ … --- ## [How mapping data without separator](https://discuss.elastic.co/t/how-mapping-data-without-separator/81579)
**Author:** [@28tomino](https://discuss.elastic.co/u/28tomino)\ **Replies:** 1\ **Last updated:** [April 7, 2017, 7:35am UTC](https://discuss.elastic.co/t/how-mapping-data-without-separator/81579 "2017-04-07T07:35:59Z")
hi, can i mapping some data without separator ? date+time mac etc etc etc ... something like this: 2017-04-03T23:56:37+02:00 xx-xx-xx-xx-xx-xx portal.sh\[527\] seq=20515 msec=580 Trace Setup proxy: (null) for url: http:… --- ## [Git plugin problems](https://discuss.elastic.co/t/git-plugin-problems/81424)
**Author:** [@adriannuix](https://discuss.elastic.co/u/adriannuix)\ **Replies:** 1\ **Last updated:** [April 7, 2017, 7:04am UTC](https://discuss.elastic.co/t/git-plugin-problems/81424 "2017-04-07T07:04:18Z")
I'm trying to use the git plugin but I get: C:\\work\\jiras\\logstash\\ls522\>bin/logstash -f gitlogstash3.conf C:\\work\\development-environment\\Tools\\GnuWin32\\bin\\stty.exe: standard input: Function not implemented Could not… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1002) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1004)