# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=882 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 883 --- ## [Logstash 5.3.2 multiple conf files](https://discuss.elastic.co/t/logstash-5-3-2-multiple-conf-files/106021)
**Author:** [@teejayuu](https://discuss.elastic.co/u/teejayuu)\ **Replies:** 2\ **Last updated:** [November 1, 2017, 3:05pm UTC](https://discuss.elastic.co/t/logstash-5-3-2-multiple-conf-files/106021 "2017-11-01T15:05:47Z")
Good morning I have setup my ELK stack (5.3.2) to gather Exchange Message Tracking logs from my Exchange Cluster with a conf file in D:\\ELK\\logstash-5.3.2\\01-inputs.conf\\exchange\_msg\_trk.conf and this is working well. I… --- ## [Plugin to generate CEF output for ARCsight](https://discuss.elastic.co/t/plugin-to-generate-cef-output-for-arcsight/106016)
**Author:** [@oscario](https://discuss.elastic.co/u/oscario)\ **Replies:** 1\ **Last updated:** [November 1, 2017, 12:09pm UTC](https://discuss.elastic.co/t/plugin-to-generate-cef-output-for-arcsight/106016 "2017-11-01T12:09:55Z")
Hi, I see a lot of answers using the cef-codes plugin to parse CEF output from i.e. ARCsight. I need the other way around: generate CEF output (from syslog but probably also from other formats) to use as input for ARCs… --- ## [Logs parsing](https://discuss.elastic.co/t/logs-parsing/106012)
**Author:** [@paul1243](https://discuss.elastic.co/u/paul1243)\ **Replies:** 1\ **Last updated:** [November 1, 2017, 11:09am UTC](https://discuss.elastic.co/t/logs-parsing/106012 "2017-11-01T11:09:09Z")
Hi - Am receiving the Build logs from the Jenkins using the Logstash plugin, and directly publishing the data to the Elastic Search. How would I parse the out the message and pull out the required data and map it as (k… --- ## [Grok filter date format](https://discuss.elastic.co/t/grok-filter-date-format/106019)
**Author:** [@TheNmaptomyHeartBeat](https://discuss.elastic.co/u/TheNmaptomyHeartBeat)\ **Replies:** 1\ **Last updated:** [November 1, 2017, 11:04am UTC](https://discuss.elastic.co/t/grok-filter-date-format/106019 "2017-11-01T11:04:40Z")
Hello. I have a date with the blow format in my log files (Nginx error log). 2016/11/15 I am trying to create a Grok filter for it. However, I can't seem to find any date stamp for it. I tried all of Grok's date stamp… --- ## [What is no MTA installed](https://discuss.elastic.co/t/what-is-no-mta-installed/105964)
**Author:** [@RajeshGarre](https://discuss.elastic.co/u/RajeshGarre)\ **Replies:** 1\ **Last updated:** [November 1, 2017, 4:15am UTC](https://discuss.elastic.co/t/what-is-no-mta-installed/105964 "2017-11-01T04:15:46Z")
Oct 31 16:55:01 elk02-logstash02 CRON\[29696\]: (root) CMD (/opt/ELK/UCC/pipelineHealthCheck/testprocess.sh) Oct 31 16:55:01 elk02-logstash02 CRON\[29695\]: (CRON) info (No MTA installed, discarding output) Oct 31 16:59:06… --- ## [Enabling X-pack monitoring for logstash](https://discuss.elastic.co/t/enabling-x-pack-monitoring-for-logstash/105951)
**Author:** [@kenny123](https://discuss.elastic.co/u/kenny123)\ **Replies:** 4\ **Last updated:** [October 31, 2017, 9:42pm UTC](https://discuss.elastic.co/t/enabling-x-pack-monitoring-for-logstash/105951 "2017-10-31T21:42:53Z")
HI, I recently started learning ELK . I am trying to setup monitoring X-pack on logstash but couldn't find the directory called plugins like in Elasticsearch or kibana . Moreover , can I please know how to configure be… --- ## [Export the CLoudera 5.8.5 cluster logs to Elastic Stack](https://discuss.elastic.co/t/export-the-cloudera-5-8-5-cluster-logs-to-elastic-stack/105954)
**Author:** [@Harman\_Gurm](https://discuss.elastic.co/u/Harman_Gurm)\ **Replies:** 2\ **Last updated:** [October 31, 2017, 9:25pm UTC](https://discuss.elastic.co/t/export-the-cloudera-5-8-5-cluster-logs-to-elastic-stack/105954 "2017-10-31T21:25:05Z")
do elastic stack supports the logs from Cludera 5.8.5 clustera and RHEL running 7.5. If yes is there any integration steps ot documentation available . Basically it is to monitor events coming from Cloudera Cluster --- ## [Logstash filter not working. Probably my fault :D](https://discuss.elastic.co/t/logstash-filter-not-working-probably-my-fault-d/105811)
**Author:** [@motts](https://discuss.elastic.co/u/motts)\ **Replies:** 4\ **Last updated:** [October 31, 2017, 8:51pm UTC](https://discuss.elastic.co/t/logstash-filter-not-working-probably-my-fault-d/105811 "2017-10-31T20:51:48Z")
I am new to logstash, but I have read a few things on it. I am experimenting with Server 2003 logs. I am stuck on a particular filter configuration, but when I run configtest, everything comes back as OK. I go to check K… --- ## [The quest for the holy-working-HTTPS-input (halloween edition)](https://discuss.elastic.co/t/the-quest-for-the-holy-working-https-input-halloween-edition/105475)
**Author:** [@EtienneDemers](https://discuss.elastic.co/u/EtienneDemers)\ **Replies:** 1\ **Last updated:** [October 31, 2017, 2:29pm UTC](https://discuss.elastic.co/t/the-quest-for-the-holy-working-https-input-halloween-edition/105475 "2017-10-31T14:29:31Z")
journal of the quest for the holy-working-HTTPS-input, day 1: innocent youth Hi! I'm trying to configure logstash to receive post json data from client-side JavaScript with SSL enable but i'm a bit new to all these con… --- ## [Duplicate data showing in interesting fields](https://discuss.elastic.co/t/duplicate-data-showing-in-interesting-fields/105831)
**Author:** [@Feedy](https://discuss.elastic.co/u/Feedy)\ **Replies:** 1\ **Last updated:** [October 31, 2017, 8:27pm UTC](https://discuss.elastic.co/t/duplicate-data-showing-in-interesting-fields/105831 "2017-10-31T20:27:32Z")
Can someone take a look at my configs to help me understand why I am getting duplicate data for some parsed fields? See screenshot of Kibana showing duplicate fields and snort-filter.conf on logstash server: \[Capture\] s… --- ## [Logstash using date filter](https://discuss.elastic.co/t/logstash-using-date-filter/105906)
**Author:** [@Guillaume\_B](https://discuss.elastic.co/u/Guillaume_B)\ **Replies:** 3\ **Last updated:** [October 31, 2017, 8:20pm UTC](https://discuss.elastic.co/t/logstash-using-date-filter/105906 "2017-10-31T20:20:10Z")
Hello everyone, I am using Logstash to parsed the message field in 3 different fields. I am trying to use the date filter to set the type of one of my field to the date type. This is an example of a line from my log fi… --- ## [Logstash 2.3 - GeoIP problem](https://discuss.elastic.co/t/logstash-2-3-geoip-problem/104974)
**Author:** [@gh0stid](https://discuss.elastic.co/u/gh0stid)\ **Replies:** 18\ **Last updated:** [October 31, 2017, 7:59pm UTC](https://discuss.elastic.co/t/logstash-2-3-geoip-problem/104974 "2017-10-31T19:59:47Z")
Hello people.. sorry to bother , once again ALL of my config is working and I mean it.. absolutely all of it. mostely ripped code from siemonster and other places, but all together " work " EXCEPT GeoIP Please gimme … --- ## [More JSON Fun](https://discuss.elastic.co/t/more-json-fun/105813)
**Author:** [@elaair](https://discuss.elastic.co/u/elaair)\ **Replies:** 2\ **Last updated:** [October 31, 2017, 5:55pm UTC](https://discuss.elastic.co/t/more-json-fun/105813 "2017-10-31T17:55:31Z")
I have managed to get some JSON into a target field in this manner: json { source =\> dataMsg target =\> justJSON }. I do not know how to access the field data to convert certain fields to floats or integers for use in Ki… --- ## [Duplicate events with filebeat -\> logstash -\> elasticsearch pipeline](https://discuss.elastic.co/t/duplicate-events-with-filebeat-logstash-elasticsearch-pipeline/105348)
**Author:** [@an.secure](https://discuss.elastic.co/u/an.secure)\ **Replies:** 5\ **Last updated:** [October 31, 2017, 4:42pm UTC](https://discuss.elastic.co/t/duplicate-events-with-filebeat-logstash-elasticsearch-pipeline/105348 "2017-10-31T16:42:27Z")
I have below pipeline to parse the log files written by an application and send them to elasticsearch : filebeat -\> logstash -\> elasticsearch For some time my elasticsearch node went down. When I restarted the elastics… --- ## [Pushing logs over Lumberjack protocol from Node.js](https://discuss.elastic.co/t/pushing-logs-over-lumberjack-protocol-from-node-js/104574)
**Author:** [@jthomas](https://discuss.elastic.co/u/jthomas)\ **Replies:** 4\ **Last updated:** [October 31, 2017, 3:55pm UTC](https://discuss.elastic.co/t/pushing-logs-over-lumberjack-protocol-from-node-js/104574 "2017-10-31T15:55:27Z")
I'm trying to send logs to Logstash using the Lumberjack protocol from Node.js. I've found this NPM module which implements a client library for the protocol but I can't seem to get it working with an instance of ELK I'm… --- ## ["Calculated" date field and Timezone problem](https://discuss.elastic.co/t/calculated-date-field-and-timezone-problem/105851)
**Author:** [@Anabella\_Cristaldi](https://discuss.elastic.co/u/Anabella_Cristaldi)\ **Replies:** 2\ **Last updated:** [October 31, 2017, 3:04pm UTC](https://discuss.elastic.co/t/calculated-date-field-and-timezone-problem/105851 "2017-10-31T15:04:47Z")
Hi all, I'm parsing some phone call logs that have a start timestamp and a duration. And I need to calculate the timestamp where the call ends. That is Start\_timestamp + duration. The date in the original log is in thi… --- ## [":error\_message=\>"undefined method \`sanitized'"](https://discuss.elastic.co/t/error-message-undefined-method-sanitized/104088)
**Author:** [@hyderNBK117](https://discuss.elastic.co/u/hyderNBK117)\ **Replies:** 3\ **Last updated:** [October 31, 2017, 2:29pm UTC](https://discuss.elastic.co/t/error-message-undefined-method-sanitized/104088 "2017-10-31T14:29:50Z")
Hi, My Elastic Stack solution works fine until, seemingly at random, logstash instances starts to log this exception and stops inputting into the elastic search cluster. \[2017-10-16T13:00:57,215\]\[ERROR\]\[logstash.output… --- ## [Early packet loss when using input-tcp with codec-netflow](https://discuss.elastic.co/t/early-packet-loss-when-using-input-tcp-with-codec-netflow/105901)
**Author:** [@hammerstefan](https://discuss.elastic.co/u/hammerstefan)\ **Replies:** 0\ **Last updated:** [October 31, 2017, 2:25pm UTC](https://discuss.elastic.co/t/early-packet-loss-when-using-input-tcp-with-codec-netflow/105901 "2017-10-31T14:25:44Z")
Hi, I am trying to use Logstash with input-tcp and codec-netflow plugin to receive IPFIX (netflow v10) from a device. The device I am using that is generating the IPFIX data has the following behavior: The first time… --- ## [Data is not update using schedule](https://discuss.elastic.co/t/data-is-not-update-using-schedule/105776)
**Author:** [@Sumit\_Gupta1](https://discuss.elastic.co/u/Sumit_Gupta1)\ **Replies:** 9\ **Last updated:** [October 31, 2017, 2:21pm UTC](https://discuss.elastic.co/t/data-is-not-update-using-schedule/105776 "2017-10-31T14:21:15Z")
I'm using Following command. This is not updating Data as schedule . It is my Script: input { jdbc { jdbc\_driver\_library =\> "/app/logstash-5.6.2/lib/ojdbc7.jar" jdbc\_driver\_class =\> "Java::oracle.jd… --- ## [Count with logstash](https://discuss.elastic.co/t/count-with-logstash/105862)
**Author:** [@citanionarrei](https://discuss.elastic.co/u/citanionarrei)\ **Replies:** 6\ **Last updated:** [October 31, 2017, 2:09pm UTC](https://discuss.elastic.co/t/count-with-logstash/105862 "2017-10-31T14:09:00Z")
Hi friends i'm using elk suite to analize log file My logs contain in any row a json file that with grok i can correctly extract it.. Now, i want count the occurrences to try, i want count a simple word (for example… --- ## [How to index all "slow log" log?](https://discuss.elastic.co/t/how-to-index-all-slow-log-log/105893)
**Author:** [@OlivierLG](https://discuss.elastic.co/u/OlivierLG)\ **Replies:** 0\ **Last updated:** [October 31, 2017, 1:42pm UTC](https://discuss.elastic.co/t/how-to-index-all-slow-log-log/105893 "2017-10-31T13:42:22Z")
Hi, We have 3 data nodes and 1 node no-data with kibana. We would like to index all log from the 3 nodes to elasticsearch in order to monitor the search. I understood that we have 1 log per node and I would not like to… --- ## [Parsing "(" in logstash](https://discuss.elastic.co/t/parsing-in-logstash/105876)
**Author:** [@Vivek\_Sharma](https://discuss.elastic.co/u/Vivek_Sharma)\ **Replies:** 2\ **Last updated:** [October 31, 2017, 12:05pm UTC](https://discuss.elastic.co/t/parsing-in-logstash/105876 "2017-10-31T12:05:06Z")
Hi I have my logs in format JOB(JobName) JobName is the name of the job in my log and JOB is the hardcoded valued in the log. When I try to parse the logs, I am not able to parse "(" bracket. Could anyone please help… --- ## [Close output file after writing](https://discuss.elastic.co/t/close-output-file-after-writing/105870)
**Author:** [@cdra](https://discuss.elastic.co/u/cdra)\ **Replies:** 2\ **Last updated:** [October 31, 2017, 11:29am UTC](https://discuss.elastic.co/t/close-output-file-after-writing/105870 "2017-10-31T11:29:48Z")
When I use output file /CSV file, I need to close it after event is writtten --- ## [Apache error log with referer](https://discuss.elastic.co/t/apache-error-log-with-referer/105770)
**Author:** [@jbelien](https://discuss.elastic.co/u/jbelien)\ **Replies:** 7\ **Last updated:** [October 31, 2017, 10:37am UTC](https://discuss.elastic.co/t/apache-error-log-with-referer/105770 "2017-10-31T10:37:17Z")
Hello everyone, I have this line in my Apache error log : \[Fri Oct 20 16:39:12.472720 2017\] \[proxy\_fcgi:error\] \[pid 5727:tid 124259626391296\] \[client 12.34.56.78:12345\] AH01071: Got error 'PHP message: PHP Notice: Unde… --- ## [Adding json objets with Logstash but I get Strings](https://discuss.elastic.co/t/adding-json-objets-with-logstash-but-i-get-strings/105440)
**Author:** [@Alberto\_Diaz](https://discuss.elastic.co/u/Alberto_Diaz)\ **Replies:** 6\ **Last updated:** [October 31, 2017, 9:36am UTC](https://discuss.elastic.co/t/adding-json-objets-with-logstash-but-i-get-strings/105440 "2017-10-31T09:36:27Z")
Hello, I'm trying to add objets to an Array named "lineas". This is my initial document: \[quote="Alberto\_Diaz, post:1, topic:105440, full:true"\] Hello, I'm trying to add objets to an Array named "lineas". This is my … --- ## [Process multiline logs from multiple docker images (multiple hosts)](https://discuss.elastic.co/t/process-multiline-logs-from-multiple-docker-images-multiple-hosts/105767)
**Author:** [@rchumarin](https://discuss.elastic.co/u/rchumarin)\ **Replies:** 3\ **Last updated:** [October 31, 2017, 9:24am UTC](https://discuss.elastic.co/t/process-multiline-logs-from-multiple-docker-images-multiple-hosts/105767 "2017-10-31T09:24:28Z")
How to configure ELK to display one string per printstacktrace Exception, that it does not lead to a mixture of streams and corrupted event data (safe threads). Now stacktrace in Kibana is splitted into multiple lines. … --- ## [Running multiple S3 ingests](https://discuss.elastic.co/t/running-multiple-s3-ingests/105855)
**Author:** [@damianconnolly](https://discuss.elastic.co/u/damianconnolly)\ **Replies:** 0\ **Last updated:** [October 31, 2017, 9:22am UTC](https://discuss.elastic.co/t/running-multiple-s3-ingests/105855 "2017-10-31T09:22:05Z")
Hi all, first post here! I have set up an ELK installation to ingest files using the S3 filter. As there is quite a large backlog of files sitting in the bucket, is it possible to run multiple instances of LS to clear t… --- ## [Logstash multiple path.config (copying files is not a valid solution!)](https://discuss.elastic.co/t/logstash-multiple-path-config-copying-files-is-not-a-valid-solution/104696)
**Author:** [@mostolog](https://discuss.elastic.co/u/mostolog)\ **Replies:** 1\ **Last updated:** [October 31, 2017, 8:29am UTC](https://discuss.elastic.co/t/logstash-multiple-path-config-copying-files-is-not-a-valid-solution/104696 "2017-10-31T08:29:48Z")
Our logstash configurations are actually splitted into 2 directories: main configuration files. group specific configuration files. According to docs: -f, --path.config CONFIG\_PATH Load the Logstash config from a spe… --- ## [How to parse a time to process field](https://discuss.elastic.co/t/how-to-parse-a-time-to-process-field/105631)
**Author:** [@geosword](https://discuss.elastic.co/u/geosword)\ **Replies:** 2\ **Last updated:** [October 31, 2017, 7:00am UTC](https://discuss.elastic.co/t/how-to-parse-a-time-to-process-field/105631 "2017-10-31T07:00:13Z")
Hi Everyone, So the logs I'm dealing with has a "time to process" field, which is being extracted by menas of a grok filter. The problem is that the field can take one of many forms: 472.767µs 1.128953ms 1.480317956s … --- ## [Document\_id overwriting all the data](https://discuss.elastic.co/t/document-id-overwriting-all-the-data/105724)
**Author:** [@Navinder](https://discuss.elastic.co/u/Navinder)\ **Replies:** 2\ **Last updated:** [October 31, 2017, 6:17am UTC](https://discuss.elastic.co/t/document-id-overwriting-all-the-data/105724 "2017-10-31T06:17:23Z")
Hi, I have a JSON field named as "payload.user\_id". Now in logstash output file I have mentioned index as well as document\_id =\> "%{payload.user\_id}". But instead of taking the value of user id as \_id. It is showing \_id … [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=881) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=883)