# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=824 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 825 --- ## [LogStash::Json::GeneratorError when using mutate copy @timestamp](https://discuss.elastic.co/t/logstash-json-generatorerror-when-using-mutate-copy-timestamp/119256)
**Author:** [@timvisher](https://discuss.elastic.co/u/timvisher)\ **Replies:** 0\ **Last updated:** [February 9, 2018, 2:37pm UTC](https://discuss.elastic.co/t/logstash-json-generatorerror-when-using-mutate-copy-timestamp/119256 "2018-02-09T14:37:14Z")
I was attempting to save off the timestamp that logstash saw an event before mutating the event to use the event's parsed timestamp with the date filter. My first attempt was: if \[@timestamp\] { mutate { copy =\> … --- ## [From the apache log timestamp have to extract the timezone and store it in elastic Db](https://discuss.elastic.co/t/from-the-apache-log-timestamp-have-to-extract-the-timezone-and-store-it-in-elastic-db/119243)
**Author:** [@ranganath\_nangineni](https://discuss.elastic.co/u/ranganath_nangineni)\ **Replies:** 1\ **Last updated:** [February 9, 2018, 2:29pm UTC](https://discuss.elastic.co/t/from-the-apache-log-timestamp-have-to-extract-the-timezone-and-store-it-in-elastic-db/119243 "2018-02-09T14:29:39Z")
Hi, I have the below logformat , I need to store the timezone of the original timestamp's timezone in characters. Ex: Log format: 10.68.36.27 - adam@gmail.com \[05/Feb/2018:14:52:23 -0600\] "GET /content/web/cnt2626024… --- ## [Logstash to read from multiple server filebeat](https://discuss.elastic.co/t/logstash-to-read-from-multiple-server-filebeat/119239)
**Author:** [@Sridhar](https://discuss.elastic.co/u/Sridhar)\ **Replies:** 3\ **Last updated:** [February 9, 2018, 2:24pm UTC](https://discuss.elastic.co/t/logstash-to-read-from-multiple-server-filebeat/119239 "2018-02-09T14:24:09Z")
Hi All, below is the file beat I'm configuring in one server with port 5044 filebeat: prospectors: - paths: - /path/to/logs/router-001-yyyymmdd.log include: "Read from QMANGER:INPUT\_IN" fields: {log\_type: input\_in… --- ## [Taking array of matching values for one pattern out of Logstash grok](https://discuss.elastic.co/t/taking-array-of-matching-values-for-one-pattern-out-of-logstash-grok/119026)
**Author:** [@hym](https://discuss.elastic.co/u/hym)\ **Replies:** 3\ **Last updated:** [February 9, 2018, 2:23pm UTC](https://discuss.elastic.co/t/taking-array-of-matching-values-for-one-pattern-out-of-logstash-grok/119026 "2018-02-09T14:23:28Z")
Hi all, I was wondering, is there any way to take an array of matching values out of Grok plugin. let's explain more with an example: I have a message like "message" = "whatever 0.0.0.0 whatever 1.1.1.1 something 2.2.2… --- ## [Integrate Nagios with Logstash](https://discuss.elastic.co/t/integrate-nagios-with-logstash/119224)
**Author:** [@charan.gandra](https://discuss.elastic.co/u/charan.gandra)\ **Replies:** 2\ **Last updated:** [February 9, 2018, 1:47pm UTC](https://discuss.elastic.co/t/integrate-nagios-with-logstash/119224 "2018-02-09T13:47:22Z")
Hi, I am new to ELK and I am looking to integrate Nagios with Logstash, however I can't find my documentation around this. Any help please? --- ## [Can't listen to syslog with Logstash](https://discuss.elastic.co/t/cant-listen-to-syslog-with-logstash/119206)
**Author:** [@Adren](https://discuss.elastic.co/u/Adren)\ **Replies:** 7\ **Last updated:** [February 9, 2018, 1:38pm UTC](https://discuss.elastic.co/t/cant-listen-to-syslog-with-logstash/119206 "2018-02-09T13:38:39Z")
Hello, I've some problems to configure Logstash for listening to syslog that a server sends me. I configured the pipleline like this: input { syslog{ port =\> "514" } } filter { geoip { source =\> "clientip" } } … --- ## [Help on Filters](https://discuss.elastic.co/t/help-on-filters/119240)
**Author:** [@Sridhar](https://discuss.elastic.co/u/Sridhar)\ **Replies:** 0\ **Last updated:** [February 9, 2018, 1:14pm UTC](https://discuss.elastic.co/t/help-on-filters/119240 "2018-02-09T13:14:54Z")
Hi All, Need help to create a logstash filter for the below log format. router#001(26322) 2018/02/09 00:00:17 -FKSPT- {2:2} Read from QMGR:INPUT\_IN\[wmqin(srid)\] \ unknown, id =\> (2)"-1", userref =\> … --- ## [Overwrite @timestamp to different format](https://discuss.elastic.co/t/overwrite-timestamp-to-different-format/119218)
**Author:** [@moriol](https://discuss.elastic.co/u/moriol)\ **Replies:** 1\ **Last updated:** [February 9, 2018, 1:11pm UTC](https://discuss.elastic.co/t/overwrite-timestamp-to-different-format/119218 "2018-02-09T13:11:32Z")
I am wondering can I change the format of the default @timestamp field, to be in milliseconds? I see I can create new field called "time", copy @timestamp to it, and then convert to milliseconds and my time value displa… --- ## [Logstash output file - storage capacity](https://discuss.elastic.co/t/logstash-output-file-storage-capacity/119058)
**Author:** [@Parthgandhi](https://discuss.elastic.co/u/Parthgandhi)\ **Replies:** 3\ **Last updated:** [February 9, 2018, 1:09pm UTC](https://discuss.elastic.co/t/logstash-output-file-storage-capacity/119058 "2018-02-09T13:09:44Z")
Hi Team, We are planning to gather IIS logs using ELK stack. We will be using filebeats to get the IIS logs to Elasticsearch server (this will also be the kibana server). Does logstash send the complete log file to elas… --- ## [How to use if else statments in logstash output pipline?](https://discuss.elastic.co/t/how-to-use-if-else-statments-in-logstash-output-pipline/119199)
**Author:** [@Robin\_Guo](https://discuss.elastic.co/u/Robin_Guo)\ **Replies:** 8\ **Last updated:** [February 9, 2018, 11:19am UTC](https://discuss.elastic.co/t/how-to-use-if-else-statments-in-logstash-output-pipline/119199 "2018-02-09T11:19:15Z")
Dear ES folks, Here is my request, I want to split the index by tags and environment. Can someone help me out? The logical like this： "system" in \[tags\] =\> index =\> "system-%{+YYYY.MM}" "nginx" in \[tags\] =\> index … --- ## [KV not including space values](https://discuss.elastic.co/t/kv-not-including-space-values/119205)
**Author:** [@tag\_v](https://discuss.elastic.co/u/tag_v)\ **Replies:** 5\ **Last updated:** [February 9, 2018, 10:53am UTC](https://discuss.elastic.co/t/kv-not-including-space-values/119205 "2018-02-09T10:53:18Z")
input message: key1=first value key2=second value key3=value3 required output: { "key1" =\> "first value", "key2" =\> "second value", "key3" =\> "value3", "message" =\> "key1=first value key2=second value key3=value3" … --- ## [Logstash: Unable to create java\_pid2361.hprof: Permission denied](https://discuss.elastic.co/t/logstash-unable-to-create-java-pid2361-hprof-permission-denied/118634)
**Author:** [@mourlos](https://discuss.elastic.co/u/mourlos)\ **Replies:** 1\ **Last updated:** [February 9, 2018, 10:47am UTC](https://discuss.elastic.co/t/logstash-unable-to-create-java-pid2361-hprof-permission-denied/118634 "2018-02-09T10:47:16Z")
Hello, Today we upgraded from 6.1.1 to 6.1.3 After the upgrade, the logstash input UDP kept on failing Looking into the logs of logstash, there wasn't any error, but on the system logs the following was coming up: lo… --- ## [Logstash filter2](https://discuss.elastic.co/t/logstash-filter2/119207)
**Author:** [@rattisyam](https://discuss.elastic.co/u/rattisyam)\ **Replies:** 2\ **Last updated:** [February 9, 2018, 10:38am UTC](https://discuss.elastic.co/t/logstash-filter2/119207 "2018-02-09T10:38:06Z")
please help me to create a filer conf file with pattern to below XML log. I am looking for the fileds "result","duration" and timestamp". \ \ 10 \ **Author:** [@Krunal\_kalaria](https://discuss.elastic.co/u/Krunal_kalaria)\ **Replies:** 4\ **Last updated:** [February 9, 2018, 10:29am UTC](https://discuss.elastic.co/t/remove-value-in-logstash/119169 "2018-02-09T10:29:54Z") Hello Everyone, I need to remove fields value because its duplicate i have following configuration file and output Symantec.conf input { stdin{} } filter { csv { separator =\> "," columns =\> \["Event Time", "Se… --- ## [DLQ with start\_timestamp is not working correctly](https://discuss.elastic.co/t/dlq-with-start-timestamp-is-not-working-correctly/118980)
**Author:** [@Shivanshu\_Bagga](https://discuss.elastic.co/u/Shivanshu_Bagga)\ **Replies:** 0\ **Last updated:** [February 8, 2018, 6:56am UTC](https://discuss.elastic.co/t/dlq-with-start-timestamp-is-not-working-correctly/118980 "2018-02-08T06:56:08Z")
Hi, Following is my configuration for DLQ input plugin- dead\_letter\_queue { path =\> "/opt/shivanshu/setup/logstash-6.0.0/data/dead\_letter\_queue" start\_timestamp =\> "2026-02-08T06:10:21" #commit\_offsets =\> … --- ## [Logstash not uploading data on ES](https://discuss.elastic.co/t/logstash-not-uploading-data-on-es/119172)
**Author:** [@niraj\_pandey](https://discuss.elastic.co/u/niraj_pandey)\ **Replies:** 3\ **Last updated:** [February 9, 2018, 8:17am UTC](https://discuss.elastic.co/t/logstash-not-uploading-data-on-es/119172 "2018-02-09T08:17:40Z")
Hi, I have created a template and now using logstash to push the data on elasticsearch. And I see it's created the index but not uploaded the data. Any idea ? Here is template: PUT \_template/ltemplsbacct { "index\_p… --- ## [Logstash 2.4 container doesnt validate package repository](https://discuss.elastic.co/t/logstash-2-4-container-doesnt-validate-package-repository/119182)
**Author:** [@roflmao](https://discuss.elastic.co/u/roflmao)\ **Replies:** 0\ **Last updated:** [February 9, 2018, 7:26am UTC](https://discuss.elastic.co/t/logstash-2-4-container-doesnt-validate-package-repository/119182 "2018-02-09T07:26:40Z")
$ docker inspect -f '{{ .Config.Image}}' 48e logstash:2.4 Inside container: # apt-get update Ign:1 http://deb.debian.org/debian stretch InRelease Hit:2 http://security.debian.org stretch/updates InRelease Hit:3 http:/… --- ## [Unable to establish connection between Logstash and Elasticsearch to parse iis log files](https://discuss.elastic.co/t/unable-to-establish-connection-between-logstash-and-elasticsearch-to-parse-iis-log-files/118587)
**Author:** [@SRIpandu1729](https://discuss.elastic.co/u/SRIpandu1729)\ **Replies:** 6\ **Last updated:** [February 9, 2018, 7:24am UTC](https://discuss.elastic.co/t/unable-to-establish-connection-between-logstash-and-elasticsearch-to-parse-iis-log-files/118587 "2018-02-09T07:24:47Z")
OS : Ubuntu 16.04 Installed : Elasticsearch, Kibana, Logstash (All 3 are of version 6.1) Log type : iis This is my config file for Logstash input { file { type =\> "iis-w3c" path =\> "/home/harsha/t… --- ## [How to update by inline script using logstash ES output](https://discuss.elastic.co/t/how-to-update-by-inline-script-using-logstash-es-output/119176)
**Author:** [@Zt\_Zeng](https://discuss.elastic.co/u/Zt_Zeng)\ **Replies:** 0\ **Last updated:** [February 9, 2018, 6:51am UTC](https://discuss.elastic.co/t/how-to-update-by-inline-script-using-logstash-es-output/119176 "2018-02-09T06:51:23Z")
I use the following config to update document by id with inlined script, but it is not working. Any one knows why? elasticsearch { index =\> "announcement%{\[@metadata\]\[suffix\]}" document\_type =\> "announcement" docu… --- ## [Logstash not reading the log data in correct sequence](https://discuss.elastic.co/t/logstash-not-reading-the-log-data-in-correct-sequence/118856)
**Author:** [@shraddha1](https://discuss.elastic.co/u/shraddha1)\ **Replies:** 3\ **Last updated:** [February 9, 2018, 6:23am UTC](https://discuss.elastic.co/t/logstash-not-reading-the-log-data-in-correct-sequence/118856 "2018-02-09T06:23:56Z")
Hi, I have an issue with data parsing through Logstash. It reads a csv file but assigns data incorrectly in the message filed. Below is the overall description: I am parsing a csv file in logstash which outputs file s… --- ## [Logstash Tshark issue](https://discuss.elastic.co/t/logstash-tshark-issue/118582)
**Author:** [@hsiuming](https://discuss.elastic.co/u/hsiuming)\ **Replies:** 0\ **Last updated:** [February 6, 2018, 8:51am UTC](https://discuss.elastic.co/t/logstash-tshark-issue/118582 "2018-02-06T08:51:47Z")
Hello ,We now get a network package data by tshark. We send these data to elasticsearch system with the step: get the package data and convert it to json file : tshark -i eth0 ... -T ek \> packets.json send this da… --- ## [The content of all logs in dead\_letter\_queue directory is 1](https://discuss.elastic.co/t/the-content-of-all-logs-in-dead-letter-queue-directory-is-1/119156)
**Author:** [@kevin\_bolton](https://discuss.elastic.co/u/kevin_bolton)\ **Replies:** 0\ **Last updated:** [February 9, 2018, 4:22am UTC](https://discuss.elastic.co/t/the-content-of-all-logs-in-dead-letter-queue-directory-is-1/119156 "2018-02-09T04:22:30Z")
Hi, After enable Persistent Queues & Dead Letter Queues, the log file in dead\_letter\_queue directory is continue to create (2 new log files every minute). And the content of all logs in dead\_letter\_queue directory is … --- ## [Csv output plugin - custom field](https://discuss.elastic.co/t/csv-output-plugin-custom-field/118744)
**Author:** [@alexus](https://discuss.elastic.co/u/alexus)\ **Replies:** 6\ **Last updated:** [February 9, 2018, 4:14am UTC](https://discuss.elastic.co/t/csv-output-plugin-custom-field/118744 "2018-02-09T04:14:26Z")
I'm following Csv output plugin | Logstash Reference \[6.1\] | Elastic and I have a question... How would I add extra field with my own text? Please advise. --- ## [Japanese field name garbled](https://discuss.elastic.co/t/japanese-field-name-garbled/119154)
**Author:** [@muraken720](https://discuss.elastic.co/u/muraken720)\ **Replies:** 0\ **Last updated:** [February 9, 2018, 3:50am UTC](https://discuss.elastic.co/t/japanese-field-name-garbled/119154 "2018-02-09T03:50:11Z")
When I use Japanese as the filed name, the filed name garbled at the output phase in Logstash. Elastic Stack（Logstash, Elasticsearch, Kibana） version: 5.6.5 and 6.1.3 expect: (dummy data) { "性別" =\> "男性", "address" … --- ## [Cannot start logstash after adding netflow module](https://discuss.elastic.co/t/cannot-start-logstash-after-adding-netflow-module/119133)
**Author:** [@imortalsolitude](https://discuss.elastic.co/u/imortalsolitude)\ **Replies:** 0\ **Last updated:** [February 8, 2018, 10:21pm UTC](https://discuss.elastic.co/t/cannot-start-logstash-after-adding-netflow-module/119133 "2018-02-08T22:21:16Z")
When i modify the logstash.yml to add netflow module my logstash never starts and gives below error" \[logstash.agent \] Failed to execute action {:action=\>LogStash::PipelineAction::Stop/pipeline\_id:main, :except… --- ## [Logstash behavior in case of disk full](https://discuss.elastic.co/t/logstash-behavior-in-case-of-disk-full/119124)
**Author:** [@ray20](https://discuss.elastic.co/u/ray20)\ **Replies:** 0\ **Last updated:** [February 8, 2018, 8:41pm UTC](https://discuss.elastic.co/t/logstash-behavior-in-case-of-disk-full/119124 "2018-02-08T20:41:13Z")
We are planning to use Logstash with persistent queues and S3 output, which creates temporary local files. While we are planning for disk space requirements for logstash nodes, I'd like to know how logstash and S3 output… --- ## [Logstash index issue after 5.x to 6.2 upgrade](https://discuss.elastic.co/t/logstash-index-issue-after-5-x-to-6-2-upgrade/119118)
**Author:** [@Diggy](https://discuss.elastic.co/u/Diggy)\ **Replies:** 2\ **Last updated:** [February 8, 2018, 8:26pm UTC](https://discuss.elastic.co/t/logstash-index-issue-after-5-x-to-6-2-upgrade/119118 "2018-02-08T20:26:58Z")
Hello, all. Yesterday, I upgraded my Elastic stack from version 5.x to 6.2, and today, my logstash (and filebeat) indices no longer work. Here's some output from elasticsearch.log: \[2018-02-08T00:03:29,150\]\[WARN \]\[log… --- ## [How to use log4jplus with logstash?](https://discuss.elastic.co/t/how-to-use-log4jplus-with-logstash/118869)
**Author:** [@durgeshtanuku](https://discuss.elastic.co/u/durgeshtanuku)\ **Replies:** 1\ **Last updated:** [February 8, 2018, 8:24pm UTC](https://discuss.elastic.co/t/how-to-use-log4jplus-with-logstash/118869 "2018-02-08T20:24:14Z")
I'm new to logstash. I'm trying to send logs of my C++ application to logstash using log4cplus library. I need to send some json data from my application to logstash. To start with I have written some piece of code tha… --- ## [Logstash Exception: Connection reset by peer](https://discuss.elastic.co/t/logstash-exception-connection-reset-by-peer/118844)
**Author:** [@Farhad\_Yousefi](https://discuss.elastic.co/u/Farhad_Yousefi)\ **Replies:** 1\ **Last updated:** [February 8, 2018, 8:18pm UTC](https://discuss.elastic.co/t/logstash-exception-connection-reset-by-peer/118844 "2018-02-08T20:18:06Z")
I want to monitor our exception table via logstash. but I have got this error. \[2018-02-07T15:43:11,344\]\[ERROR\]\[org.logstash.beats.BeatsHandler\] Exception: Connection reset by peer \[2018-02-07T15:44:00,478\]\[INFO \]\[logst… --- ## [How to have provide value for @timestamp when using JSON?](https://discuss.elastic.co/t/how-to-have-provide-value-for-timestamp-when-using-json/119071)
**Author:** [@some7](https://discuss.elastic.co/u/some7)\ **Replies:** 3\ **Last updated:** [February 8, 2018, 8:09pm UTC](https://discuss.elastic.co/t/how-to-have-provide-value-for-timestamp-when-using-json/119071 "2018-02-08T20:09:37Z")
I send a JSON object string to logstash, and one of the keys of the JSON object is @timestamp with a timestamp I've generated, but when I put it into logstash I get @timestamp which is generated by logstash, and @\_timest… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=823) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=825)