# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=654 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 655 --- ## [Syslog and Auth Logs DateParseFailure](https://discuss.elastic.co/t/syslog-and-auth-logs-dateparsefailure/155376)
**Author:** [@Miguel\_Leite](https://discuss.elastic.co/u/Miguel_Leite)\ **Replies:** 1\ **Last updated:** [November 5, 2018, 10:07am UTC](https://discuss.elastic.co/t/syslog-and-auth-logs-dateparsefailure/155376 "2018-11-05T10:07:05Z")
Hello, I have the following date filter configuration to match both timestamps from syslog and auth logs: date { locale =\> "en" match =\> \[ "\[system\]\[syslog\]\[timestamp\]", "MMM d HH:mm:ss", "MMM dd HH:… --- ## [How to restart individual pipelines?](https://discuss.elastic.co/t/how-to-restart-individual-pipelines/155360)
**Author:** [@Kvin](https://discuss.elastic.co/u/Kvin)\ **Replies:** 0\ **Last updated:** [November 5, 2018, 8:38am UTC](https://discuss.elastic.co/t/how-to-restart-individual-pipelines/155360 "2018-11-05T08:38:56Z")
I'm using Logstash 6.3. I've set up the pipelines.yml file to have multiple pipelines within one Logstash instance. In our use case we have to change the configuration of each pipeline rather often. So far I just applie… --- ## [Monitor queuedepth logstash queues](https://discuss.elastic.co/t/monitor-queuedepth-logstash-queues/155354)
**Author:** [@mortenb123](https://discuss.elastic.co/u/mortenb123)\ **Replies:** 0\ **Last updated:** [November 5, 2018, 7:54am UTC](https://discuss.elastic.co/t/monitor-queuedepth-logstash-queues/155354 "2018-11-05T07:54:58Z")
Hi Is there a way to monitor the queuedepth of logstash queues queue.type: persisted queue.max\_bytes: 4gb I can see thet the jvm usage increases when I push 50K onto the syslog port. but I do not know how much is ac… --- ## [Simple Elasticsearch and Logstash Jdbc plugin is not working](https://discuss.elastic.co/t/simple-elasticsearch-and-logstash-jdbc-plugin-is-not-working/155337)
**Author:** [@pank](https://discuss.elastic.co/u/pank)\ **Replies:** 0\ **Last updated:** [November 5, 2018, 3:45am UTC](https://discuss.elastic.co/t/simple-elasticsearch-and-logstash-jdbc-plugin-is-not-working/155337 "2018-11-05T03:45:23Z")
I already installed Logtsash and Elasticsearch version 6.4 for both and I tried simple file and run it. No error comes but index is not creted. input { jdbc { jdbc\_driver\_library =\> "/opt/share/logstash/logsta… --- ## [How to replace @timestamp with field in log](https://discuss.elastic.co/t/how-to-replace-timestamp-with-field-in-log/155253)
**Author:** [@manja](https://discuss.elastic.co/u/manja)\ **Replies:** 2\ **Last updated:** [November 4, 2018, 11:02pm UTC](https://discuss.elastic.co/t/how-to-replace-timestamp-with-field-in-log/155253 "2018-11-04T23:02:32Z")
I want collectd old access log but @timestamp logstash is a time now i want replace @timestamp with field = apachetime in log from ex. { "message" => "xxx.xx.xx.xx 222990 - - \[31/Oct/2018:10:38:33 +0700\] \\"GE… --- ## [Mutate string to geo point error (logstash)](https://discuss.elastic.co/t/mutate-string-to-geo-point-error-logstash/155301)
**Author:** [@Claudio\_Gerarduzzi](https://discuss.elastic.co/u/Claudio_Gerarduzzi)\ **Replies:** 0\ **Last updated:** [November 4, 2018, 2:02pm UTC](https://discuss.elastic.co/t/mutate-string-to-geo-point-error-logstash/155301 "2018-11-04T14:02:19Z")
Good morning community; I have the next problem in my exercise, with the next config file: input { file { path =\> "/EADIC/Chicago\_Crime\_Data\_2015.csv" start\_position =\> "beginning" sincedb\_path =\> "/dev/null" } } … --- ## [Doc\_as\_upsert with some mistake](https://discuss.elastic.co/t/doc-as-upsert-with-some-mistake/154906)
**Author:** [@bulu](https://discuss.elastic.co/u/bulu)\ **Replies:** 4\ **Last updated:** [November 4, 2018, 1:17pm UTC](https://discuss.elastic.co/t/doc-as-upsert-with-some-mistake/154906 "2018-11-04T13:17:42Z")
elasticsearch { action =\> "update" # doc\_as\_upsert =\> "true" hosts =\> "192.168.88.63" index =\> "apps\_user\_list" retry\_on\_conflict =\> 1 document\_id =\> "uid%{user\_new\_id}" ups… --- ## [Unable add a field (grok) with docker\_metadata as its value](https://discuss.elastic.co/t/unable-add-a-field-grok-with-docker-metadata-as-its-value/155278)
**Author:** [@yanivomc1](https://discuss.elastic.co/u/yanivomc1)\ **Replies:** 4\ **Last updated:** [November 4, 2018, 7:48am UTC](https://discuss.elastic.co/t/unable-add-a-field-grok-with-docker-metadata-as-its-value/155278 "2018-11-04T07:48:17Z")
Hi Guys, spent a day on this thing. i got a json log that being sent to filebeat and from there to logstash. The problem: i've added docker\_metadata in filebeat, i also see the docker metadata there in logstash oupu… --- ## [Logstash can't load the logs to ES, just looping and do nothing](https://discuss.elastic.co/t/logstash-cant-load-the-logs-to-es-just-looping-and-do-nothing/155282)
**Author:** [@KWLam](https://discuss.elastic.co/u/KWLam)\ **Replies:** 2\ **Last updated:** [November 4, 2018, 6:08am UTC](https://discuss.elastic.co/t/logstash-cant-load-the-logs-to-es-just-looping-and-do-nothing/155282 "2018-11-04T06:08:24Z")
Hi, I am a beginner, try to load logs using logstash 6.2.4, sometime work, sometime just loop there and do nothing, normal log as below: no error message at all, is my logstash issue? .conf file issue? (but it work bef… --- ## [Logstash SSL beats fail SSLV3\_ALERT\_BAD\_CERTIFICATE](https://discuss.elastic.co/t/logstash-ssl-beats-fail-sslv3-alert-bad-certificate/155225)
**Author:** [@ohkeenan](https://discuss.elastic.co/u/ohkeenan)\ **Replies:** 1\ **Last updated:** [November 3, 2018, 4:47pm UTC](https://discuss.elastic.co/t/logstash-ssl-beats-fail-sslv3-alert-bad-certificate/155225 "2018-11-03T16:47:29Z")
Hello, I can connect with openssl s\_client just fine but using metricbeat or filebeat fails (v6.4): Nov 02 11:21:27 elk.\*\*\*.com logstash\[5533\]: \[2018-11-02T11:21:27,124\]\[WARN \]\[io.netty.channel.DefaultChannelPipeline\] … --- ## [Process same file multiple times](https://discuss.elastic.co/t/process-same-file-multiple-times/155238)
**Author:** [@telebh](https://discuss.elastic.co/u/telebh)\ **Replies:** 1\ **Last updated:** [November 3, 2018, 9:32am UTC](https://discuss.elastic.co/t/process-same-file-multiple-times/155238 "2018-11-03T09:32:53Z")
Hi there, I am using logsatsh to parse our dataloading logs, which are text files but contains dynamic contents (not syslog nor apache), though for debugging i need to test the config settings i had to read this file (i… --- ## [Csv file headers are missing in logstash output](https://discuss.elastic.co/t/csv-file-headers-are-missing-in-logstash-output/155241)
**Author:** [@Chamani\_Shiranthika](https://discuss.elastic.co/u/Chamani_Shiranthika)\ **Replies:** 0\ **Last updated:** [November 3, 2018, 2:55am UTC](https://discuss.elastic.co/t/csv-file-headers-are-missing-in-logstash-output/155241 "2018-11-03T02:55:18Z")
I am generating a csv file by loading a log file from filebeat and using logstash to process the data specifying the input and output. The format of logstash output configuration is as follows. output { csv { … --- ## [RELP input plugin compression support coming?](https://discuss.elastic.co/t/relp-input-plugin-compression-support-coming/155236)
**Author:** [@jeffkirk1](https://discuss.elastic.co/u/jeffkirk1)\ **Replies:** 0\ **Last updated:** [November 2, 2018, 8:43pm UTC](https://discuss.elastic.co/t/relp-input-plugin-compression-support-coming/155236 "2018-11-02T20:43:44Z")
Hey folks, We'd very much like to be able to use the RELP input plugin for Logstash with stream compression. Does anyone know if this feature is on the road map for the RELP plugin? Thanks in advance, Jeff Kirk Staf… --- ## [Logstash-filter-verifier failing on timestamp](https://discuss.elastic.co/t/logstash-filter-verifier-failing-on-timestamp/155019)
**Author:** [@kathleen](https://discuss.elastic.co/u/kathleen)\ **Replies:** 7\ **Last updated:** [November 2, 2018, 8:14pm UTC](https://discuss.elastic.co/t/logstash-filter-verifier-failing-on-timestamp/155019 "2018-11-02T20:14:15Z")
Hi, I've started looking at the Logstash filter verifier, and I was wondering if someone could point me in the direction of what I'm doing wrong. I have a date filter in my Logstash configuration file that looks like th… --- ## [Could not find or load main class org.logstash.Logstash](https://discuss.elastic.co/t/could-not-find-or-load-main-class-org-logstash-logstash/155229)
**Author:** [@telebh](https://discuss.elastic.co/u/telebh)\ **Replies:** 0\ **Last updated:** [November 2, 2018, 7:04pm UTC](https://discuss.elastic.co/t/could-not-find-or-load-main-class-org-logstash-logstash/155229 "2018-11-02T19:04:02Z")
Hi there, I am newbie to Logstash and would need your help. I have JAVA version "1.8.0\_191" and installed logstash zip file for windows "logstash-6.4.2", and have a simple config file as follows, but it always raises er… --- ## [Help with my grok statement](https://discuss.elastic.co/t/help-with-my-grok-statement/155215)
**Author:** [@Frank\_Shearer](https://discuss.elastic.co/u/Frank_Shearer)\ **Replies:** 1\ **Last updated:** [November 2, 2018, 6:29pm UTC](https://discuss.elastic.co/t/help-with-my-grok-statement/155215 "2018-11-02T18:29:10Z")
I have this syslog output and am trying to parse it out: Login Success \[user: fshearer\] \[Source: 10.130.129.14\] \[localport: 22\] at 09:46:35 PDT Fri Nov 2 2018" Grok: "%{NOTSPACE}%{SPACE}%{CISCO\_REASON:cisco\_reason}%{S… --- ## [IPV4 Logstash](https://discuss.elastic.co/t/ipv4-logstash/155228)
**Author:** [@Hubbabubba](https://discuss.elastic.co/u/Hubbabubba)\ **Replies:** 0\ **Last updated:** [November 2, 2018, 6:46pm UTC](https://discuss.elastic.co/t/ipv4-logstash/155228 "2018-11-02T18:46:14Z")
How do I bind logstash to a IPv4 adress. It seems that it only binds to IPv6. --- ## [How input csv data without header with logstash](https://discuss.elastic.co/t/how-input-csv-data-without-header-with-logstash/155219)
**Author:** [@sutarisani](https://discuss.elastic.co/u/sutarisani)\ **Replies:** 0\ **Last updated:** [November 2, 2018, 6:12pm UTC](https://discuss.elastic.co/t/how-input-csv-data-without-header-with-logstash/155219 "2018-11-02T18:12:18Z")
Hi, I tried to input csv data with out header. How create config file data as below 02/11/2018-23:26:15,cbtcnb-smscp2p-09,KRUSTY ,8-5ba294bf,"dinni.rds\_uerror\_in","",c,0,3,3,0,0,6 02/11/2018-23:26:15,cbtcnb-smscp2… --- ## [Handling failures in ES output plugin](https://discuss.elastic.co/t/handling-failures-in-es-output-plugin/155209)
**Author:** [@niks1](https://discuss.elastic.co/u/niks1)\ **Replies:** 0\ **Last updated:** [November 2, 2018, 4:49pm UTC](https://discuss.elastic.co/t/handling-failures-in-es-output-plugin/155209 "2018-11-02T16:49:04Z")
I get this message \[2018-10-30T18:53:51,945\]\[WARN \]\[logstash.outputs.elasticsearch\] Could not index event to Elasticsearch. {:status=\>400, :action=\>\["index", {:\_id=\>nil, :\_index=\>"sunshine-txlog-2018.10.30", :\_type=\>"fo… --- ## [Manipulating array of JSON objects](https://discuss.elastic.co/t/manipulating-array-of-json-objects/154192)
**Author:** [@cappy](https://discuss.elastic.co/u/cappy)\ **Replies:** 1\ **Last updated:** [November 2, 2018, 4:39pm UTC](https://discuss.elastic.co/t/manipulating-array-of-json-objects/154192 "2018-11-02T16:39:39Z")
Hello, I have an array of objects named sections like so: { "structure": "IMAGE\_SECTION\_HEADER", "name": ".text", "flags": \[ "IMAGE\_SCN\_CNT\_CODE", "IMAGE\_SCN\_MEM\_EXECUTE", "IMAGE\_SCN\_MEM\_READ" \] }, … --- ## [Logstash: Showing some output when dropping a message](https://discuss.elastic.co/t/logstash-showing-some-output-when-dropping-a-message/155206)
**Author:** [@pkaramol](https://discuss.elastic.co/u/pkaramol)\ **Replies:** 0\ **Last updated:** [November 2, 2018, 4:37pm UTC](https://discuss.elastic.co/t/logstash-showing-some-output-when-dropping-a-message/155206 "2018-11-02T16:37:56Z")
At some point, I perform a check on incoming events and conditionally drop some of them if "to\_internal\_traffic" in \[tags\] { drop {} } My question is whether it is possible to get some debug output at the time of… --- ## [Facing input-jdbc redshift docker problem](https://discuss.elastic.co/t/facing-input-jdbc-redshift-docker-problem/153183)
**Author:** [@Julius\_JF](https://discuss.elastic.co/u/Julius_JF)\ **Replies:** 2\ **Last updated:** [November 2, 2018, 4:03pm UTC](https://discuss.elastic.co/t/facing-input-jdbc-redshift-docker-problem/153183 "2018-11-02T16:03:09Z")
Hi guys, I'm asking for an advice and assistance of logstash gurus with jdbc-input plugin :slight\_smile: There is a such structure: Logstash is running in docker container, and it's composed with docker-compose. the… --- ## [XML match IDs in different elements](https://discuss.elastic.co/t/xml-match-ids-in-different-elements/155191)
**Author:** [@ChristianCH](https://discuss.elastic.co/u/ChristianCH)\ **Replies:** 0\ **Last updated:** [November 2, 2018, 3:45pm UTC](https://discuss.elastic.co/t/xml-match-ids-in-different-elements/155191 "2018-11-02T15:45:16Z")
Hi all, I want to insert chats into Elasticsearch and have a problem in converting the chat XML transcript to the proper JSON using logstash. This is the XML: \ This is… --- ## [Mapping IDs in XML with Logstash](https://discuss.elastic.co/t/mapping-ids-in-xml-with-logstash/155188)
**Author:** [@ChristianCH](https://discuss.elastic.co/u/ChristianCH)\ **Replies:** 0\ **Last updated:** [November 2, 2018, 3:33pm UTC](https://discuss.elastic.co/t/mapping-ids-in-xml-with-logstash/155188 "2018-11-02T15:33:53Z")
Hi all, I''ve this chat text in XML format, which i want to insert into Elasticsearch using Logstash. \ \ \ \ \ **Author:** [@pkaramol](https://discuss.elastic.co/u/pkaramol)\ **Replies:** 0\ **Last updated:** [November 2, 2018, 3:25pm UTC](https://discuss.elastic.co/t/docker-compose-of-elk-stack-with-logstash-2-4-not-working/155187 "2018-11-02T15:25:20Z") I am trying to run a simple pipeline with elk using however logstash 2.4 as follows logstash: build: context: logstash/ volumes: - ./logstash/pipeline-production:/etc/logstash/conf.d:rw - ./l… --- ## [Push data into elasticsearch indices from logstash via http](https://discuss.elastic.co/t/push-data-into-elasticsearch-indices-from-logstash-via-http/155183)
**Author:** [@BoffinPanda](https://discuss.elastic.co/u/BoffinPanda)\ **Replies:** 5\ **Last updated:** [November 2, 2018, 3:17pm UTC](https://discuss.elastic.co/t/push-data-into-elasticsearch-indices-from-logstash-via-http/155183 "2018-11-02T15:17:22Z")
I am trying to push my data which I am collecting from various sensors into elasticsearch. I am able to push my data directly into elasticsearch indices using the post method. I am now attempting to push the data via l… --- ## [Convert timestamp and add it to email body](https://discuss.elastic.co/t/convert-timestamp-and-add-it-to-email-body/152165)
**Author:** [@mancharagopan](https://discuss.elastic.co/u/mancharagopan)\ **Replies:** 3\ **Last updated:** [November 2, 2018, 2:57pm UTC](https://discuss.elastic.co/t/convert-timestamp-and-add-it-to-email-body/152165 "2018-11-02T14:57:12Z")
My logstash debug message is, Oct 12 11:10:16 localhost.localdomain logstash\[17086\]: \[2018-10-12T11:10:16,683\]\[DEBUG\]\[org.logstash.beats.BeatsHandler\] \[local: 172.20.11.19:5044, remote: 172.20.11.16:51070\] Received a ne… --- ## [Struggling with field substitution syntax](https://discuss.elastic.co/t/struggling-with-field-substitution-syntax/155051)
**Author:** [@TimWard](https://discuss.elastic.co/u/TimWard)\ **Replies:** 1\ **Last updated:** [November 2, 2018, 1:40pm UTC](https://discuss.elastic.co/t/struggling-with-field-substitution-syntax/155051 "2018-11-02T13:40:11Z")
I'm using the kafka input plugin to create events from Kafka messages. This all works fine. The next step is that many of the Kafka message payloads are in fact JSON, and I want to parse the JSON into the event so as to… --- ## [How to start logstash as root](https://discuss.elastic.co/t/how-to-start-logstash-as-root/155107)
**Author:** [@mancharagopan](https://discuss.elastic.co/u/mancharagopan)\ **Replies:** 2\ **Last updated:** [November 2, 2018, 1:19pm UTC](https://discuss.elastic.co/t/how-to-start-logstash-as-root/155107 "2018-11-02T13:19:47Z")
Hello, Can anyone tell me how to start logstash as root? I am trying to use port 514 to syslog messages because some of my devices like WLC send syslog messages only to udp port 514. --- ## [Parse only files with given extension into logstash](https://discuss.elastic.co/t/parse-only-files-with-given-extension-into-logstash/155123)
**Author:** [@GitSpree23](https://discuss.elastic.co/u/GitSpree23)\ **Replies:** 1\ **Last updated:** [November 2, 2018, 12:43pm UTC](https://discuss.elastic.co/t/parse-only-files-with-given-extension-into-logstash/155123 "2018-11-02T12:43:44Z")
I am using the following input plugin to parse ".gz" files in all subfolders into logstash: input { file { #type =\> "gzip" path =\> "/home/xyz/data/sample/\*\*/\*\*/\*" … [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=653) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=655)