# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=263 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 264 --- ## [\[Logstash\] elasticsearch filter error on mark logs](https://discuss.elastic.co/t/logstash-elasticsearch-filter-error-on-mark-logs/261293)
**Author:** [@Joao\_Palma](https://discuss.elastic.co/u/Joao_Palma)\ **Replies:** 3\ **Last updated:** [January 17, 2021, 8:25pm UTC](https://discuss.elastic.co/t/logstash-elasticsearch-filter-error-on-mark-logs/261293 "2021-01-17T20:25:29Z")
Hello, I have two pipeline one main, listen filebeat and put data in elastic, and other with post process, the input is elastic, have one elastic filter (bellow), and put back data in elastic with updates. The main goa… --- ## [Logstash read previous events fields from elasticsearch in case of high events rate](https://discuss.elastic.co/t/logstash-read-previous-events-fields-from-elasticsearch-in-case-of-high-events-rate/260756)
**Author:** [@mostafaelsayed](https://discuss.elastic.co/u/mostafaelsayed)\ **Replies:** 3\ **Last updated:** [January 17, 2021, 6:29pm UTC](https://discuss.elastic.co/t/logstash-read-previous-events-fields-from-elasticsearch-in-case-of-high-events-rate/260756 "2021-01-17T18:29:06Z")
Hello In case of high events rate, if I have multiple events (with predefined order) and I want to copy field from the first event to the subsequent events using logstash elasticsearch filter Is there a way to guarante… --- ## [Need different time stamp other than default @timestamp from the indexed log](https://discuss.elastic.co/t/need-different-time-stamp-other-than-default-timestamp-from-the-indexed-log/261313)
**Author:** [@Aryaman\_Gupta](https://discuss.elastic.co/u/Aryaman_Gupta)\ **Replies:** 0\ **Last updated:** [January 16, 2021, 6:20am UTC](https://discuss.elastic.co/t/need-different-time-stamp-other-than-default-timestamp-from-the-indexed-log/261313 "2021-01-16T06:20:01Z")
Hi, My log looks like this 2021-01-13 15:30:12.543723914 node0:app:4786 APP\_ERR\_NUM Function = "DumpSample", Msg = "internal\_stats: Wed 2021-01-13 15:15:45 UTC", Val = 0 On visualising the data in the kibana dashboard… --- ## [Errors in Logstash after \`bundle install \` in plugin](https://discuss.elastic.co/t/errors-in-logstash-after-bundle-install-in-plugin/261286)
**Author:** [@a\_sharafan](https://discuss.elastic.co/u/a_sharafan)\ **Replies:** 0\ **Last updated:** [January 15, 2021, 5:26pm UTC](https://discuss.elastic.co/t/errors-in-logstash-after-bundle-install-in-plugin/261286 "2021-01-15T17:26:31Z")
I created my own filter plugin and did bundle install under directory of plugin. Also created .gem file, but after this bin/logstash doesn't work because of that problem: "No such file to load -- bundler " in ruby-kernel… --- ## [Puting multiple if conditionals into loop](https://discuss.elastic.co/t/puting-multiple-if-conditionals-into-loop/261226)
**Author:** [@woolfik](https://discuss.elastic.co/u/woolfik)\ **Replies:** 1\ **Last updated:** [January 15, 2021, 4:33pm UTC](https://discuss.elastic.co/t/puting-multiple-if-conditionals-into-loop/261226 "2021-01-15T16:33:24Z")
Good morning. So I'm into logstash filters code which contains tens of similar if conditionals like following: if ("123456" == \[ID\]) { mutate { add\_field =\> { "\[fields\]\[key1\]" =\> "value1" "\[fields\]\[ke… --- ## [Logstash exporter installed for multiple services](https://discuss.elastic.co/t/logstash-exporter-installed-for-multiple-services/261219)
**Author:** [@Robin\_Antony](https://discuss.elastic.co/u/Robin_Antony)\ **Replies:** 0\ **Last updated:** [January 15, 2021, 7:44am UTC](https://discuss.elastic.co/t/logstash-exporter-installed-for-multiple-services/261219 "2021-01-15T07:44:40Z")
I have logstash exporter used for multiple service pipelines. Some pipelines are working and some suddenly stopped giving below error which: no java in (/sbin:/bin:/usr/sbin:/usr/bin) could not find java; set JAVA\_HOME… --- ## [Logstash s3 output plugin folder structure](https://discuss.elastic.co/t/logstash-s3-output-plugin-folder-structure/261152)
**Author:** [@Ohad\_Elias](https://discuss.elastic.co/u/Ohad_Elias)\ **Replies:** 2\ **Last updated:** [January 14, 2021, 4:43pm UTC](https://discuss.elastic.co/t/logstash-s3-output-plugin-folder-structure/261152 "2021-01-14T16:43:47Z")
Our DevOps engineers have been using Logstash S3 plugin which simply puts all data in a S3 bucket location. Since we have configured files to be created in every hour on S3, the number of files in the S3 location touched… --- ## [GROK filter with KV](https://discuss.elastic.co/t/grok-filter-with-kv/261209)
**Author:** [@jerin](https://discuss.elastic.co/u/jerin)\ **Replies:** 0\ **Last updated:** [January 15, 2021, 5:44am UTC](https://discuss.elastic.co/t/grok-filter-with-kv/261209 "2021-01-15T05:44:25Z")
Hello Everyone and Badger , I am trying to do a GROK of KV.. attached the snapshot of what I am trying to do .. Expected results : Tstamp : date and time Loglevel : WARN But I am getting LogLevel: ""level\\ .. as yo… --- ## [Time parsing in logstash](https://discuss.elastic.co/t/time-parsing-in-logstash/261199)
**Author:** [@525125](https://discuss.elastic.co/u/525125)\ **Replies:** 0\ **Last updated:** [January 15, 2021, 2:48am UTC](https://discuss.elastic.co/t/time-parsing-in-logstash/261199 "2021-01-15T02:48:29Z")
I need tp parse below log line and extract the time into @apptimestamp . It throughing date parse failure. We need use the filed in kibana. We need to get whole timestamp with timezone and AM/PM log line: /start/scrip… --- ## [Blacklisted domains](https://discuss.elastic.co/t/blacklisted-domains/261033)
**Author:** [@vinu89](https://discuss.elastic.co/u/vinu89)\ **Replies:** 3\ **Last updated:** [January 14, 2021, 5:59pm UTC](https://discuss.elastic.co/t/blacklisted-domains/261033 "2021-01-14T17:59:11Z")
i have dns logs and in that i have to check whether there is any malicious domains.i have a csv file containing malicious domain how to do a comparison.i used translate field it didnt work --- ## [Get duration between 2 dates in logs](https://discuss.elastic.co/t/get-duration-between-2-dates-in-logs/261155)
**Author:** [@A\_Chichi](https://discuss.elastic.co/u/A_Chichi)\ **Replies:** 0\ **Last updated:** [January 14, 2021, 4:45pm UTC](https://discuss.elastic.co/t/get-duration-between-2-dates-in-logs/261155 "2021-01-14T16:45:47Z")
Hello, I would like to get the duration of a job depending on my logs. I've checked this link : Documentation and this one Elastic forum which is exactly what I would like to do but I don't find the solution... here is… --- ## [S3 Output Plugin tag in Filename](https://discuss.elastic.co/t/s3-output-plugin-tag-in-filename/261087)
**Author:** [@okgnae](https://discuss.elastic.co/u/okgnae)\ **Replies:** 1\ **Last updated:** [January 14, 2021, 3:31pm UTC](https://discuss.elastic.co/t/s3-output-plugin-tag-in-filename/261087 "2021-01-14T15:31:18Z")
The filenames uploaded to S3 are auto generated by logstash, but there seems to be a way to add a "tag" to the file name. What is the proper syntaxt to preform this function? The docs mention this as a possibility, bu… --- ## [Logstash-http-poller-plugin-fails-to-disable-ssl-validation](https://discuss.elastic.co/t/logstash-http-poller-plugin-fails-to-disable-ssl-validation/261135)
**Author:** [@Moataz\_abd\_el\_fattah](https://discuss.elastic.co/u/Moataz_abd_el_fattah)\ **Replies:** 0\ **Last updated:** [January 14, 2021, 2:36pm UTC](https://discuss.elastic.co/t/logstash-http-poller-plugin-fails-to-disable-ssl-validation/261135 "2021-01-14T14:36:12Z")
I have solved http-poller ssl check issue (conflict of CN name) with the below workaround Modify the below file /usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/manticore-0.7.0-java/lib/manticore/client.rb add veri… --- ## [Updated docker application logs not getting processed](https://discuss.elastic.co/t/updated-docker-application-logs-not-getting-processed/260991)
**Author:** [@codex](https://discuss.elastic.co/u/codex)\ **Replies:** 1\ **Last updated:** [January 14, 2021, 1:45pm UTC](https://discuss.elastic.co/t/updated-docker-application-logs-not-getting-processed/260991 "2021-01-14T13:45:12Z")
Hi, I'm fairly new to Elastic Search, and I'm experiencing strange behaviour when re-deploying an application that sends its logs to ELK. The application is a Java app and using Filebeat to send the logs (both running in… --- ## [7.10 packages without bundled JDK?](https://discuss.elastic.co/t/7-10-packages-without-bundled-jdk/261108)
**Author:** [@robleady](https://discuss.elastic.co/u/robleady)\ **Replies:** 1\ **Last updated:** [January 14, 2021, 11:08am UTC](https://discuss.elastic.co/t/7-10-packages-without-bundled-jdk/261108 "2021-01-14T11:08:58Z")
Are any pre-built binaries for 7.10.x available (specifically Debian) that don't have a bundled JDK? I'm in the process of creating an OS build which will require its own JDK and I'm keen on not having multiple versions… --- ## [On Kibana web interface l doesn't appear result of logstash](https://discuss.elastic.co/t/on-kibana-web-interface-l-doesnt-appear-result-of-logstash/260793)
**Author:** [@Tabriz](https://discuss.elastic.co/u/Tabriz)\ **Replies:** 13\ **Last updated:** [January 12, 2021, 12:36pm UTC](https://discuss.elastic.co/t/on-kibana-web-interface-l-doesnt-appear-result-of-logstash/260793 "2021-01-12T12:36:29Z")
What l need to do Thanks --- ## [Retrying failed action with response code 503](https://discuss.elastic.co/t/retrying-failed-action-with-response-code-503/261088)
**Author:** [@Tabriz](https://discuss.elastic.co/u/Tabriz)\ **Replies:** 1\ **Last updated:** [January 14, 2021, 7:57am UTC](https://discuss.elastic.co/t/retrying-failed-action-with-response-code-503/261088 "2021-01-14T07:57:30Z")
Sometimes the logstash service stops automatically in ELK and the data is not displayed on the kibana web service. What could be the reason?may is the reason amount of the files? --- ## [About Logstash cloud service](https://discuss.elastic.co/t/about-logstash-cloud-service/261065)
**Author:** [@sungjin.kim](https://discuss.elastic.co/u/sungjin.kim)\ **Replies:** 8\ **Last updated:** [January 14, 2021, 2:39am UTC](https://discuss.elastic.co/t/about-logstash-cloud-service/261065 "2021-01-14T02:39:45Z")
I am using elasticsearch and kibana on elastic cloud(elastic.co) i am considering to use logstash to collect user events from application server log file. for this, I am setting up filebeats on app server and about to … --- ## [Error on an external ruby script](https://discuss.elastic.co/t/error-on-an-external-ruby-script/261069)
**Author:** [@ElasticLiver](https://discuss.elastic.co/u/ElasticLiver)\ **Replies:** 0\ **Last updated:** [January 14, 2021, 2:32am UTC](https://discuss.elastic.co/t/error-on-an-external-ruby-script/261069 "2021-01-14T02:32:31Z")
Hi im getting the following error in an external ruby code: Could not process event: undefined method \`\[\]' for nil not every key in this hash, exist on the data being consumed by logstash, so I added a conditional that… --- ## [Logstash refuses to start](https://discuss.elastic.co/t/logstash-refuses-to-start/261013)
**Author:** [@Falikou1](https://discuss.elastic.co/u/Falikou1)\ **Replies:** 1\ **Last updated:** [January 14, 2021, 2:14am UTC](https://discuss.elastic.co/t/logstash-refuses-to-start/261013 "2021-01-14T02:14:44Z")
Logstash configuration parsing is: input { tcp { port =\> "5141" codec =\> json type =\> "syslog" } } filter { grok { match =\> { "message" =\> "%{SYSLOG5424PRI:syslog\_index}-\\s\*%{SYSLOGHOST:syslog\_hostnam… --- ## [Json parser error](https://discuss.elastic.co/t/json-parser-error/261061)
**Author:** [@Surjiths](https://discuss.elastic.co/u/Surjiths)\ **Replies:** 0\ **Last updated:** [January 13, 2021, 10:05pm UTC](https://discuss.elastic.co/t/json-parser-error/261061 "2021-01-13T22:05:29Z")
Hi Team, I am getting the following error from logstash \[2020-12-09T08:20:57.962Z\]\[ERROR\]\[logstash.codecs.json \] JSON parse error, original data now in message field {:error=\>#\ **Author:** [@prashant\_Rajenderan](https://discuss.elastic.co/u/prashant_Rajenderan)\ **Replies:** 2\ **Last updated:** [January 13, 2021, 9:21pm UTC](https://discuss.elastic.co/t/can-we-configure-logstash-to-listen-only-to-a-paricular-set-of-hosts/261049 "2021-01-13T21:21:32Z") Currently my logstash input is listening to filebeat on port XXXX,my requirement is to collect log data only from a particular hosts(let's say only from Webservers). I dont want to modify the filebeat configuration direc… --- ## [Parsing syslog RFC 5424 format](https://discuss.elastic.co/t/parsing-syslog-rfc-5424-format/261021)
**Author:** [@S3l3ct3d](https://discuss.elastic.co/u/S3l3ct3d)\ **Replies:** 2\ **Last updated:** [January 13, 2021, 9:07pm UTC](https://discuss.elastic.co/t/parsing-syslog-rfc-5424-format/261021 "2021-01-13T21:07:56Z")
Good morning, I have a log source that sends syslog messages in RFC 5424 format. I have searched everywhere to figure out how to parse these syslog messages. Since logstash's syslog input is for RFC 3164 format, nothing… --- ## [GROK Custom Pattern help. Drop leading and trailing spaces](https://discuss.elastic.co/t/grok-custom-pattern-help-drop-leading-and-trailing-spaces/261037)
**Author:** [@fastxl](https://discuss.elastic.co/u/fastxl)\ **Replies:** 1\ **Last updated:** [January 13, 2021, 8:41pm UTC](https://discuss.elastic.co/t/grok-custom-pattern-help-drop-leading-and-trailing-spaces/261037 "2021-01-13T20:41:36Z")
Looking to GROK the following data "(DATA: SOME STUFF)" What I am after is "SOME STUFF". When I use the following \\(DATA: (?\ \*(.+?) \*)\\) I get this result in the debugger { "DATA": "SOME STUFF" } but if my da… --- ## [Clone and display input from filebeat to logstash](https://discuss.elastic.co/t/clone-and-display-input-from-filebeat-to-logstash/261032)
**Author:** [@umesh2020](https://discuss.elastic.co/u/umesh2020)\ **Replies:** 0\ **Last updated:** [January 13, 2021, 5:59pm UTC](https://discuss.elastic.co/t/clone-and-display-input-from-filebeat-to-logstash/261032 "2021-01-13T17:59:34Z")
Hi I would like to check what input LogStash is receiving without changing output plugin. Is there a way to duplicate and display input data to LogStash coming from filebeat ? I don't have tcpdump installed, so I can't… --- ## [Lumberjack input vs beats input considerations (can I remove lumberjack intake altogether?)](https://discuss.elastic.co/t/lumberjack-input-vs-beats-input-considerations-can-i-remove-lumberjack-intake-altogether/260914)
**Author:** [@rmauri](https://discuss.elastic.co/u/rmauri)\ **Replies:** 0\ **Last updated:** [January 12, 2021, 10:11pm UTC](https://discuss.elastic.co/t/lumberjack-input-vs-beats-input-considerations-can-i-remove-lumberjack-intake-altogether/260914 "2021-01-12T22:11:34Z")
The elastic doc https://www.elastic.co/guide/en/logstash/current/plugins-inputs-lumberjack.html states " Consider using the Beats input plugin instead. The Beats input implements the Lumberjack protocol v1 and v2." Howe… --- ## [Parsing using grok](https://discuss.elastic.co/t/parsing-using-grok/260916)
**Author:** [@elasticforme](https://discuss.elastic.co/u/elasticforme)\ **Replies:** 2\ **Last updated:** [January 13, 2021, 3:40pm UTC](https://discuss.elastic.co/t/parsing-using-grok/260916 "2021-01-13T15:40:11Z")
i am running jdbc and getting following data. if I run this on SQL\> prompt --backupid "2129214415\_30120163393540,2129214415\_30120163432904" --spoolpath /system1/1\_backup but on logstash output it becomes command: "--… --- ## [How can i populate/copy an event field for every event line?](https://discuss.elastic.co/t/how-can-i-populate-copy-an-event-field-for-every-event-line/261011)
**Author:** [@EHE](https://discuss.elastic.co/u/EHE)\ **Replies:** 0\ **Last updated:** [January 13, 2021, 2:36pm UTC](https://discuss.elastic.co/t/how-can-i-populate-copy-an-event-field-for-every-event-line/261011 "2021-01-13T14:36:57Z")
Hello, I am struggling with the below I multiple log line mappings and there is one specific field i would like to populate for the logline before and after, currently i am only getting it on the exact same line where t… --- ## [Grok filter working in debugger but not getting parsed with Logstash](https://discuss.elastic.co/t/grok-filter-working-in-debugger-but-not-getting-parsed-with-logstash/261006)
**Author:** [@vijayakumar.cfis](https://discuss.elastic.co/u/vijayakumar.cfis)\ **Replies:** 0\ **Last updated:** [January 13, 2021, 1:52pm UTC](https://discuss.elastic.co/t/grok-filter-working-in-debugger-but-not-getting-parsed-with-logstash/261006 "2021-01-13T13:52:32Z")
Hi Team, I am trying to parse the below log entries using Grok. \<158\>Dec 03 04:50:03 AB-AWSA-01.abcd.xxxxxxxxxxxxxxx.com SIEMAccessLogs: Info: 10.0.0.1 "abcd\\abcd@abcd.xxxxxxxxxxxxxxx.com" - \[03/Dec/2020:04:50:01 +0000… --- ## [Kubernetes How disable SSL getting Elasticsearch Unreachable: \[https://elastic:xxxxxx@myhost.elastic-namespace.svc:9200/](https://discuss.elastic.co/t/kubernetes-how-disable-ssl-getting-elasticsearch-unreachable-https-elastic-xxxxxx-myhost-elastic-namespace-svc-9200/260985)
**Author:** [@umen](https://discuss.elastic.co/u/umen)\ **Replies:** 0\ **Last updated:** [January 13, 2021, 11:18am UTC](https://discuss.elastic.co/t/kubernetes-how-disable-ssl-getting-elasticsearch-unreachable-https-elastic-xxxxxx-myhost-elastic-namespace-svc-9200/260985 "2021-01-13T11:18:11Z")
i saw a lot of problems with this issue, looks like it is impossible to disable the SSL in logstash when connecting to elasticsearch i want to disable it , i don't what to use it this is an internal app. this configura… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=262) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=264)