# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=458 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 459 --- ## [Logstash failing with dissect when adding one field with a date](https://discuss.elastic.co/t/logstash-failing-with-dissect-when-adding-one-field-with-a-date/204911)
**Author:** [@megah3rtz](https://discuss.elastic.co/u/megah3rtz)\ **Replies:** 4\ **Last updated:** [October 23, 2019, 5:06pm UTC](https://discuss.elastic.co/t/logstash-failing-with-dissect-when-adding-one-field-with-a-date/204911 "2019-10-23T17:06:04Z")
Im using Logstash 7.2.0 that is being fed logs through filebeats. I have done a decent amount of work with parsing some of the logs coming in, but seem to keep running into this recuring issue. Sending a log like this: … --- ## [Logstash Geopoint filter](https://discuss.elastic.co/t/logstash-geopoint-filter/204905)
**Author:** [@haythem](https://discuss.elastic.co/u/haythem)\ **Replies:** 0\ **Last updated:** [October 23, 2019, 3:52pm UTC](https://discuss.elastic.co/t/logstash-geopoint-filter/204905 "2019-10-23T15:52:44Z")
Hi guys , so i'm working on a project now when i have to load data from oracle database and convert two columns(longitude,latitude) to a geo\_point , so i've created an index in elasticsearch and i created a geo\_point m… --- ## [TLSV1\_ALERT\_UNKNOWN\_CA](https://discuss.elastic.co/t/tlsv1-alert-unknown-ca/204896)
**Author:** [@mgutier](https://discuss.elastic.co/u/mgutier)\ **Replies:** 0\ **Last updated:** [October 23, 2019, 3:03pm UTC](https://discuss.elastic.co/t/tlsv1-alert-unknown-ca/204896 "2019-10-23T15:03:13Z")
Received the following error in Logstash 6.8.3 -- trying to use TCP Input with SSL. \[2019-10-22T22:30:21,891\]\[ERROR\]\[logstash.inputs.tcp \] Error in Netty pipeline: io.netty.handler.codec.DecoderException: javax.net… --- ## [Index name removes date](https://discuss.elastic.co/t/index-name-removes-date/204705)
**Author:** [@John\_McAfee1](https://discuss.elastic.co/u/John_McAfee1)\ **Replies:** 2\ **Last updated:** [October 23, 2019, 2:46pm UTC](https://discuss.elastic.co/t/index-name-removes-date/204705 "2019-10-23T14:46:07Z")
Hello, I am using Logstash 7.2.0 and have encountered an issue with naming my index with a timestamp. An index is created in Logstash, but it doesn't contain the date. It only contains the characters before the dash. … --- ## [Input kafka: the decorate\_events option overwrites @timestamp](https://discuss.elastic.co/t/input-kafka-the-decorate-events-option-overwrites-timestamp/204800)
**Author:** [@zeenon](https://discuss.elastic.co/u/zeenon)\ **Replies:** 4\ **Last updated:** [October 23, 2019, 2:15pm UTC](https://discuss.elastic.co/t/input-kafka-the-decorate-events-option-overwrites-timestamp/204800 "2019-10-23T14:15:30Z")
Version: logstash-input-kafka (8.3.1) Operating System: Red Hat 7.2 Logstash: 6.8.1 I am using decorate\_event in my kafka input section. It works nicely but unfortunatly, in combination with codec json, I lost @timest… --- ## [Logstash 7.3 question about multiple outputs](https://discuss.elastic.co/t/logstash-7-3-question-about-multiple-outputs/204877)
**Author:** [@robroy](https://discuss.elastic.co/u/robroy)\ **Replies:** 1\ **Last updated:** [October 23, 2019, 1:58pm UTC](https://discuss.elastic.co/t/logstash-7-3-question-about-multiple-outputs/204877 "2019-10-23T13:58:59Z")
Hi All :wink: I have a pipeline in which I would like to perform a s3 backup in an output in logstash. My source event has "role" and "type" fields among others. In "filter" part of my config I have this: filter { m… --- ## [Logstash plugin error while running with config file](https://discuss.elastic.co/t/logstash-plugin-error-while-running-with-config-file/204860)
**Author:** [@manu4387](https://discuss.elastic.co/u/manu4387)\ **Replies:** 7\ **Last updated:** [October 23, 2019, 1:49pm UTC](https://discuss.elastic.co/t/logstash-plugin-error-while-running-with-config-file/204860 "2019-10-23T13:49:42Z")
I am getting this error while running the github plugin in logstash. Plugin: \"a8d9de94a4535953974b37dedbea2c29254692fa33ebd1b70372cf8413e435f2", port=\>22, secret\_token=\>"3e27bb366ae773d231a… --- ## [Unable to connect to database. Tried 1 times {:error\_message=\>"Java::JavaLang::NoClassDefFoundError: javax/xml/bind/DatatypeConverter"](https://discuss.elastic.co/t/unable-to-connect-to-database-tried-1-times-error-message-java-javax-xml-bind-datatypeconverter/200901)
**Author:** [@raquelg](https://discuss.elastic.co/u/raquelg)\ **Replies:** 2\ **Last updated:** [October 23, 2019, 1:21pm UTC](https://discuss.elastic.co/t/unable-to-connect-to-database-tried-1-times-error-message-java-javax-xml-bind-datatypeconverter/200901 "2019-10-23T13:21:47Z")
Hi, I am trying to move a table form SQL Server to ElasticSearch with Logstash I have installed elasticsearch and logstash 7.3.1 with docker-compose. pipeline/logstash has the following configuration: input { tcp { … --- ## [Parsing xml parameters & values separated on new line](https://discuss.elastic.co/t/parsing-xml-parameters-values-separated-on-new-line/204823)
**Author:** [@Vijay\_Gharge](https://discuss.elastic.co/u/Vijay_Gharge)\ **Replies:** 1\ **Last updated:** [October 23, 2019, 12:52pm UTC](https://discuss.elastic.co/t/parsing-xml-parameters-values-separated-on-new-line/204823 "2019-10-23T12:52:10Z")
Hi, I am trying to parse below xml file wherein parameter name & value is covered in 2 different tags. I am looking for guidance on how to parse this xml data using logstash ? \ \parameter-1\ \10\… --- ## [\_grokparsefailure while creating field using grok filter](https://discuss.elastic.co/t/grokparsefailure-while-creating-field-using-grok-filter/204729)
**Author:** [@Vishal\_Sharma4](https://discuss.elastic.co/u/Vishal_Sharma4)\ **Replies:** 5\ **Last updated:** [October 23, 2019, 12:39pm UTC](https://discuss.elastic.co/t/grokparsefailure-while-creating-field-using-grok-filter/204729 "2019-10-23T12:39:19Z")
Hi , I am new on logstash , I have created grok pattern for below mentioned tomcat access logs. 10.0.6.35 - - \[21/Oct/2019:00:00:21 +0000\] "GET /rest/V1/productlist/category/4/ar/18/20 HTTP/1.1" 200 15917 25 My logsta… --- ## [How to avoid single line events in filebeat multiline configuration](https://discuss.elastic.co/t/how-to-avoid-single-line-events-in-filebeat-multiline-configuration/204839)
**Author:** [@ajojohnk](https://discuss.elastic.co/u/ajojohnk)\ **Replies:** 1\ **Last updated:** [October 23, 2019, 12:17pm UTC](https://discuss.elastic.co/t/how-to-avoid-single-line-events-in-filebeat-multiline-configuration/204839 "2019-10-23T12:17:45Z")
Hi I've a simple xml file which has this format Here, i succeded in extracting events at the object level using multiline. But, along with it for each line of the xml file where it finds a "\\n" it also fires a … --- ## [How can i get client source log path same filebeat](https://discuss.elastic.co/t/how-can-i-get-client-source-log-path-same-filebeat/204762)
**Author:** [@Hoang\_Duy1](https://discuss.elastic.co/u/Hoang_Duy1)\ **Replies:** 2\ **Last updated:** [October 23, 2019, 11:02am UTC](https://discuss.elastic.co/t/how-can-i-get-client-source-log-path-same-filebeat/204762 "2019-10-23T11:02:33Z")
I need create field "LogPath" to get location of log path.. like field log.file.path in filbeat but i dont know how filebeat do that. Pls help me!! My input from UDP --- ## [Github plugin logstash example to connect to github repository](https://discuss.elastic.co/t/github-plugin-logstash-example-to-connect-to-github-repository/204687)
**Author:** [@manu4387](https://discuss.elastic.co/u/manu4387)\ **Replies:** 1\ **Last updated:** [October 23, 2019, 9:20am UTC](https://discuss.elastic.co/t/github-plugin-logstash-example-to-connect-to-github-repository/204687 "2019-10-23T09:20:08Z")
I found this plugin to use to connect to github.. is there any option or example to config the file. https://www.elastic.co/guide/en/logstash/7.4/plugins-inputs-github.html --- ## [Dateparsefailure with String 2019-10-23 07:58:53](https://discuss.elastic.co/t/dateparsefailure-with-string-2019-10-23-0753/204785)
**Author:** [@J143](https://discuss.elastic.co/u/J143)\ **Replies:** 1\ **Last updated:** [October 23, 2019, 8:25am UTC](https://discuss.elastic.co/t/dateparsefailure-with-string-2019-10-23-0753/204785 "2019-10-23T08:25:29Z")
Hello, I want to parse a date String like 2019-10-23 07:58:53 and receive a dateparsefailure... The date filter looks like the following: date { match =\> \["timestamp" , "yyyy-MM-dd HH:mm:ss"\] target =\> "@timestamp" … --- ## [Logstash not stopping after completely reading the file](https://discuss.elastic.co/t/logstash-not-stopping-after-completely-reading-the-file/204805)
**Author:** [@Harish\_Chander](https://discuss.elastic.co/u/Harish_Chander)\ **Replies:** 0\ **Last updated:** [October 23, 2019, 7:34am UTC](https://discuss.elastic.co/t/logstash-not-stopping-after-completely-reading-the-file/204805 "2019-10-23T07:34:34Z")
Hi, My logstash conf is as below. input { s3 { access\_key\_id =\> "" secret\_access\_key =\> "" bucket =\> "tabprod-s3data" region =\> "us-west-2" } } filter { json { source =\> "message" remove\_field =\> \["message"\] … --- ## [Logstash - elapsed seconds to something human-readable](https://discuss.elastic.co/t/logstash-elapsed-seconds-to-something-human-readable/204602)
**Author:** [@francesc.recasens](https://discuss.elastic.co/u/francesc.recasens)\ **Replies:** 2\ **Last updated:** [October 23, 2019, 7:32am UTC](https://discuss.elastic.co/t/logstash-elapsed-seconds-to-something-human-readable/204602 "2019-10-23T07:32:40Z")
Good morning, I'm using Logstash with elapsed plugin to calculate the time between two timestamps. It works perfectly and the field is a Number in seconds. Is there a way to create a new field with that value in a huma… --- ## [AWS IAM Policy for S3 Output Plugin](https://discuss.elastic.co/t/aws-iam-policy-for-s3-output-plugin/204790)
**Author:** [@cytim](https://discuss.elastic.co/u/cytim)\ **Replies:** 0\ **Last updated:** [October 23, 2019, 6:49am UTC](https://discuss.elastic.co/t/aws-iam-policy-for-s3-output-plugin/204790 "2019-10-23T06:49:20Z")
Extending the S3 output requires PutObject on \* Resources, this post is created for other people to setup the IAM policy for S3 output plugin. { "Version": "2012-10-17", "Statement": \[ { "Sid… --- ## [Lostash-\* is not configuring in kibana](https://discuss.elastic.co/t/lostash-is-not-configuring-in-kibana/204788)
**Author:** [@Ekta1234](https://discuss.elastic.co/u/Ekta1234)\ **Replies:** 0\ **Last updated:** [October 23, 2019, 6:44am UTC](https://discuss.elastic.co/t/lostash-is-not-configuring-in-kibana/204788 "2019-10-23T06:44:42Z")
lostash-\* is not configuring in kibana. when I'm opening kibana in localhost:5601 and wanting to create index pattern "logstash-\*"...it is not happening . The log files are from springboot application. Please Help! --- ## [Logstash load my template failed with bad response error code 400](https://discuss.elastic.co/t/logstash-load-my-template-failed-with-bad-response-error-code-400/204786)
**Author:** [@terrymu](https://discuss.elastic.co/u/terrymu)\ **Replies:** 1\ **Last updated:** [October 23, 2019, 6:42am UTC](https://discuss.elastic.co/t/logstash-load-my-template-failed-with-bad-response-error-code-400/204786 "2019-10-23T06:42:31Z")
Hi Experts, I have designed a customized template will be loaded during start logstash. But logstash loads it failed with error code 400. The failed report shows as below: \[2019-10-23T14:33:27,893\]\[INFO \]\[logstash.out… --- ## [How to configure Logstash to output to dynamic list of kafka bootstrap servers?](https://discuss.elastic.co/t/how-to-configure-logstash-to-output-to-dynamic-list-of-kafka-bootstrap-servers/203002)
**Author:** [@furiabhavesh](https://discuss.elastic.co/u/furiabhavesh)\ **Replies:** 3\ **Last updated:** [October 13, 2019, 3:27pm UTC](https://discuss.elastic.co/t/how-to-configure-logstash-to-output-to-dynamic-list-of-kafka-bootstrap-servers/203002 "2019-10-13T15:27:14Z")
We need to configure logstash to output to multiple kafka brokers whose list needs to be dynamic. We need to pass this list of kafka hosts as follows: docker run -e BOOTSTRAP\_SERVERS="host1:port1,host2:port2,hostn:port… --- ## [How to get sql\_last\_value from last\_run\_metadata\_path every time](https://discuss.elastic.co/t/how-to-get-sql-last-value-from-last-run-metadata-path-every-time/204582)
**Author:** [@Olivia5](https://discuss.elastic.co/u/Olivia5)\ **Replies:** 2\ **Last updated:** [October 23, 2019, 3:55am UTC](https://discuss.elastic.co/t/how-to-get-sql-last-value-from-last-run-metadata-path-every-time/204582 "2019-10-23T03:55:05Z")
just the pipeline starts up, sql\_last\_value will be reading from the file(last\_run\_metadata\_path). so logstash is running, cant get change value from the file. ( There is a shell timing script that initializes values… --- ## [OpenLdap log parsing pipeline issue](https://discuss.elastic.co/t/openldap-log-parsing-pipeline-issue/204743)
**Author:** [@iukea](https://discuss.elastic.co/u/iukea)\ **Replies:** 0\ **Last updated:** [October 23, 2019, 12:06am UTC](https://discuss.elastic.co/t/openldap-log-parsing-pipeline-issue/204743 "2019-10-23T00:06:51Z")
Hello, I am currently trying to parse some openldap logs and need some help from the ElasticSearch gods. attached is the error message \[2019-10-23T00:03:53,115\]\[ERROR\]\[logstash.javapipeline \] Pipeline aborted due to… --- ## [How to move an Index from Server A to B](https://discuss.elastic.co/t/how-to-move-an-index-from-server-a-to-b/204552)
**Author:** [@baselai](https://discuss.elastic.co/u/baselai)\ **Replies:** 3\ **Last updated:** [October 22, 2019, 10:05pm UTC](https://discuss.elastic.co/t/how-to-move-an-index-from-server-a-to-b/204552 "2019-10-22T22:05:18Z")
I'm looking for a way to move a 2.3 GB index from Server A to Server B, that is completely lives on a different domain/host. I tried logstash code, but it fails at some point and keeps pushing the data infinetly! input… --- ## [Adding multiple nested json fields to an array field](https://discuss.elastic.co/t/adding-multiple-nested-json-fields-to-an-array-field/204279)
**Author:** [@kylemain](https://discuss.elastic.co/u/kylemain)\ **Replies:** 2\ **Last updated:** [October 22, 2019, 7:29pm UTC](https://discuss.elastic.co/t/adding-multiple-nested-json-fields-to-an-array-field/204279 "2019-10-22T19:29:17Z")
Hello, I have a nested json field in my logs called "instanceId" : "responseElements": { "instancesSet": { "items": \[ { "instanceId": "i-02669ced00e8a3701", "currentState":{ "code": 32, "name": "shutting-down" }, "prev… --- ## [Extract Hour and Day from @timestamp](https://discuss.elastic.co/t/extract-hour-and-day-from-timestamp/204542)
**Author:** [@Jim\_Thunder](https://discuss.elastic.co/u/Jim_Thunder)\ **Replies:** 7\ **Last updated:** [October 22, 2019, 7:04pm UTC](https://discuss.elastic.co/t/extract-hour-and-day-from-timestamp/204542 "2019-10-22T19:04:13Z")
Hello! I am trying to copy the day and hour from the timestamp field and add each to their own fields. Here is my code: if \[source\] =~ "tableBASEtest" { mutate{ add\_field =\> { "\[@metadata\]\[timestam… --- ## [Remove whole documents with certain value in field](https://discuss.elastic.co/t/remove-whole-documents-with-certain-value-in-field/204475)
**Author:** [@Marta\_Zagrajek](https://discuss.elastic.co/u/Marta_Zagrajek)\ **Replies:** 5\ **Last updated:** [October 22, 2019, 6:19pm UTC](https://discuss.elastic.co/t/remove-whole-documents-with-certain-value-in-field/204475 "2019-10-22T18:19:14Z")
Hello there I have apache logs in which are documents that I want to remove completely. Whole documents not fields so I do not want to use remove\_field in mutate. I have tried drop: filter { if "field\_name" == … --- ## [OpenLdap Log parsing? (strange errors)](https://discuss.elastic.co/t/openldap-log-parsing-strange-errors/204701)
**Author:** [@iukea](https://discuss.elastic.co/u/iukea)\ **Replies:** 1\ **Last updated:** [October 22, 2019, 6:05pm UTC](https://discuss.elastic.co/t/openldap-log-parsing-strange-errors/204701 "2019-10-22T18:05:16Z")
I am having a crazy hard time trying to parse logs coming from my LDAP servers. Sending data Vida filebeat to logstash. And found a nice example openldap.conf file someone was working on (it works okay but i have made… --- ## [Logstash - JSON Parsing is very slow](https://discuss.elastic.co/t/logstash-json-parsing-is-very-slow/204704)
**Author:** [@raviep](https://discuss.elastic.co/u/raviep)\ **Replies:** 0\ **Last updated:** [October 22, 2019, 5:48pm UTC](https://discuss.elastic.co/t/logstash-json-parsing-is-very-slow/204704 "2019-10-22T17:48:16Z")
Hi, I need to parse elastic search response JSON to CSV using logstash. JSON File: { "took" : 11, "timed\_out" : false, "\_shards" : { "total" : 3, "successful" : 3, "skipped" : 0, "failed" : 0 }, "hits" : { "t… --- ## [Using field value from one filter logstash to another field](https://discuss.elastic.co/t/using-field-value-from-one-filter-logstash-to-another-field/204689)
**Author:** [@billymerendina](https://discuss.elastic.co/u/billymerendina)\ **Replies:** 1\ **Last updated:** [October 22, 2019, 4:17pm UTC](https://discuss.elastic.co/t/using-field-value-from-one-filter-logstash-to-another-field/204689 "2019-10-22T16:17:10Z")
Hi everyone, I have a question for you. I use Filebeat (two different log file input) , Logstash Elastisearch and Kibana and I want to use a field value from a filter logstash conf file and use it on another field with… --- ## [Split json object into multiple Events for ES](https://discuss.elastic.co/t/split-json-object-into-multiple-events-for-es/204622)
**Author:** [@Jasonespo](https://discuss.elastic.co/u/Jasonespo)\ **Replies:** 2\ **Last updated:** [October 22, 2019, 2:47pm UTC](https://discuss.elastic.co/t/split-json-object-into-multiple-events-for-es/204622 "2019-10-22T14:47:21Z")
Hi, I need to theorise the below as it will take a while to complete through trial and error due to the speed at which these logs are sent from AWS. Is it possible to split this json object into multiple events using t… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=457) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=459)