# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=516 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 517 --- ## [New config adds mappings to other indicies](https://discuss.elastic.co/t/new-config-adds-mappings-to-other-indicies/188669)
**Author:** [@sfio](https://discuss.elastic.co/u/sfio)\ **Replies:** 8\ **Last updated:** [July 4, 2019, 1:05pm UTC](https://discuss.elastic.co/t/new-config-adds-mappings-to-other-indicies/188669 "2019-07-04T13:05:18Z")
Hi everybody, I´ve got a problem with a config file, which updates mappings of existing ones. Environment: one Debian Server with ES 7.1.1 Configs: system\_action\_job.conf input { jdbc { jdbc\_driver\_library =\> … --- ## [Convert string to date isnt working when logstash reading data from csv file. Need help!](https://discuss.elastic.co/t/convert-string-to-date-isnt-working-when-logstash-reading-data-from-csv-file-need-help/188898)
**Author:** [@prataprajasekhar](https://discuss.elastic.co/u/prataprajasekhar)\ **Replies:** 1\ **Last updated:** [July 4, 2019, 12:24pm UTC](https://discuss.elastic.co/t/convert-string-to-date-isnt-working-when-logstash-reading-data-from-csv-file-need-help/188898 "2019-07-04T12:24:32Z")
Dear All, Need help resolving to convert string into date. The logstash is configured to read data from csv file and helps converts string to integer but not string to date for certain column. Here is the sample data, … --- ## [Logstash not receiving beats](https://discuss.elastic.co/t/logstash-not-receiving-beats/188414)
**Author:** [@markov](https://discuss.elastic.co/u/markov)\ **Replies:** 14\ **Last updated:** [July 4, 2019, 9:44am UTC](https://discuss.elastic.co/t/logstash-not-receiving-beats/188414 "2019-07-04T09:44:36Z")
Hi guys, my filebeat is sending to logstash but it just sending the operating system logs, Note that i gave him the path of my logs files; those are my configs : i am runing filebeat on windows filebeat.yml \> #######… --- ## [Logstash ruby json parsing Issue](https://discuss.elastic.co/t/logstash-ruby-json-parsing-issue/188603)
**Author:** [@sukku77](https://discuss.elastic.co/u/sukku77)\ **Replies:** 2\ **Last updated:** [July 4, 2019, 8:26am UTC](https://discuss.elastic.co/t/logstash-ruby-json-parsing-issue/188603 "2019-07-04T08:26:09Z")
Hi , I need to poll http endpoint from logstash using http\_poller plugin and parse the json response to fetch the values and derive custom fields based on that. Below is my response sample json output that need to pars… --- ## [How to compare 2 numerical field values with each other?](https://discuss.elastic.co/t/how-to-compare-2-numerical-field-values-with-each-other/188840)
**Author:** [@Sjaak01](https://discuss.elastic.co/u/Sjaak01)\ **Replies:** 1\ **Last updated:** [July 4, 2019, 8:07am UTC](https://discuss.elastic.co/t/how-to-compare-2-numerical-field-values-with-each-other/188840 "2019-07-04T08:07:43Z")
Hi, Is it possible to load the value of a field and use that to compare it to another field and based on that add a new field with a value? E.g. if \[temperature\] \> \[alert1\] add field "outcome =\> normal" if else \[te… --- ## [Logstash Filter Data](https://discuss.elastic.co/t/logstash-filter-data/188847)
**Author:** [@Syed.Ubaid](https://discuss.elastic.co/u/Syed.Ubaid)\ **Replies:** 1\ **Last updated:** [July 4, 2019, 6:59am UTC](https://discuss.elastic.co/t/logstash-filter-data/188847 "2019-07-04T06:59:56Z")
How to filter data in logstash and can we add filter data into new field for example I have upload the picture if you see the column of message column there is different transaction for example MCB Debit Card Charged,… --- ## [Logstash xpack monitoring wrong config](https://discuss.elastic.co/t/logstash-xpack-monitoring-wrong-config/187574)
**Author:** [@hriegger](https://discuss.elastic.co/u/hriegger)\ **Replies:** 0\ **Last updated:** [June 26, 2019, 1:06pm UTC](https://discuss.elastic.co/t/logstash-xpack-monitoring-wrong-config/187574 "2019-06-26T13:06:16Z")
In file /usr/share/logstash/x-pack/lib/template.cfg.erb document\_type is defined, where elasticsearch \> 7.0 doesn't support this. --- ## [Logstash JDBC input plugins fetch data from mysql multiple times](https://discuss.elastic.co/t/logstash-jdbc-input-plugins-fetch-data-from-mysql-multiple-times/188844)
**Author:** [@kumaram](https://discuss.elastic.co/u/kumaram)\ **Replies:** 0\ **Last updated:** [July 4, 2019, 6:27am UTC](https://discuss.elastic.co/t/logstash-jdbc-input-plugins-fetch-data-from-mysql-multiple-times/188844 "2019-07-04T06:27:32Z")
Steps to reproduce- I have created multiple config files to fetch data from each table in mysql and put in /etc/logstash/conf.d folder Start logstash service as sudo systemctl start logstash Run following command to ex… --- ## [Does ELK support code profile like JvisualVm](https://discuss.elastic.co/t/does-elk-support-code-profile-like-jvisualvm/188837)
**Author:** [@chandramohan.j](https://discuss.elastic.co/u/chandramohan.j)\ **Replies:** 0\ **Last updated:** [July 4, 2019, 6:05am UTC](https://discuss.elastic.co/t/does-elk-support-code-profile-like-jvisualvm/188837 "2019-07-04T06:05:40Z")
Does ELK support code profile like JvisualVM... Like Call traces - JVM CPU execution for each methods. --- ## [Cannot Ingest Logs from Kafka with Logstash](https://discuss.elastic.co/t/cannot-ingest-logs-from-kafka-with-logstash/188835)
**Author:** [@ashimmahara](https://discuss.elastic.co/u/ashimmahara)\ **Replies:** 0\ **Last updated:** [July 4, 2019, 5:59am UTC](https://discuss.elastic.co/t/cannot-ingest-logs-from-kafka-with-logstash/188835 "2019-07-04T05:59:06Z")
I have been unsuccessful in Ingesting Logs from Kafka topic with Logstash 6.6.2. I have successfully pushed the logs to Kafka topic using Filebeat 6.5.4. Here is my Logstash config: input { kafka { topics =\> \["wazuh-… --- ## [Logstash converting empty string into 0.0](https://discuss.elastic.co/t/logstash-converting-empty-string-into-0-0/188797)
**Author:** [@rahulkothanath](https://discuss.elastic.co/u/rahulkothanath)\ **Replies:** 3\ **Last updated:** [July 3, 2019, 9:15pm UTC](https://discuss.elastic.co/t/logstash-converting-empty-string-into-0-0/188797 "2019-07-03T21:15:54Z")
Hi, I am converting an amount value into float as follows. mutate{ convert =\> { "AMT" =\> "float" } if "AMT"="" empty spcae ,it give 0.0 as the conversion output. this may mislead people since it is an amount field… --- ## [Catch records and erros that don't go into the elasticsearch cluster in another bucket](https://discuss.elastic.co/t/catch-records-and-erros-that-dont-go-into-the-elasticsearch-cluster-in-another-bucket/188389)
**Author:** [@Mohit\_Ruke](https://discuss.elastic.co/u/Mohit_Ruke)\ **Replies:** 5\ **Last updated:** [July 3, 2019, 9:05pm UTC](https://discuss.elastic.co/t/catch-records-and-erros-that-dont-go-into-the-elasticsearch-cluster-in-another-bucket/188389 "2019-07-03T21:05:47Z")
Hi All, Trying to transfer data from S3 bucket to ES. Using Logstash for that. Everytime a new file is deposited, it will be transferred to ES. The follow is the config file. input { s3 { … --- ## [Google cloud storage input plugin](https://discuss.elastic.co/t/google-cloud-storage-input-plugin/188805)
**Author:** [@jamesp220291](https://discuss.elastic.co/u/jamesp220291)\ **Replies:** 0\ **Last updated:** [July 3, 2019, 8:36pm UTC](https://discuss.elastic.co/t/google-cloud-storage-input-plugin/188805 "2019-07-03T20:36:18Z")
Hi all Having an issue with the google cloud storage input plugin. I am wanting to delete the history and reprocess all of the json logs again. I have tried deleting the default db file and creating a new one, however… --- ## [DateParsinfailure](https://discuss.elastic.co/t/dateparsinfailure/188798)
**Author:** [@Aymen\_Ben\_Moussa](https://discuss.elastic.co/u/Aymen_Ben_Moussa)\ **Replies:** 1\ **Last updated:** [July 3, 2019, 7:25pm UTC](https://discuss.elastic.co/t/dateparsinfailure/188798 "2019-07-03T19:25:40Z")
this is my struct "payload" =\> { "status" =\> "success", "createdAt" =\> "2019-07-01 12:34:45 +0000", "id" =\> "5d19fde56eca4555d9f61b", and this is my logstash fil… --- ## [How to write logical or(||) in logstash](https://discuss.elastic.co/t/how-to-write-logical-or-in-logstash/188794)
**Author:** [@naveenrt23](https://discuss.elastic.co/u/naveenrt23)\ **Replies:** 2\ **Last updated:** [July 3, 2019, 6:57pm UTC](https://discuss.elastic.co/t/how-to-write-logical-or-in-logstash/188794 "2019-07-03T18:57:55Z")
Hello, How can I write a logical or operator for two values? I've tried if \[type\] == "test-request" || \[type\] == "test1-request" but this gives me a syntax error. I want it to work on how a logical OR operator works.… --- ## [Need help with Parsing Multiline StackTrace](https://discuss.elastic.co/t/need-help-with-parsing-multiline-stacktrace/188573)
**Author:** [@dips](https://discuss.elastic.co/u/dips)\ **Replies:** 4\ **Last updated:** [July 3, 2019, 6:23pm UTC](https://discuss.elastic.co/t/need-help-with-parsing-multiline-stacktrace/188573 "2019-07-03T18:23:07Z")
Hello everyone, I am trying to parse multiline stack trace log file through logstash and am trying to figure out how to get this log file to load properly in elastic search. My logstash.conf file: input{ file{ … --- ## [Gelf input missing short\_message, no DEBUG info available](https://discuss.elastic.co/t/gelf-input-missing-short-message-no-debug-info-available/188792)
**Author:** [@ErisDS](https://discuss.elastic.co/u/ErisDS)\ **Replies:** 0\ **Last updated:** [July 3, 2019, 6:23pm UTC](https://discuss.elastic.co/t/gelf-input-missing-short-message-no-debug-info-available/188792 "2019-07-03T18:23:03Z")
I have an ELK setup which uses the gelf input plugin. Currently, logs are sent from Node.js using Bunyan and the dependency gelf-stream, which in term depends on gelfling. The code for creating a message is here: https… --- ## [Logstash.outputs.elasticsearch Marking url as dead. connection issue. AWS EBS datastore](https://discuss.elastic.co/t/logstash-outputs-elasticsearch-marking-url-as-dead-connection-issue-aws-ebs-datastore/185993)
**Author:** [@Harikrishna\_Sarvaiya](https://discuss.elastic.co/u/Harikrishna_Sarvaiya)\ **Replies:** 8\ **Last updated:** [July 3, 2019, 9:00am UTC](https://discuss.elastic.co/t/logstash-outputs-elasticsearch-marking-url-as-dead-connection-issue-aws-ebs-datastore/185993 "2019-07-03T09:00:01Z")
Checked all the existing threads couldn't help any. \[2019-06-17T02:23:50,125\]\[WARN \]\[logstash.outputs.elasticsearch\] Marking url as dead. Last error: \[LogStash::Outputs::ElasticSearch::HttpClient::Pool::HostUnreachableE… --- ## [Confusion on Removal of Mapping Types and document\_type in LS7.x](https://discuss.elastic.co/t/confusion-on-removal-of-mapping-types-and-document-type-in-ls7-x/188759)
**Author:** [@ruhrohshingo](https://discuss.elastic.co/u/ruhrohshingo)\ **Replies:** 0\ **Last updated:** [July 3, 2019, 3:37pm UTC](https://discuss.elastic.co/t/confusion-on-removal-of-mapping-types-and-document-type-in-ls7-x/188759 "2019-07-03T15:37:06Z")
I've read over this a few times and it's still not fully clicking for me. I've got an Elastic cluster running on 6.6.0 currently and was looking at upgrading to 7.2. For my Logstash pipeline configs I use an elasticsearc… --- ## [Repository discovery and Sync on Redhat Satellite](https://discuss.elastic.co/t/repository-discovery-and-sync-on-redhat-satellite/188682)
**Author:** [@ajaisam](https://discuss.elastic.co/u/ajaisam)\ **Replies:** 3\ **Last updated:** [July 3, 2019, 2:54pm UTC](https://discuss.elastic.co/t/repository-discovery-and-sync-on-redhat-satellite/188682 "2019-07-03T14:54:18Z")
Hi all, I am trying to get the rpms from elastic synced to my Redhat Satellite server and I keep getting url not found. The url is "https://artifacts.elastic.co/packages/7.x/yum" Does anyone on the community have expe… --- ## [Logstash to Elasticsearch - SSL Connection Error](https://discuss.elastic.co/t/logstash-to-elasticsearch-ssl-connection-error/188714)
**Author:** [@Paul\_14](https://discuss.elastic.co/u/Paul_14)\ **Replies:** 2\ **Last updated:** [July 3, 2019, 2:48pm UTC](https://discuss.elastic.co/t/logstash-to-elasticsearch-ssl-connection-error/188714 "2019-07-03T14:48:30Z")
Hello, I am attempting to enable SSL encryption between Logstash and Elasticsearch but on restarting my logstash node, I receive error: \[2019-07-03T12:36:42,992\]\[DEBUG\]\[logstash.outputs.elasticsearch\] Running health c… --- ## [Attempting to install template](https://discuss.elastic.co/t/attempting-to-install-template/188744)
**Author:** [@GedeoN](https://discuss.elastic.co/u/GedeoN)\ **Replies:** 2\ **Last updated:** [July 3, 2019, 2:47pm UTC](https://discuss.elastic.co/t/attempting-to-install-template/188744 "2019-07-03T14:47:18Z")
Bonsoir Messieurs Dames, Une nouvelle étrangeté m'est arrivée aujourd'hui. Je travaillais sur logstash, dans le but de parser un message à l'aide de grok. Pendant mes tests, j'ai dû redémarrer plusieurs fois mon conta… --- ## [Problem converting latitude and longitude from jdbc to elastic geo\_point using logstash](https://discuss.elastic.co/t/problem-converting-latitude-and-longitude-from-jdbc-to-elastic-geo-point-using-logstash/188609)
**Author:** [@roebe](https://discuss.elastic.co/u/roebe)\ **Replies:** 1\ **Last updated:** [July 3, 2019, 1:24pm UTC](https://discuss.elastic.co/t/problem-converting-latitude-and-longitude-from-jdbc-to-elastic-geo-point-using-logstash/188609 "2019-07-03T13:24:20Z")
Hi, having issues running a logstash config which is supposed to convert lat and lon into a field type geo\_point my current config file looks like: jdbc\_user =\> "elastic" statement =\> "SELECT \* FROM mChoice\_TotalPartne… --- ## [Multiple syslog input from different Device to Logstash](https://discuss.elastic.co/t/multiple-syslog-input-from-different-device-to-logstash/188672)
**Author:** [@IGSLXW](https://discuss.elastic.co/u/IGSLXW)\ **Replies:** 1\ **Last updated:** [July 3, 2019, 1:21pm UTC](https://discuss.elastic.co/t/multiple-syslog-input-from-different-device-to-logstash/188672 "2019-07-03T13:21:47Z")
We are having a logstash receiving syslog from multiple sources/hosts. Changing the syslog remote port from 514 to other is not our option. Is there any practice to separate / route the log from syslog to different pip… --- ## [Sum values of two csv inputfiles result to an index file](https://discuss.elastic.co/t/sum-values-of-two-csv-inputfiles-result-to-an-index-file/188699)
**Author:** [@subok](https://discuss.elastic.co/u/subok)\ **Replies:** 1\ **Last updated:** [July 3, 2019, 1:07pm UTC](https://discuss.elastic.co/t/sum-values-of-two-csv-inputfiles-result-to-an-index-file/188699 "2019-07-03T13:07:27Z")
Hi, I need some help and don't know how to start with it. Basically, i have two csv input files (coming from 2 different nodes) and would like to sum the values with the result going to elasticsearch.. Any help is gr… --- ## [Logstash file input plug - exception argument error](https://discuss.elastic.co/t/logstash-file-input-plug-exception-argument-error/188691)
**Author:** [@jamesp220291](https://discuss.elastic.co/u/jamesp220291)\ **Replies:** 2\ **Last updated:** [July 3, 2019, 12:49pm UTC](https://discuss.elastic.co/t/logstash-file-input-plug-exception-argument-error/188691 "2019-07-03T12:49:56Z")
Hi All Getting a weird error when trying to use the file input plugin. Some information on my environment java --version openjdk 11.0.3 2019-04-16 OpenJDK Runtime Environment (build 11.0.3+7-Ubuntu-1ubuntu218.04.1) … --- ## [Elasticsearch output plugin in logstash not working when upgraded to version 7.1.0](https://discuss.elastic.co/t/elasticsearch-output-plugin-in-logstash-not-working-when-upgraded-to-version-7-1-0/188606)
**Author:** [@kaushik.vankayala](https://discuss.elastic.co/u/kaushik.vankayala)\ **Replies:** 4\ **Last updated:** [July 3, 2019, 11:59am UTC](https://discuss.elastic.co/t/elasticsearch-output-plugin-in-logstash-not-working-when-upgraded-to-version-7-1-0/188606 "2019-07-03T11:59:16Z")
Hi Team, I am using a http\_poller input plugin and some filters and an elasticsearch output (cloud instance) plugin with Logstash 6.3.0 version. All was working fine when using a ES v6.3.0 endpoint. conf File; input {… --- ## [Unable to remove field from elastic search](https://discuss.elastic.co/t/unable-to-remove-field-from-elastic-search/188480)
**Author:** [@bhagmathi](https://discuss.elastic.co/u/bhagmathi)\ **Replies:** 6\ **Last updated:** [July 3, 2019, 6:09am UTC](https://discuss.elastic.co/t/unable-to-remove-field-from-elastic-search/188480 "2019-07-03T06:09:29Z")
Here is my index \[ { "\_index" : "ind1", "\_type" : "doc", "\_id" : "ceI6smsB5aL8Y\_rB7r5N", "\_score" : 1.0, "\_source" : { "@timestamp" : "2019-06-27T09:31:08.384Z", "id" : 2, "name" : "abc", } }, { "\_i… --- ## [Index names when using ILM](https://discuss.elastic.co/t/index-names-when-using-ilm/188116)
**Author:** [@drolfe](https://discuss.elastic.co/u/drolfe)\ **Replies:** 3\ **Last updated:** [July 3, 2019, 5:34am UTC](https://discuss.elastic.co/t/index-names-when-using-ilm/188116 "2019-07-03T05:34:58Z")
Hi All, I"m currently using stack 7.1.1, I've having trouble getting logstash to append the YYYY-MM-DD to indexes after enabling Index Lifecycle Managment (ILM) My logstash output looks like the below: output { … --- ## [Contribute to Logstash open source project](https://discuss.elastic.co/t/contribute-to-logstash-open-source-project/188323)
**Author:** [@Jasna\_Bencic1](https://discuss.elastic.co/u/Jasna_Bencic1)\ **Replies:** 0\ **Last updated:** [July 1, 2019, 1:12pm UTC](https://discuss.elastic.co/t/contribute-to-logstash-open-source-project/188323 "2019-07-01T13:12:00Z")
Hi everyone, I know somebody has asked this already but they had GSOC project and I don't. I'm interested in becoming a contributor to Logstash. I've read the documentation on Logstash's GitHub repo. Since there are 0… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=515) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=517)