# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=535 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 536 --- ## [Pattern for Specific Date and Time format](https://discuss.elastic.co/t/pattern-for-specific-date-and-time-format/184165)
**Author:** [@Wilks](https://discuss.elastic.co/u/Wilks)\ **Replies:** 1\ **Last updated:** [June 4, 2019, 6:17pm UTC](https://discuss.elastic.co/t/pattern-for-specific-date-and-time-format/184165 "2019-06-04T18:17:58Z")
Hi, I am fairly new to logtstash I am trying to find the correct Date pattern to successfully parse out the date format of this log type but I cant seem to get it to parse properly. I am guessing its something simple b… --- ## [Multiple HTTP Pipelines](https://discuss.elastic.co/t/multiple-http-pipelines/184175)
**Author:** [@salkonuhannovic](https://discuss.elastic.co/u/salkonuhannovic)\ **Replies:** 2\ **Last updated:** [June 4, 2019, 6:11pm UTC](https://discuss.elastic.co/t/multiple-http-pipelines/184175 "2019-06-04T18:11:36Z")
Hi, is it possible to have multiple pipelines which are using the HTTP plugin. Meaning to go for a setup like this: Pipeline 1: input { http { port =\> 8080 } output { elasticsearch { hosts =\> \["${ELAST… --- ## [Logstash-InfluxDB plugin doesnot send complete data to InfluxDB](https://discuss.elastic.co/t/logstash-influxdb-plugin-doesnot-send-complete-data-to-influxdb/182729)
**Author:** [@naveenrt23](https://discuss.elastic.co/u/naveenrt23)\ **Replies:** 4\ **Last updated:** [June 4, 2019, 6:03pm UTC](https://discuss.elastic.co/t/logstash-influxdb-plugin-doesnot-send-complete-data-to-influxdb/182729 "2019-06-04T18:03:55Z")
Hello, I am trying to send data from Logstash(6.2.3) to influxDB(1.7.5) using Logstash-influxDB plugin. I tried to send 10 lines of logs where 7 lines went through and 3 lines didn’t get into InfluxDB. I tried to figur… --- ## [Logstash Ruby Code string](https://discuss.elastic.co/t/logstash-ruby-code-string/184231)
**Author:** [@jonathan01](https://discuss.elastic.co/u/jonathan01)\ **Replies:** 4\ **Last updated:** [June 4, 2019, 5:38pm UTC](https://discuss.elastic.co/t/logstash-ruby-code-string/184231 "2019-06-04T17:38:34Z")
I am new to this and I am a little stuck. I have a input from metric beats "windows" =\> { "perfmon" =\> { "ica" =\> { "output\_pn\_bandwidth" =\> 0, "name" =\> "ICA-CGP 33 (jgillman)" } } I need to get the user name f… --- ## [Two fields share the same name and type, how to combine them into 1 field?](https://discuss.elastic.co/t/two-fields-share-the-same-name-and-type-how-to-combine-them-into-1-field/184229)
**Author:** [@austin0918](https://discuss.elastic.co/u/austin0918)\ **Replies:** 3\ **Last updated:** [June 4, 2019, 5:28pm UTC](https://discuss.elastic.co/t/two-fields-share-the-same-name-and-type-how-to-combine-them-into-1-field/184229 "2019-06-04T17:28:32Z")
Hi guys, I have two fields share the same name and type in an index, one is generated due to a kv{} filter, the second one is generated due to mutate {add\_field =\> {"Classification" =\> ""}} filter. Is there a way to comb… --- ## [Filter date doesn't work](https://discuss.elastic.co/t/filter-date-doesnt-work/184218)
**Author:** [@soufiane1](https://discuss.elastic.co/u/soufiane1)\ **Replies:** 4\ **Last updated:** [June 4, 2019, 5:19pm UTC](https://discuss.elastic.co/t/filter-date-doesnt-work/184218 "2019-06-04T17:19:59Z")
Hello everyone, I have a column called "Date et heure" that is formatted as \[dd/MM/YY HH:mm\] (ie 04/12/2019 17:45), I am using a date filter in order to treat this as a date when passing to ElasticSearch but it doesn't… --- ## [Logstash to logstash with SSL and lumberjack plugin](https://discuss.elastic.co/t/logstash-to-logstash-with-ssl-and-lumberjack-plugin/184223)
**Author:** [@Shawcs](https://discuss.elastic.co/u/Shawcs)\ **Replies:** 0\ **Last updated:** [June 4, 2019, 3:42pm UTC](https://discuss.elastic.co/t/logstash-to-logstash-with-ssl-and-lumberjack-plugin/184223 "2019-06-04T15:42:45Z")
Hi, I try to send data from logstash to another logstash. I followed this doc ofc https://www.elastic.co/guide/en/logstash/6.6/ls-to-ls.html but it didn't work. here is my configuration: On the recieving server: beats… --- ## [Change data field to date type](https://discuss.elastic.co/t/change-data-field-to-date-type/184185)
**Author:** [@soufiane1](https://discuss.elastic.co/u/soufiane1)\ **Replies:** 2\ **Last updated:** [June 4, 2019, 3:02pm UTC](https://discuss.elastic.co/t/change-data-field-to-date-type/184185 "2019-06-04T15:02:17Z")
Hello everyone, I am trying to load a data into kibana with logstash but still receiving the following error : Here is my config : ( "Date et heure" in my csv is like "24/06/2019 21:56") ############## mutate{ … --- ## [Country/location based data in twitter](https://discuss.elastic.co/t/country-location-based-data-in-twitter/181560)
**Author:** [@pathri](https://discuss.elastic.co/u/pathri)\ **Replies:** 1\ **Last updated:** [May 17, 2019, 10:20am UTC](https://discuss.elastic.co/t/country-location-based-data-in-twitter/181560 "2019-05-17T10:20:36Z")
Hi, i am fetching data from twitter and wants to put a geo-location filter while fetching the data. tried country\_code = \["US","IN"\] and location =\["us"\] but no luck please help. Thanks in advance --- ## [Change a Number Field Value](https://discuss.elastic.co/t/change-a-number-field-value/184008)
**Author:** [@gerard.ramos](https://discuss.elastic.co/u/gerard.ramos)\ **Replies:** 11\ **Last updated:** [June 4, 2019, 2:20pm UTC](https://discuss.elastic.co/t/change-a-number-field-value/184008 "2019-06-04T14:20:33Z")
Hi, I was trying to change a value of Windows Event Logs or add a field in a other case. I tried this: filter { if \[log.level\] == "information" { mutate { replace =\> { "\[log.level\]" =\> "información" } } } else i… --- ## [ES output plugin is slow in indexing even with 2 data nodes and 1 master node](https://discuss.elastic.co/t/es-output-plugin-is-slow-in-indexing-even-with-2-data-nodes-and-1-master-node/183561)
**Author:** [@rkhapre](https://discuss.elastic.co/u/rkhapre)\ **Replies:** 21\ **Last updated:** [June 4, 2019, 2:15pm UTC](https://discuss.elastic.co/t/es-output-plugin-is-slow-in-indexing-even-with-2-data-nodes-and-1-master-node/183561 "2019-06-04T14:15:24Z")
Hi All I have the basic configuration where my input and JDBC and output is ES. I have 1 Master node and 2 Data Nodes Case 1 : I am indexing data using ES output plugin by giving host name for both the index hosts =\>… --- ## [Bug with manticore use for time in body](https://discuss.elastic.co/t/bug-with-manticore-use-for-time-in-body/184191)
**Author:** [@Hiven](https://discuss.elastic.co/u/Hiven)\ **Replies:** 0\ **Last updated:** [June 4, 2019, 1:14pm UTC](https://discuss.elastic.co/t/bug-with-manticore-use-for-time-in-body/184191 "2019-06-04T13:14:41Z")
Hello, Just a follow up on my other post. I've followed the exact syntax described by the logstash developer here: It just prints exactly this "{ "query": { "range": { "timestamp": { "gt": "now-1h" } } } }" rather th… --- ## [Logstash is unable to send data when Elastic is configured with SSL in 7.1](https://discuss.elastic.co/t/logstash-is-unable-to-send-data-when-elastic-is-configured-with-ssl-in-7-1/183383)
**Author:** [@Prashant\_Agrawal](https://discuss.elastic.co/u/Prashant_Agrawal)\ **Replies:** 2\ **Last updated:** [June 4, 2019, 12:35pm UTC](https://discuss.elastic.co/t/logstash-is-unable-to-send-data-when-elastic-is-configured-with-ssl-in-7-1/183383 "2019-06-04T12:35:40Z")
Hello Team, I have setup one box setup of Elastic and Kibana with SSL and HTTPS. I used the ca.crt as generated by openssl pkcs12 -in elastic-stack-ca.p12 -out ca.crt -nokeys As well as tried the instance.crt generat… --- ## [Network flow data](https://discuss.elastic.co/t/network-flow-data/184148)
**Author:** [@Vikash\_Singh1](https://discuss.elastic.co/u/Vikash_Singh1)\ **Replies:** 3\ **Last updated:** [June 4, 2019, 11:38am UTC](https://discuss.elastic.co/t/network-flow-data/184148 "2019-06-04T11:38:00Z")
I am having network flow data but it is not in human readable format. How am I gonna segregate it???shaii I use nmap codec or netflow codec???I am stuck can I get some help?? --- ## [Nested XML from filebeat to logstash issue](https://discuss.elastic.co/t/nested-xml-from-filebeat-to-logstash-issue/184063)
**Author:** [@kishorerv93](https://discuss.elastic.co/u/kishorerv93)\ **Replies:** 2\ **Last updated:** [June 4, 2019, 11:09am UTC](https://discuss.elastic.co/t/nested-xml-from-filebeat-to-logstash-issue/184063 "2019-06-04T11:09:01Z")
Hi, Below is my Nested XML file. \ \ \ **Author:** [@decodeit](https://discuss.elastic.co/u/decodeit)\ **Replies:** 2\ **Last updated:** [June 4, 2019, 11:04am UTC](https://discuss.elastic.co/t/json-file-with-logstash/183971 "2019-06-04T11:04:07Z") I converted a csv file to a json file. How to read this file with logstash and put in elasticsearch. I don't see any error. My json file looks like this: \[{"LOR": "LOR ", "CGPA": "CGPA", "TOEFL Score": "TOEFL Score", "G… --- ## [Redis - Logstash - Elasticsearch is not as fast as I expected](https://discuss.elastic.co/t/redis-logstash-elasticsearch-is-not-as-fast-as-i-expected/183924)
**Author:** [@crowtit](https://discuss.elastic.co/u/crowtit)\ **Replies:** 6\ **Last updated:** [June 4, 2019, 9:45am UTC](https://discuss.elastic.co/t/redis-logstash-elasticsearch-is-not-as-fast-as-i-expected/183924 "2019-06-04T09:45:57Z")
Hi, I'm building an almost real-time data pipeline with redis, logstash and elasticsearch. I need to store 2 million documents every minute, but it is not easy for me. So I want to get some advice on my structure. Th… --- ## [Parsing Non Repetitive texts to form structured data](https://discuss.elastic.co/t/parsing-non-repetitive-texts-to-form-structured-data/183981)
**Author:** [@brendachi](https://discuss.elastic.co/u/brendachi)\ **Replies:** 9\ **Last updated:** [June 4, 2019, 9:06am UTC](https://discuss.elastic.co/t/parsing-non-repetitive-texts-to-form-structured-data/183981 "2019-06-04T09:06:04Z")
Hello. Please may i ask what is the best way to parse unstructured non repetitive text files in logstash. I have many text files that i need to process(extracting just small parts of the messages) and i have put an examp… --- ## [Logstash doesn't resume parsing of missed files in read mode](https://discuss.elastic.co/t/logstash-doesnt-resume-parsing-of-missed-files-in-read-mode/184114)
**Author:** [@apr589](https://discuss.elastic.co/u/apr589)\ **Replies:** 0\ **Last updated:** [June 4, 2019, 8:30am UTC](https://discuss.elastic.co/t/logstash-doesnt-resume-parsing-of-missed-files-in-read-mode/184114 "2019-06-04T08:30:47Z")
There are multiple files which I parse . however logstash seems to have missed 3 files , even though it has since\_db entries for it it will not resume when i stopped and restarted logstash . file { path =\> "/home/test… --- ## [Unable to assign logstash aliases to index](https://discuss.elastic.co/t/unable-to-assign-logstash-aliases-to-index/182994)
**Author:** [@wcpoon](https://discuss.elastic.co/u/wcpoon)\ **Replies:** 21\ **Last updated:** [June 4, 2019, 7:41am UTC](https://discuss.elastic.co/t/unable-to-assign-logstash-aliases-to-index/182994 "2019-06-04T07:41:40Z")
Hi Guys, my index lifecycle policies is giving below error, Index lifecycle error illegal\_argument\_exception: index.lifecycle.rollover\_alias \[logstash\] does not point to index \[logstash\] i tried to edit the index and … --- ## [Not sure the prerequisite to make a java filter plugin](https://discuss.elastic.co/t/not-sure-the-prerequisite-to-make-a-java-filter-plugin/184087)
**Author:** [@mmk1995](https://discuss.elastic.co/u/mmk1995)\ **Replies:** 0\ **Last updated:** [June 4, 2019, 4:49am UTC](https://discuss.elastic.co/t/not-sure-the-prerequisite-to-make-a-java-filter-plugin/184087 "2019-06-04T04:49:34Z")
https://www.elastic.co/guide/en/logstash/current/java-filter-plugin.html This document do not show the prerequisite to make a custom java plugin. I am using logstash 6.5.4 is it available to make a custom one? Also, … --- ## [Jruby not finding gender\_detector](https://discuss.elastic.co/t/jruby-not-finding-gender-detector/184072)
**Author:** [@marksparrish](https://discuss.elastic.co/u/marksparrish)\ **Replies:** 2\ **Last updated:** [June 4, 2019, 3:38am UTC](https://discuss.elastic.co/t/jruby-not-finding-gender-detector/184072 "2019-06-04T03:38:30Z")
I have the following ruby filter in my .conf ruby { code =\> ' require "rubygems"; require "gender\_detector"; d = GenderDetector.new(:case\_sensitive =\> false) first\_name = "MARK" if first\_name.to\_s.empty? g… --- ## [Logstash-input-wmi](https://discuss.elastic.co/t/logstash-input-wmi/184083)
**Author:** [@VamPikmin](https://discuss.elastic.co/u/VamPikmin)\ **Replies:** 0\ **Last updated:** [June 4, 2019, 3:28am UTC](https://discuss.elastic.co/t/logstash-input-wmi/184083 "2019-06-04T03:28:01Z")
I wanted to test the plugin but receive this error LoadError: load error: win32ole/win32ole -- java.lang.UnsatisfiedLinkError: /usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/jruby-win32ole-0.8.5/lib/racob-x64.dll: /… --- ## [Help with the configuration Multiples pilelines and GEO IP](https://discuss.elastic.co/t/help-with-the-configuration-multiples-pilelines-and-geo-ip/184071)
**Author:** [@lmoreira](https://discuss.elastic.co/u/lmoreira)\ **Replies:** 0\ **Last updated:** [June 4, 2019, 12:57am UTC](https://discuss.elastic.co/t/help-with-the-configuration-multiples-pilelines-and-geo-ip/184071 "2019-06-04T00:57:06Z")
Dear, how are you? I'm using the community version of elasticstack, I'm trying to setup multiple pipelines using Docker. I need to set up a pipeline for jboss and another for the nginx logs. Configuring Logstash on doc… --- ## [Csv file is being deleted after process](https://discuss.elastic.co/t/csv-file-is-being-deleted-after-process/184041)
**Author:** [@elasticforme](https://discuss.elastic.co/u/elasticforme)\ **Replies:** 2\ **Last updated:** [June 3, 2019, 7:06pm UTC](https://discuss.elastic.co/t/csv-file-is-being-deleted-after-process/184041 "2019-06-03T19:06:54Z")
I have a config which reads csv files. csv file gets to it's location via rcp from another system once a day. but that csv file gets deleted as soon it process reading it. I don't want that to be deleted. how do I do t… --- ## [Logstash exec input constantly runs out of memory](https://discuss.elastic.co/t/logstash-exec-input-constantly-runs-out-of-memory/182719)
**Author:** [@lgwapnitsky](https://discuss.elastic.co/u/lgwapnitsky)\ **Replies:** 1\ **Last updated:** [June 3, 2019, 6:04pm UTC](https://discuss.elastic.co/t/logstash-exec-input-constantly-runs-out-of-memory/182719 "2019-06-03T18:04:55Z")
I'm running a very simple, timestamp-based bash script under logstash's exec input. After a few hours, I stop receiving data, and the following appears in the log \[ERROR\]\[logstash.inputs.exec \] Error while running c… --- ## [Logstash avro reader gives empty results](https://discuss.elastic.co/t/logstash-avro-reader-gives-empty-results/184032)
**Author:** [@rahulkothanath](https://discuss.elastic.co/u/rahulkothanath)\ **Replies:** 0\ **Last updated:** [June 3, 2019, 5:45pm UTC](https://discuss.elastic.co/t/logstash-avro-reader-gives-empty-results/184032 "2019-06-03T17:45:58Z")
I want to read a avro files from kafka topics and insert into elastic search. when I am reading avro files and showing results(codec=\>json) ,it gives empty. below is my configuration: input { kafka { bootstrap\_servers… --- ## [Logstash-input-snmptrap and vendor mibs import](https://discuss.elastic.co/t/logstash-input-snmptrap-and-vendor-mibs-import/184012)
**Author:** [@Flavio\_Campana](https://discuss.elastic.co/u/Flavio_Campana)\ **Replies:** 0\ **Last updated:** [June 3, 2019, 3:25pm UTC](https://discuss.elastic.co/t/logstash-input-snmptrap-and-vendor-mibs-import/184012 "2019-06-03T15:25:54Z")
Hello Is there any "official" way to translate vendor mibs to yaml files suitable for snmptrap input? I seem to remember there was some script to convert them in the plugin folder, but i cant find it and there is no me… --- ## [Logstash-input-jmx repository does not install](https://discuss.elastic.co/t/logstash-input-jmx-repository-does-not-install/181483)
**Author:** [@ashesfall](https://discuss.elastic.co/u/ashesfall)\ **Replies:** 7\ **Last updated:** [June 3, 2019, 2:55pm UTC](https://discuss.elastic.co/t/logstash-input-jmx-repository-does-not-install/181483 "2019-06-03T14:55:40Z")
When following the instructions in the readme at this repository: logstash-plugin install --no-verify ---\> Running in ae1e0380455f LoadError: no such file to load -- bundler require at org/jruby/RubyKernel.java:98… --- ## [Logstash performance (choice)](https://discuss.elastic.co/t/logstash-performance-choice/183907)
**Author:** [@jof300](https://discuss.elastic.co/u/jof300)\ **Replies:** 3\ **Last updated:** [June 3, 2019, 2:12pm UTC](https://discuss.elastic.co/t/logstash-performance-choice/183907 "2019-06-03T14:12:23Z")
Hello everybody, We are implementing an ELK stack to manage our logs. We have all kind of logs (Firewall, syslog, application logs, hardware devices, etc...). We are running logstash on Docker swarm. The estimation of … [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=534) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=536)