# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=501 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 502 --- ## [If statement is not working for a certain path \[log\]\[file\]\[path\]](https://discuss.elastic.co/t/if-statement-is-not-working-for-a-certain-path-log-file-path/192976)
**Author:** [@nitzanm](https://discuss.elastic.co/u/nitzanm)\ **Replies:** 3\ **Last updated:** [July 31, 2019, 12:50pm UTC](https://discuss.elastic.co/t/if-statement-is-not-working-for-a-certain-path-log-file-path/192976 "2019-07-31T12:50:24Z")
can you please help me understand why this "if" is not working on this path? this path comes from filebeat to logstash: "E:\\OUM82\\EGGED\\TI\_DS\_FILES\\INFASQLS\_LOGS\\INFASQLS-Extractor-legacy\_BI\_Repository-28072019.log" i… --- ## [Logstash not pushing logs to ElasticSearch](https://discuss.elastic.co/t/logstash-not-pushing-logs-to-elasticsearch/190915)
**Author:** [@Brindha\_S](https://discuss.elastic.co/u/Brindha_S)\ **Replies:** 26\ **Last updated:** [July 31, 2019, 12:40pm UTC](https://discuss.elastic.co/t/logstash-not-pushing-logs-to-elasticsearch/190915 "2019-07-31T12:40:41Z")
I have configured syslog server and ELK in a single Windows machine. I'm getting logs via port 514 without issues, which creates a new text file everyday. The logstash's config output creates new indices every day as pe… --- ## [Create nested JSON with aggregate filter plugin](https://discuss.elastic.co/t/create-nested-json-with-aggregate-filter-plugin/192690)
**Author:** [@vschaefer](https://discuss.elastic.co/u/vschaefer)\ **Replies:** 12\ **Last updated:** [July 31, 2019, 12:37pm UTC](https://discuss.elastic.co/t/create-nested-json-with-aggregate-filter-plugin/192690 "2019-07-31T12:37:07Z")
Hi, i am trying to import data from mySQL DB by Logstash and would like to create a nested JSON. The data from the db looks as follows: | fkm\_family | s\_family | pkm\_article | | ---------- | -------- | ----------- | |… --- ## [Field type doesn't change in logstash](https://discuss.elastic.co/t/field-type-doesnt-change-in-logstash/193038)
**Author:** [@Mohammad\_hossein\_Taj](https://discuss.elastic.co/u/Mohammad_hossein_Taj)\ **Replies:** 3\ **Last updated:** [July 31, 2019, 12:24pm UTC](https://discuss.elastic.co/t/field-type-doesnt-change-in-logstash/193038 "2019-07-31T12:24:42Z")
Hi. I want to extract hour from timestamp and I do it but its type is String and I'm trying to change the type of hour to integer but it doesn't effect. here is my logstash config: input { beats { client\_inactivit… --- ## [Carapted json files if using TCP instead UDP](https://discuss.elastic.co/t/carapted-json-files-if-using-tcp-instead-udp/193106)
**Author:** [@zakabluk](https://discuss.elastic.co/u/zakabluk)\ **Replies:** 0\ **Last updated:** [July 31, 2019, 11:29am UTC](https://discuss.elastic.co/t/carapted-json-files-if-using-tcp-instead-udp/193106 "2019-07-31T11:29:18Z")
\*\*\*Logstash 7.2 in docker container. For test I used simple configuration. input { tcp { port =\> 5959 codec =\> json } } output { amazon\_es } And my logs in Kibana has tag: \_jsonparsefailure . When I used udp … --- ## [S3 csv data does not reach to elasticsearch](https://discuss.elastic.co/t/s3-csv-data-does-not-reach-to-elasticsearch/193052)
**Author:** [@shwesinhan](https://discuss.elastic.co/u/shwesinhan)\ **Replies:** 2\ **Last updated:** [July 31, 2019, 8:54am UTC](https://discuss.elastic.co/t/s3-csv-data-does-not-reach-to-elasticsearch/193052 "2019-07-31T08:54:26Z")
hi community i try to send s3 \> logstash \> aws es so, i use s3 input filter \> csv filter \> aws es output filter It seems can download s3 csv files and reach csv filter as well . but i don't see data in kibana here i… --- ## [Exploit Netflow data to verify IPAM](https://discuss.elastic.co/t/exploit-netflow-data-to-verify-ipam/193072)
**Author:** [@jonnhy\_stellmar](https://discuss.elastic.co/u/jonnhy_stellmar)\ **Replies:** 0\ **Last updated:** [July 31, 2019, 8:48am UTC](https://discuss.elastic.co/t/exploit-netflow-data-to-verify-ipam/193072 "2019-07-31T08:48:54Z")
Hi, I would like to use the Netflow data in ELK and create a list of all IP addresses that are displayed to do a check agains our IPAM database to see what is really used and what not. I would like to create a script to… --- ## [CIDR filter detect which ip matched](https://discuss.elastic.co/t/cidr-filter-detect-which-ip-matched/193051)
**Author:** [@freki](https://discuss.elastic.co/u/freki)\ **Replies:** 1\ **Last updated:** [July 31, 2019, 8:43am UTC](https://discuss.elastic.co/t/cidr-filter-detect-which-ip-matched/193051 "2019-07-31T08:43:44Z")
hey guys :slight\_smile: i wonder if it is possible to set a tag within the cidr filter depending on what ip (src or dst) matched the network. this is my code so far. cidr { address =\> \[ "%{src\_ip}", "%… --- ## [Issue in Oracle database query using jdbc for DML statement. (Sequel::DatabaseError: Java::JavaSql::SQLSyntaxErrorException: ORA-00911: invalid character)](https://discuss.elastic.co/t/issue-in-oracle-database-query-using-jdbc-for-dml-statement-sequel-java-ora-00911-invalid-character/193062)
**Author:** [@raj.teaser](https://discuss.elastic.co/u/raj.teaser)\ **Replies:** 0\ **Last updated:** [July 31, 2019, 7:53am UTC](https://discuss.elastic.co/t/issue-in-oracle-database-query-using-jdbc-for-dml-statement-sequel-java-ora-00911-invalid-character/193062 "2019-07-31T07:53:08Z")
Hi I using oracle database and performing DML operation using logstash. But getting syntax error in query. input{ jdbc { jdbc\_driver\_library =\> "{jdbc\_path}" jdbc\_driver\_class =\> "{oracle\_driver}" jd… --- ## [Error import data](https://discuss.elastic.co/t/error-import-data/192925)
**Author:** [@Axel\_MORVAN](https://discuss.elastic.co/u/Axel_MORVAN)\ **Replies:** 2\ **Last updated:** [July 31, 2019, 6:41am UTC](https://discuss.elastic.co/t/error-import-data/192925 "2019-07-31T06:41:57Z")
hello, I need to import data from a mysql databases with logstash to elasticsearch but when i import data i have this error : elasticsearch - Could not index event to Elasticsearch. {:status=\>400, :action=\>\["index", {… --- ## [Logstash output plugins and self signed certificates?](https://discuss.elastic.co/t/logstash-output-plugins-and-self-signed-certificates/192848)
**Author:** [@Sjaak01](https://discuss.elastic.co/u/Sjaak01)\ **Replies:** 2\ **Last updated:** [July 31, 2019, 6:03am UTC](https://discuss.elastic.co/t/logstash-output-plugins-and-self-signed-certificates/192848 "2019-07-31T06:03:26Z")
Hi, I want to use the Redmine output plugin but I'm running into a small problem. Our Redmine uses a self signed certificate and when I enable ssl =\> true in the output I get the following error which i assume is becaus… --- ## [How to convert the date string into datetime format](https://discuss.elastic.co/t/how-to-convert-the-date-string-into-datetime-format/192941)
**Author:** [@Karn\_Gusain](https://discuss.elastic.co/u/Karn_Gusain)\ **Replies:** 10\ **Last updated:** [July 31, 2019, 5:03am UTC](https://discuss.elastic.co/t/how-to-convert-the-date-string-into-datetime-format/192941 "2019-07-31T05:03:38Z")
Hello Folks, I have searched around all the possible elastic.co similar related posts and googled the possible stuff but did not get the answer to go with, i've below lrow og data which looks like below .. my-ldap-se… --- ## [Parsing User Agent](https://discuss.elastic.co/t/parsing-user-agent/193025)
**Author:** [@Jay-R\_Manguba](https://discuss.elastic.co/u/Jay-R_Manguba)\ **Replies:** 0\ **Last updated:** [July 31, 2019, 3:53am UTC](https://discuss.elastic.co/t/parsing-user-agent/193025 "2019-07-31T03:53:58Z")
I want to create new field for user agent using scripted fields because the user agent is in full\_log. Can someone help me how to can i separate it? --- ## [Pivot the field using csv data in LS](https://discuss.elastic.co/t/pivot-the-field-using-csv-data-in-ls/192539)
**Author:** [@rkhapre](https://discuss.elastic.co/u/rkhapre)\ **Replies:** 10\ **Last updated:** [July 31, 2019, 2:26am UTC](https://discuss.elastic.co/t/pivot-the-field-using-csv-data-in-ls/192539 "2019-07-31T02:26:58Z")
Hi All I have a csv data like this Object#,Type,Count A001,P0,5 A002,P1,3 A003,P0,4 A005,P1,2 A006,P2,2 I need to output in this below format, i tried with ruby filter, but not sure which gem to use exactly Plea… --- ## [Calling elasticseach output intermittently](https://discuss.elastic.co/t/calling-elasticseach-output-intermittently/192975)
**Author:** [@Ram\_Sharan\_Mittal](https://discuss.elastic.co/u/Ram_Sharan_Mittal)\ **Replies:** 0\ **Last updated:** [July 30, 2019, 7:04pm UTC](https://discuss.elastic.co/t/calling-elasticseach-output-intermittently/192975 "2019-07-30T19:04:11Z")
I have a file input and push to our elasticsearch . Like to reduce load on elasticserach so logstash should push file's content after a regular time (e.g 1 min - 5 min) by configuration But filecan move/delete so like to… --- ## [Files from logstash to elasticsearch multiplied](https://discuss.elastic.co/t/files-from-logstash-to-elasticsearch-multiplied/192900)
**Author:** [@luc1](https://discuss.elastic.co/u/luc1)\ **Replies:** 1\ **Last updated:** [July 30, 2019, 6:49pm UTC](https://discuss.elastic.co/t/files-from-logstash-to-elasticsearch-multiplied/192900 "2019-07-30T18:49:46Z")
Hi, I wonder if anyone encountered a problem such as mine: My files are multiplied 156 times (or a multiple of 156) when I'm running logstash on them. --- ## [Logstash Email Output](https://discuss.elastic.co/t/logstash-email-output/192770)
**Author:** [@userR](https://discuss.elastic.co/u/userR)\ **Replies:** 2\ **Last updated:** [July 30, 2019, 5:50pm UTC](https://discuss.elastic.co/t/logstash-email-output/192770 "2019-07-30T17:50:40Z")
Hi, I am wondering how to configure an email output for Outlook. In my current logstash configuration, I have the following code: output { elasticsearch { hosts =\> "localhost:9200" index =\> "somename" } i… --- ## [Logstash since dbpath= null](https://discuss.elastic.co/t/logstash-since-dbpath-null/192822)
**Author:** [@trisha\_1212](https://discuss.elastic.co/u/trisha_1212)\ **Replies:** 2\ **Last updated:** [July 30, 2019, 5:02pm UTC](https://discuss.elastic.co/t/logstash-since-dbpath-null/192822 "2019-07-30T17:02:13Z")
I am getting this logstash error while running a command.bin/logstash -f /Users/chetatri/Desktop/logstash/csv.conf and further, I figure out it because of #sincedb\_path =\> "/dev/null" and if I comment this it working … --- ## [Elastic search storage format](https://discuss.elastic.co/t/elastic-search-storage-format/192824)
**Author:** [@trisha\_1212](https://discuss.elastic.co/u/trisha_1212)\ **Replies:** 2\ **Last updated:** [July 30, 2019, 5:01pm UTC](https://discuss.elastic.co/t/elastic-search-storage-format/192824 "2019-07-30T17:01:41Z")
Hi All, I am using elastic search 6.4.3 and mac osx. I wanted help I am trying to put the content in CSV to elastic search via logstash. I am able to push the content to the Elastic search. Its display the path of CSV… --- ## [Date Filter Woes](https://discuss.elastic.co/t/date-filter-woes/192737)
**Author:** [@wwalker](https://discuss.elastic.co/u/wwalker)\ **Replies:** 1\ **Last updated:** [July 30, 2019, 4:40pm UTC](https://discuss.elastic.co/t/date-filter-woes/192737 "2019-07-30T16:40:21Z")
I have a field that tracks when a ticket was created, sys\_created\_on. The time is in local timezone, -05:00 currently. I have the following logstash config: date { match =\> \[ "sys\_created\_on", "yyyy-MM-dd HH:mm:… --- ## [Logstash Kafka Avro integration failing](https://discuss.elastic.co/t/logstash-kafka-avro-integration-failing/192499)
**Author:** [@NSK](https://discuss.elastic.co/u/NSK)\ **Replies:** 5\ **Last updated:** [July 30, 2019, 4:28pm UTC](https://discuss.elastic.co/t/logstash-kafka-avro-integration-failing/192499 "2019-07-30T16:28:03Z")
Hi - I am trying to consume a topic from Kafka using Avro Deserializer in Logstash and getting the below error. \[ERROR\]\[logstash.inputs.kafka \] Unable to create Kafka consumer from given configuration {:kafka\_error\_m… --- ## [Logstash Date and Kibana Issues](https://discuss.elastic.co/t/logstash-date-and-kibana-issues/192817)
**Author:** [@wwalker](https://discuss.elastic.co/u/wwalker)\ **Replies:** 3\ **Last updated:** [July 30, 2019, 4:14pm UTC](https://discuss.elastic.co/t/logstash-date-and-kibana-issues/192817 "2019-07-30T16:14:23Z")
Not really sure where this goes but it isn't getting any bites in the Logstash forums so... I have a field that tracks when a ticket was created, sys\_created\_on. The time is in local timezone, -05:00 currently. I have… --- ## [How to parse grok in Logstash](https://discuss.elastic.co/t/how-to-parse-grok-in-logstash/192746)
**Author:** [@Sajal](https://discuss.elastic.co/u/Sajal)\ **Replies:** 1\ **Last updated:** [July 30, 2019, 4:05pm UTC](https://discuss.elastic.co/t/how-to-parse-grok-in-logstash/192746 "2019-07-30T16:05:40Z")
Tanium 148 - \[GIS:-Startup-Programs@017472 Tanium 148 - \[WEC:-Subscription-Manager-Setting@017472 Tanium 148 - \[GIS:-Trace-Executed-Processes---Powershell@017472 Tanium 148 - \[GIS:-Scheduled-Task-Information@017472 T… --- ## [Logstash Filter to parse a string](https://discuss.elastic.co/t/logstash-filter-to-parse-a-string/192780)
**Author:** [@Gabriel\_Shen](https://discuss.elastic.co/u/Gabriel_Shen)\ **Replies:** 4\ **Last updated:** [July 30, 2019, 4:03pm UTC](https://discuss.elastic.co/t/logstash-filter-to-parse-a-string/192780 "2019-07-30T16:03:05Z")
I have a log file with each line being a long string (each being very similar). An example line is = "12345555678890,1234567890ABBCCDDEEFFGG123456789GGFFFEEDDCCBBAA" All I want is the numbers in between the two G's. Thos… --- ## [Reducing stalling\_threads on restart of logstash](https://discuss.elastic.co/t/reducing-stalling-threads-on-restart-of-logstash/192769)
**Author:** [@rsaeks](https://discuss.elastic.co/u/rsaeks)\ **Replies:** 1\ **Last updated:** [July 30, 2019, 3:56pm UTC](https://discuss.elastic.co/t/reducing-stalling-threads-on-restart-of-logstash/192769 "2019-07-30T15:56:18Z")
Good afternoon, I've been using logstash for a while and have been trying to troubleshoot the cause of stalling\_threads\_info being thrown when restarting the service. I've tried switching out a variety of inputs / filte… --- ## [Logstash jdbc input output in one conf not working](https://discuss.elastic.co/t/logstash-jdbc-input-output-in-one-conf-not-working/192929)
**Author:** [@frankfoti](https://discuss.elastic.co/u/frankfoti)\ **Replies:** 0\ **Last updated:** [July 30, 2019, 2:57pm UTC](https://discuss.elastic.co/t/logstash-jdbc-input-output-in-one-conf-not-working/192929 "2019-07-30T14:57:08Z")
Output works when input is not jdbc but this config ends in error: \[ERROR\]\[org.logstash.Logstash \] java.lang.NoClassDefFoundError: Could not initialize class com.mysql.jdbc.PreparedStatement conf: 'root@cloudcontro… --- ## [How to do split into array of object](https://discuss.elastic.co/t/how-to-do-split-into-array-of-object/192811)
**Author:** [@mrjayarajj](https://discuss.elastic.co/u/mrjayarajj)\ **Replies:** 2\ **Last updated:** [July 30, 2019, 2:48pm UTC](https://discuss.elastic.co/t/how-to-do-split-into-array-of-object/192811 "2019-07-30T14:48:21Z")
i am new to logstash my SQL output is like below FIRSTNAME-----LASTNAME----------PERMISSIONS jayaraj------------jaganathan----------BLK\_RTE\_APPRV,PLACE\_GPO,ORD\_AUTH i need to convert into { "firstName" : "Jayaraj", … --- ## [Logstash error : No implicit conversion of Integer into String](https://discuss.elastic.co/t/logstash-error-no-implicit-conversion-of-integer-into-string/192901)
**Author:** [@SalmaOua](https://discuss.elastic.co/u/SalmaOua)\ **Replies:** 3\ **Last updated:** [July 30, 2019, 2:46pm UTC](https://discuss.elastic.co/t/logstash-error-no-implicit-conversion-of-integer-into-string/192901 "2019-07-30T14:46:22Z")
Hello there ! I started diving into Elastic stack world recently, everything was going well but when I wanted to take data from a new SQL server table via Logstash, I got this error : My config file looks like this … --- ## [ELK and mysql geopoint](https://discuss.elastic.co/t/elk-and-mysql-geopoint/192888)
**Author:** [@Axel\_MORVAN](https://discuss.elastic.co/u/Axel_MORVAN)\ **Replies:** 6\ **Last updated:** [July 30, 2019, 2:42pm UTC](https://discuss.elastic.co/t/elk-and-mysql-geopoint/192888 "2019-07-30T14:42:35Z")
Hello, I need to create a kibana dashboard with a coordinate map. I import data from a mysql database with latitude and longitute. But I can't create a field type "geo\_point", and after a long search I come to ask you… --- ## [When logstash shutdown, renam file and data can be lost or duplicated](https://discuss.elastic.co/t/when-logstash-shutdown-renam-file-and-data-can-be-lost-or-duplicated/192669)
**Author:** [@Peihui\_He](https://discuss.elastic.co/u/Peihui_He)\ **Replies:** 3\ **Last updated:** [July 30, 2019, 2:06pm UTC](https://discuss.elastic.co/t/when-logstash-shutdown-renam-file-and-data-can-be-lost-or-duplicated/192669 "2019-07-30T14:06:18Z")
i'm ready to use logstah to transfer a local file to kafka. i assume the logstash is shutdown, i rename the file name (data.log -\> data.log.1) and echo one recode like '111' to data.log.1. when i restart logstash, the … [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=500) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=502)