# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=522 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 523 --- ## [Problem when loading CSV file using logstash](https://discuss.elastic.co/t/problem-when-loading-csv-file-using-logstash/187169)
**Author:** [@soufiane1](https://discuss.elastic.co/u/soufiane1)\ **Replies:** 7\ **Last updated:** [June 25, 2019, 9:39am UTC](https://discuss.elastic.co/t/problem-when-loading-csv-file-using-logstash/187169 "2019-06-25T09:39:13Z")
Hello everyone, I am trying to load a CSV file with the following config : ############################################################################################################### input { file { path =\> "C:… --- ## [Add message attributes while publishing message to SNS](https://discuss.elastic.co/t/add-message-attributes-while-publishing-message-to-sns/187284)
**Author:** [@Aakash\_Ratkal](https://discuss.elastic.co/u/Aakash_Ratkal)\ **Replies:** 0\ **Last updated:** [June 25, 2019, 9:24am UTC](https://discuss.elastic.co/t/add-message-attributes-while-publishing-message-to-sns/187284 "2019-06-25T09:24:29Z")
I am using SNS filter policy to send emails to different stake holders based on the alert types. In order to implement this using logstash plugin I need to add message attributes to the request. Is there any approach to … --- ## [Problem when loading a csv file that contain a DATE field](https://discuss.elastic.co/t/problem-when-loading-a-csv-file-that-contain-a-date-field/187278)
**Author:** [@soufiane1](https://discuss.elastic.co/u/soufiane1)\ **Replies:** 0\ **Last updated:** [June 25, 2019, 9:08am UTC](https://discuss.elastic.co/t/problem-when-loading-a-csv-file-that-contain-a-date-field/187278 "2019-06-25T09:08:42Z")
Hello everyone, I am trying to load a csv file that contains a DATE field but to load a date with a year "2019" it load "1970" like the following example : Code retour: 00 SDT: AHSCPCB2 Valeur après modif: … --- ## [Logstash stuck on file "rotation"](https://discuss.elastic.co/t/logstash-stuck-on-file-rotation/187247)
**Author:** [@LoZio](https://discuss.elastic.co/u/LoZio)\ **Replies:** 1\ **Last updated:** [June 25, 2019, 7:26am UTC](https://discuss.elastic.co/t/logstash-stuck-on-file-rotation/187247 "2019-06-25T07:26:48Z")
I'm reading the DNS debug log file from a Windows DNS server. I can't install any software on the DC, so I resorted to sharing the directory with the logs, mount it on my Linux box, and read/parse it with logstash and f… --- ## [Logstash reference error](https://discuss.elastic.co/t/logstash-reference-error/187230)
**Author:** [@vector4wang](https://discuss.elastic.co/u/vector4wang)\ **Replies:** 0\ **Last updated:** [June 25, 2019, 6:09am UTC](https://discuss.elastic.co/t/logstash-reference-error/187230 "2019-06-25T06:09:55Z")
confi content input { jdbc { jdbc\_driver\_library =\> "D:\\repo\\com\\microsoft\\sqlserver\\sqljdbc4\\4.0\\sqljdbc4-4.0.jar" jdbc\_driver\_class =\> "com.microsoft.sqlserver.jdbc.SQLServerDriver" jdb… --- ## [Performance degraded after enabling x-pack security via logstash data ingestion into elasticsearch 6.8.0](https://discuss.elastic.co/t/performance-degraded-after-enabling-x-pack-security-via-logstash-data-ingestion-into-elasticsearch-6-8-0/187222)
**Author:** [@sureshrupnar](https://discuss.elastic.co/u/sureshrupnar)\ **Replies:** 0\ **Last updated:** [June 25, 2019, 4:21am UTC](https://discuss.elastic.co/t/performance-degraded-after-enabling-x-pack-security-via-logstash-data-ingestion-into-elasticsearch-6-8-0/187222 "2019-06-25T04:21:24Z")
Hi Team, Recently we have migrated from ELK 5.5.x to ELK 6.8.x. If we run logstash configuration file without x-pack basic security, then we are getting good performance. But if we enable x-pack basic security and add… --- ## [JDBC for input and output](https://discuss.elastic.co/t/jdbc-for-input-and-output/186722)
**Author:** [@Donny\_Nightfrost](https://discuss.elastic.co/u/Donny_Nightfrost)\ **Replies:** 4\ **Last updated:** [June 25, 2019, 3:50am UTC](https://discuss.elastic.co/t/jdbc-for-input-and-output/186722 "2019-06-25T03:50:52Z")
Hello everyone, me and my friends are working in a project and stumbling upon a quiet confused method. So we want for logstash to read logs given from source (syslog) and then insert them to mysql database if it matches … --- ## [About Output JDBC plugin](https://discuss.elastic.co/t/about-output-jdbc-plugin/187211)
**Author:** [@vector4wang](https://discuss.elastic.co/u/vector4wang)\ **Replies:** 4\ **Last updated:** [June 25, 2019, 2:54am UTC](https://discuss.elastic.co/t/about-output-jdbc-plugin/187211 "2019-06-25T02:54:13Z")
我们团队正在使用logstash，从db同步数据到db，我使用的是logstash5.4 output的jdbc插件是https://github.com/theangryangel/logstash-output-jdbc，官方并没有提供，而且最新的7.1也没有。所以我想问一下，为什么官方不提供jdbc插件呢？以后会不会提供呢？ --- ## [Using log's timestamp as @timestamp](https://discuss.elastic.co/t/using-logs-timestamp-as-timestamp/186629)
**Author:** [@Artisan\_Ilustrado](https://discuss.elastic.co/u/Artisan_Ilustrado)\ **Replies:** 4\ **Last updated:** [June 25, 2019, 1:46am UTC](https://discuss.elastic.co/t/using-logs-timestamp-as-timestamp/186629 "2019-06-25T01:46:43Z")
I need to use my log's timestamp as @timestamp. I've tried different methods from the internet but none of them worked in my case input { s3 { bucket =\> "\*" access\_key\_id =\> "\*" secret\_access\_key =\> "\*" prefix … --- ## [Can Logstash filtering fix Kibana table issue?](https://discuss.elastic.co/t/can-logstash-filtering-fix-kibana-table-issue/186914)
**Author:** [@EZprogramming](https://discuss.elastic.co/u/EZprogramming)\ **Replies:** 4\ **Last updated:** [June 24, 2019, 11:31pm UTC](https://discuss.elastic.co/t/can-logstash-filtering-fix-kibana-table-issue/186914 "2019-06-24T23:31:35Z")
Initially, I wrote a Logstash configuration file to pull Balena information in JSON data and then parse it with Logstash ruby filter plugin which made the data look like this: { "@version" =\> "1", "values"… --- ## [Logstash Instance Missing from Kibana Monitoring](https://discuss.elastic.co/t/logstash-instance-missing-from-kibana-monitoring/186734)
**Author:** [@jakecramer17](https://discuss.elastic.co/u/jakecramer17)\ **Replies:** 15\ **Last updated:** [June 24, 2019, 8:35pm UTC](https://discuss.elastic.co/t/logstash-instance-missing-from-kibana-monitoring/186734 "2019-06-24T20:35:06Z")
I've configured Filebeat to output sample logs to Logstash. Through Logstash, I've set my \*.conf file to output the data to Elasticsearch. My end goal is to get my log information to appear in Kibana's Discover tab. The… --- ## [Logstash just sits and CPU Spikes](https://discuss.elastic.co/t/logstash-just-sits-and-cpu-spikes/187156)
**Author:** [@OpSecMonkey](https://discuss.elastic.co/u/OpSecMonkey)\ **Replies:** 3\ **Last updated:** [June 24, 2019, 7:45pm UTC](https://discuss.elastic.co/t/logstash-just-sits-and-cpu-spikes/187156 "2019-06-24T19:45:14Z")
Good morning everyone. Today I have downloaded Logstash, ElasticSearch and Kibana. I have everything configured and its all displaying the way it should except for when I run my config file in Powershell. It just sits … --- ## [Logstash cannot install sflow](https://discuss.elastic.co/t/logstash-cannot-install-sflow/187187)
**Author:** [@david.shirk80](https://discuss.elastic.co/u/david.shirk80)\ **Replies:** 0\ **Last updated:** [June 24, 2019, 7:10pm UTC](https://discuss.elastic.co/t/logstash-cannot-install-sflow/187187 "2019-06-24T19:10:40Z")
NOTE - I HAVE TRIED SETTING GEM SOURCE TO HTTP TEMP AND STILL NOTHING!!! TLS is set to use 1.2 all the way on up to modern. I can hit other https sites just fine. xxx@xxx:~ sudo /usr/share/logstash/bin/logstash-plugin … --- ## [How do I convert the following date stamp into integers](https://discuss.elastic.co/t/how-do-i-convert-the-following-date-stamp-into-integers/187161)
**Author:** [@kganesan](https://discuss.elastic.co/u/kganesan)\ **Replies:** 3\ **Last updated:** [June 24, 2019, 7:07pm UTC](https://discuss.elastic.co/t/how-do-i-convert-the-following-date-stamp-into-integers/187161 "2019-06-24T19:07:52Z")
Hello. I have the following date stamp info at the start of my log line and I need to convert them to integers. Log Line: 2019-06-17 14:05:00.202 Expected Output: Year: 2019 Month: 06 Date: 17 Hours: 14 Minutes:… --- ## [Classifying records into quarters based on timestamps](https://discuss.elastic.co/t/classifying-records-into-quarters-based-on-timestamps/187138)
**Author:** [@Michiele\_Ogbagabir](https://discuss.elastic.co/u/Michiele_Ogbagabir)\ **Replies:** 2\ **Last updated:** [June 24, 2019, 6:00pm UTC](https://discuss.elastic.co/t/classifying-records-into-quarters-based-on-timestamps/187138 "2019-06-24T18:00:51Z")
I have records that I would like to classify into quarters based on the timestamp value. The logic I am shooting for is: If the month of the timestamp is 1, 2, or 3 then add\_field { "Quarter" =\> "Q1"} If the month of … --- ## [I want to match one pattern multiple times in a logevent](https://discuss.elastic.co/t/i-want-to-match-one-pattern-multiple-times-in-a-logevent/186860)
**Author:** [@sonukumarsah](https://discuss.elastic.co/u/sonukumarsah)\ **Replies:** 2\ **Last updated:** [June 24, 2019, 12:45pm UTC](https://discuss.elastic.co/t/i-want-to-match-one-pattern-multiple-times-in-a-logevent/186860 "2019-06-24T12:45:11Z")
This is my sample log.. Jan 16, 2019 2:12:00 PM com.InfoProviderImpl calculateTotalExposureTime SEVERE: AOMFrameworkException occured while Executing srl: Group children ClusterNode size is 0 and index is 0 com.aomfra… --- ## [Not able to understand this grok filter for a particular code](https://discuss.elastic.co/t/not-able-to-understand-this-grok-filter-for-a-particular-code/187050)
**Author:** [@deepanshu\_goel](https://discuss.elastic.co/u/deepanshu_goel)\ **Replies:** 1\ **Last updated:** [June 24, 2019, 12:28pm UTC](https://discuss.elastic.co/t/not-able-to-understand-this-grok-filter-for-a-particular-code/187050 "2019-06-24T12:28:30Z")
Here ,i am using a log file but i am not able to understand why the grok is working can anyone please explain me this . MY ACCESS LOG IS:- 184.252.108.229 - - \[20/Sep/2017:13:22:22 +0200\] "GET /css/style.css HTTP/1.1" … --- ## [Concatenate multi arrays into one using ruby](https://discuss.elastic.co/t/concatenate-multi-arrays-into-one-using-ruby/186979)
**Author:** [@sahere37](https://discuss.elastic.co/u/sahere37)\ **Replies:** 5\ **Last updated:** [June 24, 2019, 7:07am UTC](https://discuss.elastic.co/t/concatenate-multi-arrays-into-one-using-ruby/186979 "2019-06-24T07:07:33Z")
hi all, I have following arrays as following: f1={a1,a2,a3,a4} f2={b1,b2,b3,b4} f3={c1,c2,c3,c4} f4={d1,d2,d3,d4} I want to merge these arrays into one array as following which data of each element of new array sep… --- ## [Logstash collecting SOAP messages from IoT Sensors](https://discuss.elastic.co/t/logstash-collecting-soap-messages-from-iot-sensors/187090)
**Author:** [@ladi](https://discuss.elastic.co/u/ladi)\ **Replies:** 0\ **Last updated:** [June 24, 2019, 9:18am UTC](https://discuss.elastic.co/t/logstash-collecting-soap-messages-from-iot-sensors/187090 "2019-06-24T09:18:11Z")
Hi community, I'm using ELK 6.8 stack and I am new to it. I already had my first success when sending JSON data to Logstash. I'm using it to collect the data of several IoT sensors. Now I also have an IoT sensor which i… --- ## [Ipfix templates not refreshed in logstash after modifying list of flow attributes](https://discuss.elastic.co/t/ipfix-templates-not-refreshed-in-logstash-after-modifying-list-of-flow-attributes/186276)
**Author:** [@certesrajesh](https://discuss.elastic.co/u/certesrajesh)\ **Replies:** 1\ **Last updated:** [June 24, 2019, 7:19am UTC](https://discuss.elastic.co/t/ipfix-templates-not-refreshed-in-logstash-after-modifying-list-of-flow-attributes/186276 "2019-06-24T07:19:27Z")
Hi, I need few clarifications regarding logstash templates, I have a usecase where I need to select the set of data(ipfix flow attributes) that are exported in ipfix messages. Whenever I change the set of data, I rest… --- ## [Authentication and enforcing fields](https://discuss.elastic.co/t/authentication-and-enforcing-fields/187033)
**Author:** [@tracyreed](https://discuss.elastic.co/u/tracyreed)\ **Replies:** 0\ **Last updated:** [June 24, 2019, 3:06am UTC](https://discuss.elastic.co/t/authentication-and-enforcing-fields/187033 "2019-06-24T03:06:48Z")
Hello all! Is there any way to tie certain fields to certain auth keys or any way to set a field if a client auths with a particular cert? For example, I want to be able to receive data from multiple business units and… --- ## [Logstash Multiple Inputs on same port query](https://discuss.elastic.co/t/logstash-multiple-inputs-on-same-port-query/186872)
**Author:** [@markbgale](https://discuss.elastic.co/u/markbgale)\ **Replies:** 2\ **Last updated:** [June 23, 2019, 10:25pm UTC](https://discuss.elastic.co/t/logstash-multiple-inputs-on-same-port-query/186872 "2019-06-23T22:25:12Z")
I have a novice question.... I currently have 2 pipelines for data coming into Logstash. One of which is beats, and the other is custom http data. Both of which output to seperate Elastic indexes. I've got this working… --- ## [Grok Trying to continue with pattern based on match](https://discuss.elastic.co/t/grok-trying-to-continue-with-pattern-based-on-match/186185)
**Author:** [@talqadi7](https://discuss.elastic.co/u/talqadi7)\ **Replies:** 6\ **Last updated:** [June 23, 2019, 7:07pm UTC](https://discuss.elastic.co/t/grok-trying-to-continue-with-pattern-based-on-match/186185 "2019-06-23T19:07:07Z")
Hi, I'm still new to grok and logstash, and appreciate any help. So, I'm currently setting up a grok pattern, to extract a field from a large message field with a lot of human text. The data is extremely unstructured, so… --- ## [Make @timestamp the same as Log's timestamp](https://discuss.elastic.co/t/make-timestamp-the-same-as-logs-timestamp/186980)
**Author:** [@WarriorHarb](https://discuss.elastic.co/u/WarriorHarb)\ **Replies:** 4\ **Last updated:** [June 23, 2019, 1:40pm UTC](https://discuss.elastic.co/t/make-timestamp-the-same-as-logs-timestamp/186980 "2019-06-23T13:40:56Z")
Hello, I am parsing a log with this format : 2019-04-19 12:00:05.17 FCC7-APS01-B Collector\[71\] \[Info\] Publisher.TransmitterServiceAlivePolicy - Transmitter service is Alive. I applied a grok filter : match =\>{ … --- ## [How to use Port 443 for Logstash and Filebeat? ELK Stack 7.1](https://discuss.elastic.co/t/how-to-use-port-443-for-logstash-and-filebeat-elk-stack-7-1/186801)
**Author:** [@mark.quilates](https://discuss.elastic.co/u/mark.quilates)\ **Replies:** 1\ **Last updated:** [June 23, 2019, 11:22am UTC](https://discuss.elastic.co/t/how-to-use-port-443-for-logstash-and-filebeat-elk-stack-7-1/186801 "2019-06-23T11:22:39Z")
Hi, I want to send my filebeat data using port 443 to logstash. However, I didn't receive any data to my elasticsearch. Any help please? Below is my configuration. Filebeat: output.logstash: # The Logstash hosts host… --- ## [Logstash convert data from array list to objects](https://discuss.elastic.co/t/logstash-convert-data-from-array-list-to-objects/186978)
**Author:** [@arash\_moosavi](https://discuss.elastic.co/u/arash_moosavi)\ **Replies:** 1\ **Last updated:** [June 23, 2019, 9:41am UTC](https://discuss.elastic.co/t/logstash-convert-data-from-array-list-to-objects/186978 "2019-06-23T09:41:02Z")
I send following data to Logstash: { "a": 1, "b": 2, "c": 3, "d": { \["key": "felan", "value": 123\], \["key": "bahman", "value":321\] } } I want to convert it to: { "a": 1, "b": 2, "c": 3, "d": { "felan":… --- ## [Update / Delete rows in DB using Logstash JDBC input plugin](https://discuss.elastic.co/t/update-delete-rows-in-db-using-logstash-jdbc-input-plugin/186887)
**Author:** [@vasek](https://discuss.elastic.co/u/vasek)\ **Replies:** 1\ **Last updated:** [June 23, 2019, 7:44am UTC](https://discuss.elastic.co/t/update-delete-rows-in-db-using-logstash-jdbc-input-plugin/186887 "2019-06-23T07:44:40Z")
Hi, is it possible to use SQL operations like UPDATE or DELETE in statement parameter of Logstash JDBC input plugin? Or only SELECT operation is enabled? Regads, Vasek --- ## [JDBC input scheduling issue - is it possible to run query a minute after the previous query finished](https://discuss.elastic.co/t/jdbc-input-scheduling-issue-is-it-possible-to-run-query-a-minute-after-the-previous-query-finished/185568)
**Author:** [@Andrey\_Tomilin](https://discuss.elastic.co/u/Andrey_Tomilin)\ **Replies:** 1\ **Last updated:** [June 23, 2019, 7:48am UTC](https://discuss.elastic.co/t/jdbc-input-scheduling-issue-is-it-possible-to-run-query-a-minute-after-the-previous-query-finished/185568 "2019-06-23T07:48:11Z")
I'm using the Jdbc input to retrieve data from MS SQL database once a minute. Usually it works fine. But as we know database performance is not very reliable thing and sometime it's take longer than one minute to the qu… --- ## [Logstash input JDBC plugin and MariaDB](https://discuss.elastic.co/t/logstash-input-jdbc-plugin-and-mariadb/186985)
**Author:** [@vasek](https://discuss.elastic.co/u/vasek)\ **Replies:** 0\ **Last updated:** [June 23, 2019, 6:51am UTC](https://discuss.elastic.co/t/logstash-input-jdbc-plugin-and-mariadb/186985 "2019-06-23T06:51:43Z")
Hi Elasticians, I prepared example How to connect to MariaDB using Logstash JDBC input plugin. I hope it will save some time :wink: Servers ES node - 10.88.88.231 MariaDB server - 10.88.88.150 \[10.88.88.231\] Down… --- ## [Logstash Exec Curl - cant create valid JSON](https://discuss.elastic.co/t/logstash-exec-curl-cant-create-valid-json/185301)
**Author:** [@johnwood](https://discuss.elastic.co/u/johnwood)\ **Replies:** 2\ **Last updated:** [June 23, 2019, 12:43am UTC](https://discuss.elastic.co/t/logstash-exec-curl-cant-create-valid-json/185301 "2019-06-23T00:43:09Z")
I was referred to GutHub by ES Cloud Support in ticket #00344788 but I thought I should check here first before heading over there. I am trying to get a logstash exec command to work with a curl call that needs JSON pay… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=521) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=523)