# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=499 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 500 --- ## [Mutate filter - Convert a hash data-type to something else](https://discuss.elastic.co/t/mutate-filter-convert-a-hash-data-type-to-something-else/193580)
**Author:** [@luiz.miguelsl](https://discuss.elastic.co/u/luiz.miguelsl)\ **Replies:** 4\ **Last updated:** [August 2, 2019, 4:30pm UTC](https://discuss.elastic.co/t/mutate-filter-convert-a-hash-data-type-to-something-else/193580 "2019-08-02T16:30:44Z")
Hey there guys, How are you doing? I came across this problem and thought you'd be able to help me figure it out. I created a CSV filter to parse my httpd access log file and it's working fine. We have a field "HttpSta… --- ## [UserAgent Transformation in Logstash](https://discuss.elastic.co/t/useragent-transformation-in-logstash/193548)
**Author:** [@chimbu3306](https://discuss.elastic.co/u/chimbu3306)\ **Replies:** 3\ **Last updated:** [August 2, 2019, 3:54pm UTC](https://discuss.elastic.co/t/useragent-transformation-in-logstash/193548 "2019-08-02T15:54:50Z")
I have the below useragent captured as a string in our application logs, we are publishing this logs to elastic search from logstash and currently storing it as string, We would like to convert this string to json object… --- ## [Write a Python file that is able to read a Logstash query and that execute controls on that](https://discuss.elastic.co/t/write-a-python-file-that-is-able-to-read-a-logstash-query-and-that-execute-controls-on-that/193583)
**Author:** [@Nora](https://discuss.elastic.co/u/Nora)\ **Replies:** 0\ **Last updated:** [August 2, 2019, 3:31pm UTC](https://discuss.elastic.co/t/write-a-python-file-that-is-able-to-read-a-logstash-query-and-that-execute-controls-on-that/193583 "2019-08-02T15:31:59Z")
Hi I want to write a python program that execute controls. Those controls must be execute on the results of a query. The questions are the following: if I've created a python file that includes both the query and th… --- ## [Bad JDBC connection string](https://discuss.elastic.co/t/bad-jdbc-connection-string/193551)
**Author:** [@valleram](https://discuss.elastic.co/u/valleram)\ **Replies:** 0\ **Last updated:** [August 2, 2019, 1:56pm UTC](https://discuss.elastic.co/t/bad-jdbc-connection-string/193551 "2019-08-02T13:56:20Z")
Hi, Trying to connect to a local MySql db using jdbc\_streaming filter but receiving below error. \[ERROR\] 2019-08-02 08:46:51.709 \[\[main\]-pipeline-manager\] pipeline - Error registering plugin {:pipeline\_id=\>"main", :pl… --- ## [\[Logstash 7.2\] Syslog Output message](https://discuss.elastic.co/t/logstash-7-2-syslog-output-message/193483)
**Author:** [@stestaz](https://discuss.elastic.co/u/stestaz)\ **Replies:** 1\ **Last updated:** [August 2, 2019, 1:46pm UTC](https://discuss.elastic.co/t/logstash-7-2-syslog-output-message/193483 "2019-08-02T13:46:08Z")
Hi All, syslog output is making me crazy. I have a logstash input with multiple line concatenation, the concatenated message has the field "message" containing only the last line and the field "conn\_mess" containing th… --- ## [Tls issue connecting to secure elasticsearch backend](https://discuss.elastic.co/t/tls-issue-connecting-to-secure-elasticsearch-backend/193477)
**Author:** [@asp](https://discuss.elastic.co/u/asp)\ **Replies:** 4\ **Last updated:** [August 2, 2019, 1:36pm UTC](https://discuss.elastic.co/t/tls-issue-connecting-to-secure-elasticsearch-backend/193477 "2019-08-02T13:36:18Z")
Hi, I am running elastic stack in kubernetes and I've changed my elasticsearch to use security module mow. Meaning TLS encryption on transport and http rest api and user authorization. Currently I am encountering issue… --- ## [KV filter in logstash 7.2.0 doesn't store a key having empty value](https://discuss.elastic.co/t/kv-filter-in-logstash-7-2-0-doesnt-store-a-key-having-empty-value/193461)
**Author:** [@ritesh\_singh](https://discuss.elastic.co/u/ritesh_singh)\ **Replies:** 1\ **Last updated:** [August 2, 2019, 12:56pm UTC](https://discuss.elastic.co/t/kv-filter-in-logstash-7-2-0-doesnt-store-a-key-having-empty-value/193461 "2019-08-02T12:56:38Z")
Hi I am trying to parse a log in the following format 2018-12-18 19:50:37 SNID:1|CTI:1105154514|SL:20|SMS:706C73|SMSC:1|PSI\_RF:00|SRCIP:172.25.25.119|DESTIP:|SN:|DN: i am using KV filter plugin to get it done, but i … --- ## [OpenSSL version for Beats input](https://discuss.elastic.co/t/openssl-version-for-beats-input/193522)
**Author:** [@tombar](https://discuss.elastic.co/u/tombar)\ **Replies:** 0\ **Last updated:** [August 2, 2019, 11:49am UTC](https://discuss.elastic.co/t/openssl-version-for-beats-input/193522 "2019-08-02T11:49:23Z")
I'm trying to secure the Beats input of section with SSL. I followed various tutorials to generate valid certificates, but I always end up with the error OPENSSL\_internal:WRONG\_VERSION\_NUMBER. I'm using Ubuntu 18.04.2 w… --- ## [Problem with Logstash date parse](https://discuss.elastic.co/t/problem-with-logstash-date-parse/193311)
**Author:** [@shagun](https://discuss.elastic.co/u/shagun)\ **Replies:** 2\ **Last updated:** [August 2, 2019, 11:29am UTC](https://discuss.elastic.co/t/problem-with-logstash-date-parse/193311 "2019-08-02T11:29:36Z")
I have a date in this format "event\_time" : "2019-07-02 19:23:59+0000" and need to parse it in the format of @timestamp that is "@timestamp" : "2019-08-01T11:18:34.890Z" This is my filter : date { match =\> \[ "event\_… --- ## [Logstash sending less data to ElasticSearch Cluster](https://discuss.elastic.co/t/logstash-sending-less-data-to-elasticsearch-cluster/192749)
**Author:** [@Jitendra\_Kumhar](https://discuss.elastic.co/u/Jitendra_Kumhar)\ **Replies:** 16\ **Last updated:** [August 2, 2019, 11:06am UTC](https://discuss.elastic.co/t/logstash-sending-less-data-to-elasticsearch-cluster/192749 "2019-08-02T11:06:31Z")
Hello, I am sending data from filebeats to a logstash machine and then to ElasticSearch cluster. I have persistent queue enabled on logstash.The queue gets filled very quickly.I can't figure out out reason why it is n… --- ## [Logstash not working on ubuntu 16.04](https://discuss.elastic.co/t/logstash-not-working-on-ubuntu-16-04/193490)
**Author:** [@Vikash\_Singh1](https://discuss.elastic.co/u/Vikash_Singh1)\ **Replies:** 0\ **Last updated:** [August 2, 2019, 9:05am UTC](https://discuss.elastic.co/t/logstash-not-working-on-ubuntu-16-04/193490 "2019-08-02T09:05:26Z")
I have installed logstash 7.3 on ubuntu 16.04 also java version 11 is also installed but when I am trying to start logstash it gives me an error saying: \[ERROR\] 2019-08-02 14:21:41.499 \[main\] Logstash - java.lang.Illeg… --- ## [Logstash Kafka-Input plugin not using the values supplied in \`bootstrap\_servers\` for connecting to Kafka Broker](https://discuss.elastic.co/t/logstash-kafka-input-plugin-not-using-the-values-supplied-in-bootstrap-servers-for-connecting-to-kafka-broker/193358)
**Author:** [@thomjs](https://discuss.elastic.co/u/thomjs)\ **Replies:** 2\ **Last updated:** [August 1, 2019, 4:06pm UTC](https://discuss.elastic.co/t/logstash-kafka-input-plugin-not-using-the-values-supplied-in-bootstrap-servers-for-connecting-to-kafka-broker/193358 "2019-08-01T16:06:12Z")
Elastic Stack Version: 7.2.0 Kafka Version: 5.0.1 Using docker-compose to deploy Elastic Stack, with Kafka sitting on a remote system which is reachable via direct IP (192.168.0.2) In the logstash.conf file we have: i… --- ## [What configuration is used to add newly added data in old indexes?](https://discuss.elastic.co/t/what-configuration-is-used-to-add-newly-added-data-in-old-indexes/192212)
**Author:** [@pyerunka](https://discuss.elastic.co/u/pyerunka)\ **Replies:** 7\ **Last updated:** [August 2, 2019, 4:52am UTC](https://discuss.elastic.co/t/what-configuration-is-used-to-add-newly-added-data-in-old-indexes/192212 "2019-08-02T04:52:27Z")
Hello, Can anyone help me in this? I have one index made up of database table which is created by using sql query. I want to add newly added data into it. I do not want to index whole data again. For example: database… --- ## [How to extract time and day from date](https://discuss.elastic.co/t/how-to-extract-time-and-day-from-date/193309)
**Author:** [@Syed.Ubaid](https://discuss.elastic.co/u/Syed.Ubaid)\ **Replies:** 2\ **Last updated:** [August 2, 2019, 3:09am UTC](https://discuss.elastic.co/t/how-to-extract-time-and-day-from-date/193309 "2019-08-02T03:09:21Z")
I have a column name Date in which field is 11/11/2018 11:29 i want to extract this vale 11/11/2018 11:29 into new column time in which time is 11:29 and day is also coming from new column Day for example 11/11/2018 … --- ## [Predefined parser/indexer Infoblox syslog & dns logs](https://discuss.elastic.co/t/predefined-parser-indexer-infoblox-syslog-dns-logs/193225)
**Author:** [@ray\_zuniga](https://discuss.elastic.co/u/ray_zuniga)\ **Replies:** 0\ **Last updated:** [July 31, 2019, 10:21pm UTC](https://discuss.elastic.co/t/predefined-parser-indexer-infoblox-syslog-dns-logs/193225 "2019-07-31T22:21:21Z")
Does anyone know if there is a pre-made Infoblox parser/indexer configs/grok filter ? its a DNS appliance. Edit: found these in the docs for grok filters(my previous question lol) but none for infoblox --- ## [Convert timestamps to UTC(UNIX) from PST,IST](https://discuss.elastic.co/t/convert-timestamps-to-utc-unix-from-pst-ist/193386)
**Author:** [@naveenrt23](https://discuss.elastic.co/u/naveenrt23)\ **Replies:** 8\ **Last updated:** [August 1, 2019, 8:25pm UTC](https://discuss.elastic.co/t/convert-timestamps-to-utc-unix-from-pst-ist/193386 "2019-08-01T20:25:17Z")
Hello, How can I convert my timestamps in UNIX from different timezones such as EDT, PST/IST to UTC in Logstash filters? --- ## [Error: undefined method \`to\_sym' for nil:NilClass for Postgres JDBC](https://discuss.elastic.co/t/error-undefined-method-to-sym-for-nil-nilclass-for-postgres-jdbc/193390)
**Author:** [@Ale\_Ksa](https://discuss.elastic.co/u/Ale_Ksa)\ **Replies:** 5\ **Last updated:** [August 1, 2019, 7:57pm UTC](https://discuss.elastic.co/t/error-undefined-method-to-sym-for-nil-nilclass-for-postgres-jdbc/193390 "2019-08-01T19:57:31Z")
Hello! I am getting an undefined method error when using Postgres JDBC in Logstash. I've put the debug output on Pastebin. And here is the config itself: input { # pg\_stat\_database jdbc { jdbc\_driver\_library =\> "" … --- ## [Grok then test nested field](https://discuss.elastic.co/t/grok-then-test-nested-field/193389)
**Author:** [@paniccontrol](https://discuss.elastic.co/u/paniccontrol)\ **Replies:** 1\ **Last updated:** [August 1, 2019, 6:39pm UTC](https://discuss.elastic.co/t/grok-then-test-nested-field/193389 "2019-08-01T18:39:21Z")
Hello, I'm encounter some issue on testing nested field after a grok. grok { match =\> { "message" =\> \[ ".\*(?\test) xxxx %{DATA:\[name\]}" \] } if ( "" in \[event\]\[action\] ) { mutate { remove\_field =\> "message"… --- ## [Split "message" Field using Ruby](https://discuss.elastic.co/t/split-message-field-using-ruby/193375)
**Author:** [@jakecramer17](https://discuss.elastic.co/u/jakecramer17)\ **Replies:** 2\ **Last updated:** [August 1, 2019, 5:20pm UTC](https://discuss.elastic.co/t/split-message-field-using-ruby/193375 "2019-08-01T17:20:35Z")
My input with the type "sym" contains a single string of multiple syslog logs separated by the '\\r' character (which I'm guessing represents carriage return). I'm not focused on parsing each log yet. First, I need to sep… --- ## [Gork fpr the DNS qureu log](https://discuss.elastic.co/t/gork-fpr-the-dns-qureu-log/193331)
**Author:** [@unicxs\_support](https://discuss.elastic.co/u/unicxs_support)\ **Replies:** 2\ **Last updated:** [August 1, 2019, 3:44pm UTC](https://discuss.elastic.co/t/gork-fpr-the-dns-qureu-log/193331 "2019-08-01T15:44:16Z")
Hello Experts, I've below logstream data which indeed a DNS query data where i have a literal dot after view external: query: which usually referred to be an IP address or a domain address but here its coming as dot for… --- ## [java.io.IOException: Page file size is too small to hold elements](https://discuss.elastic.co/t/java-io-ioexception-page-file-size-is-too-small-to-hold-elements/193002)
**Author:** [@kyle\_che](https://discuss.elastic.co/u/kyle_che)\ **Replies:** 4\ **Last updated:** [August 1, 2019, 4:53pm UTC](https://discuss.elastic.co/t/java-io-ioexception-page-file-size-is-too-small-to-hold-elements/193002 "2019-08-01T16:53:50Z")
I am getting the below message on 7.2.0 logstash. I use the same pipeline i had for 6.x but with 7.2.0 i get the below error. Any ideas? root@d-gp2-es46-4:/var/log/logstash# cat logstash-plain.log \[2019-07-30T22:48:1… --- ## [Delete and Re-Create Index with ES output plugin](https://discuss.elastic.co/t/delete-and-re-create-index-with-es-output-plugin/193359)
**Author:** [@rkhapre](https://discuss.elastic.co/u/rkhapre)\ **Replies:** 5\ **Last updated:** [August 1, 2019, 4:15pm UTC](https://discuss.elastic.co/t/delete-and-re-create-index-with-es-output-plugin/193359 "2019-08-01T16:15:21Z")
Hi All I have a situation where my source dataset refreshes everyday, so there is need to first drop/delete the index manually using curl command and run the LS to create the index again. Is there any option where we c… --- ## [Metricbeat data from Logstash not showing up in Kibana](https://discuss.elastic.co/t/metricbeat-data-from-logstash-not-showing-up-in-kibana/193369)
**Author:** [@carson](https://discuss.elastic.co/u/carson)\ **Replies:** 0\ **Last updated:** [August 1, 2019, 3:52pm UTC](https://discuss.elastic.co/t/metricbeat-data-from-logstash-not-showing-up-in-kibana/193369 "2019-08-01T15:52:20Z")
Hi all, I've just added Logstash to my ELK stack setup. I've got filebeat working, but metricbeat is causing trouble. When I navigate to the Infrastructure screen, I get an error in elasticsearch: "Caused by: java.lan… --- ## [XML timestamp extraction?](https://discuss.elastic.co/t/xml-timestamp-extraction/193221)
**Author:** [@eclipsed450](https://discuss.elastic.co/u/eclipsed450)\ **Replies:** 5\ **Last updated:** [August 1, 2019, 3:48pm UTC](https://discuss.elastic.co/t/xml-timestamp-extraction/193221 "2019-08-01T15:48:45Z")
I have the following in an XML event, and want to put it into it's own field, but none of the XMLPATH I've tried have worked. Help?? \ \ ... \ **Author:** [@Sourabh\_Kashyap](https://discuss.elastic.co/u/Sourabh_Kashyap)\ **Replies:** 1\ **Last updated:** [August 1, 2019, 3:46pm UTC](https://discuss.elastic.co/t/lostash-error/193357 "2019-08-01T15:46:11Z") \[2019-08-01T15:07:16,754\]\[ERROR\]\[logstash.pipeline \] A plugin had an unrecoverable error. Will restart this plugin. Plugin: \5044, codec=\>\"json\_76489cd0-0… --- ## [Creating a field of field](https://discuss.elastic.co/t/creating-a-field-of-field/191844)
**Author:** [@luc1](https://discuss.elastic.co/u/luc1)\ **Replies:** 12\ **Last updated:** [August 1, 2019, 3:45pm UTC](https://discuss.elastic.co/t/creating-a-field-of-field/191844 "2019-08-01T15:45:49Z")
Hello, I have this field in my JSON file which looks like this : "activeAlarms":\[{"id":\*\*\*\*\*, "description":"something", "repairAction": "do this"}, {"id":\*\*\*\*\*, "description":"somethingElse", "repairAction": "do that}… --- ## [Logstash XML](https://discuss.elastic.co/t/logstash-xml/192452)
**Author:** [@Imad\_Bouchakour](https://discuss.elastic.co/u/Imad_Bouchakour)\ **Replies:** 13\ **Last updated:** [August 1, 2019, 3:44pm UTC](https://discuss.elastic.co/t/logstash-xml/192452 "2019-08-01T15:44:13Z")
Hi i'am trying to parse a xml file with logstash and i having some issues this is my XML exemple : \ \ \HarryPotter\ \785\ \7\ \ \ \game of thrones\<… --- ## [Kibana shows only 1 servers logs among 2 servers sent via Logstash](https://discuss.elastic.co/t/kibana-shows-only-1-servers-logs-among-2-servers-sent-via-logstash/193354)
**Author:** [@Devendra\_Date1](https://discuss.elastic.co/u/Devendra_Date1)\ **Replies:** 0\ **Last updated:** [August 1, 2019, 2:59pm UTC](https://discuss.elastic.co/t/kibana-shows-only-1-servers-logs-among-2-servers-sent-via-logstash/193354 "2019-08-01T14:59:31Z")
--- ## [Logstash problem for loading data](https://discuss.elastic.co/t/logstash-problem-for-loading-data/192936)
**Author:** [@Abdelmadjid](https://discuss.elastic.co/u/Abdelmadjid)\ **Replies:** 11\ **Last updated:** [August 1, 2019, 12:53pm UTC](https://discuss.elastic.co/t/logstash-problem-for-loading-data/192936 "2019-08-01T12:53:17Z")
hello i need help for loading data to elasticsearch with logstash on windows 10 when i try to load data to logstash with these config file input{ file{ path =\> "C:\\Users\\pc intervalle\\Downloads\\Compressed\\logstash-7… --- ## [Ruby filter to convert date](https://discuss.elastic.co/t/ruby-filter-to-convert-date/193295)
**Author:** [@HFolly](https://discuss.elastic.co/u/HFolly)\ **Replies:** 2\ **Last updated:** [August 1, 2019, 12:38pm UTC](https://discuss.elastic.co/t/ruby-filter-to-convert-date/193295 "2019-08-01T12:38:01Z")
Hi there. In my Logstash configuration I need a Ruby filter to convert date from a given collection of strings. My collection uses an structure with 3 values containing a field name, it's value and it's data type: "C… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=498) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=500)