# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=358 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 359 --- ## [Error Code 401 retrying connection to Elastic servers](https://discuss.elastic.co/t/error-code-401-retrying-connection-to-elastic-servers/232095)
**Author:** [@edster](https://discuss.elastic.co/u/edster)\ **Replies:** 3\ **Last updated:** [May 14, 2020, 12:18pm UTC](https://discuss.elastic.co/t/error-code-401-retrying-connection-to-elastic-servers/232095 "2020-05-14T12:18:47Z")
Hello, We seem to be having random moments when logstash runs into an issue where it can't connect to the elastic servers and continuously tries to connect and even though the servers are up it continously tries to conn… --- ## [Cannot get xpack disabled on logstash.yml](https://discuss.elastic.co/t/cannot-get-xpack-disabled-on-logstash-yml/232628)
**Author:** [@Fernando\_A](https://discuss.elastic.co/u/Fernando_A)\ **Replies:** 0\ **Last updated:** [May 14, 2020, 11:37am UTC](https://discuss.elastic.co/t/cannot-get-xpack-disabled-on-logstash-yml/232628 "2020-05-14T11:37:17Z")
I am trying to run Logstash in my kubernetes-cluster but I get this error: OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release. … --- ## [Grok pattern extraction help](https://discuss.elastic.co/t/grok-pattern-extraction-help/232610)
**Author:** [@nikhilesh](https://discuss.elastic.co/u/nikhilesh)\ **Replies:** 2\ **Last updated:** [May 14, 2020, 10:47am UTC](https://discuss.elastic.co/t/grok-pattern-extraction-help/232610 "2020-05-14T10:47:06Z")
Hi All, Can any one please help to divide the message in to 3 parts using grok filter Complete message : Exception SAPClient.execute method com.sap.conn.jco.JCoException: (104) RFC\_ERROR\_SYSTEM\_FAILURE: No more memor… --- ## [Logstash cannot start because of Config Error - can't find the issue](https://discuss.elastic.co/t/logstash-cannot-start-because-of-config-error-cant-find-the-issue/232575)
**Author:** [@Moritz\_Kiesewetter](https://discuss.elastic.co/u/Moritz_Kiesewetter)\ **Replies:** 6\ **Last updated:** [May 14, 2020, 9:42am UTC](https://discuss.elastic.co/t/logstash-cannot-start-because-of-config-error-cant-find-the-issue/232575 "2020-05-14T09:42:53Z")
So i've reconfigured my logstash-config files - to seperate different logs. Now if i try to start it - i get the following error: Failed to execute action {:action=\>LogStash::PipelineAction::Create/pipeline\_id:main, … --- ## [How to change @timestamp from UTC timezone to Other timezone](https://discuss.elastic.co/t/how-to-change-timestamp-from-utc-timezone-to-other-timezone/231838)
**Author:** [@abdorrezahadi](https://discuss.elastic.co/u/abdorrezahadi)\ **Replies:** 2\ **Last updated:** [May 14, 2020, 9:01am UTC](https://discuss.elastic.co/t/how-to-change-timestamp-from-utc-timezone-to-other-timezone/231838 "2020-05-14T09:01:04Z")
Logstash generates indexes based on @timestamp with UTC time, if I want to get documents from 2020-05-08 to 2020-05-09 with my timezone, I need to search indexes logstash-2020-05-07 and logstash-2020-05-08, if I can ch… --- ## [Grok check if exists](https://discuss.elastic.co/t/grok-check-if-exists/232433)
**Author:** [@Mohammad\_Mousavi](https://discuss.elastic.co/u/Mohammad_Mousavi)\ **Replies:** 3\ **Last updated:** [May 14, 2020, 8:53am UTC](https://discuss.elastic.co/t/grok-check-if-exists/232433 "2020-05-14T08:53:48Z")
Hi, I have 2 type of nginx logs: x.x.x.x - - \[13/May/2020:16:42:27 +0430\] "GET /cache/68/fb/68fb0ff0cf190dc7fcc567e76e1e4c2b.jpg HTTP/1.1" 200 79059 x.x.x.x - - \[13/May/2020:16:40:06 +0430\] "GET /dl-apk/1585150257\_v… --- ## [Forward only ERROR or Warn log lines to Splunk](https://discuss.elastic.co/t/forward-only-error-or-warn-log-lines-to-splunk/232561)
**Author:** [@Shivhare](https://discuss.elastic.co/u/Shivhare)\ **Replies:** 4\ **Last updated:** [May 14, 2020, 8:27am UTC](https://discuss.elastic.co/t/forward-only-error-or-warn-log-lines-to-splunk/232561 "2020-05-14T08:27:59Z")
Need your expert advise on how can I configure my logstash.conf file to forward only the ERROR OR WARN log lines to Splunk. I have done some online research that a grok filter or wrapping the output with if condition can… --- ## [Imap Plugin Parsing Error](https://discuss.elastic.co/t/imap-plugin-parsing-error/232569)
**Author:** [@Stefano\_Serano](https://discuss.elastic.co/u/Stefano_Serano)\ **Replies:** 0\ **Last updated:** [May 14, 2020, 7:25am UTC](https://discuss.elastic.co/t/imap-plugin-parsing-error/232569 "2020-05-14T07:25:33Z")
Hi all. I've configured logstash multiple input: Filebeat Input Imap Plugin Input and output to two different Elasticsearch index. After upgrade form 7.2 to 7.6 logstash not index mails anymore prompting this erro… --- ## [Xml PARSING IN LOGSTASH](https://discuss.elastic.co/t/xml-parsing-in-logstash/232385)
**Author:** [@Puneeth\_S\_B\_Gowda](https://discuss.elastic.co/u/Puneeth_S_B_Gowda)\ **Replies:** 2\ **Last updated:** [May 14, 2020, 5:54am UTC](https://discuss.elastic.co/t/xml-parsing-in-logstash/232385 "2020-05-14T05:54:35Z")
Hi I am new to ELK Looking for extracting column from XML i need bedtype as a new column value as D1K from below xml D1K My logstashsetting Sample Logstash configuration for creating a simple Beats -\> Logstash -\> Ela… --- ## [Scripted Upsert with Logstash ES Output Plugin - overwrites array data with latest event instead of update on reconnection](https://discuss.elastic.co/t/scripted-upsert-with-logstash-es-output-plugin-overwrites-array-data-with-latest-event-instead-of-update-on-reconnection/232547)
**Author:** [@VIVEK\_SHARMA3](https://discuss.elastic.co/u/VIVEK_SHARMA3)\ **Replies:** 0\ **Last updated:** [May 14, 2020, 4:49am UTC](https://discuss.elastic.co/t/scripted-upsert-with-logstash-es-output-plugin-overwrites-array-data-with-latest-event-instead-of-update-on-reconnection/232547 "2020-05-14T04:49:22Z")
Scenario: We are adding object to array via logstash to elasticsearch using logstash's ES output plugin. Both are on version 6.8. It is a scripted upsert using stored script on Elasticsearch and here is the stored scrip… --- ## [How to select an index template from a logstash pipeline?](https://discuss.elastic.co/t/how-to-select-an-index-template-from-a-logstash-pipeline/232504)
**Author:** [@Incauto](https://discuss.elastic.co/u/Incauto)\ **Replies:** 1\ **Last updated:** [May 14, 2020, 1:34am UTC](https://discuss.elastic.co/t/how-to-select-an-index-template-from-a-logstash-pipeline/232504 "2020-05-14T01:34:45Z")
I have made an index template with the following index pattern: POST \_template/myindex-template { "index\_patterns": \[ "myindex-\*" \], "settings": { ..... for what I understand just by using the index pattern in th… --- ## [Logstash - JDBC Plugin - \_\_$seqval](https://discuss.elastic.co/t/logstash-jdbc-plugin-seqval/232525)
**Author:** [@LChand](https://discuss.elastic.co/u/LChand)\ **Replies:** 2\ **Last updated:** [May 14, 2020, 1:05am UTC](https://discuss.elastic.co/t/logstash-jdbc-plugin-seqval/232525 "2020-05-14T01:05:15Z")
Hello, I'm able to connect to a SQL server DB and query a view from a Linux Logstash server. The CDC feature (Change Data Capture) is turned on on the SQL side and I'm trying to use the field \_\_$seqval for :sql\_last\_val… --- ## [Can we stop logstash from "Attempting to install template..."](https://discuss.elastic.co/t/can-we-stop-logstash-from-attempting-to-install-template/232463)
**Author:** [@aviationfan](https://discuss.elastic.co/u/aviationfan)\ **Replies:** 2\ **Last updated:** [May 13, 2020, 8:45pm UTC](https://discuss.elastic.co/t/can-we-stop-logstash-from-attempting-to-install-template/232463 "2020-05-13T20:45:56Z")
I see this in my logstash log every time it starts up and it seems like an redundant and unnecessary step. Can I configure it to not do this? \[2020-05-13T09:51:10,669\]\[INFO \]\[logstash.outputs.elasticsearch\]\[main\] Attemp… --- ## [Need help with grok pattern for logstash config file for Oracle Alert log](https://discuss.elastic.co/t/need-help-with-grok-pattern-for-logstash-config-file-for-oracle-alert-log/232313)
**Author:** [@Gaurav\_Batta](https://discuss.elastic.co/u/Gaurav_Batta)\ **Replies:** 4\ **Last updated:** [May 13, 2020, 6:22pm UTC](https://discuss.elastic.co/t/need-help-with-grok-pattern-for-logstash-config-file-for-oracle-alert-log/232313 "2020-05-13T18:22:41Z")
Please help with grok pattern for following Oracle Database Alert.log message below to trap ORA- errors and its timestamp using logstash. Oracle Database alert log has following error log messages: ORA-12222: Test erro… --- ## [Nested JSON is confusing me](https://discuss.elastic.co/t/nested-json-is-confusing-me/232480)
**Author:** [@BlueTeamNinja](https://discuss.elastic.co/u/BlueTeamNinja)\ **Replies:** 0\ **Last updated:** [May 13, 2020, 4:40pm UTC](https://discuss.elastic.co/t/nested-json-is-confusing-me/232480 "2020-05-13T16:40:59Z")
I have the exact same problem in two places and I can't quite figure it out. I think I need an ELI5. I have a JSON structure coming from a giant hodge-podge of Powershell scripts that are pulling in local computer info… --- ## [Is there a way to index a text file which has the data in the following format](https://discuss.elastic.co/t/is-there-a-way-to-index-a-text-file-which-has-the-data-in-the-following-format/232396)
**Author:** [@avneshkumar98](https://discuss.elastic.co/u/avneshkumar98)\ **Replies:** 4\ **Last updated:** [May 13, 2020, 4:24pm UTC](https://discuss.elastic.co/t/is-there-a-way-to-index-a-text-file-which-has-the-data-in-the-following-format/232396 "2020-05-13T16:24:02Z")
device="ABC" date=2010-05-12 time=15:24:10 timezone="IST" device\_name="PG15625" device="ABC" date=2010-05-12 time=15:24:10 timezone="IST" device\_name="PG15625" device="ABC" date=2010-05-12 time=15:24:10 timezone="IST" … --- ## [Does logstash output to elasticsearch support pkcs#12 certificate format?](https://discuss.elastic.co/t/does-logstash-output-to-elasticsearch-support-pkcs-12-certificate-format/232410)
**Author:** [@nityaraj06](https://discuss.elastic.co/u/nityaraj06)\ **Replies:** 3\ **Last updated:** [May 13, 2020, 3:17pm UTC](https://discuss.elastic.co/t/does-logstash-output-to-elasticsearch-support-pkcs-12-certificate-format/232410 "2020-05-13T15:17:23Z")
@TimV - I have a elasticsearch certificate in .pfx format which i want to use in the logstash output config to elasticsearch like below. However it seems logstash does not support pkcs#12 format. So, do i need to reques… --- ## [Logstash multiple json field](https://discuss.elastic.co/t/logstash-multiple-json-field/232138)
**Author:** [@ohoroyoi](https://discuss.elastic.co/u/ohoroyoi)\ **Replies:** 5\ **Last updated:** [May 13, 2020, 2:30pm UTC](https://discuss.elastic.co/t/logstash-multiple-json-field/232138 "2020-05-13T14:30:28Z")
I want to send postgres data to elasticsearch using logstash. I have json column more than one in postgres table. for example, continent and country column are both json type. then, How can I configure filter option? … --- ## [Logstash - set @timestamp in .conf files](https://discuss.elastic.co/t/logstash-set-timestamp-in-conf-files/232449)
**Author:** [@cedric2](https://discuss.elastic.co/u/cedric2)\ **Replies:** 1\ **Last updated:** [May 13, 2020, 2:19pm UTC](https://discuss.elastic.co/t/logstash-set-timestamp-in-conf-files/232449 "2020-05-13T14:19:57Z")
Hi, I need to update existing logstash .conf files to generate indexes with tiemstamp\* \*Reason: ensure new indexes are regularly generated when rollup of index when ILM policy rules are reach which then allow me to clea… --- ## [How to use multiple csv files in logstash and filebeat](https://discuss.elastic.co/t/how-to-use-multiple-csv-files-in-logstash-and-filebeat/232222)
**Author:** [@Youssef\_SBAI](https://discuss.elastic.co/u/Youssef_SBAI)\ **Replies:** 10\ **Last updated:** [May 13, 2020, 11:55am UTC](https://discuss.elastic.co/t/how-to-use-multiple-csv-files-in-logstash-and-filebeat/232222 "2020-05-13T11:55:29Z")
I want to use multiple csv files in filebeat my config file beat filebeat.inputs: # Each - is an input. Most options can be set at the input level, so # you can use different inputs for various configurations. # Below… --- ## [Help configuring Logstash](https://discuss.elastic.co/t/help-configuring-logstash/232055)
**Author:** [@Joggy\_John](https://discuss.elastic.co/u/Joggy_John)\ **Replies:** 6\ **Last updated:** [May 13, 2020, 11:42am UTC](https://discuss.elastic.co/t/help-configuring-logstash/232055 "2020-05-13T11:42:13Z")
Hello, I'm new into logstash ; I am trying to input data into ELK but am struggling to get anything done. The installation is done in a windows 2016 for both ELK and Logstash. I have two major issues though : first … --- ## [Connecting MySql to Logstash Using JDBC](https://discuss.elastic.co/t/connecting-mysql-to-logstash-using-jdbc/232403)
**Author:** [@Jasmin](https://discuss.elastic.co/u/Jasmin)\ **Replies:** 2\ **Last updated:** [May 13, 2020, 10:37am UTC](https://discuss.elastic.co/t/connecting-mysql-to-logstash-using-jdbc/232403 "2020-05-13T10:37:32Z")
Hi , this is my first time working on logstash , i am trying to send data from mysql to elasticsearch using logstash , but as a first step i specify json\_line output This is my output when running /usr/share/logstash… --- ## [Inserting a dummy value for empty field while parsing using grok logstash](https://discuss.elastic.co/t/inserting-a-dummy-value-for-empty-field-while-parsing-using-grok-logstash/232408)
**Author:** [@Devyani](https://discuss.elastic.co/u/Devyani)\ **Replies:** 0\ **Last updated:** [May 13, 2020, 9:59am UTC](https://discuss.elastic.co/t/inserting-a-dummy-value-for-empty-field-while-parsing-using-grok-logstash/232408 "2020-05-13T09:59:44Z")
I want to have the dummy value for empty field while parsing grok pattern. I have used the following condition--\> if !\[exception\_stacktrace\] { mutate { replace =\> \[ "exception\_stacktrace", "-" \] } } But with this … --- ## [Geoip filter plugin dat extension and update](https://discuss.elastic.co/t/geoip-filter-plugin-dat-extension-and-update/232277)
**Author:** [@YvorL](https://discuss.elastic.co/u/YvorL)\ **Replies:** 4\ **Last updated:** [May 13, 2020, 9:46am UTC](https://discuss.elastic.co/t/geoip-filter-plugin-dat-extension-and-update/232277 "2020-05-13T09:46:23Z")
Hi, I didn't have any recent (within a year) discussions on the above (most of them are unanswered). My questions are: Can I use the ".dat" extension instead of the ".mmdb" format? Is there any way to avoid restarting… --- ## [How to bootstrap an index template with a part of the name not defined yet?](https://discuss.elastic.co/t/how-to-bootstrap-an-index-template-with-a-part-of-the-name-not-defined-yet/232332)
**Author:** [@ElasticLiver](https://discuss.elastic.co/u/ElasticLiver)\ **Replies:** 4\ **Last updated:** [May 13, 2020, 9:36am UTC](https://discuss.elastic.co/t/how-to-bootstrap-an-index-template-with-a-part-of-the-name-not-defined-yet/232332 "2020-05-13T09:36:41Z")
Hi, depending on the hostgroup field, I want to add certain string to part of the index name: filter{ if \[hostgroup\] =~ /mass/ { mutate { add\_field =\> { "indexname" =\> "mass" }} } } outp… --- ## [How to create new fields and documents based on variable length log?](https://discuss.elastic.co/t/how-to-create-new-fields-and-documents-based-on-variable-length-log/232303)
**Author:** [@san\_sun](https://discuss.elastic.co/u/san_sun)\ **Replies:** 2\ **Last updated:** [May 13, 2020, 5:03am UTC](https://discuss.elastic.co/t/how-to-create-new-fields-and-documents-based-on-variable-length-log/232303 "2020-05-13T05:03:10Z")
I have a log message with a format like: status : name message ; status: name message ; status: name message ; status: name message..... I need to split based on the semicolon, and create separate Elasticsearch docu… --- ## [Error while connecting to Amazon Elasticsearch](https://discuss.elastic.co/t/error-while-connecting-to-amazon-elasticsearch/232352)
**Author:** [@rupini](https://discuss.elastic.co/u/rupini)\ **Replies:** 0\ **Last updated:** [May 13, 2020, 3:48am UTC](https://discuss.elastic.co/t/error-while-connecting-to-amazon-elasticsearch/232352 "2020-05-13T03:48:21Z")
Hi, I am a beginner and trying to figure our ELK. I have logstash configured in an AWS EC2 instance. I am trying to transfer the data from S3 bucket to AWS ES service using logstash. I am getting the following error whe… --- ## [Can I add a month variable to the table name?](https://discuss.elastic.co/t/can-i-add-a-month-variable-to-the-table-name/232339)
**Author:** [@111335](https://discuss.elastic.co/u/111335)\ **Replies:** 0\ **Last updated:** [May 13, 2020, 2:02am UTC](https://discuss.elastic.co/t/can-i-add-a-month-variable-to-the-table-name/232339 "2020-05-13T02:02:01Z")
Hello. I want to put a specific variable in the table. Looking at the topic, it can be solved through statement\_filepath. I am not sure how. Here is what I want: ${mmm} : jan, feb, mar, apr, may ...., dec input { … --- ## [Null pointer exception in logstash filter](https://discuss.elastic.co/t/null-pointer-exception-in-logstash-filter/231446)
**Author:** [@sajjad0927](https://discuss.elastic.co/u/sajjad0927)\ **Replies:** 3\ **Last updated:** [May 13, 2020, 1:24am UTC](https://discuss.elastic.co/t/null-pointer-exception-in-logstash-filter/231446 "2020-05-13T01:24:02Z")
/Server/home/user/itbox/tools/logstash-7.6.2/vendor/bundle/jruby/2.5.0/gems/awesome\_print-1.7.0/lib/awesome\_print/formatters/base\_formatter.rb:31: warning: constant ::Fixnum is deprecated warning: thread "\[main\]\>worker1"… --- ## [Check if env variable is set](https://discuss.elastic.co/t/check-if-env-variable-is-set/232328)
**Author:** [@alxsss](https://discuss.elastic.co/u/alxsss)\ **Replies:** 1\ **Last updated:** [May 12, 2020, 11:28pm UTC](https://discuss.elastic.co/t/check-if-env-variable-is-set/232328 "2020-05-12T23:28:17Z")
Is it possible to check if an env variable is set? I tired a workaround like this In bash $set DATE=a filter{ if (\["{DATE:a}"\]=="a") { mutate { add\_field =\> { "startTime" =\> "2019-10-19" }} } else{ … [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=357) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=359)