# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=238 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 239 --- ## [Java::JavaLang::IllegalStateException\` for \`PipelineAction::Create\](https://discuss.elastic.co/t/java-illegalstateexception-for-pipelineaction-create-main/270034)
**Author:** [@prajawalkp](https://discuss.elastic.co/u/prajawalkp)\ **Replies:** 3\ **Last updated:** [April 13, 2021, 5:42pm UTC](https://discuss.elastic.co/t/java-illegalstateexception-for-pipelineaction-create-main/270034 "2021-04-13T17:42:40Z")
Hii, I am new to logstash and using logstash to syncup mysql with elasticsearch. But I can't get through this error. Please help. It is urgent. Using JAVA\_HOME defined java: /Library/Java/JavaVirtualMachines/jdk-15.… --- ## [JDBC input parameters add quotes to query](https://discuss.elastic.co/t/jdbc-input-parameters-add-quotes-to-query/270271)
**Author:** [@mlnusd](https://discuss.elastic.co/u/mlnusd)\ **Replies:** 0\ **Last updated:** [April 15, 2021, 3:32pm UTC](https://discuss.elastic.co/t/jdbc-input-parameters-add-quotes-to-query/270271 "2021-04-15T15:32:48Z")
I'm trying to use jdbc input filter to query Oracle DB and to template a schema name. My statement looks like this: select count(1) from :schema.my\_table And my logstash config is like this: input { \[...connection … --- ## [Broken grok after the update to Logstash 7.12](https://discuss.elastic.co/t/broken-grok-after-the-update-to-logstash-7-12/269863)
**Author:** [@aviadhaham](https://discuss.elastic.co/u/aviadhaham)\ **Replies:** 3\ **Last updated:** [April 15, 2021, 3:03pm UTC](https://discuss.elastic.co/t/broken-grok-after-the-update-to-logstash-7-12/269863 "2021-04-15T15:03:51Z")
Hello, In my company, after upgrading our cluster to the latest version 7.12, we noticed that one of our grok patterns stopped working. Important things to mention: The pipeline is functioning correctly, logs are com… --- ## [Multiple pipelines](https://discuss.elastic.co/t/multiple-pipelines/270238)
**Author:** [@dddddddddddddddd](https://discuss.elastic.co/u/dddddddddddddddd)\ **Replies:** 1\ **Last updated:** [April 15, 2021, 3:02pm UTC](https://discuss.elastic.co/t/multiple-pipelines/270238 "2021-04-15T15:02:05Z")
can somebody helps me 4.29} \[INFO \] 2021-04-15 07:55:35.481 \[\[beats-server\]-pipeline-manager\] beats - Starting input listener {:address=\>"0.0.0.0:5044"} \[INFO \] 2021-04-15 07:55:35.666 \[\[auth\]-pipeline-manager\] javapi… --- ## [Mapping a Data in a log to another Field](https://discuss.elastic.co/t/mapping-a-data-in-a-log-to-another-field/270250)
**Author:** [@Wilks](https://discuss.elastic.co/u/Wilks)\ **Replies:** 1\ **Last updated:** [April 15, 2021, 2:54pm UTC](https://discuss.elastic.co/t/mapping-a-data-in-a-log-to-another-field/270250 "2021-04-15T14:54:10Z")
Hey Everyone, How do you map a field from the logs to another field. So for example if I have the following log message":"03-MAR-21 00:30:46|142.122.217.10 |1| I want to parse out |1| but also then map it to an index … --- ## [Extract value from Nested message field - LogStash Kibana Grok Mutate](https://discuss.elastic.co/t/extract-value-from-nested-message-field-logstash-kibana-grok-mutate/269368)
**Author:** [@Sandun\_Akalanka](https://discuss.elastic.co/u/Sandun_Akalanka)\ **Replies:** 4\ **Last updated:** [April 15, 2021, 11:39am UTC](https://discuss.elastic.co/t/extract-value-from-nested-message-field-logstash-kibana-grok-mutate/269368 "2021-04-15T11:39:58Z")
I've configured a FileBeat service to send logs to Kibana via LogStash. FileBeat and LogStash are in different servers while Kibana is inside an AWS ElasticSearch domain. The logs are visible in the Kibana dashboard. In … --- ## [Docker container to build plugins](https://discuss.elastic.co/t/docker-container-to-build-plugins/270159)
**Author:** [@padakwaak](https://discuss.elastic.co/u/padakwaak)\ **Replies:** 1\ **Last updated:** [April 15, 2021, 11:37am UTC](https://discuss.elastic.co/t/docker-container-to-build-plugins/270159 "2021-04-15T11:37:41Z")
Hi, I'm not a Ruby/JRuby programmer and I find it quite difficult to get all the necessary build tools in place to build and test my modifications that I'm doing to some of the existing Logstash plugins. Another user c… --- ## [Logstash Avro Codec non-base64 support](https://discuss.elastic.co/t/logstash-avro-codec-non-base64-support/270225)
**Author:** [@th0ger](https://discuss.elastic.co/u/th0ger)\ **Replies:** 0\ **Last updated:** [April 15, 2021, 11:36am UTC](https://discuss.elastic.co/t/logstash-avro-codec-non-base64-support/270225 "2021-04-15T11:36:30Z")
We would like to input non-base64 encoded avro data with the avro codec. Similar to output syntax Optional base64 encoding by johanvanderkuijl · Pull Request #29 · logstash-plugins/logstash-codec-avro · GitHub code… --- ## [Multi PFsense - howto configure this](https://discuss.elastic.co/t/multi-pfsense-howto-configure-this/270172)
**Author:** [@Peque](https://discuss.elastic.co/u/Peque)\ **Replies:** 0\ **Last updated:** [April 15, 2021, 3:55am UTC](https://discuss.elastic.co/t/multi-pfsense-howto-configure-this/270172 "2021-04-15T03:55:35Z")
Hi Forum I'm quite new to the ELK solution, but learns every day :slight\_smile: We have a setup with several PF-Sense Routers that I would like get overview over. I have followed this guide : GitHub - patrickjennings/… --- ## [Multiple pipelines](https://discuss.elastic.co/t/multiple-pipelines/270120)
**Author:** [@dddddddddddddddd](https://discuss.elastic.co/u/dddddddddddddddd)\ **Replies:** 1\ **Last updated:** [April 14, 2021, 11:23pm UTC](https://discuss.elastic.co/t/multiple-pipelines/270120 "2021-04-14T23:23:41Z")
here only the first pipelines that logstash execute and the second is not, here both pipelines listen on the same port so can somebody helps me for running the 2 pipelines regards --- ## [Logstash jdbc - unnable to connect to database - plugin restart](https://discuss.elastic.co/t/logstash-jdbc-unnable-to-connect-to-database-plugin-restart/269959)
**Author:** [@g34ncarlo](https://discuss.elastic.co/u/g34ncarlo)\ **Replies:** 3\ **Last updated:** [April 14, 2021, 10:26pm UTC](https://discuss.elastic.co/t/logstash-jdbc-unnable-to-connect-to-database-plugin-restart/269959 "2021-04-14T22:26:24Z")
We're running Logstash with jdbc plugin as a Kubernetes cron job. Due to some network problem, eventually it cannot connect to database: \[ERROR\]\[logstash.inputs.jdbc \] Unable to connect to database. Tried 1 times {:… --- ## [Full logstash queue](https://discuss.elastic.co/t/full-logstash-queue/270136)
**Author:** [@d4nnyx](https://discuss.elastic.co/u/d4nnyx)\ **Replies:** 0\ **Last updated:** [April 14, 2021, 5:38pm UTC](https://discuss.elastic.co/t/full-logstash-queue/270136 "2021-04-14T17:38:48Z")
Hello there, since Im running our Elastic Stack, I have a problem with persistent queues, which time to time gets full from unknown reason. Elasticsearch setup: 3 master nodes 12 data nodes 32CPU 32GB RAM avg index … --- ## [Merge message with aggregate filter](https://discuss.elastic.co/t/merge-message-with-aggregate-filter/269115)
**Author:** [@MalfuncEddie](https://discuss.elastic.co/u/MalfuncEddie)\ **Replies:** 9\ **Last updated:** [April 14, 2021, 3:14pm UTC](https://discuss.elastic.co/t/merge-message-with-aggregate-filter/269115 "2021-04-14T15:14:14Z")
Hi, I am trying to merge 2 log lines with the aggregate filter and after a week browsing the internet and this forum I still cannot get this working. Below you have 2 loglines with an identifier ExchangeId. What I wan… --- ## [Logstash Event Size](https://discuss.elastic.co/t/logstash-event-size/269972)
**Author:** [@dawiro](https://discuss.elastic.co/u/dawiro)\ **Replies:** 4\ **Last updated:** [April 14, 2021, 2:43pm UTC](https://discuss.elastic.co/t/logstash-event-size/269972 "2021-04-14T14:43:58Z")
Hi, Can anyone tell me how I can get the size of an entire event using the ruby filter please? Thx D --- ## [Logstash](https://discuss.elastic.co/t/logstash/270107)
**Author:** [@dddddddddddddddd](https://discuss.elastic.co/u/dddddddddddddddd)\ **Replies:** 1\ **Last updated:** [April 14, 2021, 1:30pm UTC](https://discuss.elastic.co/t/logstash/270107 "2021-04-14T13:30:13Z")
can somebody help me when i excute the logstash this error appear Using bundled JDK: /usr/share/logstash/jdk OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be… --- ## [Is there any way to redo the input of beats?](https://discuss.elastic.co/t/is-there-any-way-to-redo-the-input-of-beats/268863)
**Author:** [@its-ogawa](https://discuss.elastic.co/u/its-ogawa)\ **Replies:** 2\ **Last updated:** [April 14, 2021, 10:50am UTC](https://discuss.elastic.co/t/is-there-any-way-to-redo-the-input-of-beats/268863 "2021-04-14T10:50:23Z")
I am sending logs from filebeat to logstash. However, when I added filebeat to a new server and restarted logstash, some kind of failure caused most of the logs to be thrown away. I would like to start logging again fr… --- ## [Multiple Pipelines - filebeat fails](https://discuss.elastic.co/t/multiple-pipelines-filebeat-fails/270094)
**Author:** [@Peque](https://discuss.elastic.co/u/Peque)\ **Replies:** 0\ **Last updated:** [April 14, 2021, 9:37am UTC](https://discuss.elastic.co/t/multiple-pipelines-filebeat-fails/270094 "2021-04-14T09:37:31Z")
Hi Forum. Been struggling with my ELK setup, but cannot make it work as I intend - and not sure where I'm going wrong. Using Elasticsearch,Kibana, Logstash and filebeat --\> from this guide: How To Install Elasticsearch… --- ## [Parsing multiple time series into Elasticsearch](https://discuss.elastic.co/t/parsing-multiple-time-series-into-elasticsearch/270082)
**Author:** [@tkkchan](https://discuss.elastic.co/u/tkkchan)\ **Replies:** 1\ **Last updated:** [April 14, 2021, 8:18am UTC](https://discuss.elastic.co/t/parsing-multiple-time-series-into-elasticsearch/270082 "2021-04-14T08:18:01Z")
Dear all, I am not sure if this belongs to logstash... please correct me if not. I have some data that contains a field which is a time series, of interval 60sec. Should I map it into a single ES document, or should I … --- ## [Logstash not parsing dynamic updates happening on the files present under the configured input path](https://discuss.elastic.co/t/logstash-not-parsing-dynamic-updates-happening-on-the-files-present-under-the-configured-input-path/269852)
**Author:** [@harish\_92](https://discuss.elastic.co/u/harish_92)\ **Replies:** 4\ **Last updated:** [April 14, 2021, 5:51am UTC](https://discuss.elastic.co/t/logstash-not-parsing-dynamic-updates-happening-on-the-files-present-under-the-configured-input-path/269852 "2021-04-14T05:51:21Z")
Hi Guys, I have configured a folder containing files which will be dynamically updated with some data which needs to be parsed by logstash and pushed into ES. Here, whenever new data is written into any one of the inpu… --- ## [Is there any way to reduce the size of data before indexing into elasticserch](https://discuss.elastic.co/t/is-there-any-way-to-reduce-the-size-of-data-before-indexing-into-elasticserch/270072)
**Author:** [@iammanmale](https://discuss.elastic.co/u/iammanmale)\ **Replies:** 4\ **Last updated:** [April 14, 2021, 1:49am UTC](https://discuss.elastic.co/t/is-there-any-way-to-reduce-the-size-of-data-before-indexing-into-elasticserch/270072 "2021-04-14T01:49:34Z")
As our data size is quite big, we wish to reduce the size of the record. Is it possible to do some kind of aggregation before indexing into ES in logstash? For example, it is possible to put only the record of the max va… --- ## [ILM Policy through logstash dynamic index name](https://discuss.elastic.co/t/ilm-policy-through-logstash-dynamic-index-name/268011)
**Author:** [@gbandasha](https://discuss.elastic.co/u/gbandasha)\ **Replies:** 9\ **Last updated:** [April 13, 2021, 7:05pm UTC](https://discuss.elastic.co/t/ilm-policy-through-logstash-dynamic-index-name/268011 "2021-04-13T19:05:07Z")
Hello Team, I am trying to implement the ILM Policy but I ran into an issue because my index name is dynamically created through logstash. Let me give you an explain of my setup and wat I am trying to do I create th… --- ## [Duplicated Log](https://discuss.elastic.co/t/duplicated-log/269931)
**Author:** [@Paulogbr](https://discuss.elastic.co/u/Paulogbr)\ **Replies:** 9\ **Last updated:** [April 13, 2021, 6:19pm UTC](https://discuss.elastic.co/t/duplicated-log/269931 "2021-04-13T18:19:41Z")
Hello Guys, My firewall device is sending duplicated log. I tried filter with logstash. I need write the output in file. I test with logger but don´t work, I tried use fingerprint. Can you have any ideia ? inpu… --- ## [Logstash doesn't automatically collect all Zeek fields without grok pattern](https://discuss.elastic.co/t/logstash-doesnt-automatically-collect-all-zeek-fields-without-grok-pattern/269613)
**Author:** [@Automation\_Scripts](https://discuss.elastic.co/u/Automation_Scripts)\ **Replies:** 3\ **Last updated:** [April 13, 2021, 6:01pm UTC](https://discuss.elastic.co/t/logstash-doesnt-automatically-collect-all-zeek-fields-without-grok-pattern/269613 "2021-04-13T18:01:49Z")
Hi, Is there a setting I need to provide in order to enable the automatically collection of all the Zeek's log fields? I can collect the fields message only through a grok filter. My assumption is that logstash is smar… --- ## [Xml with optionnal fields not added in the result array by logstash](https://discuss.elastic.co/t/xml-with-optionnal-fields-not-added-in-the-result-array-by-logstash/269891)
**Author:** [@meepmeep](https://discuss.elastic.co/u/meepmeep)\ **Replies:** 7\ **Last updated:** [April 13, 2021, 3:17pm UTC](https://discuss.elastic.co/t/xml-with-optionnal-fields-not-added-in-the-result-array-by-logstash/269891 "2021-04-13T15:17:09Z")
Hi :slight\_smile: I'm moving slowly in this new wold of elastic stack. I'm currently stuck on a parsing issue with my xml file (ip and nickname removed) : \ \ \ \User1\ … --- ## [Can I get a specific part of the referrer in the nginx log when parsing it via logstash](https://discuss.elastic.co/t/can-i-get-a-specific-part-of-the-referrer-in-the-nginx-log-when-parsing-it-via-logstash/269865)
**Author:** [@pradeep\_gadkari](https://discuss.elastic.co/u/pradeep_gadkari)\ **Replies:** 3\ **Last updated:** [April 13, 2021, 3:13pm UTC](https://discuss.elastic.co/t/can-i-get-a-specific-part-of-the-referrer-in-the-nginx-log-when-parsing-it-via-logstash/269865 "2021-04-13T15:13:04Z")
My grok pattern is as below: grok{ match =\> { "message" =\> \["%{IPV4:IP\_address} (?:-|(%{WORD}.%{WORD})) %{USER:ident} \\\[%{HTTPDATE:message\_timestamp}\\\] \\"(?:%{WORD:message\_type} %{NOTSPACE:request}(?: HTTP/%{NUMB… --- ## [Logstash Runs And Does Nothing](https://discuss.elastic.co/t/logstash-runs-and-does-nothing/269932)
**Author:** [@baklavadeseni](https://discuss.elastic.co/u/baklavadeseni)\ **Replies:** 2\ **Last updated:** [April 13, 2021, 8:16am UTC](https://discuss.elastic.co/t/logstash-runs-and-does-nothing/269932 "2021-04-13T08:16:39Z")
Hi there, I have simple file input/output logstash pipeline. Logstash runs but does nothing. There is no output. In logstash log file only suspicious thing is this "One or more required cgroup files or directories not f… --- ## [Csv filter plugin not working](https://discuss.elastic.co/t/csv-filter-plugin-not-working/268777)
**Author:** [@its-ogawa](https://discuss.elastic.co/u/its-ogawa)\ **Replies:** 28\ **Last updated:** [April 13, 2021, 1:07am UTC](https://discuss.elastic.co/t/csv-filter-plugin-not-working/268777 "2021-04-13T01:07:06Z")
I would like to use logstash to format my logs. Specifically, I have a log with the following format format: timestamp \[thread-name\] log-level class-name - log-message example: 2021-03-30 09:38:20.201 \[ConnectionChe… --- ## [Does logstash use openssl?](https://discuss.elastic.co/t/does-logstash-use-openssl/269827)
**Author:** [@TakaSeki](https://discuss.elastic.co/u/TakaSeki)\ **Replies:** 4\ **Last updated:** [April 13, 2021, 12:41am UTC](https://discuss.elastic.co/t/does-logstash-use-openssl/269827 "2021-04-13T00:41:09Z")
Does logstash use openssl when using the SSL / TLS protocol? If so, where can I find the version of openssl? --- ## [Replace the @TimeStamp with actual time during a bulk Filebeat Send](https://discuss.elastic.co/t/replace-the-timestamp-with-actual-time-during-a-bulk-filebeat-send/269882)
**Author:** [@Wilks](https://discuss.elastic.co/u/Wilks)\ **Replies:** 3\ **Last updated:** [April 12, 2021, 11:22pm UTC](https://discuss.elastic.co/t/replace-the-timestamp-with-actual-time-during-a-bulk-filebeat-send/269882 "2021-04-12T23:22:37Z")
Hi, We are collecting a large amount of logs through out the day and then sending them at 4:30am via Filebeat in a bulk send. The problem is when they end up in Elasticsearch/Kibana all the @TimeStamp fields are 4:30am… --- ## [Multiple pipeline](https://discuss.elastic.co/t/multiple-pipeline/269907)
**Author:** [@dddddddddddddddd](https://discuss.elastic.co/u/dddddddddddddddd)\ **Replies:** 2\ **Last updated:** [April 12, 2021, 9:35pm UTC](https://discuss.elastic.co/t/multiple-pipeline/269907 "2021-04-12T21:35:53Z")
hi I have 2 servers , the first is DHCP server and the second is DNS server so for shipping the logs to logstash i installed filebeats so in logstash I will apply the differents filtres on DNS and DHCP so how can do th… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=237) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=239)