# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=179 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 180 --- ## [Getting illegal\_state\_exception error while pushing logs to elasticsearch](https://discuss.elastic.co/t/getting-illegal-state-exception-error-while-pushing-logs-to-elasticsearch/290029)
**Author:** [@Chitrank\_Tyagi](https://discuss.elastic.co/u/Chitrank_Tyagi)\ **Replies:** 1\ **Last updated:** [November 24, 2021, 6:38pm UTC](https://discuss.elastic.co/t/getting-illegal-state-exception-error-while-pushing-logs-to-elasticsearch/290029 "2021-11-24T18:38:10Z")
I'm getting illegal\_state\_exception error, Can someone please help me this? mapping\_template.json : error: "error"=\>{"type"=\>"mapper\_parsing\_exception", "reason"=\>"failed to parse field \[message\] of type \[text\] in do… --- ## [Logstash concatinates topics](https://discuss.elastic.co/t/logstash-concatinates-topics/290033)
**Author:** [@Mamol27](https://discuss.elastic.co/u/Mamol27)\ **Replies:** 1\ **Last updated:** [November 24, 2021, 5:38pm UTC](https://discuss.elastic.co/t/logstash-concatinates-topics/290033 "2021-11-24T17:38:09Z")
Hi! I have pipeline kafka→logstash→Elasticsearch I created two topics in kafka: test-1 and test-2 Write manually to these topics messages: 67 into test-1 and 67 into test-2. I wrote two logstash workers test1.conf i… --- ## [Logstash using KV filter creating a field with an array rather than individual fields and individual values](https://discuss.elastic.co/t/logstash-using-kv-filter-creating-a-field-with-an-array-rather-than-individual-fields-and-individual-values/290007)
**Author:** [@neil6323](https://discuss.elastic.co/u/neil6323)\ **Replies:** 2\ **Last updated:** [November 24, 2021, 5:32am UTC](https://discuss.elastic.co/t/logstash-using-kv-filter-creating-a-field-with-an-array-rather-than-individual-fields-and-individual-values/290007 "2021-11-24T05:32:19Z")
Hi all, I'm working through a Logstash configuration which is parsing Cisco ISE log entries. I'm using multiline to collate entries into records and a grok filter coupled with a 'kv' filter which is mostly having the de… --- ## [Run logstash on windows -\>\> errors](https://discuss.elastic.co/t/run-logstash-on-windows-errors/289987)
**Author:** [@INS](https://discuss.elastic.co/u/INS)\ **Replies:** 2\ **Last updated:** [November 23, 2021, 9:39pm UTC](https://discuss.elastic.co/t/run-logstash-on-windows-errors/289987 "2021-11-23T21:39:53Z")
Hi I need to check the performance one of my logstash conf. Therefore I've decided to install logstash on windows the new one instance as standalone. After that I've set the variable env. At least I got the error. BTW. … --- ## [Unable to read two json](https://discuss.elastic.co/t/unable-to-read-two-json/289963)
**Author:** [@Fosco](https://discuss.elastic.co/u/Fosco)\ **Replies:** 0\ **Last updated:** [November 23, 2021, 3:34pm UTC](https://discuss.elastic.co/t/unable-to-read-two-json/289963 "2021-11-23T15:34:40Z")
Hello, I have a python file that is executed by logstash and which prints a json composed of two different data sources, the problem is that logstash only reads the first part. I tried to change the order and nothing cha… --- ## [Logstash and Beats with mTLS](https://discuss.elastic.co/t/logstash-and-beats-with-mtls/289941)
**Author:** [@widhalmt](https://discuss.elastic.co/u/widhalmt)\ **Replies:** 0\ **Last updated:** [November 23, 2021, 11:27am UTC](https://discuss.elastic.co/t/logstash-and-beats-with-mtls/289941 "2021-11-23T11:27:12Z")
Hi, I have a working TLS connection between Beats and Logstash. What I now want is mTLS so that only Beats with a valid certificate can connect to Logstash. For policy reasons I can't use the certutil Tool within Elasti… --- ## [Rightsizing elastic batch size and number of workers](https://discuss.elastic.co/t/rightsizing-elastic-batch-size-and-number-of-workers/289803)
**Author:** [@alaine](https://discuss.elastic.co/u/alaine)\ **Replies:** 4\ **Last updated:** [November 23, 2021, 9:33am UTC](https://discuss.elastic.co/t/rightsizing-elastic-batch-size-and-number-of-workers/289803 "2021-11-23T09:33:47Z")
I am trying to right size my configs for the logstashes that I have feeding my elastic stack. They are pushing filebeat and winlogbeat data. Right now they seem to be fairly underutilized and I am seeing a little bit of … --- ## [Kerberos support in Logstash Jdbc input plugin](https://discuss.elastic.co/t/kerberos-support-in-logstash-jdbc-input-plugin/289921)
**Author:** [@jitentiwari82](https://discuss.elastic.co/u/jitentiwari82)\ **Replies:** 0\ **Last updated:** [November 23, 2021, 8:45am UTC](https://discuss.elastic.co/t/kerberos-support-in-logstash-jdbc-input-plugin/289921 "2021-11-23T08:45:37Z")
Hi Guys, I am trying to use Kerberos authentication to read data from Oracle database through Logstash JDBC plug-in . Has Kerberos support for database connection through JDBC input plugin been added ? Can someone plea… --- ## [Sorting case insensitive](https://discuss.elastic.co/t/sorting-case-insensitive/289875)
**Author:** [@Lyes\_Ouchene](https://discuss.elastic.co/u/Lyes_Ouchene)\ **Replies:** 4\ **Last updated:** [November 22, 2021, 11:31pm UTC](https://discuss.elastic.co/t/sorting-case-insensitive/289875 "2021-11-22T23:31:00Z")
Hello, can i add a property to make sorting against a field case insensitive in logstash. Thank you. --- ## [Logstash JMX input plugin cannot load Cassandra configuration](https://discuss.elastic.co/t/logstash-jmx-input-plugin-cannot-load-cassandra-configuration/289273)
**Author:** [@Erkan\_SIRIN](https://discuss.elastic.co/u/Erkan_SIRIN)\ **Replies:** 4\ **Last updated:** [November 22, 2021, 7:24pm UTC](https://discuss.elastic.co/t/logstash-jmx-input-plugin-cannot-load-cassandra-configuration/289273 "2021-11-22T19:24:20Z")
I am trying to get Cassandra metrics through logstash jmx input. My logstash pipeline conf: /etc/logstash/conf.d/cassandra\_jmx.conf input { jmx { path =\> "/etc/logstash/jmxconf" polling\_frequency =\> 15 … --- ## [Filebeat \> logstash "Invalid FieldReference" in the log](https://discuss.elastic.co/t/filebeat-logstash-invalid-fieldreference-in-the-log/289747)
**Author:** [@gyurgyalag](https://discuss.elastic.co/u/gyurgyalag)\ **Replies:** 1\ **Last updated:** [November 22, 2021, 2:47pm UTC](https://discuss.elastic.co/t/filebeat-logstash-invalid-fieldreference-in-the-log/289747 "2021-11-22T14:47:38Z")
seems there are similare topics to this without answer 1 and one suggests there is no solution 2 But I ask anyway maybe somebody can help. This is the error: ^\[\[Alogstash | \[2021-11-20T16:57:20,868\]\[INFO \]\[org.log… --- ## [Filebeat -\> logstash -\> elastic](https://discuss.elastic.co/t/filebeat-logstash-elastic/289469)
**Author:** [@alaine](https://discuss.elastic.co/u/alaine)\ **Replies:** 5\ **Last updated:** [November 22, 2021, 10:47am UTC](https://discuss.elastic.co/t/filebeat-logstash-elastic/289469 "2021-11-22T10:47:27Z")
I was using logstash to forward filebeat info (zeek and suricata) to the elastic cluster I manage. Filebeat is 7.12.0 and logstash and elastic are version 7.13.4. It was working without a problem until yesterday. I am n… --- ## [JDBC logging duplicate row](https://discuss.elastic.co/t/jdbc-logging-duplicate-row/289297)
**Author:** [@LinhNT1](https://discuss.elastic.co/u/LinhNT1)\ **Replies:** 3\ **Last updated:** [November 22, 2021, 8:46am UTC](https://discuss.elastic.co/t/jdbc-logging-duplicate-row/289297 "2021-11-22T08:46:02Z")
Hi there, I have a logtash with JDBC input like below jdbc { jdbc\_driver\_library =\> "${LOGSTASH\_JDBC\_DRIVER\_JAR\_LOCATION}" jdbc\_driver\_class =\> "${LOGSTASH\_JDBC\_DRIVER}" jdbc\_connection\_string =\>… --- ## [Logstsash is running pipelines in parallel](https://discuss.elastic.co/t/logstsash-is-running-pipelines-in-parallel/289614)
**Author:** [@rodri.gz](https://discuss.elastic.co/u/rodri.gz)\ **Replies:** 4\ **Last updated:** [November 22, 2021, 8:15am UTC](https://discuss.elastic.co/t/logstsash-is-running-pipelines-in-parallel/289614 "2021-11-22T08:15:38Z")
Hello! I have a divided pipeline that generates 2 indices that enrich each other. I would like to know how I can make it do one etl first and when it finishes the other. my logstash filter wich enrichs has this filter… --- ## [Delete file in ruby filter logstash](https://discuss.elastic.co/t/delete-file-in-ruby-filter-logstash/289671)
**Author:** [@Hertz\_Drive](https://discuss.elastic.co/u/Hertz_Drive)\ **Replies:** 6\ **Last updated:** [November 22, 2021, 4:28am UTC](https://discuss.elastic.co/t/delete-file-in-ruby-filter-logstash/289671 "2021-11-22T04:28:53Z")
my code in ruby filter def register(params) print "heello ruby come in" end def filter(event) Dir.glob('/home/hz/COM.txt').each { |file| File.chmod(0777, file) File.delete(file) print(file) } #return… --- ## [MONGODB | Error checking \:\: Mongo::Error::MaxMessageSize: Message exceeds allowed max message size. The max is 50331648](https://discuss.elastic.co/t/mongodb-error-checking-ip-port-mongo-message-exceeds-allowed-max-message-size-the-max-is-50331648/289750)
**Author:** [@Bugsbee](https://discuss.elastic.co/u/Bugsbee)\ **Replies:** 2\ **Last updated:** [November 21, 2021, 2:18am UTC](https://discuss.elastic.co/t/mongodb-error-checking-ip-port-mongo-message-exceeds-allowed-max-message-size-the-max-is-50331648/289750 "2021-11-21T02:18:54Z")
hello, any idea as to why i get the error below? my ELK has been working on. its just today i chnaged he password.. and nothing else. however this time when i start elk , i get the error below anything else i need t… --- ## [Documents are getting duplicated in mongodb](https://discuss.elastic.co/t/documents-are-getting-duplicated-in-mongodb/289745)
**Author:** [@Husnain](https://discuss.elastic.co/u/Husnain)\ **Replies:** 0\ **Last updated:** [November 20, 2021, 3:37pm UTC](https://discuss.elastic.co/t/documents-are-getting-duplicated-in-mongodb/289745 "2021-11-20T15:37:26Z")
Hi, I am trying to fetch data from mysql using logstash and storing them into Elasticsearch and mongodb.Document is getting stored in Elasticsearch properly and no duplicate documents are inserted,but in mongodb same do… --- ## [New to Logstash - can't get it to connect to my Elasticsearch](https://discuss.elastic.co/t/new-to-logstash-cant-get-it-to-connect-to-my-elasticsearch/289727)
**Author:** [@aluminex](https://discuss.elastic.co/u/aluminex)\ **Replies:** 1\ **Last updated:** [November 20, 2021, 3:28am UTC](https://discuss.elastic.co/t/new-to-logstash-cant-get-it-to-connect-to-my-elasticsearch/289727 "2021-11-20T03:28:55Z")
I have Elasticsearch and Kibana on a different host. I'm trying to get my Logstash instance to output to Elasticsearch on a remote host. I can forward my logs to Filebeat and then to Elastic, but I am trying to write/t… --- ## [Logstash rewrite value for the same key instead of append](https://discuss.elastic.co/t/logstash-rewrite-value-for-the-same-key-instead-of-append/289702)
**Author:** [@ddoroshenko](https://discuss.elastic.co/u/ddoroshenko)\ **Replies:** 1\ **Last updated:** [November 19, 2021, 6:30pm UTC](https://discuss.elastic.co/t/logstash-rewrite-value-for-the-same-key-instead-of-append/289702 "2021-11-19T18:30:08Z")
Hello! I'm trying to parse CheckPoint log which contains data like this \_\_policy\_id\_tag:"product=APP1 & APP2\[db\_tag={ABCDEF};mgmt=abcd;date=123456789;policy\_name=foo\]";product:"APP3" I use the filter filter { kv { … --- ## [Add json object](https://discuss.elastic.co/t/add-json-object/289682)
**Author:** [@Roberto\_B](https://discuss.elastic.co/u/Roberto_B)\ **Replies:** 1\ **Last updated:** [November 19, 2021, 6:08pm UTC](https://discuss.elastic.co/t/add-json-object/289682 "2021-11-19T18:08:15Z")
Hi there, I would like to add a field liek this: field : {sub1 : null} which is the best way to do that? Kind Regards Roberto --- ## [Logstash information](https://discuss.elastic.co/t/logstash-information/289595)
**Author:** [@tractor\_boy](https://discuss.elastic.co/u/tractor_boy)\ **Replies:** 3\ **Last updated:** [November 19, 2021, 6:00pm UTC](https://discuss.elastic.co/t/logstash-information/289595 "2021-11-19T18:00:55Z")
I am really struggling to understand the documentation, and as such cannot work out what I need to do to get logstash to work. So logstash arrives as a zip (Is there an msi?) But nothing configured. The documentation … --- ## [Getting logstash to run](https://discuss.elastic.co/t/getting-logstash-to-run/289673)
**Author:** [@tractor\_boy](https://discuss.elastic.co/u/tractor_boy)\ **Replies:** 1\ **Last updated:** [November 19, 2021, 5:37pm UTC](https://discuss.elastic.co/t/getting-logstash-to-run/289673 "2021-11-19T17:37:15Z")
I have downloaded the 7.15 version on logstash and unzipped. Unfortunately the lack of an installer leaves the solution in a non working state. My current - first - challenge is getting logstash to use the bundled jdk.… --- ## [Parsing issues](https://discuss.elastic.co/t/parsing-issues/289681)
**Author:** [@nickel43](https://discuss.elastic.co/u/nickel43)\ **Replies:** 1\ **Last updated:** [November 19, 2021, 5:32pm UTC](https://discuss.elastic.co/t/parsing-issues/289681 "2021-11-19T17:32:13Z")
Hello, I'm new to Elastic and trying to parse JSON files in order to have multiple fields, so that I can make statistics out of it in Kibana. Here is a sample: { "info": { "generated\_on": "2017-12-03 08:41… --- ## [Logstash not sending syslog to elasticsearch](https://discuss.elastic.co/t/logstash-not-sending-syslog-to-elasticsearch/289020)
**Author:** [@MKirby](https://discuss.elastic.co/u/MKirby)\ **Replies:** 41\ **Last updated:** [November 19, 2021, 3:36pm UTC](https://discuss.elastic.co/t/logstash-not-sending-syslog-to-elasticsearch/289020 "2021-11-19T15:36:16Z")
Hello Elastic team. I have been asking tons of questions and have been slowly making my way through the final part of my ELK SIEM installation. I am running a small environment with Elasticsearch, Logstash and Kibana i… --- ## [Logstash Lookup Fields](https://discuss.elastic.co/t/logstash-lookup-fields/289357)
**Author:** [@John\_snow](https://discuss.elastic.co/u/John_snow)\ **Replies:** 35\ **Last updated:** [November 18, 2021, 6:30pm UTC](https://discuss.elastic.co/t/logstash-lookup-fields/289357 "2021-11-18T18:30:57Z")
i wanted to use a lookup table in Logstash to check if account id exisits in lookub table then it will grab the output location from the lookup file. e--g Lookup File account\_id, output\_location, secrets, 123, s3, abcde… --- ## [Rename field problem](https://discuss.elastic.co/t/rename-field-problem/289669)
**Author:** [@ddoroshenko](https://discuss.elastic.co/u/ddoroshenko)\ **Replies:** 2\ **Last updated:** [November 19, 2021, 2:12pm UTC](https://discuss.elastic.co/t/rename-field-problem/289669 "2021-11-19T14:12:09Z")
Hello! I have input data like "abc":"123";"def":"456";"ghi":"789" I use the following filter to parse data and rename "abc" field to "blabla" filter { kv { source =\> "message" field\_split =\> ";" value\_s… --- ## [How to parse duplicated keys with different values](https://discuss.elastic.co/t/how-to-parse-duplicated-keys-with-different-values/288994)
**Author:** [@ddoroshenko](https://discuss.elastic.co/u/ddoroshenko)\ **Replies:** 3\ **Last updated:** [November 19, 2021, 9:33am UTC](https://discuss.elastic.co/t/how-to-parse-duplicated-keys-with-different-values/288994 "2021-11-19T09:33:14Z")
Hello! I'm trying to parse CheckPoint log which contains duplicated keys but with different values. For example ... match\_id:"555"; match\_id:"777"; ... When I use kv plugin kv { source =\> "message" field\_split =\> … --- ## [Improve Logstash data resiliency](https://discuss.elastic.co/t/improve-logstash-data-resiliency/289378)
**Author:** [@Rodolffo](https://discuss.elastic.co/u/Rodolffo)\ **Replies:** 4\ **Last updated:** [November 19, 2021, 12:19am UTC](https://discuss.elastic.co/t/improve-logstash-data-resiliency/289378 "2021-11-19T00:19:41Z")
Hi everyone! I'm working on an architecture in which we will ingest transactions into Elasticsearch from our application servers, and want to make absolutely sure that we store 100% of the data. I just want to validate … --- ## [Date format in Elasticsearch](https://discuss.elastic.co/t/date-format-in-elasticsearch/289608)
**Author:** [@kibanauser4](https://discuss.elastic.co/u/kibanauser4)\ **Replies:** 8\ **Last updated:** [November 18, 2021, 5:30pm UTC](https://discuss.elastic.co/t/date-format-in-elasticsearch/289608 "2021-11-18T17:30:20Z")
Hello. I am trying to import a .csv file to Elasticsearch. In the .csv file there is a date field in the "yyyyMMdd" format (for example 20220326). This is what I have in my logstash conf file: filter { date { m… --- ## [About repeating parameters when using if/else conditions](https://discuss.elastic.co/t/about-repeating-parameters-when-using-if-else-conditions/289448)
**Author:** [@espala](https://discuss.elastic.co/u/espala)\ **Replies:** 4\ **Last updated:** [November 18, 2021, 5:04pm UTC](https://discuss.elastic.co/t/about-repeating-parameters-when-using-if-else-conditions/289448 "2021-11-18T17:04:44Z")
Hello there, I'm using the config I've added as "original" below on my logstash server. My configuration works fine. my purpose; I have a lot of servers and I collect the general system logs of all of them on elk. I i… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=178) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=180)