# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=171 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 172 --- ## [Logstash does not use the correct index with Elasticsearch output](https://discuss.elastic.co/t/logstash-does-not-use-the-correct-index-with-elasticsearch-output/293052)
**Author:** [@dat\_tang](https://discuss.elastic.co/u/dat_tang)\ **Replies:** 6\ **Last updated:** [December 28, 2021, 6:39pm UTC](https://discuss.elastic.co/t/logstash-does-not-use-the-correct-index-with-elasticsearch-output/293052 "2021-12-28T18:39:28Z")
Hi, I have an ELK stack which I updated 2 weeks ago from 7.2.0 to 7.16.1. The configuration did not change and is as follow: # all input will come from filebeat, no local logs input { beats { port =\> 5044 } } f… --- ## [Logstash JDBC driver](https://discuss.elastic.co/t/logstash-jdbc-driver/293041)
**Author:** [@Farid\_N](https://discuss.elastic.co/u/Farid_N)\ **Replies:** 1\ **Last updated:** [December 28, 2021, 6:16pm UTC](https://discuss.elastic.co/t/logstash-jdbc-driver/293041 "2021-12-28T18:16:09Z")
Hi dears I have upgraded my cluster form 7.6.2 to 7.16.2 There is only one problem I faced with. My Logstash has some pipelines that need to connect to a sql db and according to a statement, get a SP result from sql d… --- ## [Bug in Log4j mitigation steps in Logstash 5.x](https://discuss.elastic.co/t/bug-in-log4j-mitigation-steps-in-logstash-5-x/292612)
**Author:** [@sandeepkanabar](https://discuss.elastic.co/u/sandeepkanabar)\ **Replies:** 5\ **Last updated:** [December 21, 2021, 8:38pm UTC](https://discuss.elastic.co/t/bug-in-log4j-mitigation-steps-in-logstash-5-x/292612 "2021-12-21T20:38:23Z")
There's an excellent article on Logstash remediation to protect against Log4jshell vulnerability - Logstash 5.0.0-6.8.20 and 7.0.0-7.16.0: Log4j CVE-2021-44228, CVE-2021-45046 remediation. However, the steps suggested f… --- ## [Elasticsearch filter plugin doesn't work when using elasticsearch on cloud](https://discuss.elastic.co/t/elasticsearch-filter-plugin-doesnt-work-when-using-elasticsearch-on-cloud/293008)
**Author:** [@CasMeiron](https://discuss.elastic.co/u/CasMeiron)\ **Replies:** 1\ **Last updated:** [December 28, 2021, 12:39am UTC](https://discuss.elastic.co/t/elasticsearch-filter-plugin-doesnt-work-when-using-elasticsearch-on-cloud/293008 "2021-12-28T00:39:46Z")
Hi, I've noticed that Elasticsearch filter plugin wont work if you try to connect to a Elasticsearch cloud instance. It seems it tries to append https:// twice (one already exists on cloud host resolution). Seems like … --- ## [Logstash Ruby How to get fields of json array](https://discuss.elastic.co/t/logstash-ruby-how-to-get-fields-of-json-array/292867)
**Author:** [@chandima\_jayamina](https://discuss.elastic.co/u/chandima_jayamina)\ **Replies:** 1\ **Last updated:** [December 27, 2021, 11:07pm UTC](https://discuss.elastic.co/t/logstash-ruby-how-to-get-fields-of-json-array/292867 "2021-12-27T23:07:19Z")
Hi I have an array "testItem":\["test1":"N", "test2":"", "test3":"xyz", "items":{"itemId":"123"},"test4","null"\] In here I want to extract the itemId from Json input through logstash how can i do it. I am new to logstas… --- ## [Logstash CPU gets pegged, all processing stops with Apache logs](https://discuss.elastic.co/t/logstash-cpu-gets-pegged-all-processing-stops-with-apache-logs/292985)
**Author:** [@erihar](https://discuss.elastic.co/u/erihar)\ **Replies:** 10\ **Last updated:** [December 27, 2021, 7:27pm UTC](https://discuss.elastic.co/t/logstash-cpu-gets-pegged-all-processing-stops-with-apache-logs/292985 "2021-12-27T19:27:40Z")
We are doing Apache accesslog ingestion into Logstash, from publicly-facing web servers. These work correctly, however after a day or two, some "crafty" requests come in, either from a security scan or hack attempt, and… --- ## [Logstash as a parser for data in column](https://discuss.elastic.co/t/logstash-as-a-parser-for-data-in-column/292352)
**Author:** [@INS](https://discuss.elastic.co/u/INS)\ **Replies:** 21\ **Last updated:** [December 27, 2021, 4:57pm UTC](https://discuss.elastic.co/t/logstash-as-a-parser-for-data-in-column/292352 "2021-12-27T16:57:25Z")
Hi I'm wonder about approach for this specific data constructed in column. How I can parser only through logstash. Some of fields a specially "ENUM" "HEADER", "BLOCK", "DATE" etc, are changing dynamically. One files i… --- ## [Logstash memcached plugin does not run in set mode](https://discuss.elastic.co/t/logstash-memcached-plugin-does-not-run-in-set-mode/292894)
**Author:** [@username11](https://discuss.elastic.co/u/username11)\ **Replies:** 2\ **Last updated:** [December 27, 2021, 7:42am UTC](https://discuss.elastic.co/t/logstash-memcached-plugin-does-not-run-in-set-mode/292894 "2021-12-27T07:42:18Z")
I am using Logstash 6.8.22 and logstash-filter-memcached-0.1.2. I cannot, for the life of me, get the memcached plugin to set any values. I checked it through the command line, and memcached itself works fine and can set… --- ## [Logstash not writing data to Elasticsearch data streams](https://discuss.elastic.co/t/logstash-not-writing-data-to-elasticsearch-data-streams/292541)
**Author:** [@Rahul\_Dey](https://discuss.elastic.co/u/Rahul_Dey)\ **Replies:** 3\ **Last updated:** [December 27, 2021, 7:09am UTC](https://discuss.elastic.co/t/logstash-not-writing-data-to-elasticsearch-data-streams/292541 "2021-12-27T07:09:54Z")
We are using logstash version 7.4.2 currently. For mitigating threats of log4j vulnerability, I have upgraded the Logstash version to 7.16.1 as advised by the Elastic team. I have also changed the Elasticsearch output p… --- ## [Logstash 7.16.2(fixed Log4j2 Vulnerability) connected to ElasticSearch 7.6.2 failed](https://discuss.elastic.co/t/logstash-7-16-2-fixed-log4j2-vulnerability-connected-to-elasticsearch-7-6-2-failed/292945)
**Author:** [@swxEmily](https://discuss.elastic.co/u/swxEmily)\ **Replies:** 3\ **Last updated:** [December 27, 2021, 4:24am UTC](https://discuss.elastic.co/t/logstash-7-16-2-fixed-log4j2-vulnerability-connected-to-elasticsearch-7-6-2-failed/292945 "2021-12-27T04:24:16Z")
Third party Elasticsearch: 7.6.2 Logstash: updated from 7.10.1 to 7.16.2(fixed Log4j2 Vulnerability) logstash.conf: output { Elasticsearch{ hosts =\> \["https://10.33.27.xxx:9200","https://10.33.27.xxy:9200","https://… --- ## [Testing aggregates =\> LogStash::ConfigurationError: Aggregate plugin: more than one filter which defines timeout options. But only defining once](https://discuss.elastic.co/t/testing-aggregates-logstash-aggregate-plugin-more-than-one-filter-which-defines-timeout-options-but-only-defining-once/289644)
**Author:** [@Colin\_K](https://discuss.elastic.co/u/Colin_K)\ **Replies:** 6\ **Last updated:** [December 26, 2021, 4:33am UTC](https://discuss.elastic.co/t/testing-aggregates-logstash-aggregate-plugin-more-than-one-filter-which-defines-timeout-options-but-only-defining-once/289644 "2021-12-26T04:33:57Z")
I'm trying to test my logstash configurations that use the aggregate plugin. I have not had any issues running them locally against the binary as I write them. But now I want to test the behavior expectations of the aggr… --- ## [Logstash not indexing properly](https://discuss.elastic.co/t/logstash-not-indexing-properly/292853)
**Author:** [@eleong](https://discuss.elastic.co/u/eleong)\ **Replies:** 4\ **Last updated:** [December 25, 2021, 3:50am UTC](https://discuss.elastic.co/t/logstash-not-indexing-properly/292853 "2021-12-25T03:50:07Z")
Hi, I have a single-node Elastic stack running with the following architecture: Filebeat \> Logstash \> Elasticsearch Filebeat is doing the tagging and logstash will point the documents to be indexed on different indice… --- ## [Logstash join (aggregate?) two aggregated maps (postfix)](https://discuss.elastic.co/t/logstash-join-aggregate-two-aggregated-maps-postfix/292820)
**Author:** [@rowra](https://discuss.elastic.co/u/rowra)\ **Replies:** 3\ **Last updated:** [December 24, 2021, 5:08pm UTC](https://discuss.elastic.co/t/logstash-join-aggregate-two-aggregated-maps-postfix/292820 "2021-12-24T17:08:02Z")
Hi, I have two postfix servers. Call one 'local' and the other 'dmz'. I collect log from each and I aggregate them with taskid being the queue id postfix gives to each process. All works fine so far, I get one aggregate… --- ## [Cloudflare Logs via Logstash](https://discuss.elastic.co/t/cloudflare-logs-via-logstash/291743)
**Author:** [@ZiaulAfiq](https://discuss.elastic.co/u/ZiaulAfiq)\ **Replies:** 5\ **Last updated:** [December 24, 2021, 1:24pm UTC](https://discuss.elastic.co/t/cloudflare-logs-via-logstash/291743 "2021-12-24T13:24:43Z")
Hello & Greetings all! I'm doing some testing to retrieve Cloudflare logs using Logstash. My approach in retrieving the Cloudflare logs are by using Bash script to pull the logs from cloudflare then push them to logstas… --- ## [Load multiple tables data from oltp to elasticsearch using logstash](https://discuss.elastic.co/t/load-multiple-tables-data-from-oltp-to-elasticsearch-using-logstash/292886)
**Author:** [@umang\_t](https://discuss.elastic.co/u/umang_t)\ **Replies:** 0\ **Last updated:** [December 24, 2021, 11:17am UTC](https://discuss.elastic.co/t/load-multiple-tables-data-from-oltp-to-elasticsearch-using-logstash/292886 "2021-12-24T11:17:19Z")
What is the efficient way to load data from multiple tables (e.g. 10 to 15 tables from RDBMS) in to Elasticsearch . Note the RDBMS has records in millions so idea is to flatten the resultset and load in Elasticsearch . H… --- ## [Unit logstash.service entered failed state](https://discuss.elastic.co/t/unit-logstash-service-entered-failed-state/292865)
**Author:** [@cva](https://discuss.elastic.co/u/cva)\ **Replies:** 0\ **Last updated:** [December 24, 2021, 6:02am UTC](https://discuss.elastic.co/t/unit-logstash-service-entered-failed-state/292865 "2021-12-24T06:02:28Z")
Hi! OS Centos 7, java -version openjdk version "1.8.0\_252" Updated the version of logstash to 7.16.2, now at startup I get the following systemctl status logstash: Dec 24 01:27:08 elk systemd\[1\]: Unit logstash.servic… --- ## [Logstash configuration file in joining two csv files based on primary key](https://discuss.elastic.co/t/logstash-configuration-file-in-joining-two-csv-files-based-on-primary-key/292792)
**Author:** [@Dhamu-143](https://discuss.elastic.co/u/Dhamu-143)\ **Replies:** 1\ **Last updated:** [December 23, 2021, 2:36pm UTC](https://discuss.elastic.co/t/logstash-configuration-file-in-joining-two-csv-files-based-on-primary-key/292792 "2021-12-23T14:36:03Z")
Hi, I'm not able to join the two csv files based on the primary key(same value or same field). I'm taking two csv input files that are shown below Student Table(Table 1) Stdid,sname,fee 121,john,10000 123,glenn,120… --- ## [Xml input messy fields](https://discuss.elastic.co/t/xml-input-messy-fields/292812)
**Author:** [@abdullah144](https://discuss.elastic.co/u/abdullah144)\ **Replies:** 2\ **Last updated:** [December 23, 2021, 2:29pm UTC](https://discuss.elastic.co/t/xml-input-messy-fields/292812 "2021-12-23T14:29:52Z")
Hey Guys , I already can sent a log to my logstash as my log is xml format , but the issue is logstash creating a new fields from tag as below : I want to stop him creating the new fields from logMessage tag . My … --- ## [How to consume Logstash through the Postman](https://discuss.elastic.co/t/how-to-consume-logstash-through-the-postman/292702)
**Author:** [@abdullah144](https://discuss.elastic.co/u/abdullah144)\ **Replies:** 1\ **Last updated:** [December 22, 2021, 3:08pm UTC](https://discuss.elastic.co/t/how-to-consume-logstash-through-the-postman/292702 "2021-12-22T15:08:46Z")
Hey guys, I have a requirement to push data to Logstash through Postman where it will be automatically store it in Elasticsearch , how can I do that ? Best Regards, --- ## [Data not sent to ElasticSearch (with Logstash)](https://discuss.elastic.co/t/data-not-sent-to-elasticsearch-with-logstash/292696)
**Author:** [@Greninja\_San](https://discuss.elastic.co/u/Greninja_San)\ **Replies:** 6\ **Last updated:** [December 23, 2021, 2:03pm UTC](https://discuss.elastic.co/t/data-not-sent-to-elasticsearch-with-logstash/292696 "2021-12-23T14:03:22Z")
Hello! I'm trying to send data to Elasticsearch with logstash on a Linux server. For some reasons, I do not get any error, but nothing is sent. input { file { path =\> \["/srv/XXX/csv\_receivers/XXX\_receiver/\*"\] … --- ## [Logstash with two dissect - only the first dissect is used now and then](https://discuss.elastic.co/t/logstash-with-two-dissect-only-the-first-dissect-is-used-now-and-then/292794)
**Author:** [@stillfreem](https://discuss.elastic.co/u/stillfreem)\ **Replies:** 1\ **Last updated:** [December 23, 2021, 10:09am UTC](https://discuss.elastic.co/t/logstash-with-two-dissect-only-the-first-dissect-is-used-now-and-then/292794 "2021-12-23T10:09:14Z")
Hi All, I have the following conf { syslog { host =\> "127.0.0.1" port =\> 5000 } } filter { if \[program\] == "github\_auth" or \[program == "github\_gitauth"\] or \[progr… --- ## [Logstash new connection and connection reset](https://discuss.elastic.co/t/logstash-new-connection-and-connection-reset/292782)
**Author:** [@alfianaf](https://discuss.elastic.co/u/alfianaf)\ **Replies:** 0\ **Last updated:** [December 23, 2021, 8:35am UTC](https://discuss.elastic.co/t/logstash-new-connection-and-connection-reset/292782 "2021-12-23T08:35:38Z")
Hello, I was looking for explanation on the logstash logs about "new connection" and "connection reset", and I couldn't find any clue. Does "new connection" mean that a new connection has been established from client to… --- ## [Failed to connect to backoff while connecting Winlogbeat with Logstash](https://discuss.elastic.co/t/failed-to-connect-to-backoff-while-connecting-winlogbeat-with-logstash/292781)
**Author:** [@risshukla](https://discuss.elastic.co/u/risshukla)\ **Replies:** 0\ **Last updated:** [December 23, 2021, 8:35am UTC](https://discuss.elastic.co/t/failed-to-connect-to-backoff-while-connecting-winlogbeat-with-logstash/292781 "2021-12-23T08:35:06Z")
Hello Everyone, I am trying to update Winlogbeat to 7.16.2 to make it compatible with Logstash 7.16.2. Our configuration was already working between two stack on version 7.6.0, so we are upgrading just the minor versio… --- ## [CICD pipeline for testing the logstash config](https://discuss.elastic.co/t/cicd-pipeline-for-testing-the-logstash-config/292778)
**Author:** [@WimDH](https://discuss.elastic.co/u/WimDH)\ **Replies:** 0\ **Last updated:** [December 23, 2021, 8:24am UTC](https://discuss.elastic.co/t/cicd-pipeline-for-testing-the-logstash-config/292778 "2021-12-23T08:24:12Z")
Dears, I have my logstash config in gitlab, from where I deploy it on my production machines. As the config grows, and others contribute to it, the risk of getting errors is becoming bigger. Therefore, I would like to … --- ## [MSSQL input hanging after update](https://discuss.elastic.co/t/mssql-input-hanging-after-update/292742)
**Author:** [@tobias\_c](https://discuss.elastic.co/u/tobias_c)\ **Replies:** 0\ **Last updated:** [December 22, 2021, 9:41pm UTC](https://discuss.elastic.co/t/mssql-input-hanging-after-update/292742 "2021-12-22T21:41:45Z")
Hi all After updating from logstash 7.5.1 to 7.16.2 and updating from mssql.jdbc.4.2.jar to mssql.jdbc.9.4 (for jdk11 as the bundled jdk version) I cannot transfer my mssql data to Elasticsearch anymore. Logstash jdbc … --- ## [Logstash - grokparsefailure](https://discuss.elastic.co/t/logstash-grokparsefailure/292705)
**Author:** [@hassen\_k](https://discuss.elastic.co/u/hassen_k)\ **Replies:** 1\ **Last updated:** [December 22, 2021, 6:27pm UTC](https://discuss.elastic.co/t/logstash-grokparsefailure/292705 "2021-12-22T18:27:31Z")
Hi, Logstash version 7.16.2 I am trying to parse the following logs: amAuthWindowsDesktopSSO:12/22/2021 03:55:44:349 PM CET: Thread\[http-nio-8080-exec-3,5,main\]: TransactionId\[b742688f-6089-41c4-925d-ba08dce4d7b7-1491… --- ## [One or more required cgroup files or directories not found: /proc/self/cgroup, /sys/fs/cgroup/cpuacct, /sys/fs/cgroup/cpu](https://discuss.elastic.co/t/one-or-more-required-cgroup-files-or-directories-not-found-proc-self-cgroup-sys-fs-cgroup-cpuacct-sys-fs-cgroup-cpu/292689)
**Author:** [@rusty\_cole](https://discuss.elastic.co/u/rusty_cole)\ **Replies:** 6\ **Last updated:** [December 22, 2021, 5:21pm UTC](https://discuss.elastic.co/t/one-or-more-required-cgroup-files-or-directories-not-found-proc-self-cgroup-sys-fs-cgroup-cpuacct-sys-fs-cgroup-cpu/292689 "2021-12-22T17:21:14Z")
Hi Guys, I have logstash running on a windows machine. My inputs and parsers are set up correctly and everything works fine. The only issue I have - when I start the logstash process, it wont ingest files, instead it… --- ## [Failed test QueueTest](https://discuss.elastic.co/t/failed-test-queuetest/292719)
**Author:** [@SirBouBou](https://discuss.elastic.co/u/SirBouBou)\ **Replies:** 0\ **Last updated:** [December 22, 2021, 4:23pm UTC](https://discuss.elastic.co/t/failed-test-queuetest/292719 "2021-12-22T16:23:24Z")
Hi, I'm trying to discover Logstash and its uses to be able to contribute and help the community but I have a problem. When launching a ./gradlew test I have a timeout error for the concurrentWritesTest in the QueueTest… --- ## [SNMP input with MIB](https://discuss.elastic.co/t/snmp-input-with-mib/292673)
**Author:** [@Christer\_Palmen](https://discuss.elastic.co/u/Christer_Palmen)\ **Replies:** 2\ **Last updated:** [December 22, 2021, 3:29pm UTC](https://discuss.elastic.co/t/snmp-input-with-mib/292673 "2021-12-22T15:29:37Z")
"Hello. I´m trying to setup an snmp polling function in our Elasticsearch with Logstash. Today it is an "Proof of Concept" with only one 4G router from the vendor named Advantech. I have followed the instructions foun… --- ## [Logstash Internal API server error](https://discuss.elastic.co/t/logstash-internal-api-server-error/292707)
**Author:** [@Disha\_Bodade](https://discuss.elastic.co/u/Disha_Bodade)\ **Replies:** 0\ **Last updated:** [December 22, 2021, 3:27pm UTC](https://discuss.elastic.co/t/logstash-internal-api-server-error/292707 "2021-12-22T15:27:40Z")
I am receiving error in logstash after upgraded from 7.11.x to 7.16.1 this log is not frequent, I feel some action is generating that log but I am not sure which action is generating this error, My pipelines are proper… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=170) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=172)