# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=169 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 170 --- ## [Can't start logstash after installtion because java](https://discuss.elastic.co/t/cant-start-logstash-after-installtion-because-java/293993)
**Author:** [@aviad.co1](https://discuss.elastic.co/u/aviad.co1)\ **Replies:** 0\ **Last updated:** [January 11, 2022, 8:49am UTC](https://discuss.elastic.co/t/cant-start-logstash-after-installtion-because-java/293993 "2022-01-11T08:49:51Z")
I install this java version - java -version : openjdk version "11.0.13" 2021-10-19. OpenJDK Runtime Environment (build 11.0.13+8-Ubuntu-0ubuntu1.20.04). OpenJDK 64-Bit Server VM (build 11.0.13+8-Ubuntu-0ubuntu1.20.04… --- ## [Receiving data to the ELK database using logstash](https://discuss.elastic.co/t/receiving-data-to-the-elk-database-using-logstash/292547)
**Author:** [@San9](https://discuss.elastic.co/u/San9)\ **Replies:** 5\ **Last updated:** [January 11, 2022, 7:56am UTC](https://discuss.elastic.co/t/receiving-data-to-the-elk-database-using-logstash/292547 "2022-01-11T07:56:20Z")
I have a problem with accessing the ELK database and receiving a specific field with a token to form a further request to connect to the o365 api. I can't get data from this particular field. scheme, I connect to the EL… --- ## [Grok Pattern not working](https://discuss.elastic.co/t/grok-pattern-not-working/293159)
**Author:** [@Pranav\_M](https://discuss.elastic.co/u/Pranav_M)\ **Replies:** 5\ **Last updated:** [January 11, 2022, 4:50am UTC](https://discuss.elastic.co/t/grok-pattern-not-working/293159 "2022-01-11T04:50:00Z")
Hey Community, Received packet: Type = N5abcde8AbcAbcde9AbcdAbcde15AbcdefghAbcdefA, xx = 95, xxx = 441, xxxXxo = 2, xXxxXxx = 1, xxxx = 9 accepted. The above line is a message from my log and I want to seperate each da… --- ## [Latitude and Longitude Error in logstash?](https://discuss.elastic.co/t/latitude-and-longitude-error-in-logstash/293855)
**Author:** [@padamrai](https://discuss.elastic.co/u/padamrai)\ **Replies:** 11\ **Last updated:** [January 11, 2022, 4:35am UTC](https://discuss.elastic.co/t/latitude-and-longitude-error-in-logstash/293855 "2022-01-11T04:35:23Z")
Hello Everyone, I am trying to convert PostalCode to lat and long, I have done this task but Now I am confuse how to convert it into geopoint field I have tried in logstash but no success so for, getting error. My con… --- ## [Logstash S3 output requires bucket object permissions at the root of the bucket, regardless of prefix config](https://discuss.elastic.co/t/logstash-s3-output-requires-bucket-object-permissions-at-the-root-of-the-bucket-regardless-of-prefix-config/293951)
**Author:** [@mdebord](https://discuss.elastic.co/u/mdebord)\ **Replies:** 0\ **Last updated:** [January 10, 2022, 10:30pm UTC](https://discuss.elastic.co/t/logstash-s3-output-requires-bucket-object-permissions-at-the-root-of-the-bucket-regardless-of-prefix-config/293951 "2022-01-10T22:30:54Z")
Logstash requires write access to the root of the bucket, regardless of the prefix defined in config. There are a few issues to note with this behavior: Logstash should be isolating its work into the defined prefix p… --- ## [Logstash Servers Stopping Each Others Logging](https://discuss.elastic.co/t/logstash-servers-stopping-each-others-logging/293784)
**Author:** [@daniel.talbot](https://discuss.elastic.co/u/daniel.talbot)\ **Replies:** 4\ **Last updated:** [January 10, 2022, 8:56pm UTC](https://discuss.elastic.co/t/logstash-servers-stopping-each-others-logging/293784 "2022-01-10T20:56:40Z")
Hi all, Currently I am troubleshooting a configuration that is working in our prod environment but not in our development environment. We are running multiple servers in Dev so I am testing changes on one while I let t… --- ## [Drop previous csv lines if field value already exists](https://discuss.elastic.co/t/drop-previous-csv-lines-if-field-value-already-exists/293947)
**Author:** [@Micah\_Barsness](https://discuss.elastic.co/u/Micah_Barsness)\ **Replies:** 1\ **Last updated:** [January 10, 2022, 8:55pm UTC](https://discuss.elastic.co/t/drop-previous-csv-lines-if-field-value-already-exists/293947 "2022-01-10T20:55:38Z")
I'd like to know if there is a way to drop/overwrite previous csv documents if a field value already exists. In my screenshot above globalCallID is what I want to match on. Take globalCallID\_callID number 263504... … --- ## [How to enrich one of field in logtash](https://discuss.elastic.co/t/how-to-enrich-one-of-field-in-logtash/293579)
**Author:** [@INS](https://discuss.elastic.co/u/INS)\ **Replies:** 1\ **Last updated:** [January 10, 2022, 12:47pm UTC](https://discuss.elastic.co/t/how-to-enrich-one-of-field-in-logtash/293579 "2022-01-10T12:47:48Z")
I'm struggling on enrich (split one of field from) CSV file input in my document I have many of undefined fields so at the begging I've needed to avoid that value. so I've decided to use mutate { gsub =\> \["message"… --- ## [Can a logstash filter error be forwarded to elastic?](https://discuss.elastic.co/t/can-a-logstash-filter-error-be-forwarded-to-elastic/293742)
**Author:** [@DavidMarcu](https://discuss.elastic.co/u/DavidMarcu)\ **Replies:** 4\ **Last updated:** [January 10, 2022, 8:15am UTC](https://discuss.elastic.co/t/can-a-logstash-filter-error-be-forwarded-to-elastic/293742 "2022-01-10T08:15:44Z")
I'm having these json parsing errors from time to time: 2022-01-07T12:15:19,872\]\[WARN \]\[logstash.filters.json \] Error parsing json {:source=\>"message", :raw=\>" { the invalid json }", :exception=\>#\ **Author:** [@Hooman24](https://discuss.elastic.co/u/Hooman24)\ **Replies:** 4\ **Last updated:** [January 10, 2022, 5:35am UTC](https://discuss.elastic.co/t/convert-yyyy-mm-dd-hh-mm-ss-to-date/293820 "2022-01-10T05:35:20Z") Hi I have a dateTime field with the following format: yyyy-mm-dd-HH.mm.ss I want to use this field as a date time field in ELK. I don't know how to change 11th position of the string('-') to 'T' to convert this field … --- ## [Output to different index based on field](https://discuss.elastic.co/t/output-to-different-index-based-on-field/293804)
**Author:** [@krsecurity](https://discuss.elastic.co/u/krsecurity)\ **Replies:** 11\ **Last updated:** [January 8, 2022, 11:01pm UTC](https://discuss.elastic.co/t/output-to-different-index-based-on-field/293804 "2022-01-08T23:01:30Z")
Hi there, I have a couple of use-cases where I want to ingest filebeat module data, send it through Logstash, and then put it into different indexes based on what filebeat module that it used. So for example, the two I… --- ## [Ignore log record if log does not contain specific string "jenkins\_build\_number"](https://discuss.elastic.co/t/ignore-log-record-if-log-does-not-contain-specific-string-jenkins-build-number/293800)
**Author:** [@chandu.2035](https://discuss.elastic.co/u/chandu.2035)\ **Replies:** 2\ **Last updated:** [January 8, 2022, 4:10pm UTC](https://discuss.elastic.co/t/ignore-log-record-if-log-does-not-contain-specific-string-jenkins-build-number/293800 "2022-01-08T16:10:51Z")
Hi there, I would like to ignore inserting entry in Elasticsearch index if the parsed log pattern does not contain the specific string jenkins\_build\_number Here is my filebeat and logstash configs. filebeat.yml #=====… --- ## [Why my logstash aggregate sometimes works, sometimes not?](https://discuss.elastic.co/t/why-my-logstash-aggregate-sometimes-works-sometimes-not/293342)
**Author:** [@maoxuguang](https://discuss.elastic.co/u/maoxuguang)\ **Replies:** 7\ **Last updated:** [January 7, 2022, 4:46pm UTC](https://discuss.elastic.co/t/why-my-logstash-aggregate-sometimes-works-sometimes-not/293342 "2022-01-07T16:46:26Z")
for example, sometimes I can get the tag \[ "fordebug\_%{job\_name}" \], it indicates that the aggregate executed successfully, sometimes I can not get this tag. why? I struggled for days but no finding. I am exhausted. cou… --- ## [Error listing bucket contents: Error getting access token for service account: oauth2.googleapis.com](https://discuss.elastic.co/t/error-listing-bucket-contents-error-getting-access-token-for-service-account-oauth2-googleapis-com/293704)
**Author:** [@parmarX](https://discuss.elastic.co/u/parmarX)\ **Replies:** 0\ **Last updated:** [January 7, 2022, 4:56am UTC](https://discuss.elastic.co/t/error-listing-bucket-contents-error-getting-access-token-for-service-account-oauth2-googleapis-com/293704 "2022-01-07T04:56:14Z")
Hello , I am facing this issue while running my pipeline using configmap and deployment in docker. my plugin is unable to get access token for service accound . i am using plugin input\_google\_cloud\_storage to get file… --- ## [Logstash Date Format Failure](https://discuss.elastic.co/t/logstash-date-format-failure/293702)
**Author:** [@insu0929](https://discuss.elastic.co/u/insu0929)\ **Replies:** 2\ **Last updated:** [January 7, 2022, 4:17am UTC](https://discuss.elastic.co/t/logstash-date-format-failure/293702 "2022-01-07T04:17:53Z")
Hi I'm using filebeat to send some session count data to Logstash. The file type is csv and the data looks like this: ... "01/02/2022 00:00:27.837", "0.99961" "01/02/2022 00:00:28.853", "0" "01/02/2022 00:00:29.240", "1… --- ## [How to split sometimes?](https://discuss.elastic.co/t/how-to-split-sometimes/293484)
**Author:** [@4art4](https://discuss.elastic.co/u/4art4)\ **Replies:** 7\ **Last updated:** [January 6, 2022, 6:00pm UTC](https://discuss.elastic.co/t/how-to-split-sometimes/293484 "2022-01-06T18:00:47Z")
So I have to deal with ingesting logs that a team member is generating. I'm getting json logs, and I think I am very close to getting the data. (So close.) The error I am getting now looks like this: \[2022-01-04T18:0… --- ## [One logstash pipeline listening multiple formats (same input)](https://discuss.elastic.co/t/one-logstash-pipeline-listening-multiple-formats-same-input/293381)
**Author:** [@vee](https://discuss.elastic.co/u/vee)\ **Replies:** 16\ **Last updated:** [January 6, 2022, 7:33pm UTC](https://discuss.elastic.co/t/one-logstash-pipeline-listening-multiple-formats-same-input/293381 "2022-01-06T19:33:16Z")
Hi - Working on a new requirement to parse through one file containing two different formatted messages and send to different indices on elastic. Filebeat (same file with two different formatted messages) -\> Logstash (… --- ## [Failed to parse field \[\[observer.hostname\]\]](https://discuss.elastic.co/t/failed-to-parse-field-observer-hostname/293676)
**Author:** [@eO2H\_LG](https://discuss.elastic.co/u/eO2H_LG)\ **Replies:** 3\ **Last updated:** [January 6, 2022, 6:48pm UTC](https://discuss.elastic.co/t/failed-to-parse-field-observer-hostname/293676 "2022-01-06T18:48:07Z")
Hello All, Hoping somebody can assist me with the following error. It seems some of my DHCP data is failing to ingest due to a parsing issue with the observer.hostname field. Below is the error output from logstash-pla… --- ## [TCP Input "Too many files open" IOException](https://discuss.elastic.co/t/tcp-input-too-many-files-open-ioexception/293499)
**Author:** [@ddth\_2022](https://discuss.elastic.co/u/ddth_2022)\ **Replies:** 5\ **Last updated:** [January 6, 2022, 4:14pm UTC](https://discuss.elastic.co/t/tcp-input-too-many-files-open-ioexception/293499 "2022-01-06T16:14:58Z")
We're running Logstash 7.16.2. We're using a TCP input that opens up too many sockets without closing them and almost every 24 hours Logstash hangs up. \[2022-01-05T05:10:59,474\]\[WARN \]\[io.netty.channel.DefaultChannelPip… --- ## [Replace Log4j from 2.11.0 to 2.15.0](https://discuss.elastic.co/t/replace-log4j-from-2-11-0-to-2-15-0/291901)
**Author:** [@njain213](https://discuss.elastic.co/u/njain213)\ **Replies:** 9\ **Last updated:** [January 6, 2022, 9:01am UTC](https://discuss.elastic.co/t/replace-log4j-from-2-11-0-to-2-15-0/291901 "2022-01-06T09:01:31Z")
Hello Team, I am using logstash 7.5.1 and having log4j jar as 2.11.1. Now as per doc https://discuss.elastic.co/ it is mentioned to remove JNDI class if we don't want to upgrade logstash. Is it possible if I can upgrade… --- ## [JSON parse error](https://discuss.elastic.co/t/json-parse-error/293512)
**Author:** [@rusty\_cole](https://discuss.elastic.co/u/rusty_cole)\ **Replies:** 3\ **Last updated:** [January 6, 2022, 8:00am UTC](https://discuss.elastic.co/t/json-parse-error/293512 "2022-01-06T08:00:22Z")
Hi, I have the following error only for some of my json files, other files have no issue. I am guessing the root cause is the brackets in the json fields. I have seen some posts in the matter but no solution worked fo… --- ## [Logstash is affected by Apache Log4j2 2.17.0 Vulnerability CVE-2021-44832?](https://discuss.elastic.co/t/logstash-is-affected-by-apache-log4j2-2-17-0-vulnerability-cve-2021-44832/293162)
**Author:** [@swxEmily](https://discuss.elastic.co/u/swxEmily)\ **Replies:** 3\ **Last updated:** [January 6, 2022, 2:08am UTC](https://discuss.elastic.co/t/logstash-is-affected-by-apache-log4j2-2-17-0-vulnerability-cve-2021-44832/293162 "2022-01-06T02:08:51Z")
Apache Log4j2 2.17.0 report a new Vulnerability CVE-2021-44832, is Logstash affected or not? --- ## [Logstash not send data to Elasticsearch](https://discuss.elastic.co/t/logstash-not-send-data-to-elasticsearch/293554)
**Author:** [@mcoa](https://discuss.elastic.co/u/mcoa)\ **Replies:** 2\ **Last updated:** [January 5, 2022, 9:45pm UTC](https://discuss.elastic.co/t/logstash-not-send-data-to-elasticsearch/293554 "2022-01-05T21:45:00Z")
Hi, I've configured Logstash for send data to Elasticsearch but this not receive data and index not found (i dont have firewall/iptables/selinux.). My config logstash: Input input { file { path =\> "/var/log/zim… --- ## [Sending AWS S3 Data to AWS Cloud Watch](https://discuss.elastic.co/t/sending-aws-s3-data-to-aws-cloud-watch/293594)
**Author:** [@makimaki](https://discuss.elastic.co/u/makimaki)\ **Replies:** 1\ **Last updated:** [January 5, 2022, 9:04pm UTC](https://discuss.elastic.co/t/sending-aws-s3-data-to-aws-cloud-watch/293594 "2022-01-05T21:04:38Z")
Hi, I'm trying to send log data from AWS S3 to CloudWatch and nothing is happening This is log file {"docker": {"container\_id": "fe9802"},"kubernetes": {"container\_name": "pairwise-individual"}},"message": "2022-01-05… --- ## [Logs from AWS S3 does not go to AWS CoudWatch](https://discuss.elastic.co/t/logs-from-aws-s3-does-not-go-to-aws-coudwatch/293583)
**Author:** [@makimaki](https://discuss.elastic.co/u/makimaki)\ **Replies:** 0\ **Last updated:** [January 5, 2022, 7:31pm UTC](https://discuss.elastic.co/t/logs-from-aws-s3-does-not-go-to-aws-coudwatch/293583 "2022-01-05T19:31:56Z")
Hi, I'm trying to read logs from the S3 bucket and put them to CloudWatch. But nothing happens in the CloudWatch. Log file in S3 contain min 2 JSON strings {"docker":{"container\_id":"fe98027af6490c542e8d0321115f4c1e9… --- ## [Unable to install logstash-input-azureblob with logstash 7.16.2](https://discuss.elastic.co/t/unable-to-install-logstash-input-azureblob-with-logstash-7-16-2/293504)
**Author:** [@Natarajan](https://discuss.elastic.co/u/Natarajan)\ **Replies:** 0\ **Last updated:** [January 5, 2022, 8:20am UTC](https://discuss.elastic.co/t/unable-to-install-logstash-input-azureblob-with-logstash-7-16-2/293504 "2022-01-05T08:20:29Z")
I am trying to install logstash-input-azureblob with logstash 7.16.2. But getting the below error. I am running the logstash as docker. ERROR: Installation Aborted, message: Bundler could not find compatible versions … --- ## [Logstash Exception in posting data to Azure Loganalytics](https://discuss.elastic.co/t/logstash-exception-in-posting-data-to-azure-loganalytics/293497)
**Author:** [@stillfreem](https://discuss.elastic.co/u/stillfreem)\ **Replies:** 0\ **Last updated:** [January 5, 2022, 4:53am UTC](https://discuss.elastic.co/t/logstash-exception-in-posting-data-to-azure-loganalytics/293497 "2022-01-05T04:53:17Z")
Hi All, I've a server on which Logstash is installed and sending logs to Azure Sentinel. It works just fine but lately any other week I'd see the below entry in the logstash-plain.log file and it seems that the connect… --- ## [Logstash multiline codec plugin won't keep blank lines](https://discuss.elastic.co/t/logstash-multiline-codec-plugin-wont-keep-blank-lines/293429)
**Author:** [@saprof](https://discuss.elastic.co/u/saprof)\ **Replies:** 2\ **Last updated:** [January 4, 2022, 7:26pm UTC](https://discuss.elastic.co/t/logstash-multiline-codec-plugin-wont-keep-blank-lines/293429 "2022-01-04T19:26:57Z")
Hi, I am working on the correct settings for the multiline codec plugin. I want to join all lines between two \\r\\n into a single event and it works well, except that blank lines are dropped. I'm running Logstash v7.16.… --- ## [Logstash prune blacklist\_values](https://discuss.elastic.co/t/logstash-prune-blacklist-values/293266)
**Author:** [@4art4](https://discuss.elastic.co/u/4art4)\ **Replies:** 6\ **Last updated:** [January 4, 2022, 5:38pm UTC](https://discuss.elastic.co/t/logstash-prune-blacklist-values/293266 "2022-01-04T17:38:05Z")
So the json inputs are not always consistent. For example, one field is usually true/false, but the source occasionally tosses in a "not\_supported" just to be fun. So after some digging, I thought that I could just do: … --- ## [Logstash is running but not sending logs to elasticsearch](https://discuss.elastic.co/t/logstash-is-running-but-not-sending-logs-to-elasticsearch/293433)
**Author:** [@Othmane\_CHHAIBI](https://discuss.elastic.co/u/Othmane_CHHAIBI)\ **Replies:** 1\ **Last updated:** [January 4, 2022, 4:48pm UTC](https://discuss.elastic.co/t/logstash-is-running-but-not-sending-logs-to-elasticsearch/293433 "2022-01-04T16:48:42Z")
Hello everyone, after the upgrade of logstash from version 6.8 to 7.16.2, the status of logstash is running but it isn't sending logs to Elasticsearch. this is the logstash log file in the image below :slight\_smile: … [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=168) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=170)