# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=142 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 143 --- ## [Timestamp field issue](https://discuss.elastic.co/t/timestamp-field-issue/304126)
**Author:** [@shivendra95](https://discuss.elastic.co/u/shivendra95)\ **Replies:** 2\ **Last updated:** [May 9, 2022, 7:31am UTC](https://discuss.elastic.co/t/timestamp-field-issue/304126 "2022-05-09T07:31:04Z")
Hi there, I'm using S3 to read input into logstash. The timestamp field generated in Elasticsearch is the time when logs are read from S3 and not the timestamp field of the json log file. Below is one object of the jso… --- ## [Logstash cant start the pipeline in the conf.d file](https://discuss.elastic.co/t/logstash-cant-start-the-pipeline-in-the-conf-d-file/304172)
**Author:** [@ahmed\_barki](https://discuss.elastic.co/u/ahmed_barki)\ **Replies:** 9\ **Last updated:** [May 8, 2022, 7:49pm UTC](https://discuss.elastic.co/t/logstash-cant-start-the-pipeline-in-the-conf-d-file/304172 "2022-05-08T19:49:00Z")
Hello everyone, when i use the command /usr/share/logstash/bin/logstash -f /etc/logstash/conf.d/simple.conf here is what i get '\[INFO \] 2022-05-07 09:58:02.271 \[\[main\]-pipeline-manager\] elasticsearch - New Elasticsear… --- ## [Sending data to logstash from remote machine](https://discuss.elastic.co/t/sending-data-to-logstash-from-remote-machine/304192)
**Author:** [@safaeaz](https://discuss.elastic.co/u/safaeaz)\ **Replies:** 4\ **Last updated:** [May 8, 2022, 3:58pm UTC](https://discuss.elastic.co/t/sending-data-to-logstash-from-remote-machine/304192 "2022-05-08T15:58:58Z")
i have logstash running on a different machine and want to read csv files from anothere mechine. Is there a way to set the ip in file input of config file? --- ## [Local Logstash 7.9.3 error: (EACCES) Permission denied - NUL](https://discuss.elastic.co/t/local-logstash-7-9-3-error-eacces-permission-denied-nul/304153)
**Author:** [@JonathanHai](https://discuss.elastic.co/u/JonathanHai)\ **Replies:** 1\ **Last updated:** [May 6, 2022, 8:23pm UTC](https://discuss.elastic.co/t/local-logstash-7-9-3-error-eacces-permission-denied-nul/304153 "2022-05-06T20:23:17Z")
Hi everyone, I'm attempting to run a local instance of Logstash 7.9.3 on my windows machine. Specifically, C:\\logstash-7.9.3\\bin\>logstash.bat \[ERROR\] 2022-05-06 15:04:57.066 \[main\] Logstash - java.lang.IllegalStateExce… --- ## [Throttle Filter Logstash](https://discuss.elastic.co/t/throttle-filter-logstash/304104)
**Author:** [@Bigboy0706](https://discuss.elastic.co/u/Bigboy0706)\ **Replies:** 1\ **Last updated:** [May 6, 2022, 4:47pm UTC](https://discuss.elastic.co/t/throttle-filter-logstash/304104 "2022-05-06T16:47:21Z")
Hi As there are some deduplicated log ingressing into logstash That's why I would like to use the Throttle Filter to filter out the log The config is listed as below filter { if \[name\] == "pack\_incident\_response… --- ## [XML plugin parse](https://discuss.elastic.co/t/xml-plugin-parse/304124)
**Author:** [@adrianfusco](https://discuss.elastic.co/u/adrianfusco)\ **Replies:** 1\ **Last updated:** [May 6, 2022, 4:41pm UTC](https://discuss.elastic.co/t/xml-plugin-parse/304124 "2022-05-06T16:41:09Z")
Hello, I've been using the XML filtering plugin because I need to parse some XML data. This is a simple example: \ \ \ Desc… --- ## [Grok help needed](https://discuss.elastic.co/t/grok-help-needed/304102)
**Author:** [@sdevasy73](https://discuss.elastic.co/u/sdevasy73)\ **Replies:** 6\ **Last updated:** [May 6, 2022, 3:53pm UTC](https://discuss.elastic.co/t/grok-help-needed/304102 "2022-05-06T15:53:16Z")
i am a newbie to grok pattern, trying to parse the below mentioned cisco firewall syslog. With the try struggling with what type to use to extract the interested fields, some of them works if i paste it individually, bu… --- ## [Logstash can't connect to Elasticsearch](https://discuss.elastic.co/t/logstash-cant-connect-to-elasticsearch/303623)
**Author:** [@ozz](https://discuss.elastic.co/u/ozz)\ **Replies:** 2\ **Last updated:** [May 6, 2022, 3:37pm UTC](https://discuss.elastic.co/t/logstash-cant-connect-to-elasticsearch/303623 "2022-05-06T15:37:51Z")
Hi guys. I'm trying to connect to Elasticsearch from Logstash, but it fails due to certificate issues. To give some background data: Version 8.1.3 on Kubernetes with Rancher. I build a couple of kustomization yaml files… --- ## [Logstash hangs before indexing the last log event](https://discuss.elastic.co/t/logstash-hangs-before-indexing-the-last-log-event/303725)
**Author:** [@aviral\_srivastava](https://discuss.elastic.co/u/aviral_srivastava)\ **Replies:** 6\ **Last updated:** [May 6, 2022, 12:24pm UTC](https://discuss.elastic.co/t/logstash-hangs-before-indexing-the-last-log-event/303725 "2022-05-06T12:24:51Z")
Hi, My setup: Elasticsearch 8.1.0, kibana 8.1.0, logstash 8.1.0 My logstash config: input{ file{ path =\> "C:/Users/avisriva/Documents/Logs/TDW\_9 Logs/logs\_14082020.txt" codec =\> multiline{ pattern =\> "Log Ent… --- ## [AWS Cloudwatch \> Logstash \> Elasticsearch](https://discuss.elastic.co/t/aws-cloudwatch-logstash-elasticsearch/302938)
**Author:** [@wallace84](https://discuss.elastic.co/u/wallace84)\ **Replies:** 3\ **Last updated:** [May 6, 2022, 11:39am UTC](https://discuss.elastic.co/t/aws-cloudwatch-logstash-elasticsearch/302938 "2022-05-06T11:39:20Z")
Hello community, I was designing a log filtering process and I would like to know if there is a way to send logs from AWS Cloudwatch to our logstash docker container in a server and from there being processed to be inde… --- ## [Logstash Multiple input missbehaving](https://discuss.elastic.co/t/logstash-multiple-input-missbehaving/304092)
**Author:** [@dharminfadia](https://discuss.elastic.co/u/dharminfadia)\ **Replies:** 2\ **Last updated:** [May 6, 2022, 8:57am UTC](https://discuss.elastic.co/t/logstash-multiple-input-missbehaving/304092 "2022-05-06T08:57:33Z")
Hello Every One I m using logstash http input and tcp inout in multiple conf file in single logstash machine while I start machin I am getting tcp input DATA in to http input this is ODD how can I fix this please sugg… --- ## [Logstash s3 error Failed to open TCP connection](https://discuss.elastic.co/t/logstash-s3-error-failed-to-open-tcp-connection/304068)
**Author:** [@swev](https://discuss.elastic.co/u/swev)\ **Replies:** 0\ **Last updated:** [May 5, 2022, 9:01pm UTC](https://discuss.elastic.co/t/logstash-s3-error-failed-to-open-tcp-connection/304068 "2022-05-05T21:01:27Z")
I have been trying to get the s3 plugin to work have been running into issues connecting to the url. The interest part is the url it reports in the error message is missing the region, which was referenced in the endpo… --- ## [Udp - Exception in inputworker - Logstash (7.4.2) with netflow plugin](https://discuss.elastic.co/t/udp-exception-in-inputworker-logstash-7-4-2-with-netflow-plugin/304056)
**Author:** [@gurumu](https://discuss.elastic.co/u/gurumu)\ **Replies:** 2\ **Last updated:** [May 5, 2022, 7:58pm UTC](https://discuss.elastic.co/t/udp-exception-in-inputworker-logstash-7-4-2-with-netflow-plugin/304056 "2022-05-05T19:58:39Z")
\[ERROR\] 2022-05-05 09:24:32.009 \[\java.lang.NullPointerException, "backtrace"=\>\["org.jruby.runtime.invokedynamic.InvokeDynamicSupport.callMethodMissing(InvokeDynamicSup… --- ## [Logstash s3 input plugin not working without prefix](https://discuss.elastic.co/t/logstash-s3-input-plugin-not-working-without-prefix/304067)
**Author:** [@sindhu\_vutukuri](https://discuss.elastic.co/u/sindhu_vutukuri)\ **Replies:** 0\ **Last updated:** [May 5, 2022, 7:47pm UTC](https://discuss.elastic.co/t/logstash-s3-input-plugin-not-working-without-prefix/304067 "2022-05-05T19:47:19Z")
I am trying to ingest msk logs from s3 bucket to Elasticsearch via logstash input s3 plugin. S3 bucket holds multiple msk cluster logs. Here is my logstash input config input { s3 { "region" =\> "{{ … --- ## [Logstash user permissions](https://discuss.elastic.co/t/logstash-user-permissions/304049)
**Author:** [@ksobon](https://discuss.elastic.co/u/ksobon)\ **Replies:** 5\ **Last updated:** [May 5, 2022, 5:25pm UTC](https://discuss.elastic.co/t/logstash-user-permissions/304049 "2022-05-05T17:25:06Z")
I have configured a "logstash" user with the following set of permissions: "{ "indices": \[{ "names": \["\*beat\*"\], "privileges": \["create", "create\_index", "view\_index\_metadata"\] } \] }" That however results in … --- ## [Can't query by IDs using logstash-filter-elasticserach](https://discuss.elastic.co/t/cant-query-by-ids-using-logstash-filter-elasticserach/304024)
**Author:** [@Gerard\_Purra](https://discuss.elastic.co/u/Gerard_Purra)\ **Replies:** 1\ **Last updated:** [May 5, 2022, 4:09pm UTC](https://discuss.elastic.co/t/cant-query-by-ids-using-logstash-filter-elasticserach/304024 "2022-05-05T16:09:11Z")
Hey folks, I’m using the logstash-filter-elasticsearch and I’d like to query using IDs (IDs | Elasticsearch Guide \[8.2\] | Elastic), and I find myself unable. Does anyone have any idea on how to work around that? The pr… --- ## [Logstash Input-Path as Index](https://discuss.elastic.co/t/logstash-input-path-as-index/304030)
**Author:** [@Robsen\_Inc](https://discuss.elastic.co/u/Robsen_Inc)\ **Replies:** 1\ **Last updated:** [May 5, 2022, 4:06pm UTC](https://discuss.elastic.co/t/logstash-input-path-as-index/304030 "2022-05-05T16:06:36Z")
Hi Community, I would like to use the Path as an index. The question has been asked many times in different forums, but unfortunately the solutions presented do not work for me. Maybe you can help me. Specifically, I… --- ## [Grok ignore possible extra word/text at end of the line?](https://discuss.elastic.co/t/grok-ignore-possible-extra-word-text-at-end-of-the-line/304026)
**Author:** [@tjswe](https://discuss.elastic.co/u/tjswe)\ **Replies:** 3\ **Last updated:** [May 5, 2022, 4:03pm UTC](https://discuss.elastic.co/t/grok-ignore-possible-extra-word-text-at-end-of-the-line/304026 "2022-05-05T16:03:48Z")
Slightly embarrassed, but i can not find the answer of a what feels like simple question. Data looks like this ABC WORD1 DEF WORD2 GHI WORD3 or ABC WORD1 DEF WORD2 GHI WORD3 (nonimportanttext) This below works with … --- ## [Logstash add custom jars to java class path](https://discuss.elastic.co/t/logstash-add-custom-jars-to-java-class-path/304039)
**Author:** [@ayout](https://discuss.elastic.co/u/ayout)\ **Replies:** 0\ **Last updated:** [May 5, 2022, 1:25pm UTC](https://discuss.elastic.co/t/logstash-add-custom-jars-to-java-class-path/304039 "2022-05-05T13:25:58Z")
how can i just add jars to logstash image ( i need them in order to be able to use custom jdbc driver ,) , when trying to do so the docker runs but fails after 1min . this is the Dockerfile : COPY ./\*.jar /usr/share/lo… --- ## [Storing images , videos in elasticserch](https://discuss.elastic.co/t/storing-images-videos-in-elasticserch/303985)
**Author:** [@Naga\_Prudhvi](https://discuss.elastic.co/u/Naga_Prudhvi)\ **Replies:** 8\ **Last updated:** [May 5, 2022, 10:17am UTC](https://discuss.elastic.co/t/storing-images-videos-in-elasticserch/303985 "2022-05-05T10:17:20Z")
Can someone help me how can I store images in elasticserch? 1st way: using logstash conf file 2nd way; using directly elasticserch --- ## [Error when running pipelines.yml](https://discuss.elastic.co/t/error-when-running-pipelines-yml/303710)
**Author:** [@rrrrrrrrrrr](https://discuss.elastic.co/u/rrrrrrrrrrr)\ **Replies:** 8\ **Last updated:** [May 5, 2022, 2:33am UTC](https://discuss.elastic.co/t/error-when-running-pipelines-yml/303710 "2022-05-05T02:33:47Z")
Hello, I'm a newbie in Elastic community and I'm trying to run my pipelines.yml on /etc/logstash/ using this command /usr/share/logstash/bin/logstash -f pipelines.yml but all I'm getting is the error below... \[WARN \] 2… --- ## [Logstash Error](https://discuss.elastic.co/t/logstash-error/303872)
**Author:** [@Akhil2](https://discuss.elastic.co/u/Akhil2)\ **Replies:** 2\ **Last updated:** [May 4, 2022, 6:07pm UTC](https://discuss.elastic.co/t/logstash-error/303872 "2022-05-04T18:07:59Z")
Hello there, I am using Elasticsearch three node implementation cluster. I am using version 7.16.2. I have set up this cluster with basic settings. No security has been set up. Here is the example of data I tried to I… --- ## [Convert field to an array](https://discuss.elastic.co/t/convert-field-to-an-array/303884)
**Author:** [@JeremyP](https://discuss.elastic.co/u/JeremyP)\ **Replies:** 2\ **Last updated:** [May 4, 2022, 4:47pm UTC](https://discuss.elastic.co/t/convert-field-to-an-array/303884 "2022-05-04T16:47:32Z")
Hello, I'm looking to convert the following field to a hash of values.... Source Field: "ip\_addresses": "{192.168.1.1, 192.168.1.2, 192.168.1.3}" Desired Result: "ip\_addresses": \[ "192.168.1.1", "192… --- ## [Help with GROK Pattern](https://discuss.elastic.co/t/help-with-grok-pattern/303547)
**Author:** [@trubeat\_elk](https://discuss.elastic.co/u/trubeat_elk)\ **Replies:** 0\ **Last updated:** [April 28, 2022, 6:48pm UTC](https://discuss.elastic.co/t/help-with-grok-pattern/303547 "2022-04-28T18:48:12Z")
Hello I am very new to ELK and I am stuck at extracting fields.Below is the sample data Dec 9 06:36:01 s-login-01 CRON\[2436102\]: pam\_unix(cron:session): session closed for user mXXt Dec 9 06:34:07 s-login-01 sshd\[24… --- ## [Logstash Json Split Filter](https://discuss.elastic.co/t/logstash-json-split-filter/303955)
**Author:** [@elknick](https://discuss.elastic.co/u/elknick)\ **Replies:** 5\ **Last updated:** [May 4, 2022, 4:17pm UTC](https://discuss.elastic.co/t/logstash-json-split-filter/303955 "2022-05-04T16:17:12Z")
I am trying to use the Split filter in Logstash to separate my Json data into separate documents in elastic. I have used this same Split filter in another case and did not have this problem. I have tried targeting a few … --- ## [How to parse flatten json logs in logstash?](https://discuss.elastic.co/t/how-to-parse-flatten-json-logs-in-logstash/303802)
**Author:** [@talbehat](https://discuss.elastic.co/u/talbehat)\ **Replies:** 4\ **Last updated:** [May 4, 2022, 2:57pm UTC](https://discuss.elastic.co/t/how-to-parse-flatten-json-logs-in-logstash/303802 "2022-05-04T14:57:19Z")
Getting error :- \[2022-05-03T22:21:09,874\]\[INFO \]\[logstash.filters.flattenjson\]\[main\] Running flatten\_json filter {:event=\>#LogStash::Event:0x246b5df6} \[2022-05-03T22:21:09,880\]\[ERROR\]\[org.logstash.execution.WorkerLoop… --- ## [Logstash parse windows event nxlog](https://discuss.elastic.co/t/logstash-parse-windows-event-nxlog/303138)
**Author:** [@Chloe\_Boissavy](https://discuss.elastic.co/u/Chloe_Boissavy)\ **Replies:** 7\ **Last updated:** [May 4, 2022, 2:52pm UTC](https://discuss.elastic.co/t/logstash-parse-windows-event-nxlog/303138 "2022-05-04T14:52:15Z")
Hello, I am using nxlog to send windows data at my ELK. I would like to parse the "message" part. An example below : "message": \[ "2022-04-25 10:54:13 - DESKTOP-QMCS5UA - Security - INFO - 4672 - Privilèges… --- ## [Logstash filter plugin install No \`java.exe' executable found on PATH](https://discuss.elastic.co/t/logstash-filter-plugin-install-no-java-exe-executable-found-on-path/303534)
**Author:** [@Bavaria](https://discuss.elastic.co/u/Bavaria)\ **Replies:** 8\ **Last updated:** [May 4, 2022, 1:12pm UTC](https://discuss.elastic.co/t/logstash-filter-plugin-install-no-java-exe-executable-found-on-path/303534 "2022-05-04T13:12:25Z")
I want to get the fingerprint plugin for logstash on an airgapped windows system. For this I need to download and install it on a system, which has internet access and continue from there? Like in this shown. https://www… --- ## [Split filter logstash](https://discuss.elastic.co/t/split-filter-logstash/303925)
**Author:** [@alex\_vermex](https://discuss.elastic.co/u/alex_vermex)\ **Replies:** 0\ **Last updated:** [May 4, 2022, 9:45am UTC](https://discuss.elastic.co/t/split-filter-logstash/303925 "2022-05-04T09:45:16Z")
Hi, Split filter works fine but i have a problem here my log looks like 2312-15:44:07:813|V2.5.4|DOW |WooalsewD6/TTxrff==|ss|ss|0110006|0Succeed| 1110-14:17:40:282|V2.5.1|G|1212|444|||EXCEPTION : System.ServiceModel.… --- ## [How to handle grok optional pattern (?:)](https://discuss.elastic.co/t/how-to-handle-grok-optional-pattern/303796)
**Author:** [@marwen](https://discuss.elastic.co/u/marwen)\ **Replies:** 6\ **Last updated:** [May 4, 2022, 7:36am UTC](https://discuss.elastic.co/t/how-to-handle-grok-optional-pattern/303796 "2022-05-04T07:36:37Z")
Hi everyone I'm new to Elastick Stack, I set grok custom pattern using RegEx: MODULE\_NAME (?:((?\<=\[\\\[\])\\/\\S\[^\\\]\]+)) the problem is it's optional pattern so I want the "module\_name" field to be empty instead of showing … [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=141) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=143)