# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=246 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 247 --- ## [Logstash and Split](https://discuss.elastic.co/t/logstash-and-split/267358)
**Author:** [@riahc3](https://discuss.elastic.co/u/riahc3)\ **Replies:** 4\ **Last updated:** [March 18, 2021, 5:56pm UTC](https://discuss.elastic.co/t/logstash-and-split/267358 "2021-03-18T17:56:11Z")
Hello I have the following fields (data is example): ForwardedIpAddress 1.2.3.4,4.3.2.1 IpAddress 1.2.3.4,4.3.2.1 Would I would like to end up with is: ForwardedIpAddress1 1.2.3.4 ForwardedIpAddress2 4.3.2.1 IpAdd… --- ## [How to read Excel File with multiple Worksheet in logstash?](https://discuss.elastic.co/t/how-to-read-excel-file-with-multiple-worksheet-in-logstash/267636)
**Author:** [@padamrai](https://discuss.elastic.co/u/padamrai)\ **Replies:** 1\ **Last updated:** [March 18, 2021, 4:58pm UTC](https://discuss.elastic.co/t/how-to-read-excel-file-with-multiple-worksheet-in-logstash/267636 "2021-03-18T16:58:58Z")
I have a dataset Excel file that have 10 worksheet so I want to access particular worksheet? How I can read worksheet? --- ## [Worker loop initialization error](https://discuss.elastic.co/t/worker-loop-initialization-error/267609)
**Author:** [@Aniket\_Pant](https://discuss.elastic.co/u/Aniket_Pant)\ **Replies:** 2\ **Last updated:** [March 18, 2021, 4:56pm UTC](https://discuss.elastic.co/t/worker-loop-initialization-error/267609 "2021-03-18T16:56:22Z")
I want to use multiple pipelines for logstash for example for input section i want to have pipeline.id: source\_input and for output and filter section i want to have pipeline.id: destination\_output But i am facing this… --- ## [Part of the indices are created with this name "%{\[kubernetes\]\[namespace\]}"](https://discuss.elastic.co/t/part-of-the-indices-are-created-with-this-name-kubernetes-namespace/267426)
**Author:** [@doronshai](https://discuss.elastic.co/u/doronshai)\ **Replies:** 3\ **Last updated:** [March 18, 2021, 4:52pm UTC](https://discuss.elastic.co/t/part-of-the-indices-are-created-with-this-name-kubernetes-namespace/267426 "2021-03-18T16:52:29Z")
Hi Team, I am using this configuration for logstash: input { beats {port =\> 6600 type =\> "K4" codec =\> "json" } } filter { if \[type\] == "K4" { json {source =\> "message"} } } output { if \[type\] =… --- ## [Join various WinLogBeat Windows Event Log events into one?](https://discuss.elastic.co/t/join-various-winlogbeat-windows-event-log-events-into-one/267095)
**Author:** [@riahc3](https://discuss.elastic.co/u/riahc3)\ **Replies:** 3\ **Last updated:** [March 18, 2021, 2:41pm UTC](https://discuss.elastic.co/t/join-various-winlogbeat-windows-event-log-events-into-one/267095 "2021-03-18T14:41:52Z")
Hello Using WinLogBeat, Im collecting logs from Windows Event Logs and sending it to Logstash which then sends it to Elasticsearch. For all, working. Issue is that I want to do something a bit more complex... Lets say… --- ## [Oracle DB - JDBC Plugin high load](https://discuss.elastic.co/t/oracle-db-jdbc-plugin-high-load/267680)
**Author:** [@FALEN](https://discuss.elastic.co/u/FALEN)\ **Replies:** 0\ **Last updated:** [March 18, 2021, 2:08pm UTC](https://discuss.elastic.co/t/oracle-db-jdbc-plugin-high-load/267680 "2021-03-18T14:08:06Z")
Hi everyone, I am looking for a way to transfer logs written to the Oracle table to ELK. The plugin seems appropriate, but in the environment I will use, 1tb data per day is written to this Oracle table. In other words,… --- ## [How to fetch substring from an input in logstash](https://discuss.elastic.co/t/how-to-fetch-substring-from-an-input-in-logstash/267035)
**Author:** [@John\_sams](https://discuss.elastic.co/u/John_sams)\ **Replies:** 0\ **Last updated:** [March 12, 2021, 5:23am UTC](https://discuss.elastic.co/t/how-to-fetch-substring-from-an-input-in-logstash/267035 "2021-03-12T05:23:16Z")
"tags":{ "container\_image":"confluentinc/cp-zookeeper:5.4.0@sha256" } How can I fetch docker image tag that is 5.4.0 ? The above is an input log from telegraf. --- ## [I faced not configaration found error in logstash](https://discuss.elastic.co/t/i-faced-not-configaration-found-error-in-logstash/267641)
**Author:** [@Utsav\_Jajadiya](https://discuss.elastic.co/u/Utsav_Jajadiya)\ **Replies:** 0\ **Last updated:** [March 18, 2021, 11:01am UTC](https://discuss.elastic.co/t/i-faced-not-configaration-found-error-in-logstash/267641 "2021-03-18T11:01:24Z")
--- ## [Logstash HTTP Output Plugin Error Could not fetch URL, Network is unreachable (connect failed)](https://discuss.elastic.co/t/logstash-http-output-plugin-error-could-not-fetch-url-network-is-unreachable-connect-failed/267633)
**Author:** [@kumarvijender](https://discuss.elastic.co/u/kumarvijender)\ **Replies:** 0\ **Last updated:** [March 18, 2021, 10:04am UTC](https://discuss.elastic.co/t/logstash-http-output-plugin-error-could-not-fetch-url-network-is-unreachable-connect-failed/267633 "2021-03-18T10:04:08Z")
I have configured a couple of pipelines in logstash, In one of the pipeline I have configured an http out plugin like output { http { url =\> "url" http\_method =\> "post" request\_timeout =\> \ aut… --- ## [Issues renaming fields](https://discuss.elastic.co/t/issues-renaming-fields/267588)
**Author:** [@Sketchy](https://discuss.elastic.co/u/Sketchy)\ **Replies:** 0\ **Last updated:** [March 18, 2021, 2:18am UTC](https://discuss.elastic.co/t/issues-renaming-fields/267588 "2021-03-18T02:18:38Z")
I am having issues with renaming a large number of fields, I am trying to create them as nested fields to organise them better in elastic. I keep getting \_mutate\_error and also this in the log. {:exception=\>"class org.… --- ## [Add\_tag failed](https://discuss.elastic.co/t/add-tag-failed/267281)
**Author:** [@Christophe\_Dumont](https://discuss.elastic.co/u/Christophe_Dumont)\ **Replies:** 1\ **Last updated:** [March 17, 2021, 7:27pm UTC](https://discuss.elastic.co/t/add-tag-failed/267281 "2021-03-17T19:27:52Z")
Hello, I want to ship logs from different log source. I need to add a tag to each different log sources. This conf doesn' t work ! input { beats { port =\> 5044 } } filter { if \[ source \]=="xxx.log" { mutate { … --- ## [Clean and split email address into two new fields](https://discuss.elastic.co/t/clean-and-split-email-address-into-two-new-fields/267489)
**Author:** [@cibernicola](https://discuss.elastic.co/u/cibernicola)\ **Replies:** 3\ **Last updated:** [March 17, 2021, 6:16pm UTC](https://discuss.elastic.co/t/clean-and-split-email-address-into-two-new-fields/267489 "2021-03-17T18:16:17Z")
Hello I have a field from a csv file called email, this field contains a string like this: mailtio:user@domain.tld I'm trying to remove "mailto:" as first step, then, copying field to a temp. one split the result into … --- ## [Is positive lookbehind supported in Logstash at all?](https://discuss.elastic.co/t/is-positive-lookbehind-supported-in-logstash-at-all/267388)
**Author:** [@stillfreem](https://discuss.elastic.co/u/stillfreem)\ **Replies:** 4\ **Last updated:** [March 17, 2021, 5:39pm UTC](https://discuss.elastic.co/t/is-positive-lookbehind-supported-in-logstash-at-all/267388 "2021-03-17T17:39:54Z")
Hi there all, i started with Logstash the other day and currently getting to know the tool. I have a situation here. Consider the below Baraccuda FW log entry 2018 01 30 13:12:21 Security +01:00 Block: type=FWD|proto=… --- ## [LDAP Lookup](https://discuss.elastic.co/t/ldap-lookup/267530)
**Author:** [@wwalker](https://discuss.elastic.co/u/wwalker)\ **Replies:** 2\ **Last updated:** [March 17, 2021, 4:33pm UTC](https://discuss.elastic.co/t/ldap-lookup/267530 "2021-03-17T16:33:52Z")
Is there a plugin out there that can perform LDAP lookups? --- ## [Logstash shutdown stalled while retrying rabbitmq input connection](https://discuss.elastic.co/t/logstash-shutdown-stalled-while-retrying-rabbitmq-input-connection/267474)
**Author:** [@samyascha](https://discuss.elastic.co/u/samyascha)\ **Replies:** 1\ **Last updated:** [March 17, 2021, 3:49pm UTC](https://discuss.elastic.co/t/logstash-shutdown-stalled-while-retrying-rabbitmq-input-connection/267474 "2021-03-17T15:49:12Z")
Hi, Im trying to find a solution for a Logstash (6.8.14) stalled shutdown when a RabbitMQ input is unavailable. Starting Logstash works fine, and it starts retrying connection to the RabbitMQ endpoint. That's fine. I … --- ## [Logstash doesn't take changes from log4j2.properties after restart service](https://discuss.elastic.co/t/logstash-doesnt-take-changes-from-log4j2-properties-after-restart-service/266751)
**Author:** [@krato](https://discuss.elastic.co/u/krato)\ **Replies:** 2\ **Last updated:** [March 17, 2021, 10:37am UTC](https://discuss.elastic.co/t/logstash-doesnt-take-changes-from-log4j2-properties-after-restart-service/266751 "2021-03-17T10:37:10Z")
I've noticed that after service restart logstash doesn't apply rules modified in log4j2.properties i.e appender.rolling.policies.size.size = 10MB (I've checked and the file has 12MB so it should work) Generally speaking… --- ## [Geo\_shape: linestring with data from csv](https://discuss.elastic.co/t/geo-shape-linestring-with-data-from-csv/266977)
**Author:** [@vincedt09](https://discuss.elastic.co/u/vincedt09)\ **Replies:** 3\ **Last updated:** [March 17, 2021, 7:33am UTC](https://discuss.elastic.co/t/geo-shape-linestring-with-data-from-csv/266977 "2021-03-17T07:33:14Z")
Hi! I am working on a project that involves drawing a line in kibana using data from a csv file. Csv file: latitude,longitude 44.453538,26.086064 44.455499,26.086069 44.457815,26.086090 44.459435,26.086106 Conf… --- ## [Could not index event to elasticsearch error](https://discuss.elastic.co/t/could-not-index-event-to-elasticsearch-error/267379)
**Author:** [@atharvak](https://discuss.elastic.co/u/atharvak)\ **Replies:** 2\ **Last updated:** [March 17, 2021, 5:33am UTC](https://discuss.elastic.co/t/could-not-index-event-to-elasticsearch-error/267379 "2021-03-17T05:33:48Z")
Hello guys, I am getting following error about my filter and I have no clue why it is happening. I have included details. Filter filter { if "cost\_management" in \[tags\] { fingerprint { source =\> "message" targe… --- ## [I want to process bellow log format with the help of gork filter](https://discuss.elastic.co/t/i-want-to-process-bellow-log-format-with-the-help-of-gork-filter/267396)
**Author:** [@onkarborade](https://discuss.elastic.co/u/onkarborade)\ **Replies:** 0\ **Last updated:** [March 16, 2021, 4:01pm UTC](https://discuss.elastic.co/t/i-want-to-process-bellow-log-format-with-the-help-of-gork-filter/267396 "2021-03-16T16:01:55Z")
I have below log format want to gork it, any help will be appreciated. \[0916/131436.858:ERROR:process\_reader\_win.cc(127)\] NtOpenThread: {Access Denied} A process has requested access to an object, but has not been grant… --- ## [Logstash timeout execution grok prevent data to be indexing](https://discuss.elastic.co/t/logstash-timeout-execution-grok-prevent-data-to-be-indexing/267440)
**Author:** [@lusynda](https://discuss.elastic.co/u/lusynda)\ **Replies:** 0\ **Last updated:** [March 17, 2021, 3:37am UTC](https://discuss.elastic.co/t/logstash-timeout-execution-grok-prevent-data-to-be-indexing/267440 "2021-03-17T03:37:38Z")
Hi all I have an weird situation on logstash. A while a go when i parse the log, some of the log that i got sometime have grok time out and that log got indexed fine, but a few days ago, the log stop indexing at all, n… --- ## [Parsing repeated tags in XML](https://discuss.elastic.co/t/parsing-repeated-tags-in-xml/267405)
**Author:** [@adityaPsl](https://discuss.elastic.co/u/adityaPsl)\ **Replies:** 1\ **Last updated:** [March 16, 2021, 6:13pm UTC](https://discuss.elastic.co/t/parsing-repeated-tags-in-xml/267405 "2021-03-16T18:13:58Z")
Hello Team, I have XML data where there are repeated tags, as per the documentation it is converting it to an array, but I want to implement it like break on first match or convert array to single value. Could you plea… --- ## [Logstash configuration for multiple json (nested) files in a directory](https://discuss.elastic.co/t/logstash-configuration-for-multiple-json-nested-files-in-a-directory/267333)
**Author:** [@Razeen](https://discuss.elastic.co/u/Razeen)\ **Replies:** 3\ **Last updated:** [March 16, 2021, 4:54pm UTC](https://discuss.elastic.co/t/logstash-configuration-for-multiple-json-nested-files-in-a-directory/267333 "2021-03-16T16:54:11Z")
Hi, My logstash configuration is as follows, input { file { start\_position =\> "beginning" path =\> "/path/to/json/files\*.json" sincedb\_path =\> "/dev/null" codec =\> json } … --- ## [Retry Logstash connection to a DB after failed attempt](https://discuss.elastic.co/t/retry-logstash-connection-to-a-db-after-failed-attempt/266441)
**Author:** [@d.malacrida](https://discuss.elastic.co/u/d.malacrida)\ **Replies:** 4\ **Last updated:** [March 16, 2021, 4:30pm UTC](https://discuss.elastic.co/t/retry-logstash-connection-to-a-db-after-failed-attempt/266441 "2021-03-16T16:30:09Z")
Hello everyone, We have set up a Logstash connector to gather data from a DB every day at a fixed time (7 a.m.). Now, we have included the following parameters in the input.conf file (some parameters are masked for obvi… --- ## [LOGSTASH : output pipe](https://discuss.elastic.co/t/logstash-output-pipe/267390)
**Author:** [@Alex\_Lum](https://discuss.elastic.co/u/Alex_Lum)\ **Replies:** 2\ **Last updated:** [March 16, 2021, 4:24pm UTC](https://discuss.elastic.co/t/logstash-output-pipe/267390 "2021-03-16T16:24:10Z")
Hello there, I have a winlogbeat which send datas on logstash. On logstash : input beat, some filters and output to elasticSearch. But i want to output SOME fields on a named pipe (mkfifo) to insert them on a MariaDB. … --- ## [Delete or update document without document\_id](https://discuss.elastic.co/t/delete-or-update-document-without-document-id/267375)
**Author:** [@lele15](https://discuss.elastic.co/u/lele15)\ **Replies:** 5\ **Last updated:** [March 16, 2021, 4:18pm UTC](https://discuss.elastic.co/t/delete-or-update-document-without-document-id/267375 "2021-03-16T16:18:25Z")
Hi, i'm new on ES, i'am using logstash and csv to update or delete values into ES. I want to delete or update document based on different field not only with document\_id like this: input { stdin{} } filter { csv … --- ## [How to create periodic check in logstash](https://discuss.elastic.co/t/how-to-create-periodic-check-in-logstash/267342)
**Author:** [@anupts](https://discuss.elastic.co/u/anupts)\ **Replies:** 1\ **Last updated:** [March 16, 2021, 3:59pm UTC](https://discuss.elastic.co/t/how-to-create-periodic-check-in-logstash/267342 "2021-03-16T15:59:45Z")
Hello, We have requirement , where we need to monitor for user activity on periodic basis. We will have this information in our log file . FOr example: User has accessed some page in the application. This will be log… --- ## [Split/explode strings](https://discuss.elastic.co/t/split-explode-strings/267369)
**Author:** [@cibernicola](https://discuss.elastic.co/u/cibernicola)\ **Replies:** 2\ **Last updated:** [March 16, 2021, 2:56pm UTC](https://discuss.elastic.co/t/split-explode-strings/267369 "2021-03-16T14:56:03Z")
Hi, I have several text strings with separator patterns that I would like to be able to ingest as objects in elasticsearch, these strings are of the style: (a) text|text b) number text|number text c) mailto:text@text … --- ## [Create keystore and keys with ansible](https://discuss.elastic.co/t/create-keystore-and-keys-with-ansible/267381)
**Author:** [@Soren\_vdc](https://discuss.elastic.co/u/Soren_vdc)\ **Replies:** 0\ **Last updated:** [March 16, 2021, 1:54pm UTC](https://discuss.elastic.co/t/create-keystore-and-keys-with-ansible/267381 "2021-03-16T13:54:58Z")
Hi, I want to create ansible code to secure the password of the logstash user to send data to Elasticsearch. I have checked with the procedure with the keystore and it works fine to create the keystore but to add the ke… --- ## [Logstash](https://discuss.elastic.co/t/logstash/267348)
**Author:** [@alipujaistopo](https://discuss.elastic.co/u/alipujaistopo)\ **Replies:** 0\ **Last updated:** [March 16, 2021, 10:35am UTC](https://discuss.elastic.co/t/logstash/267348 "2021-03-16T10:35:52Z")
why my logstash don't send log to elastic? I have made a logstash's ip and elasticsearch's ip at /etc/hosts in logstash's server and i made a "output-elasticsearch.conf" at /etc/logstash/conf.d in logstash's server w… --- ## [Edit field "action"](https://discuss.elastic.co/t/edit-field-action/267291)
**Author:** [@lele15](https://discuss.elastic.co/u/lele15)\ **Replies:** 2\ **Last updated:** [March 16, 2021, 8:33am UTC](https://discuss.elastic.co/t/edit-field-action/267291 "2021-03-16T08:33:08Z")
Hi, i'm new on ES and i have a problem with csv ingestion. I want to edit data on ES when a field has a specific value. Like: Id,Name,Operation 1,Jack,Edit 2,,Delete I want something like this in my Logstash config… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=245) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=247)