# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=207 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 208 --- ## [Logstash. Mutate's split doesn't split string](https://discuss.elastic.co/t/logstash-mutates-split-doesnt-split-string/280035)
**Author:** [@ThreatInter](https://discuss.elastic.co/u/ThreatInter)\ **Replies:** 1\ **Last updated:** [July 30, 2021, 9:28am UTC](https://discuss.elastic.co/t/logstash-mutates-split-doesnt-split-string/280035 "2021-07-30T09:28:14Z")
Hi, community. I have field "temp" with this value: "2.4.3-371989". I want split it by dots. With this purpose I use mutate filter this way: mutate { split =\> \["temp", "."\] } But after split "temp" field still equals… --- ## [Logstash stopped due to an error in one of the pipeline configurations](https://discuss.elastic.co/t/logstash-stopped-due-to-an-error-in-one-of-the-pipeline-configurations/279574)
**Author:** [@subash](https://discuss.elastic.co/u/subash)\ **Replies:** 5\ **Last updated:** [July 30, 2021, 7:25am UTC](https://discuss.elastic.co/t/logstash-stopped-due-to-an-error-in-one-of-the-pipeline-configurations/279574 "2021-07-30T07:25:20Z")
We have 10 conf files running in pipeline. The problem is, if any one of the pipeline configurations has some errors then the entire Logstash instance stops though the other pipeline configurations has no errors. How do … --- ## [Timestamp changes in logstash input?](https://discuss.elastic.co/t/timestamp-changes-in-logstash-input/280027)
**Author:** [@jclemons7](https://discuss.elastic.co/u/jclemons7)\ **Replies:** 0\ **Last updated:** [July 30, 2021, 4:28am UTC](https://discuss.elastic.co/t/timestamp-changes-in-logstash-input/280027 "2021-07-30T04:28:37Z")
Hello all, I have had this index going for like at least the last 2 years.. and all of a sudden the timezone seems to not be working correctly. The pipeline has some transforms in it, but nothing touches the @timestamp … --- ## [Http Broken Pipe](https://discuss.elastic.co/t/http-broken-pipe/280003)
**Author:** [@tofubeats](https://discuss.elastic.co/u/tofubeats)\ **Replies:** 0\ **Last updated:** [July 29, 2021, 4:12pm UTC](https://discuss.elastic.co/t/http-broken-pipe/280003 "2021-07-29T16:12:36Z")
Hello, I'm new to Elasticstack. I am using HTTP output plugin but am getting the following error: \[\[main\]\>worker2\] http - \[HTTP Output Failure\] Could not fetch URL {:url=\>"https://{{fqdn}}:5059", :method=\>:post, :messa… --- ## [Drop messages that end with specific substring](https://discuss.elastic.co/t/drop-messages-that-end-with-specific-substring/279994)
**Author:** [@bgingras](https://discuss.elastic.co/u/bgingras)\ **Replies:** 2\ **Last updated:** [July 29, 2021, 5:55pm UTC](https://discuss.elastic.co/t/drop-messages-that-end-with-specific-substring/279994 "2021-07-29T17:55:35Z")
Hello everyone, I'm trying to implement a Logstash filter that drops syslogs messages that end with a specific substring, something like this: filter { if \[message\].endsWith("substring") { drop { } } } Ha… --- ## [Add geo point in Elastic](https://discuss.elastic.co/t/add-geo-point-in-elastic/280005)
**Author:** [@klynxe](https://discuss.elastic.co/u/klynxe)\ **Replies:** 0\ **Last updated:** [July 29, 2021, 4:52pm UTC](https://discuss.elastic.co/t/add-geo-point-in-elastic/280005 "2021-07-29T16:52:17Z")
Hi everyone, I use ELK and filebeat. I send a lot of logs with different fields. logstash config: input { beats { port =\> 5044 include\_codec\_tag =\> false } } filter { if \[type\] == "json" { j… --- ## [Logstash file input - output to different indices](https://discuss.elastic.co/t/logstash-file-input-output-to-different-indices/279906)
**Author:** [@zaeemmasood](https://discuss.elastic.co/u/zaeemmasood)\ **Replies:** 4\ **Last updated:** [July 29, 2021, 4:02pm UTC](https://discuss.elastic.co/t/logstash-file-input-output-to-different-indices/279906 "2021-07-29T16:02:46Z")
Hello All, I am running ELK 7.6.2 stack. In my current set up, a single file gets ingested in logstash and creates an index successfully as below (only relevant part shown): input { file { path =\> "/opt/gtal/ita… --- ## [How to read local0.info and local4.info file content in UNIX server using filebeat configuration](https://discuss.elastic.co/t/how-to-read-local0-info-and-local4-info-file-content-in-unix-server-using-filebeat-configuration/280000)
**Author:** [@muralikrishna](https://discuss.elastic.co/u/muralikrishna)\ **Replies:** 1\ **Last updated:** [July 29, 2021, 3:18pm UTC](https://discuss.elastic.co/t/how-to-read-local0-info-and-local4-info-file-content-in-unix-server-using-filebeat-configuration/280000 "2021-07-29T15:18:07Z")
Hello Everyone, I have some applications writing the logs to local0.info & local4.info file in UNIX servers. I have to read the log content from above files using filebeat.yml and send the same to Logstash. Could you … --- ## [XML Filter: How import data from XML file](https://discuss.elastic.co/t/xml-filter-how-import-data-from-xml-file/279969)
**Author:** [@Fabio1](https://discuss.elastic.co/u/Fabio1)\ **Replies:** 1\ **Last updated:** [July 29, 2021, 2:42pm UTC](https://discuss.elastic.co/t/xml-filter-how-import-data-from-xml-file/279969 "2021-07-29T14:42:27Z")
Hello everyone, I'm trying to upload my xml file to logstash/elasticsearch to improve my research capabilites. I already tried a lot of ways but I just can import all the file with the xml tags (very confusing). I woul… --- ## [Add a field to logs between 2 specific logs](https://discuss.elastic.co/t/add-a-field-to-logs-between-2-specific-logs/279743)
**Author:** [@Loulou](https://discuss.elastic.co/u/Loulou)\ **Replies:** 11\ **Last updated:** [July 29, 2021, 2:10pm UTC](https://discuss.elastic.co/t/add-a-field-to-logs-between-2-specific-logs/279743 "2021-07-29T14:10:12Z")
Hi, What I am trying to do is, when I encounter a specific log (with a message including "blockOpened"), I save somewhere the fact that I encountered it, and then for all the following logs I add to them a field or tag.… --- ## [SQL Server data to Elasticsearch using LogStash](https://discuss.elastic.co/t/sql-server-data-to-elasticsearch-using-logstash/279996)
**Author:** [@shharukh](https://discuss.elastic.co/u/shharukh)\ **Replies:** 0\ **Last updated:** [July 29, 2021, 1:50pm UTC](https://discuss.elastic.co/t/sql-server-data-to-elasticsearch-using-logstash/279996 "2021-07-29T13:50:55Z")
hey ELK fam, Im trying to connect SQL sever with logstash, but unable to connect with SQLsever here are log can anyone please help me out. regards shahrukh \*\*\[2021-07-29T18:21:41,590\]\[ERROR\]\[logstash.inputs.jdbc … --- ## [Logstash cycle in ruby filter](https://discuss.elastic.co/t/logstash-cycle-in-ruby-filter/279974)
**Author:** [@ThreatInter](https://discuss.elastic.co/u/ThreatInter)\ **Replies:** 0\ **Last updated:** [July 29, 2021, 11:34am UTC](https://discuss.elastic.co/t/logstash-cycle-in-ruby-filter/279974 "2021-07-29T11:34:55Z")
Hi, community. I have message in CEF format, that may have cs1...csn and cs1Label...csnLabel fields. I can't use CEF codec, because I want to keep my original message too. So I decided to use "for" loop in ruby filter an… --- ## [Grok match regex](https://discuss.elastic.co/t/grok-match-regex/279957)
**Author:** [@Soren\_vdc](https://discuss.elastic.co/u/Soren_vdc)\ **Replies:** 2\ **Last updated:** [July 29, 2021, 11:11am UTC](https://discuss.elastic.co/t/grok-match-regex/279957 "2021-07-29T11:11:18Z")
Hi, I want to create grok match and have the 2 first words and the last part of the message. For example: ERROR;AuthenticationController ;processMessage ;THR=11107194 R=760a52d97983874 T=6031a0711ec3 U=Unauthenticated… --- ## [How to output a specific fields of JSON data in logstash](https://discuss.elastic.co/t/how-to-output-a-specific-fields-of-json-data-in-logstash/279944)
**Author:** [@Takky](https://discuss.elastic.co/u/Takky)\ **Replies:** 0\ **Last updated:** [July 29, 2021, 7:28am UTC](https://discuss.elastic.co/t/how-to-output-a-specific-fields-of-json-data-in-logstash/279944 "2021-07-29T07:28:45Z")
I want to know how to select and output some fields from JSON data with many fields. For example, there is the following input JSON data. \[input\] { "layers1": { "field01": { "sample01": "5", "… --- ## [Logstash-paloalto loss of data](https://discuss.elastic.co/t/logstash-paloalto-loss-of-data/279504)
**Author:** [@Betorov](https://discuss.elastic.co/u/Betorov)\ **Replies:** 2\ **Last updated:** [July 29, 2021, 7:17am UTC](https://discuss.elastic.co/t/logstash-paloalto-loss-of-data/279504 "2021-07-29T07:17:17Z")
Hi evreyone, I'm not sure if I had to post this here or in a paloalto forum. Problem: My logstash take the logs that are coming from paloalto, this type of log have a precise structure explained here : Threat Log Fie… --- ## [Dissect filter sometimes not working](https://discuss.elastic.co/t/dissect-filter-sometimes-not-working/279927)
**Author:** [@jeffrey008](https://discuss.elastic.co/u/jeffrey008)\ **Replies:** 1\ **Last updated:** [July 29, 2021, 4:43am UTC](https://discuss.elastic.co/t/dissect-filter-sometimes-not-working/279927 "2021-07-29T04:43:32Z")
Logstash cannot find the pattern in my log. It sometimes can dissect successfully but sometimes not. \[2021-07-29T04:18:35,253\]\[WARN \]\[org.logstash.dissect.Dissector\]\[main\]\[0b28d6955374719d4eec09ce396e5505f458a3d25e49b40… --- ## [Logstash - Error parsing csv :exception=\>java.lang.ArrayIndexOutOfBoundsException](https://discuss.elastic.co/t/logstash-error-parsing-csv-exception-java-lang-arrayindexoutofboundsexception/279884)
**Author:** [@deadloss](https://discuss.elastic.co/u/deadloss)\ **Replies:** 2\ **Last updated:** [July 29, 2021, 4:41am UTC](https://discuss.elastic.co/t/logstash-error-parsing-csv-exception-java-lang-arrayindexoutofboundsexception/279884 "2021-07-29T04:41:50Z")
I am trying to sync logs from csv, sometimes I get this error \[2021-07-28T14:49:03,857\]\[WARN \]\[logstash.filters.csv \] Error parsing csv {:field=\>"message", :source=\>"M5804240304", :exception=\>java.lang.ArrayIndexOut… --- ## [Logstash does not create index in Elasticsearch (Windows 10)](https://discuss.elastic.co/t/logstash-does-not-create-index-in-elasticsearch-windows-10/279796)
**Author:** [@Lucas\_Yoshioka](https://discuss.elastic.co/u/Lucas_Yoshioka)\ **Replies:** 2\ **Last updated:** [July 29, 2021, 12:35am UTC](https://discuss.elastic.co/t/logstash-does-not-create-index-in-elasticsearch-windows-10/279796 "2021-07-29T00:35:13Z")
Hi guys, I am trying run ELK, but for some reason Logstash does not create my indexes in Elasticsearch. I installed both and added their /bin path to my environment variables. Here is the conf file I am using: input { … --- ## [Filtering syslog message](https://discuss.elastic.co/t/filtering-syslog-message/279877)
**Author:** [@Dovan](https://discuss.elastic.co/u/Dovan)\ **Replies:** 4\ **Last updated:** [July 28, 2021, 8:18pm UTC](https://discuss.elastic.co/t/filtering-syslog-message/279877 "2021-07-28T20:18:19Z")
hello, I'm new to ELK and I'm trying to use logstash to consume logs from a syslog server I have, how do I extract the messages that arrive inside the message field? here is a log sample: "\<133\>Jul 28 09:59:11 10.128.x… --- ## [Multiple Pipeline not working](https://discuss.elastic.co/t/multiple-pipeline-not-working/279907)
**Author:** [@Dovan](https://discuss.elastic.co/u/Dovan)\ **Replies:** 0\ **Last updated:** [July 28, 2021, 8:04pm UTC](https://discuss.elastic.co/t/multiple-pipeline-not-working/279907 "2021-07-28T20:04:01Z")
I'm trying to configure several pipelines, but only the first pipeline is working, below are my configuration files and the latest logstash log messages pipelines.yml: # This file is where you define your pipelines. Yo… --- ## [Logstash workflow design - pipelines and configuration files](https://discuss.elastic.co/t/logstash-workflow-design-pipelines-and-configuration-files/279865)
**Author:** [@Icaka](https://discuss.elastic.co/u/Icaka)\ **Replies:** 1\ **Last updated:** [July 28, 2021, 7:09pm UTC](https://discuss.elastic.co/t/logstash-workflow-design-pipelines-and-configuration-files/279865 "2021-07-28T19:09:05Z")
Hi all, New to Elastic and Logstash, however my scenario is large IT environment and I would like to collect logs from various sources. First I would like to design most basic ones such as filebeat collecting syslogs, s… --- ## [Aggregation filter Logstash](https://discuss.elastic.co/t/aggregation-filter-logstash/279868)
**Author:** [@Roberto\_B](https://discuss.elastic.co/u/Roberto_B)\ **Replies:** 0\ **Last updated:** [July 28, 2021, 1:44pm UTC](https://discuss.elastic.co/t/aggregation-filter-logstash/279868 "2021-07-28T13:44:37Z")
Hi all, i'm trying to use aggregation filter, i have a big result set (more than 140k rows from mySql). And the aggregation stopped always at row 53. In the log I can see this error: \[ERROR\]\[logstash.javapipeline … --- ## [RuntimeError: Invalid FieldReference: \`\[\]\`](https://discuss.elastic.co/t/runtimeerror-invalid-fieldreference/279856)
**Author:** [@Andrew\_Harris](https://discuss.elastic.co/u/Andrew_Harris)\ **Replies:** 0\ **Last updated:** [July 28, 2021, 12:43pm UTC](https://discuss.elastic.co/t/runtimeerror-invalid-fieldreference/279856 "2021-07-28T12:43:53Z")
Hi, I have two servers running Logstash 7.10 pushing a series of CSV file into Elastic. Now I thought that both server were configured exactly the same, but it looks like they aren't as one is repeatedly giving the foll… --- ## [Logstash lumberjack to beats certificate unknown](https://discuss.elastic.co/t/logstash-lumberjack-to-beats-certificate-unknown/279844)
**Author:** [@muffin](https://discuss.elastic.co/u/muffin)\ **Replies:** 0\ **Last updated:** [July 28, 2021, 10:56am UTC](https://discuss.elastic.co/t/logstash-lumberjack-to-beats-certificate-unknown/279844 "2021-07-28T10:56:14Z")
Hi, We're using Logstash 7.12, we want to use the lumberjack output plugin to relay traffic to the Beats input of another Logstash instance (also 7.12). When using a self-signed certificate (identical on both sides) th… --- ## [How to mention elasticsearch credentials in logstash config?](https://discuss.elastic.co/t/how-to-mention-elasticsearch-credentials-in-logstash-config/279840)
**Author:** [@Shreesh\_Narayanan](https://discuss.elastic.co/u/Shreesh_Narayanan)\ **Replies:** 1\ **Last updated:** [July 28, 2021, 10:54am UTC](https://discuss.elastic.co/t/how-to-mention-elasticsearch-credentials-in-logstash-config/279840 "2021-07-28T10:54:12Z")
Hello, I recently enabled x-pack security on elasticsearch and set up passwords for built-in users successfully . However now i get an error when trying to ingest some logs \[2021-07-23T13:02:58,225\]\[WARN \]\[logstash.o… --- ## [Logstash JMS Input with Weblogic JMS server](https://discuss.elastic.co/t/logstash-jms-input-with-weblogic-jms-server/279838)
**Author:** [@AlessandroKP](https://discuss.elastic.co/u/AlessandroKP)\ **Replies:** 0\ **Last updated:** [July 28, 2021, 10:15am UTC](https://discuss.elastic.co/t/logstash-jms-input-with-weblogic-jms-server/279838 "2021-07-28T10:15:17Z")
Hi, I'm having problems configuring the Logstash JMS input plugin in order to read messages from a queue on Oracle Weblogic JMS server. I'm using Logstash v7.9.3 and the version of the jms input plugin is v3.1.2 (the o… --- ## [Logstash on virtualbox/docker + cisco](https://discuss.elastic.co/t/logstash-on-virtualbox-docker-cisco/279828)
**Author:** [@Margo](https://discuss.elastic.co/u/Margo)\ **Replies:** 0\ **Last updated:** [July 28, 2021, 8:40am UTC](https://discuss.elastic.co/t/logstash-on-virtualbox-docker-cisco/279828 "2021-07-28T08:40:59Z")
Hi! I've started ELK with docker-stack on virtualbox, ports has 514:5044. Pipeline input - udp/5044, output - stdout. Tcpdump catch syslog from cisco - 514/udp. But there is no stdout or index pattern in kibana. Can you… --- ## [Error using https in filter section](https://discuss.elastic.co/t/error-using-https-in-filter-section/279793)
**Author:** [@michaelv](https://discuss.elastic.co/u/michaelv)\ **Replies:** 0\ **Last updated:** [July 28, 2021, 4:47am UTC](https://discuss.elastic.co/t/error-using-https-in-filter-section/279793 "2021-07-28T04:47:05Z")
Hi All, I've gotten my logstash script working without https to port 9200. However, I have turned on encryption but I cannot figure out why the filter section fails but the output section for the https works. … --- ## [Running Logstash on cloud](https://discuss.elastic.co/t/running-logstash-on-cloud/279678)
**Author:** [@Rickenson\_Robert](https://discuss.elastic.co/u/Rickenson_Robert)\ **Replies:** 6\ **Last updated:** [July 27, 2021, 6:56pm UTC](https://discuss.elastic.co/t/running-logstash-on-cloud/279678 "2021-07-27T18:56:53Z")
Hi all, I am new to Logstash and was wondering if there was a way to use the program in a hosted environment? It’s my understanding that the binaries must be downloaded to a machine and executed from it. Is there a way … --- ## [Logstash configure custom ilm policy per elasticsearch datastream output](https://discuss.elastic.co/t/logstash-configure-custom-ilm-policy-per-elasticsearch-datastream-output/279757)
**Author:** [@anupshrestha](https://discuss.elastic.co/u/anupshrestha)\ **Replies:** 0\ **Last updated:** [July 27, 2021, 5:20pm UTC](https://discuss.elastic.co/t/logstash-configure-custom-ilm-policy-per-elasticsearch-datastream-output/279757 "2021-07-27T17:20:11Z")
Is there a way to specify a custom ilm policy when using datastream with elasticsearch output in logstash. The goal here is to have separate data retention period per datastream in elasticsearch. Any help or feedback is… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=206) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=208)