# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=230 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 231 --- ## [How to split all xml fields by default](https://discuss.elastic.co/t/how-to-split-all-xml-fields-by-default/272601)
**Author:** [@mangeshmj1992](https://discuss.elastic.co/u/mangeshmj1992)\ **Replies:** 1\ **Last updated:** [May 14, 2021, 3:05pm UTC](https://discuss.elastic.co/t/how-to-split-all-xml-fields-by-default/272601 "2021-05-14T15:05:23Z")
Hello All, is there any way to split all xml fields by default without using split function.? --- ## [Logstash started failing after making elasticsearch secure](https://discuss.elastic.co/t/logstash-started-failing-after-making-elasticsearch-secure/273020)
**Author:** [@rohitarorait82](https://discuss.elastic.co/u/rohitarorait82)\ **Replies:** 0\ **Last updated:** [May 14, 2021, 1:18pm UTC](https://discuss.elastic.co/t/logstash-started-failing-after-making-elasticsearch-secure/273020 "2021-05-14T13:18:25Z")
Hi All , My elk cluster of one node was working fine. I tried to make it secure then elasticsearch and kibana are working fine , however logstash is not able to connect with elasticsearch. I have done below changes in … --- ## [Logstash import date](https://discuss.elastic.co/t/logstash-import-date/272992)
**Author:** [@nikssssss](https://discuss.elastic.co/u/nikssssss)\ **Replies:** 0\ **Last updated:** [May 14, 2021, 8:30am UTC](https://discuss.elastic.co/t/logstash-import-date/272992 "2021-05-14T08:30:02Z")
hello, i want to pass a field ("datetime") as date with this format 1/19/21 12:00 so i add in the conf file the following : date { match =\> \["datetime", "MM/dd/yy HH:mm"\] target =\> "Dateread" … --- ## [How to get variable names in nested JSON format](https://discuss.elastic.co/t/how-to-get-variable-names-in-nested-json-format/272768)
**Author:** [@its-ogawa](https://discuss.elastic.co/u/its-ogawa)\ **Replies:** 4\ **Last updated:** [May 14, 2021, 1:31am UTC](https://discuss.elastic.co/t/how-to-get-variable-names-in-nested-json-format/272768 "2021-05-14T01:31:48Z")
I am using a JSON plugin to parse logs in JSON format. Eventually, the nested variable names are retrieved along with their parent names. How can I simply get just the variable names? target log: ... snip ... {... sn… --- ## [Logstash Elasticsearch Filter Lookup No Results](https://discuss.elastic.co/t/logstash-elasticsearch-filter-lookup-no-results/272933)
**Author:** [@MakoWish](https://discuss.elastic.co/u/MakoWish)\ **Replies:** 1\ **Last updated:** [May 13, 2021, 6:02pm UTC](https://discuss.elastic.co/t/logstash-elasticsearch-filter-lookup-no-results/272933 "2021-05-13T18:02:01Z")
I am having an issue with the Logstash Elasticsearch filter plugin not returning results, when I know there should be. I am sure it is an escape character, or something along those lines, but I cannot for the life of me … --- ## ["Dead ES instance" errors from Logstash](https://discuss.elastic.co/t/dead-es-instance-errors-from-logstash/272927)
**Author:** [@josephmiano](https://discuss.elastic.co/u/josephmiano)\ **Replies:** 0\ **Last updated:** [May 13, 2021, 3:19pm UTC](https://discuss.elastic.co/t/dead-es-instance-errors-from-logstash/272927 "2021-05-13T15:19:55Z")
We have a very busy ES instance with 200+ servers streaming via logstash. At peak volume times, we are seeing these "dead ES instance" errors when logstash attempts to connect to the instance: \[2021-05-12T14:49:46,293… --- ## [Not able to import data in elastic through logstash](https://discuss.elastic.co/t/not-able-to-import-data-in-elastic-through-logstash/271912)
**Author:** [@ajay8310](https://discuss.elastic.co/u/ajay8310)\ **Replies:** 7\ **Last updated:** [May 13, 2021, 3:19pm UTC](https://discuss.elastic.co/t/not-able-to-import-data-in-elastic-through-logstash/271912 "2021-05-13T15:19:16Z")
I am using logstash to import data in elastic search. I am using .conf file to import data in index in elastic. Every time I run the .conf file I get the following message repeatedly and index DOES NOT get created. \< \[… --- ## [Detecting empty fields not working as before in logstash 7.12.1](https://discuss.elastic.co/t/detecting-empty-fields-not-working-as-before-in-logstash-7-12-1/271764)
**Author:** [@mario\_wood](https://discuss.elastic.co/u/mario_wood)\ **Replies:** 3\ **Last updated:** [May 13, 2021, 3:18pm UTC](https://discuss.elastic.co/t/detecting-empty-fields-not-working-as-before-in-logstash-7-12-1/271764 "2021-05-13T15:18:34Z")
Hi, I'm migrating our logstash from 6.8 to 7.12 and I'm finding that the detection of empty fields is not working as it used to. For example given the input { "foo":"", "bar":"baz" } and the logstash filter if \[… --- ## [How to deal with brackets json input filter](https://discuss.elastic.co/t/how-to-deal-with-brackets-json-input-filter/272848)
**Author:** [@Magnuss](https://discuss.elastic.co/u/Magnuss)\ **Replies:** 8\ **Last updated:** [May 13, 2021, 11:02am UTC](https://discuss.elastic.co/t/how-to-deal-with-brackets-json-input-filter/272848 "2021-05-13T11:02:15Z")
Hi there, I'm trying to figure out how to deal with brackets in logstash. The code is as following: { "test1": "text", "images": \[ "https://test2.com/images/I/test.jpg", "https://test2.com/images/I/test.jpg", … --- ## [Reindex data on old ES version using Logstash](https://discuss.elastic.co/t/reindex-data-on-old-es-version-using-logstash/272496)
**Author:** [@rschirin](https://discuss.elastic.co/u/rschirin)\ **Replies:** 5\ **Last updated:** [May 13, 2021, 9:48am UTC](https://discuss.elastic.co/t/reindex-data-on-old-es-version-using-logstash/272496 "2021-05-13T09:48:01Z")
Hey there, I am trying to reindex data from an old ES version to another old ES version using Logstash with elasticsearch input. Anyway I have some doubts. If I schedule the action, during the second execution it will … --- ## [Kafka to kibana](https://discuss.elastic.co/t/kafka-to-kibana/272883)
**Author:** [@SRIGURU\_VEL](https://discuss.elastic.co/u/SRIGURU_VEL)\ **Replies:** 0\ **Last updated:** [May 13, 2021, 6:34am UTC](https://discuss.elastic.co/t/kafka-to-kibana/272883 "2021-05-13T06:34:44Z")
hi, how to push to topic from kafka into logstash with two different index? can anyone suggest me that conf file? --- ## [Remove single quotes from a string](https://discuss.elastic.co/t/remove-single-quotes-from-a-string/272863)
**Author:** [@daniel\_a](https://discuss.elastic.co/u/daniel_a)\ **Replies:** 5\ **Last updated:** [May 13, 2021, 1:48am UTC](https://discuss.elastic.co/t/remove-single-quotes-from-a-string/272863 "2021-05-13T01:48:37Z")
How do I remove single quotes from the message field using Logstash gsub filter? I've tried the syntax below, and it didn't work. mutate { gsub =\> \[ "\[message\]", "'", "" \] } mutate { gsub =\> \[ "\[message\]… --- ## [Logstash: Ingesting CSV, filtered by CSV columns](https://discuss.elastic.co/t/logstash-ingesting-csv-filtered-by-csv-columns/272831)
**Author:** [@hsalim](https://discuss.elastic.co/u/hsalim)\ **Replies:** 4\ **Last updated:** [May 12, 2021, 7:30pm UTC](https://discuss.elastic.co/t/logstash-ingesting-csv-filtered-by-csv-columns/272831 "2021-05-12T19:30:20Z")
Is it possible to ingest data via logstash by applying a filter on data in the CSV file being ingested? For example: input { file { path =\> "blah/blah/\*.gz" max\_open\_files =\> 16000 mode =\> "read" start\_position =\> "b… --- ## [Ingest several csv and create geo\_point](https://discuss.elastic.co/t/ingest-several-csv-and-create-geo-point/272816)
**Author:** [@RichaMax](https://discuss.elastic.co/u/RichaMax)\ **Replies:** 0\ **Last updated:** [May 12, 2021, 1:47pm UTC](https://discuss.elastic.co/t/ingest-several-csv-and-create-geo-point/272816 "2021-05-12T13:47:59Z")
Hello everyone, I'm new to ELK and I have some questions regarding the ingestion of csv files and I couldn't find what I was looking for on the forum. I want to ingest several csv into elasticsearch in different index … --- ## [Stalled Logstash Pipeline and Centralized Pipeline Management](https://discuss.elastic.co/t/stalled-logstash-pipeline-and-centralized-pipeline-management/272815)
**Author:** [@e.sharshenaliev](https://discuss.elastic.co/u/e.sharshenaliev)\ **Replies:** 0\ **Last updated:** [May 12, 2021, 1:30pm UTC](https://discuss.elastic.co/t/stalled-logstash-pipeline-and-centralized-pipeline-management/272815 "2021-05-12T13:30:03Z")
We are using one instance of Logstash (running on Debian VM via "systemctl start logstash") with x-pack Centralized Pipeline Management (CPM). We've encountered the following problem: if for whatever reason pipelines is… --- ## [Logstash multiline pattern not working](https://discuss.elastic.co/t/logstash-multiline-pattern-not-working/272635)
**Author:** [@Saravana37](https://discuss.elastic.co/u/Saravana37)\ **Replies:** 2\ **Last updated:** [May 12, 2021, 1:04pm UTC](https://discuss.elastic.co/t/logstash-multiline-pattern-not-working/272635 "2021-05-12T13:04:18Z")
Hello All , My logstash.conf file .. input { exec { command =\> "E:\\ELK\\logstash\\scripts\\srvrmgr.bat" interval =\> 300 codec =\> multiline { # Grok pattern names are valid! :slight\_smile: pattern =\> "^(D122|T… --- ## [Logstash aggregate problem](https://discuss.elastic.co/t/logstash-aggregate-problem/271760)
**Author:** [@fabryx87](https://discuss.elastic.co/u/fabryx87)\ **Replies:** 5\ **Last updated:** [May 12, 2021, 9:35am UTC](https://discuss.elastic.co/t/logstash-aggregate-problem/271760 "2021-05-12T09:35:27Z")
Hi all, I am trying to do an aggregate in logstash, but probably i am not understanding how it works.... I want to copy the field contenent of elevated\_token inside the map, and create a new field with this value on t… --- ## [How to send logs to logstash using log4js?](https://discuss.elastic.co/t/how-to-send-logs-to-logstash-using-log4js/272783)
**Author:** [@K\_N](https://discuss.elastic.co/u/K_N)\ **Replies:** 0\ **Last updated:** [May 12, 2021, 8:32am UTC](https://discuss.elastic.co/t/how-to-send-logs-to-logstash-using-log4js/272783 "2021-05-12T08:32:37Z")
I want to send logs from my local node.js application and I used this https://github.com/log4js-node/logstashHTTP. But the examples that the documentation mention send logs directly to elasticsearch (I'm able to see the… --- ## [Access Logsatsh via HTTPS](https://discuss.elastic.co/t/access-logsatsh-via-https/272780)
**Author:** [@Abdelhalim](https://discuss.elastic.co/u/Abdelhalim)\ **Replies:** 0\ **Last updated:** [May 12, 2021, 8:18am UTC](https://discuss.elastic.co/t/access-logsatsh-via-https/272780 "2021-05-12T08:18:27Z")
Hello, I would like to monitor Logstash with Metricbeat, and TLS is activated in all my cluster (beats, kibana, Elasticsearch) that's why why I run metricbeat, I get an error: error making http request: Get "https://10… --- ## [Config File doesn't work](https://discuss.elastic.co/t/config-file-doesnt-work/272712)
**Author:** [@qttv](https://discuss.elastic.co/u/qttv)\ **Replies:** 3\ **Last updated:** [May 12, 2021, 7:32am UTC](https://discuss.elastic.co/t/config-file-doesnt-work/272712 "2021-05-12T07:32:26Z")
Good evening. I'm trying to run Logstash for the first time. I already downloaded and runned with success ElasticSearch and Kibana. I created the config file named "logstash-simple.conf" following the istructions on th… --- ## [How to add a specific key to a field from a JSON format part](https://discuss.elastic.co/t/how-to-add-a-specific-key-to-a-field-from-a-json-format-part/272384)
**Author:** [@its-ogawa](https://discuss.elastic.co/u/its-ogawa)\ **Replies:** 3\ **Last updated:** [May 12, 2021, 3:05am UTC](https://discuss.elastic.co/t/how-to-add-a-specific-key-to-a-field-from-a-json-format-part/272384 "2021-05-12T03:05:04Z")
I was able to get a JSON part from a message in grok. I want to keep a specific key from the JSON part in a field, how can I do that? target log: ... snip ... { ... snip ..., "api\_code":"40216", ... snip ... } ... sni… --- ## [Conditional with regex failing](https://discuss.elastic.co/t/conditional-with-regex-failing/272743)
**Author:** [@ElasticLiver](https://discuss.elastic.co/u/ElasticLiver)\ **Replies:** 5\ **Last updated:** [May 12, 2021, 2:22am UTC](https://discuss.elastic.co/t/conditional-with-regex-failing/272743 "2021-05-12T02:22:22Z")
Hi Im trying to replace a value whenever the the characters ":" are present in the category field, but I get an error, I tried to scape the colon, but I get an error too if \[category\] =~ /:\\\\/ { mutate { replace =\> \[ "… --- ## [Logstash not capturing SNMP traps from remote machine](https://discuss.elastic.co/t/logstash-not-capturing-snmp-traps-from-remote-machine/272754)
**Author:** [@ssunkara](https://discuss.elastic.co/u/ssunkara)\ **Replies:** 0\ **Last updated:** [May 12, 2021, 1:33am UTC](https://discuss.elastic.co/t/logstash-not-capturing-snmp-traps-from-remote-machine/272754 "2021-05-12T01:33:22Z")
Hi, I have logstash configured with snmptrap input plugin on a CentOS7 machine. Below is my .conf file that I have at /etc/logstash/conf.d input { snmptrap { community =\> "public" port =\> 10… --- ## [Convert mapping in the table logstash](https://discuss.elastic.co/t/convert-mapping-in-the-table-logstash/272753)
**Author:** [@Juan\_David\_Jaramillo](https://discuss.elastic.co/u/Juan_David_Jaramillo)\ **Replies:** 7\ **Last updated:** [May 11, 2021, 11:50pm UTC](https://discuss.elastic.co/t/convert-mapping-in-the-table-logstash/272753 "2021-05-11T23:50:12Z")
good, I need help with the mapping in logstash, I need to change certain metrics to integer, but the metrics go together in a table so it does not let me change them. I attach the configuration in logstash. input { ht… --- ## [Unable to find valid certification path to requested target](https://discuss.elastic.co/t/unable-to-find-valid-certification-path-to-requested-target/272692)
**Author:** [@osher.l](https://discuss.elastic.co/u/osher.l)\ **Replies:** 0\ **Last updated:** [May 11, 2021, 12:06pm UTC](https://discuss.elastic.co/t/unable-to-find-valid-certification-path-to-requested-target/272692 "2021-05-11T12:06:38Z")
We have ELK stack on Kubernetes(version 7.0.0), Logstash is failing to interact with elasticsearch with the following warning and errors, all other components like filebeat and kibana are well functioning. \[2021-05-11T1… --- ## [Add message\_length field](https://discuss.elastic.co/t/add-message-length-field/272737)
**Author:** [@lag](https://discuss.elastic.co/u/lag)\ **Replies:** 2\ **Last updated:** [May 11, 2021, 6:39pm UTC](https://discuss.elastic.co/t/add-message-length-field/272737 "2021-05-11T18:39:40Z")
I am trying to add calculated "message\_length" field that would be a representation on character count in "message" field. Info is coming in from winlogbeat/filebeat/syslog if that makes any difference. This is the code… --- ## [Custom message filter with logstash](https://discuss.elastic.co/t/custom-message-filter-with-logstash/272626)
**Author:** [@cuongnp](https://discuss.elastic.co/u/cuongnp)\ **Replies:** 6\ **Last updated:** [May 11, 2021, 5:31pm UTC](https://discuss.elastic.co/t/custom-message-filter-with-logstash/272626 "2021-05-11T17:31:47Z")
Hi all, I have a message log below: \<85\>1 2021-05-11T09:25:02+07:00 172.19.16.241 CP-GW - Log \[Fields@1.3.6.1.4.1.2620 Log delay="1620699902" src="172.19.9.18" dst="172.19.11.13" proto="6" UP\_match\_table="TABLE\_START" … --- ## [Logstash to output syslog in Snare format](https://discuss.elastic.co/t/logstash-to-output-syslog-in-snare-format/271405)
**Author:** [@Ravi342883](https://discuss.elastic.co/u/Ravi342883)\ **Replies:** 4\ **Last updated:** [May 11, 2021, 5:10pm UTC](https://discuss.elastic.co/t/logstash-to-output-syslog-in-snare-format/271405 "2021-05-11T17:10:58Z")
Hi All, For a PoC portion, I would like to know whether logstash can output the syslogs in Snare format. Any reference? If anyone has got insights around this, please shed some light here. Thanks, Ravi --- ## [Standalone cluster appears when monitoring logstash with 27 pipeines ELK 7.12](https://discuss.elastic.co/t/standalone-cluster-appears-when-monitoring-logstash-with-27-pipeines-elk-7-12/272730)
**Author:** [@Philip\_Thomson1](https://discuss.elastic.co/u/Philip_Thomson1)\ **Replies:** 0\ **Last updated:** [May 11, 2021, 4:57pm UTC](https://discuss.elastic.co/t/standalone-cluster-appears-when-monitoring-logstash-with-27-pipeines-elk-7-12/272730 "2021-05-11T16:57:29Z")
I have upgraded to 7.12, and then dutifully setup metricbeats for monitoring ES, Kibana and logstash Its a simple setup, with one instance of logstash, however in the monitoring page I see my cluster and a "standalone c… --- ## [Logstash JSON parse error](https://discuss.elastic.co/t/logstash-json-parse-error/272533)
**Author:** [@Richard\_Phillips\_Roy](https://discuss.elastic.co/u/Richard_Phillips_Roy)\ **Replies:** 3\ **Last updated:** [May 11, 2021, 4:59pm UTC](https://discuss.elastic.co/t/logstash-json-parse-error/272533 "2021-05-11T16:59:49Z")
Hi, how do i fix this parsing error in logstash \[ERROR\] 2021-05-08 15:30:46.678 \[\[main\]\#\