# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=903 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 904 --- ## [Issues parsing xml data](https://discuss.elastic.co/t/issues-parsing-xml-data/101760)
**Author:** [@jspeer](https://discuss.elastic.co/u/jspeer)\ **Replies:** 0\ **Last updated:** [September 25, 2017, 9:18pm UTC](https://discuss.elastic.co/t/issues-parsing-xml-data/101760 "2017-09-25T21:18:39Z")
I've setup a filter to try to split an XML and assign the fields to fields I can use but the split isn't working. I'm using http\_poller to pull the following xml file over \\ **Author:** [@subhasdan](https://discuss.elastic.co/u/subhasdan)\ **Replies:** 11\ **Last updated:** [September 25, 2017, 10:18pm UTC](https://discuss.elastic.co/t/logstash-5-4-repeated-deserialization-errors-and-losing-data-with-persistent-queues/91503 "2017-09-25T22:18:06Z") Hi, We are getting repeated deserialization errors with Persistent Queues. Environment: - Logstash 5.4.1 (unmodified CentOS image from elastic repos) - 200GB SSD mounted disk for the queue - Persistent queues enabled … --- ## [How to parse multiline json containing multiple arrays](https://discuss.elastic.co/t/how-to-parse-multiline-json-containing-multiple-arrays/101764)
**Author:** [@jag959](https://discuss.elastic.co/u/jag959)\ **Replies:** 0\ **Last updated:** [September 25, 2017, 10:13pm UTC](https://discuss.elastic.co/t/how-to-parse-multiline-json-containing-multiple-arrays/101764 "2017-09-25T22:13:15Z")
Very new to ELK. Need assistance figuring out out to import the following json input which contains several arrays within it. { "Status": "0", "Message": "OK", "Count": "507", "BookingRecords": \[ { "BookDate": "1… --- ## [PLAIN Codec vs. #fix\_streaming\_codecs - Breaking Change](https://discuss.elastic.co/t/plain-codec-vs-fix-streaming-codecs-breaking-change/101758)
**Author:** [@muellerg82](https://discuss.elastic.co/u/muellerg82)\ **Replies:** 0\ **Last updated:** [September 25, 2017, 8:51pm UTC](https://discuss.elastic.co/t/plain-codec-vs-fix-streaming-codecs-breaking-change/101758 "2017-09-25T20:51:28Z")
After happily deploying ElasticSearch Upgrade to 5.6.1 with only minor issues, got in this morning to find that all TCP input pipelines for logging messages that do not adhere to the requirements by the syslog input had … --- ## [Logstash: Parse Mongo Collection Name To Add As 'Type' in ElasticSearch](https://discuss.elastic.co/t/logstash-parse-mongo-collection-name-to-add-as-type-in-elasticsearch/101218)
**Author:** [@Jeffrey\_Blayney](https://discuss.elastic.co/u/Jeffrey_Blayney)\ **Replies:** 2\ **Last updated:** [September 25, 2017, 7:40pm UTC](https://discuss.elastic.co/t/logstash-parse-mongo-collection-name-to-add-as-type-in-elasticsearch/101218 "2017-09-25T19:40:03Z")
I could use some help. I am using logstash with the logstash-input-mongodb plugin and the logstash-output-elasticsearch plugin. Problems: Sending documents to elasticsearch takes a "type" and an "id" in the URI. Ex: PUT… --- ## [Drop fields/Remove Fields in bulk is not working using mutate filter](https://discuss.elastic.co/t/drop-fields-remove-fields-in-bulk-is-not-working-using-mutate-filter/101710)
**Author:** [@rkhapre](https://discuss.elastic.co/u/rkhapre)\ **Replies:** 2\ **Last updated:** [September 25, 2017, 3:38pm UTC](https://discuss.elastic.co/t/drop-fields-remove-fields-in-bulk-is-not-working-using-mutate-filter/101710 "2017-09-25T15:38:28Z")
Hi All I have almost 50 fields which starts with parsed.abc or \[parsed\]\[abc\] parsed.xyz OR \[parsed\]\[xyz\] parsed.dfg OR \[parsed\]\[dfg \] etc.... I need to drop all those using mutate. I am able to drop them one by on… --- ## [Logstash will not start](https://discuss.elastic.co/t/logstash-will-not-start/101535)
**Author:** [@ajstark123](https://discuss.elastic.co/u/ajstark123)\ **Replies:** 4\ **Last updated:** [September 25, 2017, 2:49pm UTC](https://discuss.elastic.co/t/logstash-will-not-start/101535 "2017-09-25T14:49:53Z")
We install logstash v5.5.2 from the tar file. We are running logstash from our /opt/pki/logstash/bin directory. Logstast is not start and getting the following error below. It appears that logstash is having an issue… --- ## [Anonymize plugin](https://discuss.elastic.co/t/anonymize-plugin/101675)
**Author:** [@lucrug](https://discuss.elastic.co/u/lucrug)\ **Replies:** 4\ **Last updated:** [September 25, 2017, 12:27pm UTC](https://discuss.elastic.co/t/anonymize-plugin/101675 "2017-09-25T12:27:05Z")
For what motive the "anonymize" plugin was deprecated with the "fingerprint"? --- ## [Running my first Logstash \*.conf file and need your help](https://discuss.elastic.co/t/running-my-first-logstash-conf-file-and-need-your-help/101474)
**Author:** [@hackilberry](https://discuss.elastic.co/u/hackilberry)\ **Replies:** 5\ **Last updated:** [September 25, 2017, 10:03am UTC](https://discuss.elastic.co/t/running-my-first-logstash-conf-file-and-need-your-help/101474 "2017-09-25T10:03:03Z")
Hi all, I'm new with it, so be patient to me. I successfuly succeeded runing my first "test" in Logstash (or maybe I didn't .. this is my question). I tried to read an XML file from a directory, and output the result … --- ## [Dateparsefailure reason not known](https://discuss.elastic.co/t/dateparsefailure-reason-not-known/101650)
**Author:** [@raja\_gopal](https://discuss.elastic.co/u/raja_gopal)\ **Replies:** 7\ **Last updated:** [September 25, 2017, 9:57am UTC](https://discuss.elastic.co/t/dateparsefailure-reason-not-known/101650 "2017-09-25T09:57:10Z")
I am running logstash 5 on rhel 6. I am trying to get xml data as input and format it through xml filter plugin. I store the output into a target field and i try to parse a UNIX type date in one of the fields inside the … --- ## [Fail to connect logstash 5.6.0 to Oracle with jdbc input](https://discuss.elastic.co/t/fail-to-connect-logstash-5-6-0-to-oracle-with-jdbc-input/101610)
**Author:** [@Yohann\_Nicolas](https://discuss.elastic.co/u/Yohann_Nicolas)\ **Replies:** 3\ **Last updated:** [September 25, 2017, 9:16am UTC](https://discuss.elastic.co/t/fail-to-connect-logstash-5-6-0-to-oracle-with-jdbc-input/101610 "2017-09-25T09:16:35Z")
Hello, I fail to connect my logstash v5.6.0 to an Oracle 12c database. I tried both ojdbc6.jar and ojdbc7.jar I checked that the jdbc\_driver\_class is Java::oracle.jdbc.driver.OracleDriver (with uppercase "J") I am run… --- ## [Date filter fails to convert String value](https://discuss.elastic.co/t/date-filter-fails-to-convert-string-value/101258)
**Author:** [@glipinski](https://discuss.elastic.co/u/glipinski)\ **Replies:** 2\ **Last updated:** [September 25, 2017, 8:38am UTC](https://discuss.elastic.co/t/date-filter-fails-to-convert-string-value/101258 "2017-09-25T08:38:24Z")
Hi, My logs are in xml format. I am using xml filter to parse them and select specific fields. One of the fields is Time xpath =\> \[" .//Time/text()", "timestamp"\] I want to use this filed as a base timestamp (the fi… --- ## [Logstash Getting Filename and Writing to that file](https://discuss.elastic.co/t/logstash-getting-filename-and-writing-to-that-file/101649)
**Author:** [@ranjith](https://discuss.elastic.co/u/ranjith)\ **Replies:** 0\ **Last updated:** [September 25, 2017, 7:19am UTC](https://discuss.elastic.co/t/logstash-getting-filename-and-writing-to-that-file/101649 "2017-09-25T07:19:55Z")
Hello all, I am new to Logstash can anyone help me for the following scenario. Is it possible to get filename and store it in the variable and processing the different files and storing it in the filename which we alre… --- ## [Logstash 5.6.1 incremental refresh is failing, but clean refresh works fine!](https://discuss.elastic.co/t/logstash-5-6-1-incremental-refresh-is-failing-but-clean-refresh-works-fine/101641)
**Author:** [@vishnyk](https://discuss.elastic.co/u/vishnyk)\ **Replies:** 1\ **Last updated:** [September 25, 2017, 6:39am UTC](https://discuss.elastic.co/t/logstash-5-6-1-incremental-refresh-is-failing-but-clean-refresh-works-fine/101641 "2017-09-25T06:39:29Z")
Hi all; I am facing this logstash5.6.1 issue in Prod environment, but it works fine in QA. 1- Incremetal refresh (oracle drivers) using parameter " :sql\_last\_value" is failing. It loads only one record as per debug lo… --- ## [No log4j2 configuration file found: logging only errors to the console](https://discuss.elastic.co/t/no-log4j2-configuration-file-found-logging-only-errors-to-the-console/101295)
**Author:** [@karan\_singh](https://discuss.elastic.co/u/karan_singh)\ **Replies:** 4\ **Last updated:** [September 25, 2017, 6:16am UTC](https://discuss.elastic.co/t/no-log4j2-configuration-file-found-logging-only-errors-to-the-console/101295 "2017-09-25T06:16:03Z")
Elasticsearch and kibana are running perfectly fine but whenver i try to run my logstash configuration file it throws me the following error: C:\\Users\\Dell -\\Desktop\\ELK\\logstash-5.5.2\\bin\>logstash -f logstash-simple.co… --- ## [Logstash-elastic do not support array based columns?](https://discuss.elastic.co/t/logstash-elastic-do-not-support-array-based-columns/101433)
**Author:** [@rkhapre](https://discuss.elastic.co/u/rkhapre)\ **Replies:** 14\ **Last updated:** [September 25, 2017, 5:58am UTC](https://discuss.elastic.co/t/logstash-elastic-do-not-support-array-based-columns/101433 "2017-09-25T05:58:37Z")
Hi All I have a json file like this { "name":"John", "age":30, "cars": \[ { "name":"Ford", "models":\[ "Fiesta", "Focus", "Mustang" \] }, { "name":"BMW", "models":\[ "320", "X3", "X5" \] }, … --- ## [Logstash error read](https://discuss.elastic.co/t/logstash-error-read/101231)
**Author:** [@Erick\_Garcia\_Perez](https://discuss.elastic.co/u/Erick_Garcia_Perez)\ **Replies:** 5\ **Last updated:** [September 25, 2017, 5:37am UTC](https://discuss.elastic.co/t/logstash-error-read/101231 "2017-09-25T05:37:54Z")
i'm trying read log of apache\_Access and apache\_error, those are remote log, when i insert, logstash read again the file and insert agin the entries. My configuracion file of logstash: input { file{ path =\> "/home/cl… --- ## [Geo\_point in logstash](https://discuss.elastic.co/t/geo-point-in-logstash/101373)
**Author:** [@DILIP\_SHARMA](https://discuss.elastic.co/u/DILIP_SHARMA)\ **Replies:** 4\ **Last updated:** [September 25, 2017, 4:35am UTC](https://discuss.elastic.co/t/geo-point-in-logstash/101373 "2017-09-25T04:35:45Z")
hello , currently to define any field as geo\_point i using following command in kibana dev tool - PUT t5 { "mappings": { "dataa": { "properties": { "G": { "type": "geo\_point" } … --- ## [Grok fields not indexed in elasticsearch](https://discuss.elastic.co/t/grok-fields-not-indexed-in-elasticsearch/101611)
**Author:** [@Midiman](https://discuss.elastic.co/u/Midiman)\ **Replies:** 1\ **Last updated:** [September 24, 2017, 8:59pm UTC](https://discuss.elastic.co/t/grok-fields-not-indexed-in-elasticsearch/101611 "2017-09-24T20:59:56Z")
Hi, I am trying to get grok field values into elasticsearch, but they are just not showing up. Here's the scenario: ### First, an example syslog message: \<30\>Sep 24 19:25:12 win-poc-05t.soca.local MSWinEventLog\\t6\\tMicr… --- ## [Syntax in logstash for plugin](https://discuss.elastic.co/t/syntax-in-logstash-for-plugin/101606)
**Author:** [@Ram\_Sharan\_Mittal](https://discuss.elastic.co/u/Ram_Sharan_Mittal)\ **Replies:** 1\ **Last updated:** [September 24, 2017, 8:47pm UTC](https://discuss.elastic.co/t/syntax-in-logstash-for-plugin/101606 "2017-09-24T20:47:02Z")
I am starting with ruby and logstash. seeing following statements in http output plugin(https://github.com/logstash-plugins/logstash-output-http) at , What are these systax. Is it ruby syntax or logstash or jruby: RET… --- ## [Logstash templates? Are they needed?](https://discuss.elastic.co/t/logstash-templates-are-they-needed/101512)
**Author:** [@lucasjkr](https://discuss.elastic.co/u/lucasjkr)\ **Replies:** 1\ **Last updated:** [September 24, 2017, 8:42pm UTC](https://discuss.elastic.co/t/logstash-templates-are-they-needed/101512 "2017-09-24T20:42:31Z")
Might seem like a dumb question. I've been trying to learn the ELK stack, and keep shifting focus between each component. At work, I'm "playing" with a small cluster in attempt to ingest and analyze BRO data. Outside o… --- ## [Sprintf not working within single conf file](https://discuss.elastic.co/t/sprintf-not-working-within-single-conf-file/101567)
**Author:** [@rkhapre](https://discuss.elastic.co/u/rkhapre)\ **Replies:** 1\ **Last updated:** [September 24, 2017, 8:39pm UTC](https://discuss.elastic.co/t/sprintf-not-working-within-single-conf-file/101567 "2017-09-24T20:39:02Z")
Hi All I have 2 Input and 1 Output, i need to pass the field value from Input 1 (elastic) to Input 2 ( http) and then do the processing in pipeline. Below is my conf file, please let me know what is the best way to pro… --- ## [Select output plugin based on a field](https://discuss.elastic.co/t/select-output-plugin-based-on-a-field/101210)
**Author:** [@Ram\_Sharan\_Mittal](https://discuss.elastic.co/u/Ram_Sharan_Mittal)\ **Replies:** 4\ **Last updated:** [September 24, 2017, 8:35pm UTC](https://discuss.elastic.co/t/select-output-plugin-based-on-a-field/101210 "2017-09-24T20:35:03Z")
In my requirement, event have to send to different output plugin based on a field in input --- ## [Connect mariadb database to elasticsearch using logstash](https://discuss.elastic.co/t/connect-mariadb-database-to-elasticsearch-using-logstash/101320)
**Author:** [@surajiiitm](https://discuss.elastic.co/u/surajiiitm)\ **Replies:** 1\ **Last updated:** [September 24, 2017, 8:31pm UTC](https://discuss.elastic.co/t/connect-mariadb-database-to-elasticsearch-using-logstash/101320 "2017-09-24T20:31:02Z")
I am able to import mysql data using logstash to elasticsearch but when i try connecting mariadb data to elasticsearch, it fails. As there is no jdbc driver for ubuntu mariadb to install and i have tried some other jdb… --- ## [How to navigate with event,get method of Event API](https://discuss.elastic.co/t/how-to-navigate-with-event-get-method-of-event-api/101306)
**Author:** [@rkhapre](https://discuss.elastic.co/u/rkhapre)\ **Replies:** 1\ **Last updated:** [September 24, 2017, 8:24pm UTC](https://discuss.elastic.co/t/how-to-navigate-with-event-get-method-of-event-api/101306 "2017-09-24T20:24:16Z")
Hi I have a json file , for which i am not able to navigate to the events with logstash 5.0. I want to extract the name of cars from below using get and set method of ruby as per the documentation event.get('\[message… --- ## [How can I create 12h based index in logstash](https://discuss.elastic.co/t/how-can-i-create-12h-based-index-in-logstash/101287)
**Author:** [@sanketdarji](https://discuss.elastic.co/u/sanketdarji)\ **Replies:** 2\ **Last updated:** [September 24, 2017, 8:22pm UTC](https://discuss.elastic.co/t/how-can-i-create-12h-based-index-in-logstash/101287 "2017-09-24T20:22:00Z")
I want to create a 12h based logstash index. can you please let me know any way how can I achieve it? --- ## [To retrieve two particular values from logstash and perform mathematical operation](https://discuss.elastic.co/t/to-retrieve-two-particular-values-from-logstash-and-perform-mathematical-operation/101286)
**Author:** [@RASHMI\_S](https://discuss.elastic.co/u/RASHMI_S)\ **Replies:** 1\ **Last updated:** [September 24, 2017, 8:19pm UTC](https://discuss.elastic.co/t/to-retrieve-two-particular-values-from-logstash-and-perform-mathematical-operation/101286 "2017-09-24T20:19:50Z")
{"xn":"CSI\_IUCPBySL","var":"DEFAULT","c":"olpv081040817257150214","x":"olpv081040817257150214","xst":"20170914T150218.840","xet":"20170914T150218.936","in":{"CallID":"olpv081040817257150214","SearchBy":"TN","TNorBAN":"40… --- ## [Grokparse failure on simple grok](https://discuss.elastic.co/t/grokparse-failure-on-simple-grok/101200)
**Author:** [@cchooks2](https://discuss.elastic.co/u/cchooks2)\ **Replies:** 1\ **Last updated:** [September 24, 2017, 7:47pm UTC](https://discuss.elastic.co/t/grokparse-failure-on-simple-grok/101200 "2017-09-24T19:47:49Z")
Hello ... me again... I am trying to apply a grok filter to the following: Time: 9/20/2017 11:11:01 AM MSG: Somedata Severity: Critical somedata I first attempted to parse the time stamp and the Serverity using: TIMES… --- ## [Help parsing nested JSON from VirusTotal](https://discuss.elastic.co/t/help-parsing-nested-json-from-virustotal/98900)
**Author:** [@kolmai](https://discuss.elastic.co/u/kolmai)\ **Replies:** 11\ **Last updated:** [September 24, 2017, 12:52pm UTC](https://discuss.elastic.co/t/help-parsing-nested-json-from-virustotal/98900 "2017-09-24T12:52:20Z")
Hello everyone, I’m trying to get started with Elastic stack and my first attempt is to index a complex JSON into ES using the following config has failed. input { beats { port =\> 5044 tags =\> "beats" } } … --- ## [Logstash 100% utilization processing no logs](https://discuss.elastic.co/t/logstash-100-utilization-processing-no-logs/99625)
**Author:** [@bryanm](https://discuss.elastic.co/u/bryanm)\ **Replies:** 1\ **Last updated:** [September 24, 2017, 10:48am UTC](https://discuss.elastic.co/t/logstash-100-utilization-processing-no-logs/99625 "2017-09-24T10:48:47Z")
I’m on Centos 7 using logstash 5.5.2-1 with java 1.8.0 openjdk. I have removed everything in the conf.d directory so logstash isn’t loading any inputs or processing any logs. I have increased the heap size in jvm.optio… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=902) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=904)