# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=657 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 658 --- ## [Filter in grok - or \_](https://discuss.elastic.co/t/filter-in-grok-or/154474)
**Author:** [@ldelpozo](https://discuss.elastic.co/u/ldelpozo)\ **Replies:** 2\ **Last updated:** [October 30, 2018, 2:10pm UTC](https://discuss.elastic.co/t/filter-in-grok-or/154474 "2018-10-30T14:10:38Z")
Hello!!! Is there a way to filter this in grok: BMDL\_DEV/4.BMDL\_TEST\_HELLOWORLD\_DOCKER" I am trying to break it in pieces but no luck match =\> \["jenkins\_jobname", "(?\.(\\S\*.))(?\<\_\_forwardslash\>.(/+… --- ## [Grok log](https://discuss.elastic.co/t/grok-log/154634)
**Author:** [@KAmerad\_Jubei\_Kibaga](https://discuss.elastic.co/u/KAmerad_Jubei_Kibaga)\ **Replies:** 2\ **Last updated:** [October 30, 2018, 1:53pm UTC](https://discuss.elastic.co/t/grok-log/154634 "2018-10-30T13:53:30Z")
Version: logstash 6.4.2 Operating System: Ubuntu I have log: 2018-10-30 11:45:02 act\_id: 76698461 st:SED\_AGENT\_SIGN\_ACCEPTED request:c01df4fe1cd62d867934d1903456bfc1 My grok filter { if \[type\] == "act-process" { g… --- ## [GrokParse Failure Reason](https://discuss.elastic.co/t/grokparse-failure-reason/154608)
**Author:** [@Miguel\_Leite](https://discuss.elastic.co/u/Miguel_Leite)\ **Replies:** 2\ **Last updated:** [October 30, 2018, 12:17pm UTC](https://discuss.elastic.co/t/grokparse-failure-reason/154608 "2018-10-30T12:17:29Z")
I have the following logstash configuration: xml { source =\> "message" target =\> "parsed\_listen" force\_array =\> false add\_… --- ## [Multiple Nested JSON Parse](https://discuss.elastic.co/t/multiple-nested-json-parse/154559)
**Author:** [@Anand\_Upadhyay](https://discuss.elastic.co/u/Anand_Upadhyay)\ **Replies:** 2\ **Last updated:** [October 30, 2018, 12:01pm UTC](https://discuss.elastic.co/t/multiple-nested-json-parse/154559 "2018-10-30T12:01:31Z")
Hi... i'm having issue with parsing of multiple nested JSON can any1 help.. I just dont want identify key field.. JSON Format { "billingDetails": { "POSBILL": \[ { "BillId": 49, "BillNo": "POS/16Oct18", "BillDate… --- ## [I try to input csv file into elasticsearch. but always debuging](https://discuss.elastic.co/t/i-try-to-input-csv-file-into-elasticsearch-but-always-debuging/154641)
**Author:** [@levi\_munthe](https://discuss.elastic.co/u/levi_munthe)\ **Replies:** 0\ **Last updated:** [October 30, 2018, 11:53am UTC](https://discuss.elastic.co/t/i-try-to-input-csv-file-into-elasticsearch-but-always-debuging/154641 "2018-10-30T11:53:18Z")
i try input csv file using logstash to elaticsearch. i am using centos. input { file { path =\> "/etc/logstash/sql/geotester.csv" start\_position =\> "beginning" } } filter { csv{ separator =\> "," columns =\> \["id"… --- ## [Problem parsing with grok](https://discuss.elastic.co/t/problem-parsing-with-grok/154461)
**Author:** [@Miguel\_Leite](https://discuss.elastic.co/u/Miguel_Leite)\ **Replies:** 2\ **Last updated:** [October 30, 2018, 10:38am UTC](https://discuss.elastic.co/t/problem-parsing-with-grok/154461 "2018-10-30T10:38:49Z")
I'm trying to parse the following string using grok: (ADDRESS=(PROTOCOL=tcp)(HOST=172.18.213.2)(PORT=52737)) I tried using the kv filter but I was having problems regarding the parenthesis... With the following patter… --- ## [How to add xpack logstash to kibina with basic licence](https://discuss.elastic.co/t/how-to-add-xpack-logstash-to-kibina-with-basic-licence/153596)
**Author:** [@ahmedfarouk](https://discuss.elastic.co/u/ahmedfarouk)\ **Replies:** 3\ **Last updated:** [October 30, 2018, 10:36am UTC](https://discuss.elastic.co/t/how-to-add-xpack-logstash-to-kibina-with-basic-licence/153596 "2018-10-30T10:36:03Z")
hello any one can help me regarding to add log-stash monitor to kibana --- ## [Logstash Filter Use](https://discuss.elastic.co/t/logstash-filter-use/154188)
**Author:** [@Miguel\_Leite](https://discuss.elastic.co/u/Miguel_Leite)\ **Replies:** 4\ **Last updated:** [October 30, 2018, 10:10am UTC](https://discuss.elastic.co/t/logstash-filter-use/154188 "2018-10-30T10:10:48Z")
Hello. I'm trying to parse a specific log event in xml. I used the xml filter with the target =\> "parsed\_listen". After running it creates a field "parsed\_listen.txt" with the message from the log... Now I want to pars… --- ## [Ruby to expand nested array field into nested JSON objects](https://discuss.elastic.co/t/ruby-to-expand-nested-array-field-into-nested-json-objects/153753)
**Author:** [@dx123](https://discuss.elastic.co/u/dx123)\ **Replies:** 5\ **Last updated:** [October 30, 2018, 9:33am UTC](https://discuss.elastic.co/t/ruby-to-expand-nested-array-field-into-nested-json-objects/153753 "2018-10-30T09:33:36Z")
This is an example format of a JSON field that I have now and wish to convert: { "target": \[ "Sector1 - Subsector1", "Sector2 - Subsector2" \] } The end result desired JSON Objects Output format that I need: { … --- ## [Inconsistent XML - Conditional fields check](https://discuss.elastic.co/t/inconsistent-xml-conditional-fields-check/154588)
**Author:** [@mrncgirc](https://discuss.elastic.co/u/mrncgirc)\ **Replies:** 1\ **Last updated:** [October 30, 2018, 9:22am UTC](https://discuss.elastic.co/t/inconsistent-xml-conditional-fields-check/154588 "2018-10-30T09:22:21Z")
Hi, I am importing an XML file from an external provider. The XML has many inconsistencies. Often fields vary. This means that I often index fields with incorrect data (I index the mapping expression). I would like to … --- ## [How many elapsed filter can use?](https://discuss.elastic.co/t/how-many-elapsed-filter-can-use/154594)
**Author:** [@shwesinhan](https://discuss.elastic.co/u/shwesinhan)\ **Replies:** 0\ **Last updated:** [October 30, 2018, 9:19am UTC](https://discuss.elastic.co/t/how-many-elapsed-filter-can-use/154594 "2018-10-30T09:19:18Z")
hi community i use several elapsed filters in my logstash config. (estimate 30 elapsed filters) so far, it works properly may i know what is max limit of elapsed filter? appreciate!! --- ## [Logstash doesn't apply my ES Template when I upload data](https://discuss.elastic.co/t/logstash-doesnt-apply-my-es-template-when-i-upload-data/153813)
**Author:** [@romainrom](https://discuss.elastic.co/u/romainrom)\ **Replies:** 15\ **Last updated:** [October 30, 2018, 7:50am UTC](https://discuss.elastic.co/t/logstash-doesnt-apply-my-es-template-when-i-upload-data/153813 "2018-10-30T07:50:05Z")
Hello, I'm trying to use Logstash to load Oracle data to ES applying a specific template. logstash version : 6.4.2 Here are the steps followed: Step 1: retrieve data from Oracle and send them to ES without any templa… --- ## [One or more required cgroup files or directories not found](https://discuss.elastic.co/t/one-or-more-required-cgroup-files-or-directories-not-found/153152)
**Author:** [@Haitianisgood](https://discuss.elastic.co/u/Haitianisgood)\ **Replies:** 7\ **Last updated:** [October 30, 2018, 7:24am UTC](https://discuss.elastic.co/t/one-or-more-required-cgroup-files-or-directories-not-found/153152 "2018-10-30T07:24:22Z")
ENV: CentOS release 6.8 (Final) (2.6.32-642.el6.x86\_64) Logstash 6.4.2 logstash : INPUT: from kafka 6.4.2 OUTPUT: to Elastashsearch 6.4.2 There is some output to ES when I start logstash using /usr/share/logstash/b… --- ## [Can't run logstash on Windows](https://discuss.elastic.co/t/cant-run-logstash-on-windows/154333)
**Author:** [@joubeh](https://discuss.elastic.co/u/joubeh)\ **Replies:** 5\ **Last updated:** [October 30, 2018, 7:21am UTC](https://discuss.elastic.co/t/cant-run-logstash-on-windows/154333 "2018-10-30T07:21:28Z")
Hello, what I'm facing that I've downloaded logstash.zip, I've unzipped it to C drive I've Elastic search and Kibana running on my machine but when I'm executing the following command on cmd C:\\logstash-6.4.2\\bin\>logs… --- ## [Enriching using Elasticsearch plugin , request and response message does not match if not enough delay between messages](https://discuss.elastic.co/t/enriching-using-elasticsearch-plugin-request-and-response-message-does-not-match-if-not-enough-delay-between-messages/154564)
**Author:** [@Petr.Simik](https://discuss.elastic.co/u/Petr.Simik)\ **Replies:** 0\ **Last updated:** [October 30, 2018, 5:42am UTC](https://discuss.elastic.co/t/enriching-using-elasticsearch-plugin-request-and-response-message-does-not-match-if-not-enough-delay-between-messages/154564 "2018-10-30T05:42:32Z")
I need to match response and request message so I decided to enrich response message with data from request. I use elastic plugin Code checks if message is response it finds request with same ID and enrich line with re… --- ## [How to retrieve n number of previous lines in log file using logstash configuration?](https://discuss.elastic.co/t/how-to-retrieve-n-number-of-previous-lines-in-log-file-using-logstash-configuration/153350)
**Author:** [@Vyankatesh\_S\_Repal](https://discuss.elastic.co/u/Vyankatesh_S_Repal)\ **Replies:** 2\ **Last updated:** [October 30, 2018, 5:19am UTC](https://discuss.elastic.co/t/how-to-retrieve-n-number-of-previous-lines-in-log-file-using-logstash-configuration/153350 "2018-10-30T05:19:37Z")
I am creating an application where I need to put previous and after n number of lines in elasticsearch with current log. This is my current logstash configuraion. What changes I need to make so that I can retrieve last … --- ## [Syslog input plugin in Docker vs Baremetal](https://discuss.elastic.co/t/syslog-input-plugin-in-docker-vs-baremetal/154312)
**Author:** [@James\_Summers](https://discuss.elastic.co/u/James_Summers)\ **Replies:** 0\ **Last updated:** [October 28, 2018, 2:24am UTC](https://discuss.elastic.co/t/syslog-input-plugin-in-docker-vs-baremetal/154312 "2018-10-28T02:24:24Z")
For the past few weeks I've been setting up logging for my local network infrastructure. I have managed to get all the pieces set up but would like to move from running the tar version to docker versions to ease upgradi… --- ## [Logstash input for Microsoft Teams Channel](https://discuss.elastic.co/t/logstash-input-for-microsoft-teams-channel/154529)
**Author:** [@Jackson\_Pollock](https://discuss.elastic.co/u/Jackson_Pollock)\ **Replies:** 0\ **Last updated:** [October 29, 2018, 10:54pm UTC](https://discuss.elastic.co/t/logstash-input-for-microsoft-teams-channel/154529 "2018-10-29T22:54:24Z")
I would like to push "Microsoft Teams Channel" data streams into es. Has anyone successfully accomplished this? Thanks, --- ## [Logstash deleting records from Db Table after running logstash.conf file](https://discuss.elastic.co/t/logstash-deleting-records-from-db-table-after-running-logstash-conf-file/154405)
**Author:** [@groverjatin17](https://discuss.elastic.co/u/groverjatin17)\ **Replies:** 2\ **Last updated:** [October 29, 2018, 9:50pm UTC](https://discuss.elastic.co/t/logstash-deleting-records-from-db-table-after-running-logstash-conf-file/154405 "2018-10-29T21:50:51Z")
HI all, I am using ORACLE Db in my case. I have created a sample table with user\_id and names as the only 2 columns in them. I inserted some 2 records with these values(\[(1,"sam"),(2,"smith") and so on \]). I try to e… --- ## [Logstash keeps starting with errors](https://discuss.elastic.co/t/logstash-keeps-starting-with-errors/154524)
**Author:** [@lcui\_dxc](https://discuss.elastic.co/u/lcui_dxc)\ **Replies:** 0\ **Last updated:** [October 29, 2018, 9:40pm UTC](https://discuss.elastic.co/t/logstash-keeps-starting-with-errors/154524 "2018-10-29T21:40:55Z")
Hello there, (Sorry I mistakenly put this one under beats... now I moved it here under logstash). All the sudden, our developers reported that logstash started working abnormally. I took a look on the all nodes of ELK… --- ## [Auto-Generate Reports with Basic License](https://discuss.elastic.co/t/auto-generate-reports-with-basic-license/154054)
**Author:** [@wwalker](https://discuss.elastic.co/u/wwalker)\ **Replies:** 4\ **Last updated:** [October 29, 2018, 8:01pm UTC](https://discuss.elastic.co/t/auto-generate-reports-with-basic-license/154054 "2018-10-29T20:01:42Z")
I have a number of reports I'd like to auto-generate and have emailed out. Unless there is a function I am not aware of, I was thinking I could use PowerShell to make the API call and email generation. Unfortunately, I… --- ## [Mysqld grok](https://discuss.elastic.co/t/mysqld-grok/154497)
**Author:** [@szgaljic](https://discuss.elastic.co/u/szgaljic)\ **Replies:** 0\ **Last updated:** [October 29, 2018, 7:28pm UTC](https://discuss.elastic.co/t/mysqld-grok/154497 "2018-10-29T19:28:50Z")
Example mysqld lines: 2015-06-02 16:27:31 10013 \[Note\] Blah 2017-09-08T18:39:29.306811Z 281026 \[Note\] - Blah My grok pattern: MYSQLD\_LINE %{TIMESTAMP\_ISO8601:\[mysql\]\[error\]\[timestamp\]}%{SPACE}%{NUMBER:\[mysql\]\[error\]\[… --- ## [XML import](https://discuss.elastic.co/t/xml-import/154273)
**Author:** [@mrncgirc](https://discuss.elastic.co/u/mrncgirc)\ **Replies:** 9\ **Last updated:** [October 29, 2018, 6:04pm UTC](https://discuss.elastic.co/t/xml-import/154273 "2018-10-29T18:04:18Z")
XML Import with Logstash Hi there I have read about all XML filter import examples I can find. However, I just cannot figure out how to get this specific xml format imported through logstash. Use case The government of… --- ## [After ES upgrade to 5.6 logstash only gets 400 bad request \[SOLVED\]](https://discuss.elastic.co/t/after-es-upgrade-to-5-6-logstash-only-gets-400-bad-request-solved/152843)
**Author:** [@deltadelta](https://discuss.elastic.co/u/deltadelta)\ **Replies:** 1\ **Last updated:** [October 29, 2018, 5:11pm UTC](https://discuss.elastic.co/t/after-es-upgrade-to-5-6-logstash-only-gets-400-bad-request-solved/152843 "2018-10-29T17:11:44Z")
Hi, Not sure if this should be in LS or ES, but thought I'd try here first. Up until recently, I was running ES 2.4 and LS 5.6 and all was well. Recently updated ES to 5.6 and now LS only ever gets a 400 bad request res… --- ## [Multiline / Aggregate Filter Plugin - Mixed & Muddled Log File](https://discuss.elastic.co/t/multiline-aggregate-filter-plugin-mixed-muddled-log-file/153822)
**Author:** [@LloydA](https://discuss.elastic.co/u/LloydA)\ **Replies:** 1\ **Last updated:** [October 29, 2018, 4:47pm UTC](https://discuss.elastic.co/t/multiline-aggregate-filter-plugin-mixed-muddled-log-file/153822 "2018-10-29T16:47:12Z")
Hi All, TL;DR Can the Multiline filter plugin handle a log file that has multi-line events occurring concurrently and interweavingly ? Explanation I have a complex query about multi-line logs that are interwoven with… --- ## [Logstash Split Filter Error](https://discuss.elastic.co/t/logstash-split-filter-error/154452)
**Author:** [@michael1412](https://discuss.elastic.co/u/michael1412)\ **Replies:** 1\ **Last updated:** [October 29, 2018, 2:42pm UTC](https://discuss.elastic.co/t/logstash-split-filter-error/154452 "2018-10-29T14:42:36Z")
System: CentOS 7 Logstash Version: 5.5 Config: if \[stats\] { # check type of stats if is array ruby { code =\> 'case event.get("\[stats\]") when Ha… --- ## [Checking length of a message](https://discuss.elastic.co/t/checking-length-of-a-message/154044)
**Author:** [@Parvatayya\_Malimath](https://discuss.elastic.co/u/Parvatayya_Malimath)\ **Replies:** 4\ **Last updated:** [October 29, 2018, 2:40pm UTC](https://discuss.elastic.co/t/checking-length-of-a-message/154044 "2018-10-29T14:40:23Z")
Hi I am creating logstash filter, for a message like below 10.134.246.236 - username \[24/Oct/2018:15:51:39 +0200\] "POST /xyz/xyz/xyz/ i need to create a filter, which will check the length of username and create a ta… --- ## [How to process old logs?](https://discuss.elastic.co/t/how-to-process-old-logs/154454)
**Author:** [@tomerl](https://discuss.elastic.co/u/tomerl)\ **Replies:** 0\ **Last updated:** [October 29, 2018, 2:14pm UTC](https://discuss.elastic.co/t/how-to-process-old-logs/154454 "2018-10-29T14:14:38Z")
Hi, Just starting out with ELK. I need to take old syslog files and process them in ELK so I will be able to see the messages in the date of the events, i.e, the timestamp that is shown in the log will be the actual ti… --- ## [Configuration error in Logstash under Kubernetes](https://discuss.elastic.co/t/configuration-error-in-logstash-under-kubernetes/154348)
**Author:** [@tomerl](https://discuss.elastic.co/u/tomerl)\ **Replies:** 1\ **Last updated:** [October 29, 2018, 1:45pm UTC](https://discuss.elastic.co/t/configuration-error-in-logstash-under-kubernetes/154348 "2018-10-29T13:45:25Z")
Hi, I'm trying to deploy logstash in my Kubernetes environment using the stable helm chart: https://github.com/helm/charts/blob/master/stable/logstash Using the default values, everything works fine, but when I've trie… --- ## [JSON parse failure. Falling back to plain-text](https://discuss.elastic.co/t/json-parse-failure-falling-back-to-plain-text/154445)
**Author:** [@Nilesh\_Shashtri](https://discuss.elastic.co/u/Nilesh_Shashtri)\ **Replies:** 0\ **Last updated:** [October 29, 2018, 1:21pm UTC](https://discuss.elastic.co/t/json-parse-failure-falling-back-to-plain-text/154445 "2018-10-29T13:21:58Z")
I have show error in my Logstash log:- rating root-level values at \[Source: \[B@25cc18c4; line: 1, column: 4\]\>, :data=\>"29/Oct/2018:07:22:33 +0000 103.216.238.37 IN 0.169 172.69.135.147 - GET /account/subscription HTTP/… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=656) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=658)