# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=687 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 688 --- ## [Logstash KV filter for multiline message](https://discuss.elastic.co/t/logstash-kv-filter-for-multiline-message/147950)
**Author:** [@eswarloges](https://discuss.elastic.co/u/eswarloges)\ **Replies:** 1\ **Last updated:** [September 11, 2018, 10:52am UTC](https://discuss.elastic.co/t/logstash-kv-filter-for-multiline-message/147950 "2018-09-11T10:52:32Z")
Hi Team, I am creating a KV filter for the following message 1535125123989 error targetResponse m=GET, u=/favicon.ico, h=test.com, r=192.168.1.5:2383, s=404, name=Error, message=no match found for /favicon.ico, code=u… --- ## [Split nested json array](https://discuss.elastic.co/t/split-nested-json-array/147969)
**Author:** [@slavo3](https://discuss.elastic.co/u/slavo3)\ **Replies:** 8\ **Last updated:** [September 11, 2018, 9:27am UTC](https://discuss.elastic.co/t/split-nested-json-array/147969 "2018-09-11T09:27:36Z")
I would like to to retrieve every element in below JSON to be a field so as to visualize in kibana by applying metrics in dashboard. { "message": "{"SegmentId":4,"VectorList":\[{"LocalizationId":60,"ServiceId":1,"Date"… --- ## [Can we use multiline plugin with logstash input journald plugins in logstash?](https://discuss.elastic.co/t/can-we-use-multiline-plugin-with-logstash-input-journald-plugins-in-logstash/148096)
**Author:** [@talbehat](https://discuss.elastic.co/u/talbehat)\ **Replies:** 0\ **Last updated:** [September 11, 2018, 9:06am UTC](https://discuss.elastic.co/t/can-we-use-multiline-plugin-with-logstash-input-journald-plugins-in-logstash/148096 "2018-09-11T09:06:29Z")
input { journald { lowercase =\> true seekto =\> "tail" thisboot =\> true type =\> "journald" codec =\> multiline { pattern =\> "(^%{TIMESTAMP\_ISO8601})" negate =\> "true" what =\> previous } tags =\> \[ "journald" \] … --- ## [Error in logstash-plugin-wmi](https://discuss.elastic.co/t/error-in-logstash-plugin-wmi/148086)
**Author:** [@jarry](https://discuss.elastic.co/u/jarry)\ **Replies:** 0\ **Last updated:** [September 11, 2018, 8:33am UTC](https://discuss.elastic.co/t/error-in-logstash-plugin-wmi/148086 "2018-09-11T08:33:59Z")
Hello. Im trying to run wmi input plugin, but i got only errors "Bad file descriptor" logstash-plugin-wmi install： error message： f:\\nginx\\logstash-6.4.0\\logstash-6.4.0\\bin\>logstash -f win.conf Sending Logstash … --- ## [Could not index event to Elasticsearch. - failed to parse \[host\]](https://discuss.elastic.co/t/could-not-index-event-to-elasticsearch-failed-to-parse-host/147908)
**Author:** [@Victor\_Sava](https://discuss.elastic.co/u/Victor_Sava)\ **Replies:** 4\ **Last updated:** [September 11, 2018, 8:01am UTC](https://discuss.elastic.co/t/could-not-index-event-to-elasticsearch-failed-to-parse-host/147908 "2018-09-11T08:01:32Z")
\[WARN \]\[logstash.outputs.elasticsearch\] Could not index event to Elasticsearch. {:status=\>400, :action=\>\["index", {:\_id=\>nil, :\_index=\>"point2homesiis-2018.09.10", :\_type=\>"iis", :\_routing=\>nil}, #\ **Author:** [@pranay\_sankpal](https://discuss.elastic.co/u/pranay_sankpal)\ **Replies:** 1\ **Last updated:** [September 11, 2018, 7:35am UTC](https://discuss.elastic.co/t/multiple-hosts-of-logstash/148060 "2018-09-11T07:35:22Z") Could you suggest the better approach to have multiple logstash servers ? My Approach : filebeat points to two host entries : host : \["172.0.0.1", "172.0.0.2"\] filebeat points to domain name host : \["https:\\logs… --- ## [Execute pipeline on a specific host only](https://discuss.elastic.co/t/execute-pipeline-on-a-specific-host-only/148074)
**Author:** [@darefilz](https://discuss.elastic.co/u/darefilz)\ **Replies:** 0\ **Last updated:** [September 11, 2018, 7:34am UTC](https://discuss.elastic.co/t/execute-pipeline-on-a-specific-host-only/148074 "2018-09-11T07:34:38Z")
Hello, we run Logstash in a Docker Swarm like mode. Sometimes, there is only one instance running, sometimes eight depending on various parameters. There only one config for all instances. One of our inputs is a schedu… --- ## [Prevent upsert to merge internal objects in logstash configuration](https://discuss.elastic.co/t/prevent-upsert-to-merge-internal-objects-in-logstash-configuration/148061)
**Author:** [@yalpanian](https://discuss.elastic.co/u/yalpanian)\ **Replies:** 0\ **Last updated:** [September 11, 2018, 6:45am UTC](https://discuss.elastic.co/t/prevent-upsert-to-merge-internal-objects-in-logstash-configuration/148061 "2018-09-11T06:45:27Z")
consider this record is stored in elastic search : { "id" : 1 "Name" : "a", "Obj" : { "old" : 123 } } and i want to update that record by this : { "id" : 1 "Family" : "b", "Obj" : { "new": 321 } } finally i want to… --- ## [How to parse a particular log file](https://discuss.elastic.co/t/how-to-parse-a-particular-log-file/147138)
**Author:** [@anon69830709](https://discuss.elastic.co/u/anon69830709)\ **Replies:** 3\ **Last updated:** [September 11, 2018, 6:36am UTC](https://discuss.elastic.co/t/how-to-parse-a-particular-log-file/147138 "2018-09-11T06:36:47Z")
So i have 2 log file say log1 and log2.I want to parse them using separate filter. so can i use if statement on file source like if \[source\] == ///log1 { do something } else { do something } --- ## [How to extract the data in the target to the root directory](https://discuss.elastic.co/t/how-to-extract-the-data-in-the-target-to-the-root-directory/148054)
**Author:** [@sun\_changlong](https://discuss.elastic.co/u/sun_changlong)\ **Replies:** 0\ **Last updated:** [September 11, 2018, 6:17am UTC](https://discuss.elastic.co/t/how-to-extract-the-data-in-the-target-to-the-root-directory/148054 "2018-09-11T06:17:00Z")
After I use the kv plugin to parse the log, I want to filter the value further. I want to further process the value of cmd. Replace \\" with ".how to use ruby filter to loop through k,v pairs? I used the target in the kv… --- ## [Logstash latest version 6.4 does not show stdout on csv filter](https://discuss.elastic.co/t/logstash-latest-version-6-4-does-not-show-stdout-on-csv-filter/148053)
**Author:** [@fillic2002](https://discuss.elastic.co/u/fillic2002)\ **Replies:** 0\ **Last updated:** [September 11, 2018, 6:16am UTC](https://discuss.elastic.co/t/logstash-latest-version-6-4-does-not-show-stdout-on-csv-filter/148053 "2018-09-11T06:16:45Z")
I tried deleting the sincedb.log file as well but still there is no output to stdout. Also there is no error in the trace log. No pointer added to sincedb as well, its empty file. if i copy same pipeline in 6.1.1... it w… --- ## [Need to limit number of characters to be read using grok](https://discuss.elastic.co/t/need-to-limit-number-of-characters-to-be-read-using-grok/147840)
**Author:** [@rgovinda](https://discuss.elastic.co/u/rgovinda)\ **Replies:** 3\ **Last updated:** [September 11, 2018, 6:05am UTC](https://discuss.elastic.co/t/need-to-limit-number-of-characters-to-be-read-using-grok/147840 "2018-09-11T06:05:45Z")
Question: how can I make grok read only the first 18 characters from this string "nameOfItem=Oranges of California and Florida". With the way I have, I am getting the last 18 cut off. Please help. grok pattern used: … --- ## [Associating log lines with timestamp in log line](https://discuss.elastic.co/t/associating-log-lines-with-timestamp-in-log-line/144979)
**Author:** [@kraogv](https://discuss.elastic.co/u/kraogv)\ **Replies:** 6\ **Last updated:** [September 11, 2018, 5:07am UTC](https://discuss.elastic.co/t/associating-log-lines-with-timestamp-in-log-line/144979 "2018-09-11T05:07:19Z")
Hi, I am currently trying to associate lines after I encounter the time field in log lines to that time field. Sample log: 06-26-18 03:58:51 AM top - 03:58:51 up 84 days, 2:05, 0 users, load average: 0.01, 0.08, 0… --- ## [Nested json parsing dynamic types](https://discuss.elastic.co/t/nested-json-parsing-dynamic-types/148042)
**Author:** [@Malli](https://discuss.elastic.co/u/Malli)\ **Replies:** 0\ **Last updated:** [September 11, 2018, 4:33am UTC](https://discuss.elastic.co/t/nested-json-parsing-dynamic-types/148042 "2018-09-11T04:33:51Z")
I am parsing json format log message and that structure is dynamic. Example if i parse the log message some like {"file":"this is sample file"} parsing works fine and can search the log also in kibana. Then tried to p… --- ## [Date Filter Configuration Error Help Required](https://discuss.elastic.co/t/date-filter-configuration-error-help-required/147951)
**Author:** [@JKCode](https://discuss.elastic.co/u/JKCode)\ **Replies:** 2\ **Last updated:** [September 11, 2018, 3:52am UTC](https://discuss.elastic.co/t/date-filter-configuration-error-help-required/147951 "2018-09-11T03:52:22Z")
Hi All, Version ELK Stack 6.4 / Windows 10 / zip downloads Trying to ingest data from csv file into elasticsearch 6.4. Want to parse the csv data columns containing dates in specific format with Logstash date filter p… --- ## [Pipeline IDs as variables?](https://discuss.elastic.co/t/pipeline-ids-as-variables/146252)
**Author:** [@ceekay](https://discuss.elastic.co/u/ceekay)\ **Replies:** 2\ **Last updated:** [September 10, 2018, 11:20pm UTC](https://discuss.elastic.co/t/pipeline-ids-as-variables/146252 "2018-09-10T23:20:15Z")
Is it possible to get the pipeline ID from inside a Logstash config? I have a monitoring script that gets the pipeline IDs from the Logstash API, and checks a file with the same name to see if the heartbeat is current. … --- ## [Recommendation for date in index name](https://discuss.elastic.co/t/recommendation-for-date-in-index-name/148022)
**Author:** [@javadevmtl](https://discuss.elastic.co/u/javadevmtl)\ **Replies:** 0\ **Last updated:** [September 10, 2018, 9:35pm UTC](https://discuss.elastic.co/t/recommendation-for-date-in-index-name/148022 "2018-09-10T21:35:45Z")
Hi, in logstash I'm currently using the following pattern... index =\> "some-index-name-here\_%{+YYYY.MM}" I was thinking that I should base the YYYY.MM by parsing the actual log line time instead of using the calculated… --- ## [Extract a specific part of a kv filter](https://discuss.elastic.co/t/extract-a-specific-part-of-a-kv-filter/148021)
**Author:** [@nilsb](https://discuss.elastic.co/u/nilsb)\ **Replies:** 0\ **Last updated:** [September 10, 2018, 9:22pm UTC](https://discuss.elastic.co/t/extract-a-specific-part-of-a-kv-filter/148021 "2018-09-10T21:22:00Z")
Hi experts, I am trying to use kv filter but getting landed into a new problem. Surely no issue with filter, but my lack of knowledge :slight\_smile: PFB the details: Problem statement: I am trying to parse an applica… --- ## [Parse the same input in two different ways](https://discuss.elastic.co/t/parse-the-same-input-in-two-different-ways/147999)
**Author:** [@DanV](https://discuss.elastic.co/u/DanV)\ **Replies:** 2\ **Last updated:** [September 10, 2018, 9:07pm UTC](https://discuss.elastic.co/t/parse-the-same-input-in-two-different-ways/147999 "2018-09-10T21:07:09Z")
Hello, i have the following situation, i'm receiving logs from UDP port 6514 as the input for logstash, what i wanna do is apply two different filters to the same logs (parallely) and send them off to elasticsearch in … --- ## [Is it possible to have separate log file for each Logstash pipeline?](https://discuss.elastic.co/t/is-it-possible-to-have-separate-log-file-for-each-logstash-pipeline/147867)
**Author:** [@Piotr\_Wasilewski](https://discuss.elastic.co/u/Piotr_Wasilewski)\ **Replies:** 1\ **Last updated:** [September 10, 2018, 8:33pm UTC](https://discuss.elastic.co/t/is-it-possible-to-have-separate-log-file-for-each-logstash-pipeline/147867 "2018-09-10T20:33:30Z")
Hi, Is it possible to have separate log file for each logstash pipeline? We have a number of pipelines on single node and debugging is cumbersome if not impossible at times, because one of pipelines makes so much noise … --- ## [Logstash 6.4.0 initial permission issue](https://discuss.elastic.co/t/logstash-6-4-0-initial-permission-issue/148015)
**Author:** [@derekmizak](https://discuss.elastic.co/u/derekmizak)\ **Replies:** 0\ **Last updated:** [September 10, 2018, 7:56pm UTC](https://discuss.elastic.co/t/logstash-6-4-0-initial-permission-issue/148015 "2018-09-10T19:56:17Z")
On Ubuntu 16.04 LTS after default installation of Logstash - there seams to be some permission issues. for e.g dmadmin@es01:/usr/share/logstash/bin$ ls -l /var/lib/logstash/ total 12 drwxr-xr-x 2 root root 4096 Sep 10… --- ## [Mapping error logs](https://discuss.elastic.co/t/mapping-error-logs/148012)
**Author:** [@KZ\_Win](https://discuss.elastic.co/u/KZ_Win)\ **Replies:** 0\ **Last updated:** [September 10, 2018, 7:48pm UTC](https://discuss.elastic.co/t/mapping-error-logs/148012 "2018-09-10T19:48:38Z")
Some of our logs from logstash to elasticsearch ends up in dead letter queue and I believe logstash used to dump the exception it gets before it ends up in dlq. For example, it will say one of the field is of the wrong… --- ## [Index not created in widnows](https://discuss.elastic.co/t/index-not-created-in-widnows/147990)
**Author:** [@Damodara\_Gottipalli](https://discuss.elastic.co/u/Damodara_Gottipalli)\ **Replies:** 2\ **Last updated:** [September 10, 2018, 7:29pm UTC](https://discuss.elastic.co/t/index-not-created-in-widnows/147990 "2018-09-10T19:29:02Z")
Not sure what's wrong. I am using 6.3.1 This is the config file::: input { file { path =\> \["C:\\study\\elasticsearch\\data.csv"\] start\_position =\> "beginning" sincedb\_path =\> "nul" codec =\> rubydebug } } filter { … --- ## [Elapsed Filter - Save tags/fields from start event](https://discuss.elastic.co/t/elapsed-filter-save-tags-fields-from-start-event/148004)
**Author:** [@AquaX](https://discuss.elastic.co/u/AquaX)\ **Replies:** 0\ **Last updated:** [September 10, 2018, 7:00pm UTC](https://discuss.elastic.co/t/elapsed-filter-save-tags-fields-from-start-event/148004 "2018-09-10T19:00:25Z")
Hey everyone. First time poster, long time reader. I'm curious about the Logstash 'elapsed' filter. I have two different patterns that I am successfully able to grok and generate start and end events for. However, the… --- ## [JSON Array To Object](https://discuss.elastic.co/t/json-array-to-object/147606)
**Author:** [@manwithoutsleep](https://discuss.elastic.co/u/manwithoutsleep)\ **Replies:** 3\ **Last updated:** [September 7, 2018, 5:55pm UTC](https://discuss.elastic.co/t/json-array-to-object/147606 "2018-09-07T17:55:30Z")
Hi - Each record of input to my Logstash instance is a JSON array of key-value pairs. I need to translate this array to a single object with the value of the "key" as the field name and the value of the "value" as the f… --- ## [Logstash web-hdfs plugin problem](https://discuss.elastic.co/t/logstash-web-hdfs-plugin-problem/147996)
**Author:** [@John\_McAfee1](https://discuss.elastic.co/u/John_McAfee1)\ **Replies:** 0\ **Last updated:** [September 10, 2018, 5:02pm UTC](https://discuss.elastic.co/t/logstash-web-hdfs-plugin-problem/147996 "2018-09-10T17:02:37Z")
Good morning: I am using ELK 6.2.2 to parse many kinds of files. All the other plugins work just fine. But I am getting two errors when attempting to use the webhdfs output plugin. It's not very specific(at least to m… --- ## [Using jdbc\_static to iterate over all event fields](https://discuss.elastic.co/t/using-jdbc-static-to-iterate-over-all-event-fields/147995)
**Author:** [@manwithoutsleep](https://discuss.elastic.co/u/manwithoutsleep)\ **Replies:** 0\ **Last updated:** [September 10, 2018, 4:48pm UTC](https://discuss.elastic.co/t/using-jdbc-static-to-iterate-over-all-event-fields/147995 "2018-09-10T16:48:59Z")
Is it possible to use jdbc\_static to iterate over all fields of the event and replace the field names, perhaps with the mutate filter? I have jdbc\_static set up correctly, but I'm having trouble with the iteration over a… --- ## [Jdbc and consistence of data](https://discuss.elastic.co/t/jdbc-and-consistence-of-data/147756)
**Author:** [@elasticforme](https://discuss.elastic.co/u/elasticforme)\ **Replies:** 4\ **Last updated:** [September 10, 2018, 4:19pm UTC](https://discuss.elastic.co/t/jdbc-and-consistence-of-data/147756 "2018-09-10T16:19:46Z")
I am doing simple test. connected to oracle db using jdbc and pulled two field name,date total 3000 record got pulled in to logstash. which are same when I check on oracle db. ran that automatically few time as my conf… --- ## [How to drop a line if it does not have all fields in it?](https://discuss.elastic.co/t/how-to-drop-a-line-if-it-does-not-have-all-fields-in-it/147090)
**Author:** [@vikas\_gopal](https://discuss.elastic.co/u/vikas_gopal)\ **Replies:** 3\ **Last updated:** [September 10, 2018, 1:15pm UTC](https://discuss.elastic.co/t/how-to-drop-a-line-if-it-does-not-have-all-fields-in-it/147090 "2018-09-10T13:15:52Z")
Hi Experts, I have csv file and it has 4 columns , unfortunately it has data like "abc", "123", "res","vg" "abc", "123", "res","vg" "abc", "123", "res "abc", "123", "res "abc", "123", "res","vg" "abc", "123", "res… --- ## [Logstash pipeline block in start\_input](https://discuss.elastic.co/t/logstash-pipeline-block-in-start-input/147940)
**Author:** [@anji](https://discuss.elastic.co/u/anji)\ **Replies:** 1\ **Last updated:** [September 10, 2018, 1:02pm UTC](https://discuss.elastic.co/t/logstash-pipeline-block-in-start-input/147940 "2018-09-10T13:02:48Z")
Hi Team, i have setup like Elasticsearch , logstash and kibana in one machine when i running my logstash file i am getting Error pipeline block in start\_input can anyone suggest what needs to be done, i have tried to … [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=686) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=688)