# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=861 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 862 --- ## [Logstash Multiple Pipelines Doesn't work](https://discuss.elastic.co/t/logstash-multiple-pipelines-doesnt-work/110251)
**Author:** [@pavuk](https://discuss.elastic.co/u/pavuk)\ **Replies:** 15\ **Last updated:** [December 7, 2017, 7:40am UTC](https://discuss.elastic.co/t/logstash-multiple-pipelines-doesnt-work/110251 "2017-12-07T07:40:28Z")
Unfortunately announced Multiple Pipelines feature doesn't work. I have created file # cat /etc/logstash/pipeline.yml - pipeline.id: nginx path.config: "/etc/logstash/conf.d/nginx.conf" #queue.type: persisted - pip… --- ## [Working with large XML's - logstash/elastic can not handle](https://discuss.elastic.co/t/working-with-large-xmls-logstash-elastic-can-not-handle/110610)
**Author:** [@hornbill81](https://discuss.elastic.co/u/hornbill81)\ **Replies:** 1\ **Last updated:** [December 7, 2017, 7:31am UTC](https://discuss.elastic.co/t/working-with-large-xmls-logstash-elastic-can-not-handle/110610 "2017-12-07T07:31:17Z")
Hi All, First time posting so may edit a few times if the formatting sucks. I have a log file from tibco process that spits out small, medium and large XML messages. filebeat sends the messages to elastic (just to be a… --- ## [Reindexing/Updating Elasticsearch using Logstash on Jenkins](https://discuss.elastic.co/t/reindexing-updating-elasticsearch-using-logstash-on-jenkins/108607)
**Author:** [@kseth](https://discuss.elastic.co/u/kseth)\ **Replies:** 5\ **Last updated:** [December 7, 2017, 7:11am UTC](https://discuss.elastic.co/t/reindexing-updating-elasticsearch-using-logstash-on-jenkins/108607 "2017-12-07T07:11:01Z")
I would like to automate the process of updating the elasticsearch with latest data on demand and secondly, recreating the index along with feeding data using a Jenkins job. I am using jdbc input plugin for fetching dat… --- ## [Curl in logstash conf](https://discuss.elastic.co/t/curl-in-logstash-conf/110457)
**Author:** [@Shekhar\_Pandey](https://discuss.elastic.co/u/Shekhar_Pandey)\ **Replies:** 10\ **Last updated:** [December 7, 2017, 6:10am UTC](https://discuss.elastic.co/t/curl-in-logstash-conf/110457 "2017-12-07T06:10:28Z")
Can we use curl option in logstash conf...is this possible or not.... Thank you. --- ## [Get year month and day from date field](https://discuss.elastic.co/t/get-year-month-and-day-from-date-field/110600)
**Author:** [@DILIP\_SHARMA](https://discuss.elastic.co/u/DILIP_SHARMA)\ **Replies:** 1\ **Last updated:** [December 7, 2017, 4:40am UTC](https://discuss.elastic.co/t/get-year-month-and-day-from-date-field/110600 "2017-12-07T04:40:22Z")
hi my csv data - "s1","salesman01","2017-06-22 00:00:00.000","2017-06-22 16:10:23.000","2017-06-22 16:11:44.000","5","25.252130,55.349060" i want to create new field which will call computeid for that i used this - m… --- ## [Something went wrong when installing logstash-filter-prune, message: certificate verify failed](https://discuss.elastic.co/t/something-went-wrong-when-installing-logstash-filter-prune-message-certificate-verify-failed/110575)
**Author:** [@Samuel\_Lima1](https://discuss.elastic.co/u/Samuel_Lima1)\ **Replies:** 0\ **Last updated:** [December 6, 2017, 8:54pm UTC](https://discuss.elastic.co/t/something-went-wrong-when-installing-logstash-filter-prune-message-certificate-verify-failed/110575 "2017-12-06T20:54:51Z")
Hi folks, I´m trying to install a plugin with logstash-plugin as folow: sudo /usr/share/logstash/bin/logstash-plugin install --no-verify logstash-filter-prune ERROR: Something went wrong when installing logstash-filt… --- ## [Strange JSON Codec Behavior](https://discuss.elastic.co/t/strange-json-codec-behavior/110570)
**Author:** [@code\_diva](https://discuss.elastic.co/u/code_diva)\ **Replies:** 2\ **Last updated:** [December 6, 2017, 9:03pm UTC](https://discuss.elastic.co/t/strange-json-codec-behavior/110570 "2017-12-06T21:03:49Z")
Hi, I have been using logstash for quite some time and I have never seen behavior quite like this. I am attributing it to the JSON codec because I have never used it before. When I run logstash with the information belo… --- ## [X-Pack Logstash Pipelines with basic License](https://discuss.elastic.co/t/x-pack-logstash-pipelines-with-basic-license/110100)
**Author:** [@babelr](https://discuss.elastic.co/u/babelr)\ **Replies:** 1\ **Last updated:** [December 6, 2017, 8:24pm UTC](https://discuss.elastic.co/t/x-pack-logstash-pipelines-with-basic-license/110100 "2017-12-06T20:24:53Z")
Hi there! I set up an ELK Stack 6.0 with X-Pack (running in trial mode). I configured a rather large logstash pipeline using X-Pack and let it pick up by a few logstash instances automatically. Finally, I applied for a… --- ## [Grok filter for different type of messages](https://discuss.elastic.co/t/grok-filter-for-different-type-of-messages/110555)
**Author:** [@ShruthiS](https://discuss.elastic.co/u/ShruthiS)\ **Replies:** 0\ **Last updated:** [December 6, 2017, 6:06pm UTC](https://discuss.elastic.co/t/grok-filter-for-different-type-of-messages/110555 "2017-12-06T18:06:04Z")
Hi, I am trying to parse cloud foundry log messages. there are different types of messages added by cloud foundry and one type is APP. there are different APP messages like the standart APACHELOG format, stack traces an… --- ## [Logstash 5.6.2 Crashing](https://discuss.elastic.co/t/logstash-5-6-2-crashing/110549)
**Author:** [@shashwat](https://discuss.elastic.co/u/shashwat)\ **Replies:** 2\ **Last updated:** [December 6, 2017, 5:48pm UTC](https://discuss.elastic.co/t/logstash-5-6-2-crashing/110549 "2017-12-06T17:48:06Z")
Hi, I am running Logstash 5.6.2 on Linux environment. It runs for some time, sometimes for 2-3 days but then it suddenly stops. Following are the logs: \] An unexpected error occurred! {:error=\>#\ **Author:** [@moriol](https://discuss.elastic.co/u/moriol)\ **Replies:** 1\ **Last updated:** [December 6, 2017, 4:10pm UTC](https://discuss.elastic.co/t/add-new-field-based-on-the-result-of-array/110537 "2017-12-06T16:10:42Z") I have part of JSON appearing as below: { "SampleName" =\> \[ \[0\] "name1", \[1\] "name2", \[2\] "name3", \[3\] "name4" \], } I want to do the following: If the SampleName\[0\] is equal to name1, create a field called… --- ## [CSV Logstash filter for big files](https://discuss.elastic.co/t/csv-logstash-filter-for-big-files/109265)
**Author:** [@ELK\_GB](https://discuss.elastic.co/u/ELK_GB)\ **Replies:** 4\ **Last updated:** [December 6, 2017, 3:18pm UTC](https://discuss.elastic.co/t/csv-logstash-filter-for-big-files/109265 "2017-12-06T15:18:34Z")
Hi Guys, I have a big CSV file that has about 500+ csv columns, I managed to get them into the LS CSV filter, but those fields/columns will be dynamic. Meaning that sometimes we may get more/less columns produced by the… --- ## [Logstah-6.0.0 logstash-input-jdbc intall error](https://discuss.elastic.co/t/logstah-6-0-0-logstash-input-jdbc-intall-error/110530)
**Author:** [@S3ns3o](https://discuss.elastic.co/u/S3ns3o)\ **Replies:** 0\ **Last updated:** [December 6, 2017, 2:41pm UTC](https://discuss.elastic.co/t/logstah-6-0-0-logstash-input-jdbc-intall-error/110530 "2017-12-06T14:41:34Z")
Hi, I'm trying to install the jdbc input plugin for logstash-6.0.0 but get the following error : An error occurred while installing logstash-core (6.0.0), and Bundler cannot continue. Make sure that gem install logst… --- ## [Split XML array Logstash](https://discuss.elastic.co/t/split-xml-array-logstash/110494)
**Author:** [@moriol](https://discuss.elastic.co/u/moriol)\ **Replies:** 2\ **Last updated:** [December 6, 2017, 12:52pm UTC](https://discuss.elastic.co/t/split-xml-array-logstash/110494 "2017-12-06T12:52:55Z")
I have data in the following format, repeating each time as follows: \< Data ID="test" ShortDescription="some description" type="this is a type"\> \< LongDescription\>this is a long description\< /LongDescription\> \< Values\>… --- ## [Help with grok filter](https://discuss.elastic.co/t/help-with-grok-filter/109637)
**Author:** [@Rafael\_Pereira\_Silva](https://discuss.elastic.co/u/Rafael_Pereira_Silva)\ **Replies:** 6\ **Last updated:** [December 6, 2017, 10:30am UTC](https://discuss.elastic.co/t/help-with-grok-filter/109637 "2017-12-06T10:30:07Z")
Hello, I am dificult with Grok Filter this message: loc=45671|time=2017-11-27 18:35:02|action=HTTPS Inspect|orig=10.212.12.31|i/f\_dir=inbound|i/f\_name=eth1.315|has\_accounting=0|uuid=\<00000000,00000000,00000000,0000000… --- ## [Help grokking a syslog](https://discuss.elastic.co/t/help-grokking-a-syslog/109862)
**Author:** [@ajhstn](https://discuss.elastic.co/u/ajhstn)\ **Replies:** 6\ **Last updated:** [December 6, 2017, 10:26am UTC](https://discuss.elastic.co/t/help-grokking-a-syslog/109862 "2017-12-06T10:26:12Z")
Hello, i need some help to grok this. Should i use the CEF pattern or just manually hack away at my own? I cant seem to find any built-in patterns that work.. The message looks like part of it is syslog, part is key=v… --- ## [Lgstash not loading data](https://discuss.elastic.co/t/lgstash-not-loading-data/110458)
**Author:** [@DILIP\_SHARMA](https://discuss.elastic.co/u/DILIP_SHARMA)\ **Replies:** 5\ **Last updated:** [December 6, 2017, 10:14am UTC](https://discuss.elastic.co/t/lgstash-not-loading-data/110458 "2017-12-06T10:14:00Z")
hi i am trying to load data to ES from logstash using template json my data - "s1","salesman01","2017-06-22 00:00:00.000","2017-06-22 16:10:23.000","2017-06-22 16:11:44.000","5","25.252130,55.349060" "s2","salesman01"… --- ## [Pipelines.yml doesn't work](https://discuss.elastic.co/t/pipelines-yml-doesnt-work/110478)
**Author:** [@Andrey\_Tomilin](https://discuss.elastic.co/u/Andrey_Tomilin)\ **Replies:** 0\ **Last updated:** [December 6, 2017, 9:29am UTC](https://discuss.elastic.co/t/pipelines-yml-doesnt-work/110478 "2017-12-06T09:29:51Z")
I'm using Windows Server 2012 R2 (can't check if it's ok on Linux). The pipelines.yml contains two these lines: - pipeline.id: test path.config: "C:/logstash-6.0.0/config/test.conf" The content of the test.conf do… --- ## [Kv filter has no support for this type of data](https://discuss.elastic.co/t/kv-filter-has-no-support-for-this-type-of-data/109868)
**Author:** [@Sjaak01](https://discuss.elastic.co/u/Sjaak01)\ **Replies:** 9\ **Last updated:** [December 6, 2017, 6:39am UTC](https://discuss.elastic.co/t/kv-filter-has-no-support-for-this-type-of-data/109868 "2017-12-06T06:39:24Z")
Hi, I've got the following setup: Netflow -\> remote logstash -\> file -\> file transfer -\> server logstash -\> Elasticsearch. On the server logstash I'm trying to parse the netflow file but the log outputs kv filter has n… --- ## [Filter Logstash syslog](https://discuss.elastic.co/t/filter-logstash-syslog/110429)
**Author:** [@yayitaing](https://discuss.elastic.co/u/yayitaing)\ **Replies:** 2\ **Last updated:** [December 6, 2017, 6:30am UTC](https://discuss.elastic.co/t/filter-logstash-syslog/110429 "2017-12-06T06:30:08Z")
I have the following message that is displayed in the kibana brought by filebeat from a linux syslog: \[image\] However, I need each field of the message to be shown in the kibana as an independent field to be able to pe… --- ## [Elastic to csv using logstash](https://discuss.elastic.co/t/elastic-to-csv-using-logstash/110103)
**Author:** [@Shekhar\_Pandey](https://discuss.elastic.co/u/Shekhar_Pandey)\ **Replies:** 4\ **Last updated:** [December 6, 2017, 5:47am UTC](https://discuss.elastic.co/t/elastic-to-csv-using-logstash/110103 "2017-12-06T05:47:52Z")
I use the following conf file to import data from elasticsearch to csv using logstash... input { elasticsearch { hosts =\> "localhost:9200" index =\> "croma" query =\> ' {"query": { "match\_phrase" : { "search\_data.… --- ## [Output merged Logfile data to MongoDB using Logstash](https://discuss.elastic.co/t/output-merged-logfile-data-to-mongodb-using-logstash/110368)
**Author:** [@edna](https://discuss.elastic.co/u/edna)\ **Replies:** 0\ **Last updated:** [December 5, 2017, 1:57pm UTC](https://discuss.elastic.co/t/output-merged-logfile-data-to-mongodb-using-logstash/110368 "2017-12-05T13:57:28Z")
Hi everyone, I am quite new to Logstash and have been learning to use it for some time. I have a log file as follows logfile.txt 01012014:123456789:1:INFO:Some Text Type1:TranLine 01012014:123456790:1:INFO:Some Text … --- ## [Timestamp grok pattern help](https://discuss.elastic.co/t/timestamp-grok-pattern-help/110393)
**Author:** [@Ravikumar\_G](https://discuss.elastic.co/u/Ravikumar_G)\ **Replies:** 7\ **Last updated:** [December 5, 2017, 9:08pm UTC](https://discuss.elastic.co/t/timestamp-grok-pattern-help/110393 "2017-12-05T21:08:00Z")
Hello, I'm trying to ingest timestamp with millisecond events but failing in kibana any help would be great actual log 2017-12-05T16:14:10.543997+00:00 ccdn-ats-tk-40405-02 pump4\[40837\]: Level=Error, subSystem=SESSION… --- ## [Grok parse faliure filebeat 6.0](https://discuss.elastic.co/t/grok-parse-faliure-filebeat-6-0/110243)
**Author:** [@ramzey1981](https://discuss.elastic.co/u/ramzey1981)\ **Replies:** 4\ **Last updated:** [December 5, 2017, 8:15pm UTC](https://discuss.elastic.co/t/grok-parse-faliure-filebeat-6-0/110243 "2017-12-05T20:15:50Z")
hello folks i have the following filebeat log going to logstash https://pastebin.com/raw/hAGrDNLc and the following logstash filter but the filter is throwing a grok parse failure. I am not certain why the if conditio… --- ## [Grok issues in 5.6.3 -\> 6.0](https://discuss.elastic.co/t/grok-issues-in-5-6-3-6-0/110410)
**Author:** [@robscott27](https://discuss.elastic.co/u/robscott27)\ **Replies:** 2\ **Last updated:** [December 5, 2017, 8:02pm UTC](https://discuss.elastic.co/t/grok-issues-in-5-6-3-6-0/110410 "2017-12-05T20:02:18Z")
Recently, I've made the move to LS 6.0. I was coming from LS 5.6.3 and had a filter working perfectly parsing pfsense log files. I copied over the same file from the 5.6.3 install to use in LS 6.0 install but in 6.0 it's… --- ## [Kv filter plugin trim\_key option seems stopped working](https://discuss.elastic.co/t/kv-filter-plugin-trim-key-option-seems-stopped-working/110161)
**Author:** [@Nikolay\_Shushkin](https://discuss.elastic.co/u/Nikolay_Shushkin)\ **Replies:** 2\ **Last updated:** [December 5, 2017, 7:22pm UTC](https://discuss.elastic.co/t/kv-filter-plugin-trim-key-option-seems-stopped-working/110161 "2017-12-05T19:22:25Z")
Hello experts, After upgrade logstash from 1.5 to 5.6 we started having field names after kv with leading spaces. Configuration in 1.5: kv { source =\> "headers" field\_split =\> "\[\\r\\n\]+" value\_split =\> "… --- ## [Ingest Specific Json fields to Elastichsearch](https://discuss.elastic.co/t/ingest-specific-json-fields-to-elastichsearch/110324)
**Author:** [@mussa572](https://discuss.elastic.co/u/mussa572)\ **Replies:** 2\ **Last updated:** [December 5, 2017, 4:47pm UTC](https://discuss.elastic.co/t/ingest-specific-json-fields-to-elastichsearch/110324 "2017-12-05T16:47:22Z")
Hi I am ingesting following Json data to elasticseach through Logstash . There are around 100 different fields in that document. I am only interested in sending 4 fields ie linux.domain, linux.environment\_id, linux.hostn… --- ## [Multiple Indexes Created Instead of One](https://discuss.elastic.co/t/multiple-indexes-created-instead-of-one/110348)
**Author:** [@HilariousGil](https://discuss.elastic.co/u/HilariousGil)\ **Replies:** 2\ **Last updated:** [December 5, 2017, 2:32pm UTC](https://discuss.elastic.co/t/multiple-indexes-created-instead-of-one/110348 "2017-12-05T14:32:14Z")
Hi, I'm using Logstash to create a new index and the index name is set like this: - index =\> "name-%{+YYYY-MM-dd.HH-mm}" The execution of Logstash takes a bit more than one minute to process so once it is finished I e… --- ## [Urldecode useragent cloudfront](https://discuss.elastic.co/t/urldecode-useragent-cloudfront/110200)
**Author:** [@qmaerik](https://discuss.elastic.co/u/qmaerik)\ **Replies:** 1\ **Last updated:** [December 5, 2017, 2:02pm UTC](https://discuss.elastic.co/t/urldecode-useragent-cloudfront/110200 "2017-12-05T14:02:59Z")
I' m having problems with the urldecode filter and can't get it to work on any field so far and I can't figure out what I am doing wrong. I have a grok filter that is parsing the agent field just fine, but when adding u… --- ## [Modify field content](https://discuss.elastic.co/t/modify-field-content/110274)
**Author:** [@Walter\_Xue](https://discuss.elastic.co/u/Walter_Xue)\ **Replies:** 2\ **Last updated:** [December 5, 2017, 1:51pm UTC](https://discuss.elastic.co/t/modify-field-content/110274 "2017-12-05T13:51:22Z")
I need to add new field at logstsh ! the original field is request content "/example/example/example/getMessageList.json?\_t=1512453462541" , now I don't need "?\_t=1512453462541" so how to do it ? config is: mutate… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=860) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=862)