# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=799 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 800 --- ## [Parsing problems with IPv6 in the event](https://discuss.elastic.co/t/parsing-problems-with-ipv6-in-the-event/124231)
**Author:** [@Akaren](https://discuss.elastic.co/u/Akaren)\ **Replies:** 2\ **Last updated:** [March 19, 2018, 2:04pm UTC](https://discuss.elastic.co/t/parsing-problems-with-ipv6-in-the-event/124231 "2018-03-19T14:04:33Z")
Hi, we are having 5.4.3. version of ELK. The field "source" is set to be type "ip". Before it work with any problem until we started to send data with ipv6 (not ipv4). The IP address is in correct format. What can be wr… --- ## [Installation of elasticsearch kibana and logstash in windows 10](https://discuss.elastic.co/t/installation-of-elasticsearch-kibana-and-logstash-in-windows-10/124564)
**Author:** [@mayanksachan007](https://discuss.elastic.co/u/mayanksachan007)\ **Replies:** 2\ **Last updated:** [March 19, 2018, 1:45pm UTC](https://discuss.elastic.co/t/installation-of-elasticsearch-kibana-and-logstash-in-windows-10/124564 "2018-03-19T13:45:40Z")
Windows command for elasticsearch : C:\\Users\\ABC\\Desktop\\ELK\\elasticsearch-6.2.2\\bin\>elasticsearch-plugin.bat install file:///C:\\Users\\ABC\\Desktop\\ELK\\x-pack-6.2.2.zip Windows command for kibana: C:\\Users\\ABC\\Desktop\\… --- ## [In Logstash when we given start command it is not able to push the data in Elastic Search](https://discuss.elastic.co/t/in-logstash-when-we-given-start-command-it-is-not-able-to-push-the-data-in-elastic-search/124529)
**Author:** [@rakeshroshan](https://discuss.elastic.co/u/rakeshroshan)\ **Replies:** 1\ **Last updated:** [March 19, 2018, 1:32pm UTC](https://discuss.elastic.co/t/in-logstash-when-we-given-start-command-it-is-not-able-to-push-the-data-in-elastic-search/124529 "2018-03-19T13:32:37Z")
Doubt:-1: I am trying to do ELK demo in Windows. I want to load files into elastic search. I started the beats. First time when i try to load a file in elastic search , it loads successfully. But second time when i try… --- ## [Logstash JSON iteration failure](https://discuss.elastic.co/t/logstash-json-iteration-failure/124492)
**Author:** [@satya\_sandeep](https://discuss.elastic.co/u/satya_sandeep)\ **Replies:** 11\ **Last updated:** [March 19, 2018, 1:19pm UTC](https://discuss.elastic.co/t/logstash-json-iteration-failure/124492 "2018-03-19T13:19:19Z")
I have a logstash script filter { json{ source=\>"message" target=\>"dat" } split { field =\> "\[dat\]" } split { field =\> "\[dat\]\[friends\]" } } json message is : \[ { "\_id": "5aaf44772d97f524d9d11519", "index": 0, "gu… --- ## [Grok Filter for JIRA Access Logs](https://discuss.elastic.co/t/grok-filter-for-jira-access-logs/124545)
**Author:** [@deepesh\_goel](https://discuss.elastic.co/u/deepesh_goel)\ **Replies:** 0\ **Last updated:** [March 19, 2018, 11:46am UTC](https://discuss.elastic.co/t/grok-filter-for-jira-access-logs/124545 "2018-03-19T11:46:10Z")
Hi Everyone, Lately I been working on JIRA access logs parsing using grok filter but did not able to write a one. Can anyone help me on writing a grok filter for my JIRA access logs. Example: 10.12.68.155 0x14430011x… --- ## [Import CSV and update multi value field](https://discuss.elastic.co/t/import-csv-and-update-multi-value-field/124514)
**Author:** [@kevkos236](https://discuss.elastic.co/u/kevkos236)\ **Replies:** 0\ **Last updated:** [March 19, 2018, 9:17am UTC](https://discuss.elastic.co/t/import-csv-and-update-multi-value-field/124514 "2018-03-19T09:17:48Z")
Greetings, I'm importing large CSV files into elasticsearch using logstash. I'm prefixing specific values like this: filter { if \[path\] =~ "xyz" { csv { separator =\> ";" … --- ## [Illegal\_argument\_exception mapping dates](https://discuss.elastic.co/t/illegal-argument-exception-mapping-dates/124513)
**Author:** [@ventrca](https://discuss.elastic.co/u/ventrca)\ **Replies:** 0\ **Last updated:** [March 19, 2018, 9:14am UTC](https://discuss.elastic.co/t/illegal-argument-exception-mapping-dates/124513 "2018-03-19T09:14:37Z")
Hi, I've this simple filter to parse data, the problem is that sometimes logs are are saved in ES but are not correctly mapped. date{ match =\> \[ "acc-log-timestamp", "yyyy-MM-dd HH:mm:ss", "ISO8… --- ## [Logstash error when running config file](https://discuss.elastic.co/t/logstash-error-when-running-config-file/124502)
**Author:** [@Padmapriya](https://discuss.elastic.co/u/Padmapriya)\ **Replies:** 0\ **Last updated:** [March 19, 2018, 8:47am UTC](https://discuss.elastic.co/t/logstash-error-when-running-config-file/124502 "2018-03-19T08:47:45Z")
Hi, I am trying to run the config file in logstash by using command like logstash -f D:\\Data\\IIB\_log.config but receiving the below error. 2018-03-19T09:00:26.353Z admin-PC " 2018-03-19T09:00:26.368Z admin-PC Error,1… --- ## [Running LS in two user sessions of the same server](https://discuss.elastic.co/t/running-ls-in-two-user-sessions-of-the-same-server/124307)
**Author:** [@Loic\_Bellot](https://discuss.elastic.co/u/Loic_Bellot)\ **Replies:** 8\ **Last updated:** [March 19, 2018, 8:58am UTC](https://discuss.elastic.co/t/running-ls-in-two-user-sessions-of-the-same-server/124307 "2018-03-19T08:58:25Z")
Hi everybody, I'm trying to run LS in two user sessions of the same server. No problem to run the first LS instance in the first session. No problem to run the first AND second LS instances in the first session. But … --- ## [How to extract first line of a multiline message?](https://discuss.elastic.co/t/how-to-extract-first-line-of-a-multiline-message/124163)
**Author:** [@proximator](https://discuss.elastic.co/u/proximator)\ **Replies:** 2\ **Last updated:** [March 19, 2018, 4:01am UTC](https://discuss.elastic.co/t/how-to-extract-first-line-of-a-multiline-message/124163 "2018-03-19T04:01:23Z")
I need to add a new field as the first line of a multline message coming from filebeat to logstash? Then from this line, a substring of this line as new field which I need to parse it as timestamp? Thanks for your help… --- ## [Error when using multi pipeline config](https://discuss.elastic.co/t/error-when-using-multi-pipeline-config/124207)
**Author:** [@br14nb0](https://discuss.elastic.co/u/br14nb0)\ **Replies:** 8\ **Last updated:** [March 19, 2018, 3:40am UTC](https://discuss.elastic.co/t/error-when-using-multi-pipeline-config/124207 "2018-03-19T03:40:00Z")
I have this error when I try to run multi pipeline config "ERROR: Failed to read pipelines yaml file. Location: C:/elk/logstash/config/pipelines.yml" And this is the content of my pipelines.yml pipeline.id: sharepoin… --- ## [DNS Filter](https://discuss.elastic.co/t/dns-filter/124379)
**Author:** [@wwalker](https://discuss.elastic.co/u/wwalker)\ **Replies:** 9\ **Last updated:** [March 19, 2018, 3:38am UTC](https://discuss.elastic.co/t/dns-filter/124379 "2018-03-19T03:38:22Z")
I setup a filter to do some internal DNS resolution but it seems that when I give it a proper DNS server, it stops processing events. If I give it a bogus DNS server I, obviously, get non-stop resolution failures in the… --- ## [How logstash change the id to custom with kv](https://discuss.elastic.co/t/how-logstash-change-the-id-to-custom-with-kv/124189)
**Author:** [@zqc0512](https://discuss.elastic.co/u/zqc0512)\ **Replies:** 4\ **Last updated:** [March 19, 2018, 12:50am UTC](https://discuss.elastic.co/t/how-logstash-change-the-id-to-custom-with-kv/124189 "2018-03-19T00:50:03Z")
with kv configure now in kibana the data is like \_source": { "1": "1", "2": "0", "3": "0", "4": "0", "5": "1", "6": "0", "7": "1", "8": "0", "9": "0", "10": "0", "11": "0", "12": "0", "13": "0", "14": "0"… --- ## [Moving multiple MySQL tables to ElasticSearch in the same index](https://discuss.elastic.co/t/moving-multiple-mysql-tables-to-elasticsearch-in-the-same-index/124450)
**Author:** [@limafoxtrottango](https://discuss.elastic.co/u/limafoxtrottango)\ **Replies:** 1\ **Last updated:** [March 18, 2018, 8:56pm UTC](https://discuss.elastic.co/t/moving-multiple-mysql-tables-to-elasticsearch-in-the-same-index/124450 "2018-03-18T20:56:18Z")
I am migrating my database to ElasticSearch. Its current size is less than 3G, and since many fields are common across tables, I have decided to use a single index to store them. Here's my Logstash configuration: input… --- ## [How can i check data of mapping field in elasticsearch with logstash](https://discuss.elastic.co/t/how-can-i-check-data-of-mapping-field-in-elasticsearch-with-logstash/121372)
**Author:** [@yaser](https://discuss.elastic.co/u/yaser)\ **Replies:** 5\ **Last updated:** [March 18, 2018, 8:51pm UTC](https://discuss.elastic.co/t/how-can-i-check-data-of-mapping-field-in-elasticsearch-with-logstash/121372 "2018-03-18T20:51:32Z")
hi every one , can i check the duplicate username and return true or false in logstash input part { is rabbitmq server } / username is send by json filter { what's happen here?} output { what's happen here? } --- ## [XML filter issues](https://discuss.elastic.co/t/xml-filter-issues/124416)
**Author:** [@arisbanach](https://discuss.elastic.co/u/arisbanach)\ **Replies:** 1\ **Last updated:** [March 18, 2018, 1:26am UTC](https://discuss.elastic.co/t/xml-filter-issues/124416 "2018-03-18T01:26:20Z")
I am using the XML filter to filter data from S3 files and it is working for some of the documents, but I am getting a lot of errors on other documents. From Elasticsearch, I see it rejects a lot of them with this error: … --- ## [Webpage Input Plugin](https://discuss.elastic.co/t/webpage-input-plugin/124399)
**Author:** [@Salma](https://discuss.elastic.co/u/Salma)\ **Replies:** 19\ **Last updated:** [March 17, 2018, 8:19pm UTC](https://discuss.elastic.co/t/webpage-input-plugin/124399 "2018-03-17T20:19:26Z")
Hi, I'm wondering if there is a way to input webpage data into Elasticsearch using Logstash pipeline. I have a webpage that is publicly accessible like website/logs/access.log what input plugin should I use? Thanks, … --- ## [Urls to whitelist for logstash](https://discuss.elastic.co/t/urls-to-whitelist-for-logstash/124006)
**Author:** [@saneejmc](https://discuss.elastic.co/u/saneejmc)\ **Replies:** 1\ **Last updated:** [March 17, 2018, 8:15pm UTC](https://discuss.elastic.co/t/urls-to-whitelist-for-logstash/124006 "2018-03-17T20:15:11Z")
Hi All, I am trying to install plugin logstash-output-email , logstash-filter-elapsed , logstash-filter-aggregate and logstash-filter-date , in my secured AWS cluster. But I am getting 'ERROR: Something went wrong… --- ## [Percentage Formatting](https://discuss.elastic.co/t/percentage-formatting/124417)
**Author:** [@wwalker](https://discuss.elastic.co/u/wwalker)\ **Replies:** 1\ **Last updated:** [March 17, 2018, 8:04pm UTC](https://discuss.elastic.co/t/percentage-formatting/124417 "2018-03-17T20:04:51Z")
Not really sure where this belongs to be honest. I am ingesting a field that has a numeric value that represents a percentage. So 5 is 5%, 50 is 50%, 100 is 100%. Unfortunately, I can't figure out the Kibana formattin… --- ## [Logstash kafka rename topic name](https://discuss.elastic.co/t/logstash-kafka-rename-topic-name/124334)
**Author:** [@rajkrishmoorthy](https://discuss.elastic.co/u/rajkrishmoorthy)\ **Replies:** 2\ **Last updated:** [March 17, 2018, 5:39pm UTC](https://discuss.elastic.co/t/logstash-kafka-rename-topic-name/124334 "2018-03-17T17:39:25Z")
below is my logstash config, i need to rename topic to new topicname, how can i achieve that ?, i tried following config it seems not working input { kafka { bootstrap\_servers =\> "boostrap1:9092" topics =\> \["global-h… --- ## [Logstash on windows](https://discuss.elastic.co/t/logstash-on-windows/124391)
**Author:** [@kanchan\_pingale](https://discuss.elastic.co/u/kanchan_pingale)\ **Replies:** 3\ **Last updated:** [March 17, 2018, 5:01pm UTC](https://discuss.elastic.co/t/logstash-on-windows/124391 "2018-03-17T17:01:26Z")
Hello! I want to know how to install xpack on windows !an you please help me with the commands . I am new to this. --- ## [Could nprobe pass netflow to logstash?](https://discuss.elastic.co/t/could-nprobe-pass-netflow-to-logstash/124404)
**Author:** [@f26227279](https://discuss.elastic.co/u/f26227279)\ **Replies:** 0\ **Last updated:** [March 17, 2018, 2:14pm UTC](https://discuss.elastic.co/t/could-nprobe-pass-netflow-to-logstash/124404 "2018-03-17T14:14:05Z")
Hi everyone, After passing netflow to nprobe, I want to pass the flow(from nprobe) to logstash. Is it feasible? because I found that logstash support nprobe: \[image\] this is my nprobe command: nprobe /c --zmq "tcp:… --- ## [Logstash from Mongo To Elastic - Array Problem](https://discuss.elastic.co/t/logstash-from-mongo-to-elastic-array-problem/124340)
**Author:** [@Pierpaolo\_Rai](https://discuss.elastic.co/u/Pierpaolo_Rai)\ **Replies:** 4\ **Last updated:** [March 17, 2018, 9:32am UTC](https://discuss.elastic.co/t/logstash-from-mongo-to-elastic-array-problem/124340 "2018-03-17T09:32:08Z")
Hello guys, I'm running Logstash to sync my ElasticSearch with my MongoDB operation. I used the mongdb plugin to read from mongo, this is my initial config file: input { mongodb { uri =\> 'mongodb://localhost:27017/db… --- ## [Grok pattern filter Sharepoint](https://discuss.elastic.co/t/grok-pattern-filter-sharepoint/124049)
**Author:** [@br14nb0](https://discuss.elastic.co/u/br14nb0)\ **Replies:** 1\ **Last updated:** [March 17, 2018, 8:41am UTC](https://discuss.elastic.co/t/grok-pattern-filter-sharepoint/124049 "2018-03-17T08:41:38Z")
Hello guys I have this Sharepoint log line: " 02/02/2017 11:30:19.53 w3wp.exe (0x23E8) 0x0DD8 SharePoint Foundation General ad74" So how can I parse "Sharepoint Foundation" ? This field means the category so … --- ## [Updated data from jdbc(mysql) to elasticsearch by using logstash](https://discuss.elastic.co/t/updated-data-from-jdbc-mysql-to-elasticsearch-by-using-logstash/124058)
**Author:** [@songhee](https://discuss.elastic.co/u/songhee)\ **Replies:** 1\ **Last updated:** [March 17, 2018, 8:27am UTC](https://discuss.elastic.co/t/updated-data-from-jdbc-mysql-to-elasticsearch-by-using-logstash/124058 "2018-03-17T08:27:16Z")
Hello, I am also newbie in elk stack. I just connected mysql(DBMS) and elasticsearch by using logstash and successed to send bulk data from mysql server. I want to manage data on mysql. like create, update, delete pr… --- ## [Logstash doesn't store in elasticsearch](https://discuss.elastic.co/t/logstash-doesnt-store-in-elasticsearch/124317)
**Author:** [@franco.federico](https://discuss.elastic.co/u/franco.federico)\ **Replies:** 3\ **Last updated:** [March 17, 2018, 7:55am UTC](https://discuss.elastic.co/t/logstash-doesnt-store-in-elasticsearch/124317 "2018-03-17T07:55:43Z")
I'm trying to do a script to load an apache file log. The configuration files is correct because I started different time to load this file with the command logstash -f \ and I viewed the dot on the scree… --- ## [Have Some Questions For ELK+Kafka](https://discuss.elastic.co/t/have-some-questions-for-elk-kafka/124219)
**Author:** [@guanzhiwei](https://discuss.elastic.co/u/guanzhiwei)\ **Replies:** 1\ **Last updated:** [March 17, 2018, 6:51am UTC](https://discuss.elastic.co/t/have-some-questions-for-elk-kafka/124219 "2018-03-17T06:51:26Z")
已经解决 谢谢平台 问题已经得到解决 --- ## [Logstash-output-mongodb update document mode?](https://discuss.elastic.co/t/logstash-output-mongodb-update-document-mode/124233)
**Author:** [@YvesHamel](https://discuss.elastic.co/u/YvesHamel)\ **Replies:** 1\ **Last updated:** [March 17, 2018, 5:56am UTC](https://discuss.elastic.co/t/logstash-output-mongodb-update-document-mode/124233 "2018-03-17T05:56:06Z")
Hi, I use logstash for loading csv file content to mongodb database using logstash-output-mongodb. It works fine for loading new document in the mongodb collection but I can't update existing document. My question is … --- ## [IF with Mutate Filter](https://discuss.elastic.co/t/if-with-mutate-filter/124386)
**Author:** [@wwalker](https://discuss.elastic.co/u/wwalker)\ **Replies:** 2\ **Last updated:** [March 17, 2018, 5:51am UTC](https://discuss.elastic.co/t/if-with-mutate-filter/124386 "2018-03-17T05:51:44Z")
I have a couple fields that will come in one of two values, r or s. I'd like to convert them to Relaxed or Strict as they come, but there may be instances where the field has no value. Is there a more efficient way tha… --- ## [Get the field count of a csv file](https://discuss.elastic.co/t/get-the-field-count-of-a-csv-file/124161)
**Author:** [@pfseth](https://discuss.elastic.co/u/pfseth)\ **Replies:** 1\ **Last updated:** [March 16, 2018, 7:33pm UTC](https://discuss.elastic.co/t/get-the-field-count-of-a-csv-file/124161 "2018-03-16T19:33:47Z")
Sample CSV file 2018-03-05 21:29:31 -0500|188c9263-2519-4a8d-825c-4605c53b720e|email|1000|0|7702831801|start|89029287801|1273721801|end 2018-03-05 21:23:10 -0500|23k8w23y-27k3-293s-le22-825c29l23us2|fax|21000|0|238923… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=798) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=800)