# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=612 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 613 --- ## [Multiline not working as expected](https://discuss.elastic.co/t/multiline-not-working-as-expected/165743)
**Author:** [@Arjunkrishna\_Srirama](https://discuss.elastic.co/u/Arjunkrishna_Srirama)\ **Replies:** 3\ **Last updated:** [January 25, 2019, 10:08am UTC](https://discuss.elastic.co/t/multiline-not-working-as-expected/165743 "2019-01-25T10:08:15Z")
Hi, I have an application which logs for each events. So for each event the application logs multiple lines which are nothing but the steps taken by the application to do its job. So I tried to use multiline filter … --- ## [Logstash Error "Error: Address already in use"](https://discuss.elastic.co/t/logstash-error-error-address-already-in-use/165601)
**Author:** [@sandy6](https://discuss.elastic.co/u/sandy6)\ **Replies:** 3\ **Last updated:** [January 25, 2019, 4:38am UTC](https://discuss.elastic.co/t/logstash-error-error-address-already-in-use/165601 "2019-01-25T04:38:35Z")
Hi, I have apache pipeline with multiple match patterns. Here is the code input { beats { port =\> 5044 id =\> "apache-access" } } filter { if \[fields\]\[log\] == "apache-access" { grok { match =\> \[ "message", '%{… --- ## [Http Poller call authenticate api issue](https://discuss.elastic.co/t/http-poller-call-authenticate-api-issue/164528)
**Author:** [@Haseeb\_Wali](https://discuss.elastic.co/u/Haseeb_Wali)\ **Replies:** 2\ **Last updated:** [January 25, 2019, 2:54am UTC](https://discuss.elastic.co/t/http-poller-call-authenticate-api-issue/164528 "2019-01-25T02:54:37Z")
I am trying to read data from a REST API then i will put it in elastic search. So the problem i am having is how to call 2 seperate APIs in input component of logStash. below is explanation of the problem i am facing. I… --- ## [Xml file parsing](https://discuss.elastic.co/t/xml-file-parsing/165550)
**Author:** [@fadihaddad](https://discuss.elastic.co/u/fadihaddad)\ **Replies:** 4\ **Last updated:** [January 24, 2019, 11:48pm UTC](https://discuss.elastic.co/t/xml-file-parsing/165550 "2019-01-24T23:48:22Z")
here is a sample of my xml file this is the first time I want to use an xml filter and I am lost. how can I use an xsd structure for it and how can I map value types --- ## [Finding fields within square brackets](https://discuss.elastic.co/t/finding-fields-within-square-brackets/165499)
**Author:** [@Freebird](https://discuss.elastic.co/u/Freebird)\ **Replies:** 10\ **Last updated:** [January 24, 2019, 11:18pm UTC](https://discuss.elastic.co/t/finding-fields-within-square-brackets/165499 "2019-01-24T23:18:30Z")
I'm brand new to this so please bear with me.. I have a log from an application that has inconsistent lines - as in the content of each line is different - so that might be a challenge in itself...but later. For now I'… --- ## [How to extract several words as a phrase?](https://discuss.elastic.co/t/how-to-extract-several-words-as-a-phrase/165526)
**Author:** [@Freebird](https://discuss.elastic.co/u/Freebird)\ **Replies:** 1\ **Last updated:** [January 24, 2019, 6:23am UTC](https://discuss.elastic.co/t/how-to-extract-several-words-as-a-phrase/165526 "2019-01-24T06:23:16Z")
I'm trying to learn how to use grok. I've managed to make some progress, but I am stuck again :frowning: Example of my log: 2019-01-12 02:59:54.324 Trace \[T24ServiceConnector\] Sending OFS request \[Tx be17fc06-f7c4-… --- ## [Unable to connect to elasticsearch with x-pack](https://discuss.elastic.co/t/unable-to-connect-to-elasticsearch-with-x-pack/165639)
**Author:** [@ashalabh](https://discuss.elastic.co/u/ashalabh)\ **Replies:** 1\ **Last updated:** [January 24, 2019, 9:41pm UTC](https://discuss.elastic.co/t/unable-to-connect-to-elasticsearch-with-x-pack/165639 "2019-01-24T21:41:31Z")
Sending Logstash logs to /home/elastic/logstash/logs which is now configured via log4j2.properties \[2019-01-24T12:50:23,035\]\[WARN \]\[logstash.config.source.multilocal\] Ignoring the 'pipelines.yml' file because modules or… --- ## [Elasticsearch's output script running only for updated documents](https://discuss.elastic.co/t/elasticsearchs-output-script-running-only-for-updated-documents/165635)
**Author:** [@ffknob](https://discuss.elastic.co/u/ffknob)\ **Replies:** 0\ **Last updated:** [January 24, 2019, 5:36pm UTC](https://discuss.elastic.co/t/elasticsearchs-output-script-running-only-for-updated-documents/165635 "2019-01-24T17:36:37Z")
My pipeline receives two kind of unordered events: a parent event and possibly some children. Both will be grouped by a same id in one document in the index. The parent event will be written to the "parent" field and the… --- ## [Unable to parse datetime from log message](https://discuss.elastic.co/t/unable-to-parse-datetime-from-log-message/165490)
**Author:** [@cganesan](https://discuss.elastic.co/u/cganesan)\ **Replies:** 4\ **Last updated:** [January 24, 2019, 3:24pm UTC](https://discuss.elastic.co/t/unable-to-parse-datetime-from-log-message/165490 "2019-01-24T15:24:16Z")
I'm unable to extract the datetime field from the log streamed from filebeat to elsaticsearch via logstash. I'm using date filter to extract the timestamp from the log message and set it to @timestamp. Since I already sp… --- ## [Grok filter not working properly](https://discuss.elastic.co/t/grok-filter-not-working-properly/165618)
**Author:** [@MiddlewareTeam](https://discuss.elastic.co/u/MiddlewareTeam)\ **Replies:** 2\ **Last updated:** [January 24, 2019, 3:24pm UTC](https://discuss.elastic.co/t/grok-filter-not-working-properly/165618 "2019-01-24T15:24:16Z")
\ \[ "message" , "%{IP:access-ip1} %{IP:access-ip2} - - \[%{NOTSPACE:access-timestamp} +%{INT}\] "%{WORD:access-httpmethod} %{NOTSPACE:access-request} %{WORD:acc ess-protocol}/%{NUMBER:access-pro… --- ## [How to separate fields of a log on logstash with beats](https://discuss.elastic.co/t/how-to-separate-fields-of-a-log-on-logstash-with-beats/161818)
**Author:** [@Askia\_Mohamed\_Kadri](https://discuss.elastic.co/u/Askia_Mohamed_Kadri)\ **Replies:** 2\ **Last updated:** [January 24, 2019, 3:18pm UTC](https://discuss.elastic.co/t/how-to-separate-fields-of-a-log-on-logstash-with-beats/161818 "2019-01-24T15:18:35Z")
Hi everybody, I'm trying to handle a multiline stacktrace using Filebeat and Logstash. I found the good pattern to use on Filebeat, but now I don't know how to separate the fields on Logstash after shipping the logs on … --- ## [GEOIP woes](https://discuss.elastic.co/t/geoip-woes/165467)
**Author:** [@nsutton](https://discuss.elastic.co/u/nsutton)\ **Replies:** 6\ **Last updated:** [January 24, 2019, 2:52pm UTC](https://discuss.elastic.co/t/geoip-woes/165467 "2019-01-24T14:52:26Z")
I've trawled through post after post about GEOIP to on avail. I think my problem might lie in a minimul understanding of mappings. I have a fortigate firewall that is sending its logs via syslog to logstash. This all … --- ## [Test multiline codec](https://discuss.elastic.co/t/test-multiline-codec/164451)
**Author:** [@imaad](https://discuss.elastic.co/u/imaad)\ **Replies:** 2\ **Last updated:** [January 24, 2019, 2:04pm UTC](https://discuss.elastic.co/t/test-multiline-codec/164451 "2019-01-24T14:04:50Z")
Hello, This is my configuration file : input { file { path =\> "/usr/share/logstash/testmultiline.log" start\_position =\> "beginning" #type =\> "log" sincedb\_path =\> "/dev/null" codec =\> multiline { … --- ## [Host field missing on received documents using syslog input](https://discuss.elastic.co/t/host-field-missing-on-received-documents-using-syslog-input/165585)
**Author:** [@victor.nilsson](https://discuss.elastic.co/u/victor.nilsson)\ **Replies:** 1\ **Last updated:** [January 24, 2019, 1:55pm UTC](https://discuss.elastic.co/t/host-field-missing-on-received-documents-using-syslog-input/165585 "2019-01-24T13:55:13Z")
Hi, I have the following logstash input: input { udp { port =\> 5514 tags =\> "syslog" id =\> "syslog-udp" } tcp { port =\> 5514 tags =\> "syslog" id =\> "syslog-tc… --- ## [Parsing an array of KV pairs](https://discuss.elastic.co/t/parsing-an-array-of-kv-pairs/165591)
**Author:** [@Ash\_Powell](https://discuss.elastic.co/u/Ash_Powell)\ **Replies:** 1\ **Last updated:** [January 24, 2019, 1:49pm UTC](https://discuss.elastic.co/t/parsing-an-array-of-kv-pairs/165591 "2019-01-24T13:49:04Z")
I am trying to parse some new data through Logstash, it comes through as an array of KV pairs "custom.fields": \[ { "Field\_A": "Value A" }, { "Field\_B": "Value B" }, { "Field\_C": "Value C" }, { "Field\_D": "Val… --- ## [Output to multiple sources is causing missing logs](https://discuss.elastic.co/t/output-to-multiple-sources-is-causing-missing-logs/165108)
**Author:** [@Hamil](https://discuss.elastic.co/u/Hamil)\ **Replies:** 2\ **Last updated:** [January 24, 2019, 1:48pm UTC](https://discuss.elastic.co/t/output-to-multiple-sources-is-causing-missing-logs/165108 "2019-01-24T13:48:00Z")
I have a conf file that inputs different files and assigns them a \[type\] so that they can be accessed individually with the filter and output sections of the conf file. Here is my conf file: As you can see, I am send… --- ## [Could not get aggregate filter to... well, aggregate](https://discuss.elastic.co/t/could-not-get-aggregate-filter-to-well-aggregate/165461)
**Author:** [@ffknob](https://discuss.elastic.co/u/ffknob)\ **Replies:** 5\ **Last updated:** [January 24, 2019, 1:27pm UTC](https://discuss.elastic.co/t/could-not-get-aggregate-filter-to-well-aggregate/165461 "2019-01-24T13:27:36Z")
I have a collection of XML file, each one with one event... Most of them are "parent" events, but some of them are "child" events related to some parent. I'd like to aggregate them, creating a "nfe" field for the parent … --- ## [Mutate, copy, destination field name expansion](https://discuss.elastic.co/t/mutate-copy-destination-field-name-expansion/165428)
**Author:** [@ffknob](https://discuss.elastic.co/u/ffknob)\ **Replies:** 3\ **Last updated:** [January 24, 2019, 1:19pm UTC](https://discuss.elastic.co/t/mutate-copy-destination-field-name-expansion/165428 "2019-01-24T13:19:21Z")
I'm trying to copy a field to a nested one with mutate-copy with a dynamic destination field name being (e. g. not hard coded in the pipeline conf). The thing is, the mutate-copy operation doesn't seens to expand the des… --- ## [\[ERROR\] Unable to retrieve license information](https://discuss.elastic.co/t/error-unable-to-retrieve-license-information/165609)
**Author:** [@bigdamhero](https://discuss.elastic.co/u/bigdamhero)\ **Replies:** 0\ **Last updated:** [January 24, 2019, 1:11pm UTC](https://discuss.elastic.co/t/error-unable-to-retrieve-license-information/165609 "2019-01-24T13:11:39Z")
I'm hoping for some insights on where to look for the error. I think if logstash could talk to my elastic cluster it would get the licence information. I'm not sure why that matters as Logstash is supposed to be free to… --- ## [Import json files to elasticsearch using logstash](https://discuss.elastic.co/t/import-json-files-to-elasticsearch-using-logstash/165608)
**Author:** [@itsron143](https://discuss.elastic.co/u/itsron143)\ **Replies:** 0\ **Last updated:** [January 24, 2019, 1:08pm UTC](https://discuss.elastic.co/t/import-json-files-to-elasticsearch-using-logstash/165608 "2019-01-24T13:08:11Z")
i have around 300000 json files in a single folder and I want to index them to elasticsearch. How do I do this using the bulk api efficiently or How do I use logstash to pass the folder name as file input plugin, json a… --- ## [Multiple inputs on the same port](https://discuss.elastic.co/t/multiple-inputs-on-the-same-port/165391)
**Author:** [@hoff](https://discuss.elastic.co/u/hoff)\ **Replies:** 4\ **Last updated:** [January 24, 2019, 11:45am UTC](https://discuss.elastic.co/t/multiple-inputs-on-the-same-port/165391 "2019-01-24T11:45:03Z")
I have five servers who generate 5 other jsons with two recognize field type (example value server1) and source (example value server\_name). My question is whether I can send data of five server on one input port and re… --- ## [How all ntopng flow fields gets restricted while passing through Logstash](https://discuss.elastic.co/t/how-all-ntopng-flow-fields-gets-restricted-while-passing-through-logstash/165582)
**Author:** [@swathikothapu](https://discuss.elastic.co/u/swathikothapu)\ **Replies:** 1\ **Last updated:** [January 24, 2019, 11:18am UTC](https://discuss.elastic.co/t/how-all-ntopng-flow-fields-gets-restricted-while-passing-through-logstash/165582 "2019-01-24T11:18:15Z")
Identify and capture all ntopng flow fields through lua script and also analyze how all ntopng flow fields gets restricted while passing through Logstash --- ## [Logs sending to logstash not showing in any index](https://discuss.elastic.co/t/logs-sending-to-logstash-not-showing-in-any-index/165556)
**Author:** [@nitinamrutkar](https://discuss.elastic.co/u/nitinamrutkar)\ **Replies:** 3\ **Last updated:** [January 24, 2019, 10:33am UTC](https://discuss.elastic.co/t/logs-sending-to-logstash-not-showing-in-any-index/165556 "2019-01-24T10:33:17Z")
Hi Team , I am sending my application logs from windows VM using filebeat to logstash and then to elastic search. We are using custom logstash config and I am trying to send my filebeat logs in category server logs but … --- ## [Single line xml using logstash](https://discuss.elastic.co/t/single-line-xml-using-logstash/165569)
**Author:** [@Alaoui](https://discuss.elastic.co/u/Alaoui)\ **Replies:** 0\ **Last updated:** [January 24, 2019, 9:58am UTC](https://discuss.elastic.co/t/single-line-xml-using-logstash/165569 "2019-01-24T09:58:53Z")
hi , how can i read just a single line on xml file using logstash ???? --- ## [Configure kv plugin to handle multi line messages](https://discuss.elastic.co/t/configure-kv-plugin-to-handle-multi-line-messages/165549)
**Author:** [@yasko](https://discuss.elastic.co/u/yasko)\ **Replies:** 2\ **Last updated:** [January 24, 2019, 10:15am UTC](https://discuss.elastic.co/t/configure-kv-plugin-to-handle-multi-line-messages/165549 "2019-01-24T10:15:51Z")
I'm collecting logs using filebeat and sending them to logstash for processing. The log format is: timestamp: 2019-01-24T07:04:42.322Z, level: INFO, message: something happened In logstash I've setup the kv plugin to … --- ## [Most efficient grok approach for ISE logs](https://discuss.elastic.co/t/most-efficient-grok-approach-for-ise-logs/165083)
**Author:** [@MarcDAFrame](https://discuss.elastic.co/u/MarcDAFrame)\ **Replies:** 2\ **Last updated:** [January 24, 2019, 9:10am UTC](https://discuss.elastic.co/t/most-efficient-grok-approach-for-ise-logs/165083 "2019-01-24T09:10:58Z")
grok{match =\> {"message" =\> "User-Name=%{DATA:username}\[,;\]"}} grok{match =\> {"message" =\> "NAS-IP-Address=%{IP:nas\_ip}\[,;\]"}} grok{match =\> {"message" =\> "NAS-Port=%{NUMBER:nas\_port}\[,;\]"}} grok{match =\> {"message"… --- ## [Cisco ISE Log logstash Profile](https://discuss.elastic.co/t/cisco-ise-log-logstash-profile/165560)
**Author:** [@xbug](https://discuss.elastic.co/u/xbug)\ **Replies:** 0\ **Last updated:** [January 24, 2019, 9:08am UTC](https://discuss.elastic.co/t/cisco-ise-log-logstash-profile/165560 "2019-01-24T09:08:26Z")
Because I am not very skilled, consult the CISCO ISE log, how to write the logstash configuration file of the format log。 --- ## [KV filter don't split on field\_split\_pattern once](https://discuss.elastic.co/t/kv-filter-dont-split-on-field-split-pattern-once/165431)
**Author:** [@lyubava](https://discuss.elastic.co/u/lyubava)\ **Replies:** 4\ **Last updated:** [January 24, 2019, 7:43am UTC](https://discuss.elastic.co/t/kv-filter-dont-split-on-field-split-pattern-once/165431 "2019-01-24T07:43:58Z")
Hello, I am trying to use KV filter for my task. It works almost correctly, but i have one mistake in output result. Here is the input logs, which i want to analyze: sid: S-1-5-21-2526317496-1893015908-99665470-1000, b… --- ## [Custom grok pattern file parsing error](https://discuss.elastic.co/t/custom-grok-pattern-file-parsing-error/165493)
**Author:** [@cganesan](https://discuss.elastic.co/u/cganesan)\ **Replies:** 2\ **Last updated:** [January 24, 2019, 1:16am UTC](https://discuss.elastic.co/t/custom-grok-pattern-file-parsing-error/165493 "2019-01-24T01:16:44Z")
I have a custom grok pattern file with following regex patterns. I successfully tested these using regex online tool and it works fine however logstash fails to load the custom patterns file - see below. Appreciate your … --- ## [Logstash netflow module - no logstash.yml?](https://discuss.elastic.co/t/logstash-netflow-module-no-logstash-yml/165505)
**Author:** [@Ross\_T](https://discuss.elastic.co/u/Ross_T)\ **Replies:** 0\ **Last updated:** [January 24, 2019, 1:16am UTC](https://discuss.elastic.co/t/logstash-netflow-module-no-logstash-yml/165505 "2019-01-24T01:16:37Z")
Hello, I recently installed the elastic stack following the information on the site. Debian packages following the instructions; 1,2 and 3 https://www.elastic.co/guide/en/elastic-stack/current/installing-elastic-stac… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=611) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=613)