# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=422 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 423 --- ## [(NoMethodError) undefined method \`deep\_merge'?](https://discuss.elastic.co/t/nomethoderror-undefined-method-deep-merge/214367)
**Author:** [@Kenneth\_M\_Kolano](https://discuss.elastic.co/u/Kenneth_M_Kolano)\ **Replies:** 6\ **Last updated:** [January 9, 2020, 8:37pm UTC](https://discuss.elastic.co/t/nomethoderror-undefined-method-deep-merge/214367 "2020-01-09T20:37:56Z")
I'm unexpectedly seeing this error when starting my Logstash 7.5.1 service on a new deployment on Ubuntu 19.04, and can find no relevant info online. \[ERROR\] 2020-01-09 08:18:37.084 \[main\] Logstash - java.lang.IllegalSt… --- ## [Problem with reading aggregation map file](https://discuss.elastic.co/t/problem-with-reading-aggregation-map-file/213375)
**Author:** [@lens](https://discuss.elastic.co/u/lens)\ **Replies:** 1\ **Last updated:** [January 9, 2020, 7:48pm UTC](https://discuss.elastic.co/t/problem-with-reading-aggregation-map-file/213375 "2020-01-09T19:48:39Z")
I'm using the aggregate plugin and currently testing to see if using the aggregate\_maps\_path option works to write the aggregation maps to a file when I stop logstash and read them when I restart it. I'm getting these e… --- ## [Help needed in grok](https://discuss.elastic.co/t/help-needed-in-grok/213827)
**Author:** [@adeel109](https://discuss.elastic.co/u/adeel109)\ **Replies:** 4\ **Last updated:** [January 9, 2020, 5:35pm UTC](https://discuss.elastic.co/t/help-needed-in-grok/213827 "2020-01-09T17:35:12Z")
I've been searching this and am directed to the same page most of the time. I am trying to write grok patterns for my custom logs. For example, below is a log line: 2020/01/02 08:40:16 UUID: 5E82093B:7550\_B0092619:01BB… --- ## [Logstash parsing problem](https://discuss.elastic.co/t/logstash-parsing-problem/214379)
**Author:** [@Vladpov](https://discuss.elastic.co/u/Vladpov)\ **Replies:** 4\ **Last updated:** [January 9, 2020, 7:01pm UTC](https://discuss.elastic.co/t/logstash-parsing-problem/214379 "2020-01-09T19:01:33Z")
Hello guys! I'm trying to parse the following type of log message: 111.22.333.444 - - \[08/Jan/2020:11:50:15 +0100\] \[https://awdasfe.asfeaf.cas:111\] "POST /VFQ3P/asfiheasfhe/v2/safiehjafe/check HTTP/1.1" 204 0 "-" "-" (… --- ## [TCP input plugin generate additional port field](https://discuss.elastic.co/t/tcp-input-plugin-generate-additional-port-field/214459)
**Author:** [@lantern77](https://discuss.elastic.co/u/lantern77)\ **Replies:** 2\ **Last updated:** [January 9, 2020, 7:00pm UTC](https://discuss.elastic.co/t/tcp-input-plugin-generate-additional-port-field/214459 "2020-01-09T19:00:12Z")
Hello, For some reason, I can't seem to figure out why, but whenever we use the tcp input plugin, an addition "port" field gets added to our documents. For example "port": 39750. Can anyone confirm if this is default b… --- ## [Using Upsert across index](https://discuss.elastic.co/t/using-upsert-across-index/214465)
**Author:** [@smo](https://discuss.elastic.co/u/smo)\ **Replies:** 4\ **Last updated:** [January 9, 2020, 5:22pm UTC](https://discuss.elastic.co/t/using-upsert-across-index/214465 "2020-01-09T17:22:27Z")
I use this config to insert document in ES :\`\`\` output { if \[type\] == "usage" { elasticsearch { hosts =\> \["elastic4:9204"\] index =\> "usage-%{+YYYY-MM}" document\_id =\> "%{IDSU}" action =\> "update" doc\_as\_upsert =\> … --- ## [Help with combining events from a log](https://discuss.elastic.co/t/help-with-combining-events-from-a-log/214441)
**Author:** [@chapmanjc](https://discuss.elastic.co/u/chapmanjc)\ **Replies:** 1\ **Last updated:** [January 9, 2020, 5:40pm UTC](https://discuss.elastic.co/t/help-with-combining-events-from-a-log/214441 "2020-01-09T17:40:47Z")
Hello all, I am a total newbie to the ELK Stack (Please be kind!) and I am hoping to get some assistance. Essentially what I am wanting to do is when I go into Discovery in Kibana, it takes a "sessionID" and combines i… --- ## [Type string in a multiple input Http\_Poller for different Grok pattern](https://discuss.elastic.co/t/type-string-in-a-multiple-input-http-poller-for-different-grok-pattern/211481)
**Author:** [@kamalesh](https://discuss.elastic.co/u/kamalesh)\ **Replies:** 3\ **Last updated:** [January 9, 2020, 5:10pm UTC](https://discuss.elastic.co/t/type-string-in-a-multiple-input-http-poller-for-different-grok-pattern/211481 "2020-01-09T17:10:10Z")
Hi All, I am trying to using multiple Url's for the http\_poller and need to use a different Grok patterns for each of the Url's I am aware that i can make use of type string in the Input plugins and make Grok patterns … --- ## [Not able to write in file with particular tag](https://discuss.elastic.co/t/not-able-to-write-in-file-with-particular-tag/214440)
**Author:** [@srbhklkrn](https://discuss.elastic.co/u/srbhklkrn)\ **Replies:** 3\ **Last updated:** [January 9, 2020, 4:46pm UTC](https://discuss.elastic.co/t/not-able-to-write-in-file-with-particular-tag/214440 "2020-01-09T16:46:40Z")
I'm trying to write all logs with tag Prod to file but it's not working. When I remove tag condition it works but not with tags. Any inputs? Here is my pipeline: input { tcp { port =\> 10514 type =\> syslog… --- ## [Persisted queue and Zabbix Ouput Plugin](https://discuss.elastic.co/t/persisted-queue-and-zabbix-ouput-plugin/214395)
**Author:** [@Halgrim](https://discuss.elastic.co/u/Halgrim)\ **Replies:** 1\ **Last updated:** [January 9, 2020, 3:58pm UTC](https://discuss.elastic.co/t/persisted-queue-and-zabbix-ouput-plugin/214395 "2020-01-09T15:58:10Z")
Hello! I'm using logstash with a persisted queue option. And it works fine when I using elasticsearch as output plugin. (If the connection with ES is lost messages goes to persisted queue and when the connection is up a… --- ## [Index Templates for older filebeat indices...how?](https://discuss.elastic.co/t/index-templates-for-older-filebeat-indices-how/214461)
**Author:** [@hueyg](https://discuss.elastic.co/u/hueyg)\ **Replies:** 0\ **Last updated:** [January 9, 2020, 3:52pm UTC](https://discuss.elastic.co/t/index-templates-for-older-filebeat-indices-how/214461 "2020-01-09T15:52:21Z")
I migrated to a 7.4 cluster from a stand alone 5.x single test instance. I may have carried over some legacy bad habits so to speak. Attempting to implement ILM policies but exposing other issues. I followed the same … --- ## [Logstash 7.5.1 High CPU Usage](https://discuss.elastic.co/t/logstash-7-5-1-high-cpu-usage/214120)
**Author:** [@chris\_g](https://discuss.elastic.co/u/chris_g)\ **Replies:** 6\ **Last updated:** [January 9, 2020, 3:46pm UTC](https://discuss.elastic.co/t/logstash-7-5-1-high-cpu-usage/214120 "2020-01-09T15:46:02Z")
I recently upgraded Elastic from 7.0.1 to 7.5.1. Elasticsearch and Kibana had no issues but I am having some trouble with Logstash. When I run Logstash 7.0.1, no issues. Everything works as normal. Then, I upgraded to 7… --- ## [What security layer logstash uses](https://discuss.elastic.co/t/what-security-layer-logstash-uses/214021)
**Author:** [@Gauti](https://discuss.elastic.co/u/Gauti)\ **Replies:** 3\ **Last updated:** [January 9, 2020, 3:19pm UTC](https://discuss.elastic.co/t/what-security-layer-logstash-uses/214021 "2020-01-09T15:19:34Z")
HI All, We are using logstash to transfer data from our source to elasticsearch, just wondering when data is transferred is there any kind of encryption happens to send the data securely? Thanks Gauti --- ## [Logstash aggregate - JDBC SQL Relations](https://discuss.elastic.co/t/logstash-aggregate-jdbc-sql-relations/214424)
**Author:** [@Ahmed\_Ahmed](https://discuss.elastic.co/u/Ahmed_Ahmed)\ **Replies:** 0\ **Last updated:** [January 9, 2020, 12:02pm UTC](https://discuss.elastic.co/t/logstash-aggregate-jdbc-sql-relations/214424 "2020-01-09T12:02:36Z")
Hi, I have been following your responses and found them always helpful and spot on. I would like to ask With reference to the following where there is a parent-child relation is pipelined Logstash aggregate-example4 I… --- ## [Indexing csv with header in elasticsearch](https://discuss.elastic.co/t/indexing-csv-with-header-in-elasticsearch/214229)
**Author:** [@johnkary](https://discuss.elastic.co/u/johnkary)\ **Replies:** 8\ **Last updated:** [January 9, 2020, 10:41am UTC](https://discuss.elastic.co/t/indexing-csv-with-header-in-elasticsearch/214229 "2020-01-09T10:41:40Z")
Hi all! I' m using Elasticsearch 6.6 and Kibana 6.4 installed on my Google Cloud account. Here is my issue: I have a folder there where csv logs arrive from some IoT devices and with a Logstash pipeline that i created, … --- ## [Multiple logstash servers - unique pipeline name per server?](https://discuss.elastic.co/t/multiple-logstash-servers-unique-pipeline-name-per-server/214417)
**Author:** [@probson](https://discuss.elastic.co/u/probson)\ **Replies:** 0\ **Last updated:** [January 9, 2020, 10:39am UTC](https://discuss.elastic.co/t/multiple-logstash-servers-unique-pipeline-name-per-server/214417 "2020-01-09T10:39:12Z")
Hi, I have 2 logstash servers with the same pipeline, all of the beats are going to 1 server at the moment, if I go into stack monitoring I see 1 pipeline (main) from there I want to see the pipeline and the performance… --- ## [Logstash 7.4.0 "grok " filter not work or might some problem with Logstash](https://discuss.elastic.co/t/logstash-7-4-0-grok-filter-not-work-or-might-some-problem-with-logstash/214408)
**Author:** [@Krunal\_Patel](https://discuss.elastic.co/u/Krunal_Patel)\ **Replies:** 0\ **Last updated:** [January 9, 2020, 10:13am UTC](https://discuss.elastic.co/t/logstash-7-4-0-grok-filter-not-work-or-might-some-problem-with-logstash/214408 "2020-01-09T10:13:01Z")
Here, I have tried to send MySQL's error log data through filebeat 7.4.0 to Logstash 7.4.0 and after filter, the required information, send these all data to Elastic and then viewing through Kibana. There has some prob… --- ## [Logstash Aggregate filter plugin works in 7.5.0, not in 6.6.1](https://discuss.elastic.co/t/logstash-aggregate-filter-plugin-works-in-7-5-0-not-in-6-6-1/214260)
**Author:** [@bjosve](https://discuss.elastic.co/u/bjosve)\ **Replies:** 4\ **Last updated:** [January 9, 2020, 8:26am UTC](https://discuss.elastic.co/t/logstash-aggregate-filter-plugin-works-in-7-5-0-not-in-6-6-1/214260 "2020-01-09T08:26:31Z")
Hi, I have a problem with Logstash Aggregate filter. The exact same filter works in Logstash 7.5.0 but not in Logstash 6.6.1 and I can't find out why. The Aggregate filter version in 7.5.0 is 2.9.1 and in 6.6.1 it is 2.… --- ## [Read csv file automatically without using 'logstash -f'](https://discuss.elastic.co/t/read-csv-file-automatically-without-using-logstash-f/214298)
**Author:** [@kumar4](https://discuss.elastic.co/u/kumar4)\ **Replies:** 5\ **Last updated:** [January 9, 2020, 7:09am UTC](https://discuss.elastic.co/t/read-csv-file-automatically-without-using-logstash-f/214298 "2020-01-09T07:09:19Z")
hello, i have a csv file that i view on kibana (i have already created an index), the problem is that every time i have to type the command 'logstash -f myfile.csv', I want to automate this task, that is to say that the… --- ## [Multiple logstash instance need to run on window 10](https://discuss.elastic.co/t/multiple-logstash-instance-need-to-run-on-window-10/214198)
**Author:** [@Pramod\_Thakur](https://discuss.elastic.co/u/Pramod_Thakur)\ **Replies:** 2\ **Last updated:** [January 9, 2020, 6:59am UTC](https://discuss.elastic.co/t/multiple-logstash-instance-need-to-run-on-window-10/214198 "2020-01-09T06:59:31Z")
Hi Folks, On my local machine logstash is working fine when I am running only one logstash pipeline, but when I am trying to run second logstash pipeline (config file) it says that Logstash could not be started becaus… --- ## [Logstash Sql Server](https://discuss.elastic.co/t/logstash-sql-server/213979)
**Author:** [@Syed.Ubaid](https://discuss.elastic.co/u/Syed.Ubaid)\ **Replies:** 5\ **Last updated:** [January 9, 2020, 4:37am UTC](https://discuss.elastic.co/t/logstash-sql-server/213979 "2020-01-09T04:37:17Z")
I have integrate elk stack with sql server. When i run logstash to parse the data the query parse the data and logstash service shutdown after parsing the records. Is there any way around that the service will run in the… --- ## [Seeing io.netty.util.internal.OutOfDirectMemoryError in my logtash logs](https://discuss.elastic.co/t/seeing-io-netty-util-internal-outofdirectmemoryerror-in-my-logtash-logs/214334)
**Author:** [@kommineni24](https://discuss.elastic.co/u/kommineni24)\ **Replies:** 0\ **Last updated:** [January 9, 2020, 2:16am UTC](https://discuss.elastic.co/t/seeing-io-netty-util-internal-outofdirectmemoryerror-in-my-logtash-logs/214334 "2020-01-09T02:16:27Z")
Hi, Currently I am using 6.4.2 version of Elastic Stack. Started seeing our logstash docker container exited with code 1. Upon troubleshooting logstash logs notices erros releated to memory. Exited with Status code 1 … --- ## [How to auto-set zabbix\_host?](https://discuss.elastic.co/t/how-to-auto-set-zabbix-host/213961)
**Author:** [@Philip\_Brown](https://discuss.elastic.co/u/Philip_Brown)\ **Replies:** 17\ **Last updated:** [January 8, 2020, 9:13pm UTC](https://discuss.elastic.co/t/how-to-auto-set-zabbix-host/213961 "2020-01-08T21:13:01Z")
I'm trying to set up a logstash-\> zabbix gateway, that collects inputs from multiple servers running filebeat. I gather that zabbix\_host is a mandatory field. It also seems like it would be set to the ACTUAL generating… --- ## [Cannot write to a field alias \[user.domain\]](https://discuss.elastic.co/t/cannot-write-to-a-field-alias-user-domain/213944)
**Author:** [@Tony\_Chirillo](https://discuss.elastic.co/u/Tony_Chirillo)\ **Replies:** 1\ **Last updated:** [January 8, 2020, 9:08pm UTC](https://discuss.elastic.co/t/cannot-write-to-a-field-alias-user-domain/213944 "2020-01-08T21:08:33Z")
I recently installed a single instance of Winlogbeat 7.5.0 on a host to test log ingestion from the 7.5.0 beater version into Logstash and then into our production Elasticsearch environment. Our Elastic environment is on… --- ## [Logstash file output](https://discuss.elastic.co/t/logstash-file-output/214288)
**Author:** [@rahul\_cheekoti](https://discuss.elastic.co/u/rahul_cheekoti)\ **Replies:** 1\ **Last updated:** [January 8, 2020, 7:38pm UTC](https://discuss.elastic.co/t/logstash-file-output/214288 "2020-01-08T19:38:46Z")
Hi all My Logstash file generates a output file every day . It will modified for every few minutes. But when I check the last modified date of file it is updating once the file is created and when new file is create… --- ## [Logstash-core-plugin-api gem compatibility to 7.x](https://discuss.elastic.co/t/logstash-core-plugin-api-gem-compatibility-to-7-x/214295)
**Author:** [@pawankt](https://discuss.elastic.co/u/pawankt)\ **Replies:** 0\ **Last updated:** [January 8, 2020, 7:38pm UTC](https://discuss.elastic.co/t/logstash-core-plugin-api-gem-compatibility-to-7-x/214295 "2020-01-08T19:38:42Z")
Hi, Issue noticed for api gem compatibility which still looking older logstash core. I was tried to place 5.6.4 gem in vendor/cache still having problem. If we can sort logstash-core-plugin-api to 7.x i can rebuild with … --- ## [Cant access fields from log using query template for Elasticsearch filter in Logstash](https://discuss.elastic.co/t/cant-access-fields-from-log-using-query-template-for-elasticsearch-filter-in-logstash/214291)
**Author:** [@james.garside](https://discuss.elastic.co/u/james.garside)\ **Replies:** 0\ **Last updated:** [January 8, 2020, 6:20pm UTC](https://discuss.elastic.co/t/cant-access-fields-from-log-using-query-template-for-elasticsearch-filter-in-logstash/214291 "2020-01-08T18:20:42Z")
Cant access fields from log using query template I have an instance of logstash which I want to use to enrich a field on incomming logs by querying elasticsearch then add the result to the recieved log. I have configure… --- ## [DMARC XML FIle](https://discuss.elastic.co/t/dmarc-xml-file/214248)
**Author:** [@Wilks](https://discuss.elastic.co/u/Wilks)\ **Replies:** 2\ **Last updated:** [January 8, 2020, 5:09pm UTC](https://discuss.elastic.co/t/dmarc-xml-file/214248 "2020-01-08T17:09:33Z")
Hi All, I am trying to send dmarc logs to elasticsearch and running into some issues with the logstash config. Logstash only appears to process the first few lines of the xml log file as per below. It seems to be s… --- ## [How to forward logstash logs to Rapid7?](https://discuss.elastic.co/t/how-to-forward-logstash-logs-to-rapid7/214141)
**Author:** [@srbhklkrn](https://discuss.elastic.co/u/srbhklkrn)\ **Replies:** 6\ **Last updated:** [January 8, 2020, 4:15pm UTC](https://discuss.elastic.co/t/how-to-forward-logstash-logs-to-rapid7/214141 "2020-01-08T16:15:20Z")
I'm trying to forward logs using Logstash to Rapid7 SIEM tool with particular tags, below is the output plugin but it's not working, what am I doing wrong? output { if \[type\] == "syslog" { elasticsearch { h… --- ## [Logtash ignores some lines randomly](https://discuss.elastic.co/t/logtash-ignores-some-lines-randomly/214259)
**Author:** [@JustDevZero](https://discuss.elastic.co/u/JustDevZero)\ **Replies:** 1\ **Last updated:** [January 8, 2020, 3:18pm UTC](https://discuss.elastic.co/t/logtash-ignores-some-lines-randomly/214259 "2020-01-08T15:18:41Z")
Logtash apparently ignores some of my log lines done by nginx. I'm trying to parse my customized nginx logs with logstash, I'm adding the response time by upstream, etc... and a few more fields that aren't normally on t… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=421) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=423)