# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=785 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 786 --- ## [Merge events](https://discuss.elastic.co/t/merge-events/126994)
**Author:** [@las](https://discuss.elastic.co/u/las)\ **Replies:** 1\ **Last updated:** [April 6, 2018, 7:45pm UTC](https://discuss.elastic.co/t/merge-events/126994 "2018-04-06T19:45:29Z")
I am using logstash to merge events - how do I stop individual events from going to elastic search and only have aggregated ones go? Am using grok and aggregate plugins. Aggregate is configured with timeout. --- ## [Add field for a group of events](https://discuss.elastic.co/t/add-field-for-a-group-of-events/127122)
**Author:** [@las](https://discuss.elastic.co/u/las)\ **Replies:** 2\ **Last updated:** [April 6, 2018, 7:44pm UTC](https://discuss.elastic.co/t/add-field-for-a-group-of-events/127122 "2018-04-06T19:44:19Z")
For a group of events, I want to add a unique identifier. I am using aggregate plugin. The problem is that I don't want additional events from the aggregate filter and I just want to use aggregate for grouping the events… --- ## [Support list for hosts environment variable](https://discuss.elastic.co/t/support-list-for-hosts-environment-variable/126961)
**Author:** [@kpe](https://discuss.elastic.co/u/kpe)\ **Replies:** 1\ **Last updated:** [April 6, 2018, 7:44pm UTC](https://discuss.elastic.co/t/support-list-for-hosts-environment-variable/126961 "2018-04-06T19:44:18Z")
Hello, Any news about this issue =\> https://github.com/elastic/logstash/issues/8157 ? I can reproduce it on LS 5.6.3. Regards, Kevin --- ## [XML to Elasticsearch through Logstash](https://discuss.elastic.co/t/xml-to-elasticsearch-through-logstash/126830)
**Author:** [@seungtae](https://discuss.elastic.co/u/seungtae)\ **Replies:** 2\ **Last updated:** [April 6, 2018, 7:27pm UTC](https://discuss.elastic.co/t/xml-to-elasticsearch-through-logstash/126830 "2018-04-06T19:27:47Z")
I'm trying to parse in XML source which is a few 800 lines, and looks like this : \ \ **Author:** [@Sara\_Hemmi](https://discuss.elastic.co/u/Sara_Hemmi)\ **Replies:** 1\ **Last updated:** [April 6, 2018, 3:54pm UTC](https://discuss.elastic.co/t/cant-parse-my-data-logs-getting-grokparsefailure/127056 "2018-04-06T15:54:17Z") Hello , any body can help me please to parse this log file (https\_access files from apache tomcat) , i've try with many patterns but i always get \["\_grokparsefailure"\] Those are two lines from my data logs : 172.23.10.… --- ## [Trigger email event(output) on file input which having csv filter](https://discuss.elastic.co/t/trigger-email-event-output-on-file-input-which-having-csv-filter/126673)
**Author:** [@adg](https://discuss.elastic.co/u/adg)\ **Replies:** 6\ **Last updated:** [April 6, 2018, 12:55pm UTC](https://discuss.elastic.co/t/trigger-email-event-output-on-file-input-which-having-csv-filter/126673 "2018-04-06T12:55:54Z")
while using logstash i stuck for sending email on event when i found list of specific expression in input file which is filter by csv plugin --- ## [Extract field from source](https://discuss.elastic.co/t/extract-field-from-source/126984)
**Author:** [@las](https://discuss.elastic.co/u/las)\ **Replies:** 6\ **Last updated:** [April 6, 2018, 12:50pm UTC](https://discuss.elastic.co/t/extract-field-from-source/126984 "2018-04-06T12:50:03Z")
The event from filebeat contains the source field which contains the log filename. How do I use grok to extract some data from it? --- ## [How to convert uuid into string in logstash](https://discuss.elastic.co/t/how-to-convert-uuid-into-string-in-logstash/127069)
**Author:** [@aarthinim](https://discuss.elastic.co/u/aarthinim)\ **Replies:** 4\ **Last updated:** [April 6, 2018, 12:38pm UTC](https://discuss.elastic.co/t/how-to-convert-uuid-into-string-in-logstash/127069 "2018-04-06T12:38:09Z")
My table structure is create table dev.emp ( empid int, emp\_first varchar, emp\_last varchar, empno uuid primary key, ); Logstash configuration is input { jdbc{ jdbc\_connection\_string =\> "jdbc:cassandra://local… --- ## [Parse the middle of a loog](https://discuss.elastic.co/t/parse-the-middle-of-a-loog/127004)
**Author:** [@Sebastian\_Herrera](https://discuss.elastic.co/u/Sebastian_Herrera)\ **Replies:** 1\ **Last updated:** [April 6, 2018, 12:27pm UTC](https://discuss.elastic.co/t/parse-the-middle-of-a-loog/127004 "2018-04-06T12:27:38Z")
hello i am new with the filtering, i am having a log and i want to parse it but i am having an issue. My log is Mar 23 11:59:11 CLL21DCIDS01-IDS snort\[26109\]: \[137:1:2\] (spp\_ssl) Invalid Client HELLO after Server HELLO … --- ## [How to pass temporary data from filter to output in logstash](https://discuss.elastic.co/t/how-to-pass-temporary-data-from-filter-to-output-in-logstash/126903)
**Author:** [@EloaCardoso](https://discuss.elastic.co/u/EloaCardoso)\ **Replies:** 10\ **Last updated:** [April 6, 2018, 10:29am UTC](https://discuss.elastic.co/t/how-to-pass-temporary-data-from-filter-to-output-in-logstash/126903 "2018-04-06T10:29:39Z")
How can I pass temporary data from a filter to output in logstash? I did like this: filter { mutate { add\_field =\> { "\[@metadata\]\[TEMP\_DATA\]" =\> "%{custom\_index}" } } mutate { remove\_field =\> \["custom\_index"\] } } … --- ## [Error occurred while running the logstash installation command in windows](https://discuss.elastic.co/t/error-occurred-while-running-the-logstash-installation-command-in-windows/120230)
**Author:** [@sandy\_19](https://discuss.elastic.co/u/sandy_19)\ **Replies:** 3\ **Last updated:** [April 6, 2018, 10:28am UTC](https://discuss.elastic.co/t/error-occurred-while-running-the-logstash-installation-command-in-windows/120230 "2018-04-06T10:28:14Z")
Hi, I am new to ELK stack. I downloaded logstash, elasticsearch and kibana 6.2.1 versions. I am successful in installing elasticsearch and kibana in windows 10. But while installing logstash i am getting error. I execu… --- ## [Logstash 6.2 out of memory : suspects](https://discuss.elastic.co/t/logstash-6-2-out-of-memory-suspects/126798)
**Author:** [@anselme](https://discuss.elastic.co/u/anselme)\ **Replies:** 3\ **Last updated:** [April 6, 2018, 8:38am UTC](https://discuss.elastic.co/t/logstash-6-2-out-of-memory-suspects/126798 "2018-04-06T08:38:49Z")
Dear all, Any help welcome, since I did all I could but now I need specialists... I have 1 logstash instance fed by 1 filebeat and outputting to 1 ES. I get an OOM logstash crash after a few hours. I tried to increase… --- ## [XML Dynamic Parsing](https://discuss.elastic.co/t/xml-dynamic-parsing/126957)
**Author:** [@AloysiusParedes](https://discuss.elastic.co/u/AloysiusParedes)\ **Replies:** 8\ **Last updated:** [April 6, 2018, 8:24am UTC](https://discuss.elastic.co/t/xml-dynamic-parsing/126957 "2018-04-06T08:24:51Z")
Still new to ELK. I'd like to parse an XML with Logstash and output to Elasticsearch. XML: \ \O\ \false\ . . . \ **Author:** [@Masahiro\_Shimizu](https://discuss.elastic.co/u/Masahiro_Shimizu)\ **Replies:** 3\ **Last updated:** [April 6, 2018, 8:07am UTC](https://discuss.elastic.co/t/convert-timestamp-to-epoctime-with-milisecond/126899 "2018-04-06T08:07:37Z") Hi , this is my first question. I would like convert @timestamp to epoch-time with millisecond Because I want millisecond not second , I try check the latency through the messaging queue. Here is my code, ruby { co… --- ## [Problem with reading logs](https://discuss.elastic.co/t/problem-with-reading-logs/126727)
**Author:** [@pablo\_gracia](https://discuss.elastic.co/u/pablo_gracia)\ **Replies:** 4\ **Last updated:** [April 6, 2018, 8:00am UTC](https://discuss.elastic.co/t/problem-with-reading-logs/126727 "2018-04-06T08:00:40Z")
First of all sorry for my english I know it´s not the best one. Well so, I installed logstash, elasicsearch and kibana on my debian 8 in Virtual box, and i Hvae a WatchWard where im receiving the logs from. The problem… --- ## [Logstash 6.2.2 with JDBC data on not updated](https://discuss.elastic.co/t/logstash-6-2-2-with-jdbc-data-on-not-updated/126949)
**Author:** [@Ongun](https://discuss.elastic.co/u/Ongun)\ **Replies:** 5\ **Last updated:** [April 6, 2018, 7:54am UTC](https://discuss.elastic.co/t/logstash-6-2-2-with-jdbc-data-on-not-updated/126949 "2018-04-06T07:54:02Z")
Hello all, I have an ELK Stack 6.2.2 with X-pack trial license. I am running a multiple pipeline configuration with JDBC drivers and sometimes I notice that my data on Elasticsearch is not updated (particularly when mul… --- ## [How to rename elasticsearch columns?](https://discuss.elastic.co/t/how-to-rename-elasticsearch-columns/126930)
**Author:** [@dyl](https://discuss.elastic.co/u/dyl)\ **Replies:** 7\ **Last updated:** [April 6, 2018, 7:38am UTC](https://discuss.elastic.co/t/how-to-rename-elasticsearch-columns/126930 "2018-04-06T07:38:22Z")
Hey community, I just import CSV into ElasticSearch but I would like to know if I had possibility to rename these columns : \[image\] "Bloqué par le filtrage par réputation" : { "type" : "text", … --- ## [Logstash starting damn slow](https://discuss.elastic.co/t/logstash-starting-damn-slow/125708)
**Author:** [@asp](https://discuss.elastic.co/u/asp)\ **Replies:** 8\ **Last updated:** [April 6, 2018, 7:05am UTC](https://discuss.elastic.co/t/logstash-starting-damn-slow/125708 "2018-04-06T07:05:26Z")
Hi, I need some guidiance to speed up the starting of logstash - or at least I would like to know why it takes so long and if that is really ok. \[2018-03-27T07:18:48,795\]\[WARN \]\[logstash.outputs.elasticsearch\] Restored… --- ## [Problems with logstsah multiple pipelines](https://discuss.elastic.co/t/problems-with-logstsah-multiple-pipelines/127017)
**Author:** [@Robin\_Guo](https://discuss.elastic.co/u/Robin_Guo)\ **Replies:** 2\ **Last updated:** [April 6, 2018, 2:22am UTC](https://discuss.elastic.co/t/problems-with-logstsah-multiple-pipelines/127017 "2018-04-06T02:22:17Z")
Dear folks, Recently I'm trying to enable multiple pipelines with logstash version 6.2.3. but we got the problems from the logstash log as follows, I tested my pipline with version 5.5.x , It's working well with single… --- ## [Asterisk in Mutate with GSUB](https://discuss.elastic.co/t/asterisk-in-mutate-with-gsub/126300)
**Author:** [@Suman\_Reddy1](https://discuss.elastic.co/u/Suman_Reddy1)\ **Replies:** 7\ **Last updated:** [April 6, 2018, 1:33am UTC](https://discuss.elastic.co/t/asterisk-in-mutate-with-gsub/126300 "2018-04-06T01:33:36Z")
@magnusbaeck Here is the thing, which broken in our production. mutate { gsub =\> \["message", "%{docNumber}", "%{docNumberMasked}"\] } I have to type all the configuration, if there is any typo or syntactical error… --- ## [Adding further logging for flexlm](https://discuss.elastic.co/t/adding-further-logging-for-flexlm/127007)
**Author:** [@gmarquez](https://discuss.elastic.co/u/gmarquez)\ **Replies:** 0\ **Last updated:** [April 6, 2018, 12:12am UTC](https://discuss.elastic.co/t/adding-further-logging-for-flexlm/127007 "2018-04-06T00:12:42Z")
Hi, I was able to get logstash working with flexlm-license-elk (found on github). It does a great job of finding IN/OUT and providing data! Credit to sergioedo for this! However, I would like to add also DENIED & Warni… --- ## [Logstash lost connectivity to ElasticSearch](https://discuss.elastic.co/t/logstash-lost-connectivity-to-elasticsearch/127006)
**Author:** [@mrubin](https://discuss.elastic.co/u/mrubin)\ **Replies:** 0\ **Last updated:** [April 5, 2018, 11:33pm UTC](https://discuss.elastic.co/t/logstash-lost-connectivity-to-elasticsearch/127006 "2018-04-05T23:33:34Z")
My logstash server has been working fine for weeks. No-one has touched it. All of a sudden, it has lost connectivity to ElasticSearch. The log is filled with "Marking url as dead. Last error: \[LogStash::Outputs::Elastic… --- ## [Unable to start logstash service error code in log](https://discuss.elastic.co/t/unable-to-start-logstash-service-error-code-in-log/126812)
**Author:** [@heskez](https://discuss.elastic.co/u/heskez)\ **Replies:** 9\ **Last updated:** [April 5, 2018, 10:29pm UTC](https://discuss.elastic.co/t/unable-to-start-logstash-service-error-code-in-log/126812 "2018-04-05T22:29:07Z")
Hi there, When I try to start logstash "systemctl start logstash" I see these error messages in the log: \[2018-04-04T21:46:39,189\]\[WARN \]\[logstash.outputs.elasticsearch\] Restored connection to ES instance {:url=\>"http:… --- ## [Filter Error ASCII-8BIT to UTF-8](https://discuss.elastic.co/t/filter-error-ascii-8bit-to-utf-8/124156)
**Author:** [@Kareny](https://discuss.elastic.co/u/Kareny)\ **Replies:** 3\ **Last updated:** [April 5, 2018, 10:22pm UTC](https://discuss.elastic.co/t/filter-error-ascii-8bit-to-utf-8/124156 "2018-04-05T22:22:50Z")
Hi, I got the following error when running the logstash with jdbc plugin to fetch logs from a oracle database. Mar 13 17:09:31 exabeamlms01 docker\[10496\]: \[2018-03-13T17:09:31,010\]\[ERROR\] \[logstash.filters.ruby \] Ruby… --- ## [Logstash file as input plugin isn't working](https://discuss.elastic.co/t/logstash-file-as-input-plugin-isnt-working/126806)
**Author:** [@aysala](https://discuss.elastic.co/u/aysala)\ **Replies:** 10\ **Last updated:** [April 5, 2018, 10:06pm UTC](https://discuss.elastic.co/t/logstash-file-as-input-plugin-isnt-working/126806 "2018-04-05T22:06:43Z")
Hi, I am currently running ELK stack on my dcos cluster. I wanted to add few more log paths to Logstash configuration using file input plugin as follows. input { file { path =\> \['/var/lib/mesos/slave/slaves/… --- ## [Beats Received on Port 11001 Are Being Processed By Port 11000 Config](https://discuss.elastic.co/t/beats-received-on-port-11001-are-being-processed-by-port-11000-config/126603)
**Author:** [@mn\_dev](https://discuss.elastic.co/u/mn_dev)\ **Replies:** 18\ **Last updated:** [April 5, 2018, 9:09pm UTC](https://discuss.elastic.co/t/beats-received-on-port-11001-are-being-processed-by-port-11000-config/126603 "2018-04-05T21:09:36Z")
From the guidance of Two filebeat instances, I have successfully setup two instances of Filebeat on one machine. Instance1 reads logs for AppA and ships them to Logstash:11000 while Instance2 reads logs for AppB and ship… --- ## [Send data to right pipeline from two CSV files](https://discuss.elastic.co/t/send-data-to-right-pipeline-from-two-csv-files/126700)
**Author:** [@mladen](https://discuss.elastic.co/u/mladen)\ **Replies:** 5\ **Last updated:** [April 5, 2018, 8:42pm UTC](https://discuss.elastic.co/t/send-data-to-right-pipeline-from-two-csv-files/126700 "2018-04-05T20:42:10Z")
Hello, I am trying to understand workflow Filebeat ---\> Beats/Logstash ---\> Right pipeline. For example, we can have two CSVfiles with the same structure. Both data stream is sending to the Beats who listen on port 504… --- ## [Logstash: s3 input plugin](https://discuss.elastic.co/t/logstash-s3-input-plugin/126349)
**Author:** [@harkamals](https://discuss.elastic.co/u/harkamals)\ **Replies:** 1\ **Last updated:** [April 5, 2018, 7:43pm UTC](https://discuss.elastic.co/t/logstash-s3-input-plugin/126349 "2018-04-05T19:43:22Z")
Hello, I am to work on a use case of ingesting aws cloud trail logs from a s3 bucket into logstash. A number of logstash servers are spun by the auto scaling group Do I need to apply s3 input config to every one of the… --- ## [Store a field as a global var](https://discuss.elastic.co/t/store-a-field-as-a-global-var/126774)
**Author:** [@Khaled\_Saidi](https://discuss.elastic.co/u/Khaled_Saidi)\ **Replies:** 3\ **Last updated:** [April 5, 2018, 7:41pm UTC](https://discuss.elastic.co/t/store-a-field-as-a-global-var/126774 "2018-04-05T19:41:42Z")
Hello everyone, I am a new user of the ELK stack and I am facing a problem that I can not solve. i'm using filebeat to collect datas from several logs files and send them to logstash for parsing. Is it possible for lo… --- ## [Unable to install logstash-core gem](https://discuss.elastic.co/t/unable-to-install-logstash-core-gem/126619)
**Author:** [@annk](https://discuss.elastic.co/u/annk)\ **Replies:** 5\ **Last updated:** [April 5, 2018, 5:43pm UTC](https://discuss.elastic.co/t/unable-to-install-logstash-core-gem/126619 "2018-04-05T17:43:28Z")
Hi , I am trying to install xpack on logstash server , but it complained about missing logstash-core gem. Tried to install logstash-core-6.2.2 after installing all dependencies . $ gem list \*\*\* LOCAL GEMS \*\*\* jar-de… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=784) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=786)