# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=749 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 750 --- ## [How do i Parse the particular value](https://discuss.elastic.co/t/how-do-i-parse-the-particular-value/134055)
**Author:** [@Chinmaya\_Mishra](https://discuss.elastic.co/u/Chinmaya_Mishra)\ **Replies:** 1\ **Last updated:** [May 31, 2018, 1:53pm UTC](https://discuss.elastic.co/t/how-do-i-parse-the-particular-value/134055 "2018-05-31T13:53:10Z")
How do i parsed \_dv -\> \[0\] -\> \_v value from below message "message" =\> "{\_dv=\[{\_t=SMU\_EVENTS, \_r=1527684757285, \_v={severity=INFO, analysisModule=MAC\_ANALYSIS, headerValue=, apMac=74:67:f7:63:9a:12, pa ramlist=\[{Value=… --- ## [How can I send logs to elasticsearch](https://discuss.elastic.co/t/how-can-i-send-logs-to-elasticsearch/134059)
**Author:** [@fabrice.tagne](https://discuss.elastic.co/u/fabrice.tagne)\ **Replies:** 1\ **Last updated:** [May 31, 2018, 1:44pm UTC](https://discuss.elastic.co/t/how-can-i-send-logs-to-elasticsearch/134059 "2018-05-31T13:44:55Z")
Logstash configuration is in /etc/logstash/logstash.yml the rest in default # ------------ Metrics Settings -------------- # # Bind address for the metrics REST endpoint # http.host: "127.0.0.1" # # Bind port for the me… --- ## [Filter grok logstash email failed](https://discuss.elastic.co/t/filter-grok-logstash-email-failed/133915)
**Author:** [@rouchad\_rouchad](https://discuss.elastic.co/u/rouchad_rouchad)\ **Replies:** 3\ **Last updated:** [May 31, 2018, 1:30pm UTC](https://discuss.elastic.co/t/filter-grok-logstash-email-failed/133915 "2018-05-31T13:30:56Z")
hello everyone please I need your help! I connected my mailbox to elasticsearch using imap the problem is that I want to apply grok type filters on I want to make condtions in my logstash config ie filters type grok =… --- ## [TypeError when setting tracking\_column\_type to "timestamp" and use\_column\_value to true](https://discuss.elastic.co/t/typeerror-when-setting-tracking-column-type-to-timestamp-and-use-column-value-to-true/132822)
**Author:** [@Garrick\_Hensberg](https://discuss.elastic.co/u/Garrick_Hensberg)\ **Replies:** 1\ **Last updated:** [May 31, 2018, 1:27pm UTC](https://discuss.elastic.co/t/typeerror-when-setting-tracking-column-type-to-timestamp-and-use-column-value-to-true/132822 "2018-05-31T13:27:16Z")
I am experiencing an issue when trying to use the tracking\_column\_type="timestamp" and use\_column\_value=true properties with the jdbc input plugin. I keep getting the below error: { 2012 rufus-scheduler intercepted an … --- ## [What does this 6.x warning mean?](https://discuss.elastic.co/t/what-does-this-6-x-warning-mean/134062)
**Author:** [@TimWard](https://discuss.elastic.co/u/TimWard)\ **Replies:** 0\ **Last updated:** [May 31, 2018, 1:23pm UTC](https://discuss.elastic.co/t/what-does-this-6-x-warning-mean/134062 "2018-05-31T13:23:32Z")
Detected a 6.x and above cluster: the type event field won't be used to determine the document \_type {:es\_version=\>6} What does this message mean? - as far as I can see there are no usages of "type" left in my Logstas… --- ## [Logstash 6.2.4 File input plugin spawning a lot of child .sincedb files](https://discuss.elastic.co/t/logstash-6-2-4-file-input-plugin-spawning-a-lot-of-child-sincedb-files/134018)
**Author:** [@ld\_pvl](https://discuss.elastic.co/u/ld_pvl)\ **Replies:** 4\ **Last updated:** [May 31, 2018, 1:09pm UTC](https://discuss.elastic.co/t/logstash-6-2-4-file-input-plugin-spawning-a-lot-of-child-sincedb-files/134018 "2018-05-31T13:09:40Z")
Hi Team, I've noticed this strange behaviour where let's say my original .sincedb-foo file was created by logstash under user A - i.e. the .sincedb-foo file is owned by user A. Now if I stop then start logstash up agai… --- ## [Kafka Input terminates pipeline after cluster restart with SASL\_SSL](https://discuss.elastic.co/t/kafka-input-terminates-pipeline-after-cluster-restart-with-sasl-ssl/133879)
**Author:** [@JohanRask](https://discuss.elastic.co/u/JohanRask)\ **Replies:** 0\ **Last updated:** [May 30, 2018, 1:21pm UTC](https://discuss.elastic.co/t/kafka-input-terminates-pipeline-after-cluster-restart-with-sasl-ssl/133879 "2018-05-30T13:21:47Z")
Hi, We are using logstash-6.2.4 + kafka 1.1 (confluent) successfully with multiple topics and multiple logstash pipelines. If we use anonymous (PLAINTEXT on default port 9092) there are no issues when taking the comple… --- ## [Send data from Logstash to hadoop](https://discuss.elastic.co/t/send-data-from-logstash-to-hadoop/133976)
**Author:** [@anush\_jayan](https://discuss.elastic.co/u/anush_jayan)\ **Replies:** 1\ **Last updated:** [May 31, 2018, 11:11am UTC](https://discuss.elastic.co/t/send-data-from-logstash-to-hadoop/133976 "2018-05-31T11:11:46Z")
hi can help me how can i send data to hadoop or hive using logstash this is my config file input{ stdin{} } filter{ grok { # patterns\_dir =\> \["/etc/logstash/conf.d/patterns"\] match =\> { "message" =\> "%{TIMESTAMP\_… --- ## [Starting logstash with index provided by user from UI screen](https://discuss.elastic.co/t/starting-logstash-with-index-provided-by-user-from-ui-screen/133996)
**Author:** [@Ganesha\_Venkanna](https://discuss.elastic.co/u/Ganesha_Venkanna)\ **Replies:** 3\ **Last updated:** [May 31, 2018, 11:00am UTC](https://discuss.elastic.co/t/starting-logstash-with-index-provided-by-user-from-ui-screen/133996 "2018-05-31T11:00:10Z")
Hi, I understand that currently the index name is supplied in the logstash.conf,i have a requirement where,where user will supply a index name through UI and i need to run logstash with the name provided. Or Can we su… --- ## [Handle the xml parsed data, met an issue as ruby exception occurred: can't convert String into Integer](https://discuss.elastic.co/t/handle-the-xml-parsed-data-met-an-issue-as-ruby-exception-occurred-cant-convert-string-into-integer/133642)
**Author:** [@terrymu](https://discuss.elastic.co/u/terrymu)\ **Replies:** 3\ **Last updated:** [May 31, 2018, 10:29am UTC](https://discuss.elastic.co/t/handle-the-xml-parsed-data-met-an-issue-as-ruby-exception-occurred-cant-convert-string-into-integer/133642 "2018-05-31T10:29:52Z")
Hi experts, Here is a xml parsed data show as below: "parsed" =\> { "PMSetup" =\> \[ \[0\] { "startTime" =\> "2018-03-09T08:40:00.000+01:00", "interval" =\> "5", … --- ## [How to understand an error-line at docker log outout](https://discuss.elastic.co/t/how-to-understand-an-error-line-at-docker-log-outout/133994)
**Author:** [@Robert\_D](https://discuss.elastic.co/u/Robert_D)\ **Replies:** 1\ **Last updated:** [May 31, 2018, 10:18am UTC](https://discuss.elastic.co/t/how-to-understand-an-error-line-at-docker-log-outout/133994 "2018-05-31T10:18:03Z")
I have an error like logstash\_1 | \[2018-05-31T07:47:02,627\]\[ERROR\]\[logstash.pipeline \] Error registering plugin {:pipeline\_id=\>"main", :plugin=\>"#\ **Author:** [@dr01](https://discuss.elastic.co/u/dr01)\ **Replies:** 4\ **Last updated:** [May 31, 2018, 9:17am UTC](https://discuss.elastic.co/t/parsing-only-a-limited-set-of-fields-in-json/133885 "2018-05-31T09:17:42Z") I need to parse a JSON log message but keep only a few fields. Is it possible to couple the JSON filter plugin (https://www.elastic.co/guide/en/logstash/current/plugins-filters-json.html) with the KV filter plugin (http… --- ## [Trying to get values in array](https://discuss.elastic.co/t/trying-to-get-values-in-array/133736)
**Author:** [@Shivom\_Kumar](https://discuss.elastic.co/u/Shivom_Kumar)\ **Replies:** 5\ **Last updated:** [May 31, 2018, 9:10am UTC](https://discuss.elastic.co/t/trying-to-get-values-in-array/133736 "2018-05-31T09:10:06Z")
hi . i was doing something similar to getting values as an array. i have my log something like this fab 20 gds 30 rt 21 i want to create two array : one contating {fab,dgs,rt} and other containing their respective valu… --- ## [Not able to see the out when i run logstash](https://discuss.elastic.co/t/not-able-to-see-the-out-when-i-run-logstash/133967)
**Author:** [@naga\_kunchala](https://discuss.elastic.co/u/naga_kunchala)\ **Replies:** 6\ **Last updated:** [May 31, 2018, 8:30am UTC](https://discuss.elastic.co/t/not-able-to-see-the-out-when-i-run-logstash/133967 "2018-05-31T08:30:53Z")
when i run logstash withe ./logstash -f /etc/logstash/conf.d/logstash-filter.conf. not geeting any output and see the below error. \[root@localhost bin\]# ./logstash -f /etc/logstash/conf.d/logstash-filter.conf OpenJDK … --- ## [Problem setting "data" type](https://discuss.elastic.co/t/problem-setting-data-type/133881)
**Author:** [@Samuele\_Colmi](https://discuss.elastic.co/u/Samuele_Colmi)\ **Replies:** 4\ **Last updated:** [May 31, 2018, 8:00am UTC](https://discuss.elastic.co/t/problem-setting-data-type/133881 "2018-05-31T08:00:00Z")
Hello everyone! I'm trying to index a .csv file with Logstash. One of the field of the file is a string that rappresent a date ("yyyy-MM-dd HH:mm"), and i want to convert this field into a date type at the momento of u… --- ## [trustAnchors parameter must be non-empty](https://discuss.elastic.co/t/trustanchors-parameter-must-be-non-empty/131761)
**Author:** [@sarlacpit](https://discuss.elastic.co/u/sarlacpit)\ **Replies:** 4\ **Last updated:** [May 31, 2018, 7:29am UTC](https://discuss.elastic.co/t/trustanchors-parameter-must-be-non-empty/131761 "2018-05-31T07:29:29Z")
Hi folks, Using Ubuntu 18:04 LTS and the Elasticstack v6.1.3 and I seem to be running in to an error starting logstash: /usr/share/logstash/bin/logstash --debug --path.settings /etc/logstash OpenSSL::X509::StoreError: … --- ## [Logstash Java Heap Error](https://discuss.elastic.co/t/logstash-java-heap-error/133983)
**Author:** [@JoseJR](https://discuss.elastic.co/u/JoseJR)\ **Replies:** 0\ **Last updated:** [May 31, 2018, 6:58am UTC](https://discuss.elastic.co/t/logstash-java-heap-error/133983 "2018-05-31T06:58:53Z")
Need assistance on the below Error from logstash logs We have enough jvm.options settings xms xmx both 8gb, memory is not getting used up. CPU is maxed then logstash will restart. Any thoughts? \[ERROR\]\[logstash.pipelin… --- ## [Logstash Issue: Glob did not work, collector name {:name=\>"ConcurrentMarkSweep"}](https://discuss.elastic.co/t/logstash-issue-glob-did-not-work-collector-name-name-concurrentmarksweep/133964)
**Author:** [@rahulnama](https://discuss.elastic.co/u/rahulnama)\ **Replies:** 5\ **Last updated:** [May 31, 2018, 6:22am UTC](https://discuss.elastic.co/t/logstash-issue-glob-did-not-work-collector-name-name-concurrentmarksweep/133964 "2018-05-31T06:22:59Z")
Hello, I'm parsing sql logs with logstash version 6.2.3. Windows 10. this is my logstash.conf file. I've used it many times but unfortunately something happend and it stopped working input { file{ path =\> "C:\\Users… --- ## [Logstash 6.2.4 crashing due to no timestamp error](https://discuss.elastic.co/t/logstash-6-2-4-crashing-due-to-no-timestamp-error/133944)
**Author:** [@rreilly-edr](https://discuss.elastic.co/u/rreilly-edr)\ **Replies:** 1\ **Last updated:** [May 31, 2018, 6:17am UTC](https://discuss.elastic.co/t/logstash-6-2-4-crashing-due-to-no-timestamp-error/133944 "2018-05-31T06:17:35Z")
I have a pipeline that has been running for some time, it had a config that has been defined but we just started using. sending nginx logs using example nginx config from 6.x logstash docs example. \[2018-05-30T20:59:52,… --- ## [How to run postgres in Ruby filter?](https://discuss.elastic.co/t/how-to-run-postgres-in-ruby-filter/133796)
**Author:** [@veerendranath\_lukkan](https://discuss.elastic.co/u/veerendranath_lukkan)\ **Replies:** 7\ **Last updated:** [May 31, 2018, 6:06am UTC](https://discuss.elastic.co/t/how-to-run-postgres-in-ruby-filter/133796 "2018-05-31T06:06:41Z")
hi , i am trying to check jdbc connection is available or not in ruby filter it is possible or not. --- ## [SNMP Trap Integration](https://discuss.elastic.co/t/snmp-trap-integration/130220)
**Author:** [@ysumit](https://discuss.elastic.co/u/ysumit)\ **Replies:** 10\ **Last updated:** [May 31, 2018, 5:48am UTC](https://discuss.elastic.co/t/snmp-trap-integration/130220 "2018-05-31T05:48:57Z")
Hi Team Do we have any document for Integration of SNMP Traps with Logstash? We are trying to send SNMP traps to Logstash. Regards Sumit --- ## [Can't get auth to work LS -\> ES](https://discuss.elastic.co/t/cant-get-auth-to-work-ls-es/133943)
**Author:** [@jonrust](https://discuss.elastic.co/u/jonrust)\ **Replies:** 1\ **Last updated:** [May 30, 2018, 11:12pm UTC](https://discuss.elastic.co/t/cant-get-auth-to-work-ls-es/133943 "2018-05-30T23:12:18Z")
I can connect using curl (running on cdlelk01): curl -u logstash\_system:mypass http://localhost:9200/ { "name" : ... My outputs have the same user and password: output { elasticsearch { hosts =\> \[ "cdlelk01… --- ## [Enriching elasticsearch inserts with elasticsearch lookups](https://discuss.elastic.co/t/enriching-elasticsearch-inserts-with-elasticsearch-lookups/133936)
**Author:** [@cbarton](https://discuss.elastic.co/u/cbarton)\ **Replies:** 2\ **Last updated:** [May 30, 2018, 8:23pm UTC](https://discuss.elastic.co/t/enriching-elasticsearch-inserts-with-elasticsearch-lookups/133936 "2018-05-30T20:23:54Z")
I'm not sure exactly where to pose the question, but 2/3 of my possible approaches involve logstash, so it seemed the right place to start. I have incoming apache logs, and need to extract the IPs from them, then do a l… --- ## [Grokparsefailure PostgreSQL](https://discuss.elastic.co/t/grokparsefailure-postgresql/133783)
**Author:** [@Jose\_Calvelo](https://discuss.elastic.co/u/Jose_Calvelo)\ **Replies:** 4\ **Last updated:** [May 30, 2018, 7:43pm UTC](https://discuss.elastic.co/t/grokparsefailure-postgresql/133783 "2018-05-30T19:43:51Z")
Hi, im trying to build a filter for PostgreSQL and I have managed to make it work with the debugger. But when I receive it in my application I see that logstash failed to parse it. Log example: 2018-05-29 11:09:44 -03 … --- ## [Strange error with gsub](https://discuss.elastic.co/t/strange-error-with-gsub/133827)
**Author:** [@br14nb0](https://discuss.elastic.co/u/br14nb0)\ **Replies:** 1\ **Last updated:** [May 30, 2018, 6:51pm UTC](https://discuss.elastic.co/t/strange-error-with-gsub/133827 "2018-05-30T18:51:42Z")
I have my mutate config mutate { gsub =\> \[ "service","\\x7f\\\\",""\] } I want to replace "DEL" string with NULL. But look likes it misunderstands with my escape. --- ## [Grok Filter Works With 5.4.0 ELK Stack, Not 6.2.4 on Windows Server 2012](https://discuss.elastic.co/t/grok-filter-works-with-5-4-0-elk-stack-not-6-2-4-on-windows-server-2012/133891)
**Author:** [@reromnek](https://discuss.elastic.co/u/reromnek)\ **Replies:** 1\ **Last updated:** [May 30, 2018, 6:50pm UTC](https://discuss.elastic.co/t/grok-filter-works-with-5-4-0-elk-stack-not-6-2-4-on-windows-server-2012/133891 "2018-05-30T18:50:02Z")
I have created a grok filter in a logstash.json file shown here: input { beats { port =\> 5044 type =\> "log" } } filter { Ignore the comments that IIS will add to the start of the W3C logs if \[message\] =~ "^#" { … --- ## [Metricbeat data not available on Elastic cluster through logstash](https://discuss.elastic.co/t/metricbeat-data-not-available-on-elastic-cluster-through-logstash/133911)
**Author:** [@niraj\_pandey](https://discuss.elastic.co/u/niraj_pandey)\ **Replies:** 0\ **Last updated:** [May 30, 2018, 4:03pm UTC](https://discuss.elastic.co/t/metricbeat-data-not-available-on-elastic-cluster-through-logstash/133911 "2018-05-30T16:03:22Z")
Hi, I have deployed a logstash server in one site (lets say A) and this logstash server push the data on elastic cluster on remote site (lets say B). On site A I have around 500 hosts and I have installed the metricbea… --- ## [Aggregations response in logstash Elastic search filter](https://discuss.elastic.co/t/aggregations-response-in-logstash-elastic-search-filter/133907)
**Author:** [@UdaySingh\_Ghadage](https://discuss.elastic.co/u/UdaySingh_Ghadage)\ **Replies:** 0\ **Last updated:** [May 30, 2018, 3:29pm UTC](https://discuss.elastic.co/t/aggregations-response-in-logstash-elastic-search-filter/133907 "2018-05-30T15:29:36Z")
Hi, Can you please let me know how my elastic search filter plugin configuration should be: My query in Query.JSON: { "size": 0, "\_source": { "excludes": \[\] }, "aggs": { "2": { "filters": { "filters": { "JobI… --- ## [Indexing JSON files from a local directory to elastic](https://discuss.elastic.co/t/indexing-json-files-from-a-local-directory-to-elastic/133667)
**Author:** [@Yaniv\_Boneh](https://discuss.elastic.co/u/Yaniv_Boneh)\ **Replies:** 4\ **Last updated:** [May 30, 2018, 2:33pm UTC](https://discuss.elastic.co/t/indexing-json-files-from-a-local-directory-to-elastic/133667 "2018-05-30T14:33:10Z")
Hello, I wish to configure my Logstash pipline so it will index JSON files from a local directory. Of course, I already went through relevant topics and found no satisfying solution for my case, for instance, this issu… --- ## [Unable to start Logstash 6.2](https://discuss.elastic.co/t/unable-to-start-logstash-6-2/133884)
**Author:** [@Makra](https://discuss.elastic.co/u/Makra)\ **Replies:** 0\ **Last updated:** [May 30, 2018, 1:51pm UTC](https://discuss.elastic.co/t/unable-to-start-logstash-6-2/133884 "2018-05-30T13:51:46Z")
Hi I am having a problem with running Logstash 6.2 in one of the Windows 7 machine. If i execute logstash from command line with options it produces following error. Although Logstash 5.6 does not produces any error in … [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=748) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=750)