# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=930 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 931 --- ## [Logstash unable to identify tags from filebeat ( grokparsefailure)](https://discuss.elastic.co/t/logstash-unable-to-identify-tags-from-filebeat-grokparsefailure/96129)
**Author:** [@Nikhilpawar1985](https://discuss.elastic.co/u/Nikhilpawar1985)\ **Replies:** 5\ **Last updated:** [August 8, 2017, 3:09pm UTC](https://discuss.elastic.co/t/logstash-unable-to-identify-tags-from-filebeat-grokparsefailure/96129 "2017-08-08T15:09:52Z")
Hi, i am sending two different logs files with filebeat agent with different tags on filebeat side to logstash but logstash only able to parse one file and failing for other . Below is my config. Can someone help . i… --- ## [Logstash dosn't import CSV file](https://discuss.elastic.co/t/logstash-dosnt-import-csv-file/96286)
**Author:** [@Tals](https://discuss.elastic.co/u/Tals)\ **Replies:** 0\ **Last updated:** [August 8, 2017, 2:44pm UTC](https://discuss.elastic.co/t/logstash-dosnt-import-csv-file/96286 "2017-08-08T14:44:43Z")
Hi I’m trying to import CSV with Logstash. My config file looking like this: input{ file{ path=\>"/Users/office/Desktop/Elasticsearch data/ufo.csv" start\_position =\> "beginning" sincedb\_path =\> “/dev/null” } } … --- ## [JDBC error: NoMethodError: undefined method \`new' for #\"](https://discuss.elastic.co/t/jdbc-error-nomethoderror-undefined-method-new-for-array-0x5031fa45/96285)
**Author:** [@nikaalv](https://discuss.elastic.co/u/nikaalv)\ **Replies:** 0\ **Last updated:** [August 8, 2017, 2:33pm UTC](https://discuss.elastic.co/t/jdbc-error-nomethoderror-undefined-method-new-for-array-0x5031fa45/96285 "2017-08-08T14:33:25Z")
Hi! I’m trying to connect to Sybase SQLAnywhere My Logstash input plugin is: input { jdbc { jdbc\_connection\_string =\> "jdbc:sqlanywhere://host:port/db\_name" jdbc\_user =\> "user" jdbc\_password =\> "passs" jdbc\_valid… --- ## [Logging form folder with subfolders](https://discuss.elastic.co/t/logging-form-folder-with-subfolders/95907)
**Author:** [@Alex\_Kovalyov](https://discuss.elastic.co/u/Alex_Kovalyov)\ **Replies:** 2\ **Last updated:** [August 8, 2017, 1:27pm UTC](https://discuss.elastic.co/t/logging-form-folder-with-subfolders/95907 "2017-08-08T13:27:29Z")
Hello! I have the next structure of logs /var/log/stash/site1/.log /var/log/stash/site1/accounts/account1/.log /var/log/stash/site1/otherKind1/.log /var/log/stash/site1/otherKind2/.log so in site can be many subfol… --- ## [Replace @timestamp with Time field in my log](https://discuss.elastic.co/t/replace-timestamp-with-time-field-in-my-log/94526)
**Author:** [@Sri\_Divya\_Ayalasomay](https://discuss.elastic.co/u/Sri_Divya_Ayalasomay)\ **Replies:** 4\ **Last updated:** [August 8, 2017, 1:22pm UTC](https://discuss.elastic.co/t/replace-timestamp-with-time-field-in-my-log/94526 "2017-08-08T13:22:27Z")
I am trying to replace time field in my log time : here is my config file: I am new to elastic search , tried couple of things but failed. nput { file{ path=\>"C:\\logs\\testing1.csv" start\_position=\>"beginning" } } fi… --- ## [Logstash JDBC connectivity with UniVerse Database](https://discuss.elastic.co/t/logstash-jdbc-connectivity-with-universe-database/96253)
**Author:** [@arunvijayam](https://discuss.elastic.co/u/arunvijayam)\ **Replies:** 0\ **Last updated:** [August 8, 2017, 11:24am UTC](https://discuss.elastic.co/t/logstash-jdbc-connectivity-with-universe-database/96253 "2017-08-08T11:24:46Z")
Hi All, Is there any opportunity to connect UniVerse Database 11.2.5 with logstash jdbc connectivity or any other work around to do this to pull records and store them in elastic search? --- ## [Logstash, how to define unique id for tweets?](https://discuss.elastic.co/t/logstash-how-to-define-unique-id-for-tweets/96159)
**Author:** [@Lbr1](https://discuss.elastic.co/u/Lbr1)\ **Replies:** 1\ **Last updated:** [August 8, 2017, 11:18am UTC](https://discuss.elastic.co/t/logstash-how-to-define-unique-id-for-tweets/96159 "2017-08-08T11:18:17Z")
Good evening, I have the following output in my logstash file output { stdout { codec =\> dots } #stdout { codec =\> rubydebug } #stdout { codec =\> json } elasticsearch { hosts =\> "localhost:9200" index =\> "game" … --- ## [Separate logfiles for Logstash](https://discuss.elastic.co/t/separate-logfiles-for-logstash/96249)
**Author:** [@samia](https://discuss.elastic.co/u/samia)\ **Replies:** 2\ **Last updated:** [August 8, 2017, 11:12am UTC](https://discuss.elastic.co/t/separate-logfiles-for-logstash/96249 "2017-08-08T11:12:55Z")
I’m using Logstash for 4 different sources on a single node. The config file for these sources are very similar and look like this : ##################### input { file { path =\> "/....../source1/\*.log" codec =\> "… --- ## [Can ELK meets my need: make the subdirectory name a index key?](https://discuss.elastic.co/t/can-elk-meets-my-need-make-the-subdirectory-name-a-index-key/96241)
**Author:** [@wuchang](https://discuss.elastic.co/u/wuchang)\ **Replies:** 1\ **Last updated:** [August 8, 2017, 11:12am UTC](https://discuss.elastic.co/t/can-elk-meets-my-need-make-the-subdirectory-name-a-index-key/96241 "2017-08-08T11:12:49Z")
I am doing a reasearch on Logstash+ElasticSearch+Kibana to make sure they meet my needs .Below is the usage scenario: I want to use them for my Hadoop application logs search in a WEB UI. Since a hadoop application can … --- ## [How work 's3 output plugin' to segment files in parts?](https://discuss.elastic.co/t/how-work-s3-output-plugin-to-segment-files-in-parts/96252)
**Author:** [@JaviiPG](https://discuss.elastic.co/u/JaviiPG)\ **Replies:** 0\ **Last updated:** [August 8, 2017, 11:08am UTC](https://discuss.elastic.co/t/how-work-s3-output-plugin-to-segment-files-in-parts/96252 "2017-08-08T11:08:48Z")
Hi everyone, I have an ELK environment installed. One of the client requirements is to have a backup of the records. For this reason I created a bucket S3 and I'm using a 'logstash-output-s3' plugin. At the moment I ha… --- ## [Logstash is running but we are getting error in the logs](https://discuss.elastic.co/t/logstash-is-running-but-we-are-getting-error-in-the-logs/95827)
**Author:** [@JAYALAKSHMI](https://discuss.elastic.co/u/JAYALAKSHMI)\ **Replies:** 2\ **Last updated:** [August 8, 2017, 9:38am UTC](https://discuss.elastic.co/t/logstash-is-running-but-we-are-getting-error-in-the-logs/95827 "2017-08-08T09:38:53Z")
Hi All, We have configured the logstash as below. input { elasticsearch { hosts =\> "localhost:9202" index =\> "index\_name" size =\> 100 scroll =\> "1m" docinfo =\> true } } filter { mutate { remove\_field =\> \[ “@… --- ## [Confusion between logstash and beats](https://discuss.elastic.co/t/confusion-between-logstash-and-beats/96105)
**Author:** [@Blason](https://discuss.elastic.co/u/Blason)\ **Replies:** 3\ **Last updated:** [August 8, 2017, 8:55am UTC](https://discuss.elastic.co/t/confusion-between-logstash-and-beats/96105 "2017-08-08T08:55:59Z")
Hi Guys, Since I am a novice would like to know if shipping logs from Linux system like apache/nginx filebeat would be better as compare to logstash? or I need to have logstash to injest the logs? I believe I can direc… --- ## [Logstash conditionals](https://discuss.elastic.co/t/logstash-conditionals/96139)
**Author:** [@zozo6015](https://discuss.elastic.co/u/zozo6015)\ **Replies:** 3\ **Last updated:** [August 8, 2017, 8:52am UTC](https://discuss.elastic.co/t/logstash-conditionals/96139 "2017-08-08T08:52:44Z")
Hello, I am trying to parse auth.log into the elasticsearch and since different actions have different formats I cannot parse the whole log as one. I was thinking to check if there are certain words in the message part … --- ## [\_jsonparsefailure - What is the problem?](https://discuss.elastic.co/t/-jsonparsefailure-what-is-the-problem/96110)
**Author:** [@ssasporta](https://discuss.elastic.co/u/ssasporta)\ **Replies:** 2\ **Last updated:** [August 8, 2017, 8:18am UTC](https://discuss.elastic.co/t/-jsonparsefailure-what-is-the-problem/96110 "2017-08-08T08:18:13Z")
My input looks like that: 10.108.2.44 "73.91.231.150, 10.10.10.165" - - \[26/Jul/2017:00:00:00 -0500\] "GET /wod/landing?c=e&macId=90%3Ac7%3Ad8%3Afa%3A78%3Ae9&location=WODTrial&apMacId=10%3A56%3A11%3A15%3Ad3%3A8d&a=as&bn=… --- ## [How to migrate elasticsearch data](https://discuss.elastic.co/t/how-to-migrate-elasticsearch-data/96218)
**Author:** [@Trung\_Do](https://discuss.elastic.co/u/Trung_Do)\ **Replies:** 0\ **Last updated:** [August 8, 2017, 8:13am UTC](https://discuss.elastic.co/t/how-to-migrate-elasticsearch-data/96218 "2017-08-08T08:13:10Z")
Hi all Now i’m using es 5.5.1 and i have requirement to migrate data. Last month i index a data with below structure: { "username": "user\_03", "firstName":"Trung", "lastName":"Do", "post\_date": "2010-1… --- ## [Not able to parse the data having quote in middle in csv file](https://discuss.elastic.co/t/not-able-to-parse-the-data-having-quote-in-middle-in-csv-file/96210)
**Author:** [@deepanshu\_mittal](https://discuss.elastic.co/u/deepanshu_mittal)\ **Replies:** 0\ **Last updated:** [August 8, 2017, 7:40am UTC](https://discuss.elastic.co/t/not-able-to-parse-the-data-having-quote-in-middle-in-csv-file/96210 "2017-08-08T07:40:48Z")
Hi, I am trying to parse csv file in logstash but it is failing some messages as some field contains double quotes either at start or middle of mesage. Eg: 4491750\*\*,\*\*“CRITICAL: Value for OSCS - Sydney aucom-east-1 r… --- ## [Logstash to Confluent kafka rest proxy](https://discuss.elastic.co/t/logstash-to-confluent-kafka-rest-proxy/96213)
**Author:** [@rvnrajesh](https://discuss.elastic.co/u/rvnrajesh)\ **Replies:** 0\ **Last updated:** [August 8, 2017, 7:47am UTC](https://discuss.elastic.co/t/logstash-to-confluent-kafka-rest-proxy/96213 "2017-08-08T07:47:49Z")
Hi I’d like to use Confluent’s REST proxy to push the json data to kafka. But according to the documentation http://docs.confluent.io/3.0.0/kafka-rest/docs/intro.html#produce-and-consume-json-messages7 I’d need to wra… --- ## [Empty message field in exec input field](https://discuss.elastic.co/t/empty-message-field-in-exec-input-field/96205)
**Author:** [@Amir\_Hassani](https://discuss.elastic.co/u/Amir_Hassani)\ **Replies:** 0\ **Last updated:** [August 8, 2017, 7:22am UTC](https://discuss.elastic.co/t/empty-message-field-in-exec-input-field/96205 "2017-08-08T07:22:44Z")
Hello, I’m trying to get information from Glusterfs. when I use exec command, sometimes it returns empty message. what is the reason and how can I fix that? my command is like this: gluster volume profile dkv info | g… --- ## [Extracting data from multiple events from Elasticsearch using single logstash filter](https://discuss.elastic.co/t/extracting-data-from-multiple-events-from-elasticsearch-using-single-logstash-filter/96195)
**Author:** [@sushil1016](https://discuss.elastic.co/u/sushil1016)\ **Replies:** 0\ **Last updated:** [August 8, 2017, 6:54am UTC](https://discuss.elastic.co/t/extracting-data-from-multiple-events-from-elasticsearch-using-single-logstash-filter/96195 "2017-08-08T06:54:01Z")
I have log lines loaded in ElasticSearch which has the data scattered in multiple events, say event\_id is in event (line) number 5 and event\_action is available in event number 88, further event\_port information is avail… --- ## [Unknown setting copy for mutate](https://discuss.elastic.co/t/unknown-setting-copy-for-mutate/96187)
**Author:** [@ryac](https://discuss.elastic.co/u/ryac)\ **Replies:** 2\ **Last updated:** [August 8, 2017, 6:49am UTC](https://discuss.elastic.co/t/unknown-setting-copy-for-mutate/96187 "2017-08-08T06:49:05Z")
Hi all, Having some difficulty in copying a field in Logstash. In my filter, I have this: mutate { copy =\> { "original\_field" =\> "new\_field" } } Testing the configs, I get this: ERROR logstash.filters.mutate - Unk… --- ## [CSV file indexed to %{\[@metadata\] instead of target](https://discuss.elastic.co/t/csv-file-indexed-to-metadata-instead-of-target/95957)
**Author:** [@bulsac](https://discuss.elastic.co/u/bulsac)\ **Replies:** 7\ **Last updated:** [August 8, 2017, 5:18am UTC](https://discuss.elastic.co/t/csv-file-indexed-to-metadata-instead-of-target/95957 "2017-08-08T05:18:41Z")
I am attempting to ingest a CSV file using logstash into an elasticsearch index that I have pre-defined using a PUT request. It appears that logstash is ingesting the CSV file not into the intended elasticsearch index, b… --- ## [Logstash generating duplicated index](https://discuss.elastic.co/t/logstash-generating-duplicated-index/96172)
**Author:** [@julioeduar](https://discuss.elastic.co/u/julioeduar)\ **Replies:** 0\ **Last updated:** [August 8, 2017, 2:56am UTC](https://discuss.elastic.co/t/logstash-generating-duplicated-index/96172 "2017-08-08T02:56:33Z")
Hi everyone, I have the next scenario: One application log that is being loaded to ES through logstash (insert operation), the id is unique and extracted of the log and taken as \_id in ES Other application log that is… --- ## [Where does logstash put its own logging information?](https://discuss.elastic.co/t/where-does-logstash-put-its-own-logging-information/96155)
**Author:** [@jayunit100](https://discuss.elastic.co/u/jayunit100)\ **Replies:** 1\ **Last updated:** [August 8, 2017, 2:55am UTC](https://discuss.elastic.co/t/where-does-logstash-put-its-own-logging-information/96155 "2017-08-08T02:55:21Z")
I’m running logstash, and from standard out, I see no logging info after the server finally starts: 21:02:11.228 \[\[main\]-pipeline-manager\] INFO logstash.inputs.beats - Beats inputs: Starting input listener {:address=\>“… --- ## [Illegal latitude value](https://discuss.elastic.co/t/illegal-latitude-value/96163)
**Author:** [@ssasporta](https://discuss.elastic.co/u/ssasporta)\ **Replies:** 1\ **Last updated:** [August 8, 2017, 12:17am UTC](https://discuss.elastic.co/t/illegal-latitude-value/96163 "2017-08-08T00:17:43Z")
Hi, Three of my events are missing in Elastic search. I found the following warning in the Logstash log: \[WARN \]\[logstash.outputs.elasticsearch\] Could not index event to Elasticsearch. In the first event the IP is: 7… --- ## [Geo\_point is not getting mapped correctly](https://discuss.elastic.co/t/geo-point-is-not-getting-mapped-correctly/96156)
**Author:** [@zozo6015](https://discuss.elastic.co/u/zozo6015)\ **Replies:** 5\ **Last updated:** [August 7, 2017, 11:39pm UTC](https://discuss.elastic.co/t/geo-point-is-not-getting-mapped-correctly/96156 "2017-08-07T23:39:56Z")
Hello, I am not sure if this is the proper place to ask this question. I am parsing my log file with the following grok pattern taken from the logstash documentation as it is: input { beats { # The port to list… --- ## [Logshtash 5.5.1 Kafka output SASL\_SSL Error](https://discuss.elastic.co/t/logshtash-5-5-1-kafka-output-sasl-ssl-error/95518)
**Author:** [@luis.carvalho](https://discuss.elastic.co/u/luis.carvalho)\ **Replies:** 1\ **Last updated:** [August 7, 2017, 11:29pm UTC](https://discuss.elastic.co/t/logshtash-5-5-1-kafka-output-sasl-ssl-error/95518 "2017-08-07T23:29:01Z")
Hi, I want to produce logs to a Kafka cluster with SASL/Kerberos security. The Kafka cluster is working and tested with KafkaConsoleProducer, but with my Logstash configs it returns the error: \[ERROR\]\[logstash.outputs… --- ## [Pattern matching in Logstash](https://discuss.elastic.co/t/pattern-matching-in-logstash/95622)
**Author:** [@Deepthi\_V](https://discuss.elastic.co/u/Deepthi_V)\ **Replies:** 4\ **Last updated:** [August 7, 2017, 7:03pm UTC](https://discuss.elastic.co/t/pattern-matching-in-logstash/95622 "2017-08-07T19:03:50Z")
Hi All, I am calling Logstash using stdin like shown below: /opt/logstash/bin/logstash agent -e ‘input { file { path =\> “/home/dv/Desktop/log\_output/input/\*\*/\*.log” codec =\> multiline { pattern =\> “^\\s” what =\> “previo… --- ## [Using hash variable in output](https://discuss.elastic.co/t/using-hash-variable-in-output/96143)
**Author:** [@Exocomp](https://discuss.elastic.co/u/Exocomp)\ **Replies:** 0\ **Last updated:** [August 7, 2017, 6:37pm UTC](https://discuss.elastic.co/t/using-hash-variable-in-output/96143 "2017-08-07T18:37:06Z")
Is it possible to use hash as a variable in the output section in logstash. Here is a simple example to demonstrate what I mean: ruby { code =\> " count = { 'metric1' =\> 1, 'metric2' =\> 2 }; event.set('statsd\_count… --- ## [Issues with Logstash forwarder](https://discuss.elastic.co/t/issues-with-logstash-forwarder/96142)
**Author:** [@srivarunlokku](https://discuss.elastic.co/u/srivarunlokku)\ **Replies:** 1\ **Last updated:** [August 7, 2017, 6:17pm UTC](https://discuss.elastic.co/t/issues-with-logstash-forwarder/96142 "2017-08-07T18:17:14Z")
Following is the link, you can find the issue I am facing. Please go through it and help ASAP --- ## [Running logstash in the background in production environment](https://discuss.elastic.co/t/running-logstash-in-the-background-in-production-environment/95913)
**Author:** [@shroh](https://discuss.elastic.co/u/shroh)\ **Replies:** 5\ **Last updated:** [August 7, 2017, 6:10pm UTC](https://discuss.elastic.co/t/running-logstash-in-the-background-in-production-environment/95913 "2017-08-07T18:10:47Z")
Hi, I think i am having difficulty on starting logstash as a service which runs continuously in the background to read all the config files under the directory /etc/logstash/conf.d/ Although when i run it like /usr/sha… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=929) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=931)