# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=567 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 568 --- ## [Logsatsh issue](https://discuss.elastic.co/t/logsatsh-issue/176475)
**Author:** [@arun.kumar92](https://discuss.elastic.co/u/arun.kumar92)\ **Replies:** 1\ **Last updated:** [April 11, 2019, 7:04pm UTC](https://discuss.elastic.co/t/logsatsh-issue/176475 "2019-04-11T19:04:35Z")
My logstash conatiner throwing below error. \[ERROR\] 2019-04-11 16:12:59.325 \[\[main\]-pipeline-manager\] elasticsearch - Failed to install template. {:message=\>"Template file '' could not be found!", :class=\>"ArgumentError… --- ## [Processing and comparing events in batches](https://discuss.elastic.co/t/processing-and-comparing-events-in-batches/176443)
**Author:** [@Mudit\_Mehta](https://discuss.elastic.co/u/Mudit_Mehta)\ **Replies:** 9\ **Last updated:** [April 11, 2019, 6:57pm UTC](https://discuss.elastic.co/t/processing-and-comparing-events-in-batches/176443 "2019-04-11T18:57:27Z")
I have a setup in which logstash forwarder is sending log events to logstash. Instead of checking filters on a single log event. I want to apply filters on a batch of events. Here an batch of events is basically the col… --- ## [Not able to index timestamp from input filename](https://discuss.elastic.co/t/not-able-to-index-timestamp-from-input-filename/176474)
**Author:** [@Kalyan\_MB](https://discuss.elastic.co/u/Kalyan_MB)\ **Replies:** 0\ **Last updated:** [April 11, 2019, 6:38pm UTC](https://discuss.elastic.co/t/not-able-to-index-timestamp-from-input-filename/176474 "2019-04-11T18:38:03Z")
Hi, input file name format: free\_data\_09\_09\_18\_10.txt \ i want this time stamp also to be indexed along with input file data, trying with below logstash filter config which is not working … --- ## [Issues installing custom plugin on Logstash](https://discuss.elastic.co/t/issues-installing-custom-plugin-on-logstash/176462)
**Author:** [@prodrg](https://discuss.elastic.co/u/prodrg)\ **Replies:** 0\ **Last updated:** [April 11, 2019, 5:10pm UTC](https://discuss.elastic.co/t/issues-installing-custom-plugin-on-logstash/176462 "2019-04-11T17:10:40Z")
I am writing a filter plugin for Logstash and I'm following this guide. So, I have finished the code, written everything and done some testing with rspec. Now, I have built the gem with gem build logstash-filter-e164.ge… --- ## [Manually Setting Geoip.Location](https://discuss.elastic.co/t/manually-setting-geoip-location/176429)
**Author:** [@David\_Berry](https://discuss.elastic.co/u/David_Berry)\ **Replies:** 4\ **Last updated:** [April 11, 2019, 4:41pm UTC](https://discuss.elastic.co/t/manually-setting-geoip-location/176429 "2019-04-11T16:41:48Z")
Hello Folks, I am looking for a way to manually set the geo location of various Logstash servers. While these Logstash server are all separated geographically, they are collecting logs and beats from servers that don't … --- ## [REST API, http\_poller and ruby code](https://discuss.elastic.co/t/rest-api-http-poller-and-ruby-code/176447)
**Author:** [@nandrik](https://discuss.elastic.co/u/nandrik)\ **Replies:** 2\ **Last updated:** [April 11, 2019, 4:18pm UTC](https://discuss.elastic.co/t/rest-api-http-poller-and-ruby-code/176447 "2019-04-11T16:18:27Z")
I wonder if the following functionality, which I've successfully implemented in the form of a client-side Python script, can be completely replaced by server-side Logstash configuration, using a combination of http\_polle… --- ## [Remove Latitude and Longitude from a cell in a CSV for geo\_point visualization](https://discuss.elastic.co/t/remove-latitude-and-longitude-from-a-cell-in-a-csv-for-geo-point-visualization/174838)
**Author:** [@Ryan\_Downey](https://discuss.elastic.co/u/Ryan_Downey)\ **Replies:** 11\ **Last updated:** [April 11, 2019, 3:31pm UTC](https://discuss.elastic.co/t/remove-latitude-and-longitude-from-a-cell-in-a-csv-for-geo-point-visualization/174838 "2019-04-11T15:31:42Z")
I'm trying to parse data from a CSV cell so that I can use the latitude and longitude present in the cell for a geo\_point visualization. As of right now, the column is titled Location and the cells after that have the l… --- ## [Logstash not picking up conf file](https://discuss.elastic.co/t/logstash-not-picking-up-conf-file/176287)
**Author:** [@elasticforme](https://discuss.elastic.co/u/elasticforme)\ **Replies:** 1\ **Last updated:** [April 11, 2019, 3:25pm UTC](https://discuss.elastic.co/t/logstash-not-picking-up-conf-file/176287 "2019-04-11T15:25:50Z")
I have weird problem. I have three node cluster, one node runs all the logstash pipeline, other two just for test. Setup conf file, tested it on one of the server from command line. when all good, moved to primary serv… --- ## [Couchdb\_Changes fetch posts multiple times](https://discuss.elastic.co/t/couchdb-changes-fetch-posts-multiple-times/176413)
**Author:** [@eebee](https://discuss.elastic.co/u/eebee)\ **Replies:** 0\ **Last updated:** [April 11, 2019, 1:50pm UTC](https://discuss.elastic.co/t/couchdb-changes-fetch-posts-multiple-times/176413 "2019-04-11T13:50:35Z")
I have connected logstash to a coudant database using the couchdb\_changes input plugin. Everything works just fine, but after a while, logstash re-fetch some of the posts in the cloudant db. Over and over again causing … --- ## [Use default configuration](https://discuss.elastic.co/t/use-default-configuration/176411)
**Author:** [@andurif](https://discuss.elastic.co/u/andurif)\ **Replies:** 0\ **Last updated:** [April 11, 2019, 1:32pm UTC](https://discuss.elastic.co/t/use-default-configuration/176411 "2019-04-11T13:32:43Z")
Hello, I use logstash to populate some elastic indexes. Even if this works, I have this warning when I run the command: WARNING: Could not find logstash.yml which is typically located in $LS\_HOME/config or /etc/logst… --- ## [How to change UCS-2 LE to UTF-8](https://discuss.elastic.co/t/how-to-change-ucs-2-le-to-utf-8/176181)
**Author:** [@peterch](https://discuss.elastic.co/u/peterch)\ **Replies:** 3\ **Last updated:** [April 11, 2019, 12:47pm UTC](https://discuss.elastic.co/t/how-to-change-ucs-2-le-to-utf-8/176181 "2019-04-11T12:47:24Z")
Dear All, I use filebeat to get a log file to logstash. The log is encoded with UCS-2 LE. I tried to change to UTF-8 but fail. Anyone can help? The following is my logstash config for reference. input { beats { port… --- ## [Time Filter field name: @timestamp](https://discuss.elastic.co/t/time-filter-field-name-timestamp/176232)
**Author:** [@Guy\_Shar](https://discuss.elastic.co/u/Guy_Shar)\ **Replies:** 1\ **Last updated:** [April 11, 2019, 12:35pm UTC](https://discuss.elastic.co/t/time-filter-field-name-timestamp/176232 "2019-04-11T12:35:38Z")
Am trying to setup a graph using the date and time values from my log file. The date and time format is as follow: 16/09/2014 11:54:55 I would like this value to be populated into the field: @timestamp, as at the mome… --- ## [Quote\_char not working in csv filter logstash](https://discuss.elastic.co/t/quote-char-not-working-in-csv-filter-logstash/176386)
**Author:** [@Smack\_Alpha](https://discuss.elastic.co/u/Smack_Alpha)\ **Replies:** 1\ **Last updated:** [April 11, 2019, 12:16pm UTC](https://discuss.elastic.co/t/quote-char-not-working-in-csv-filter-logstash/176386 "2019-04-11T12:16:20Z")
I am working on the csv filter. The Separator and the Columns works fine. I came to know that the quotes\_char assign the charater to every output i get. So in the quotes\_char, i added a "^" string. But It doesn't made an… --- ## [Custom template for logstash](https://discuss.elastic.co/t/custom-template-for-logstash/176225)
**Author:** [@Vijayakumar\_Kannan](https://discuss.elastic.co/u/Vijayakumar_Kannan)\ **Replies:** 3\ **Last updated:** [April 11, 2019, 11:10am UTC](https://discuss.elastic.co/t/custom-template-for-logstash/176225 "2019-04-11T11:10:45Z")
My custom logs are sent from filebeat (ingest pipeline) -\> logstash -\> elasticsearch . Now i need to make custom index for retention period purpose. when i try the custom index my geo\_ip not mapping correct one. Logsta… --- ## [Updating data in CSV logstash is pushing the entire CSV file again with updated data which duplicates my records in index. But I just want to sync the data](https://discuss.elastic.co/t/updating-data-in-csv-logstash-is-pushing-the-entire-csv-file-again-with-updated-data-which-duplicates-my-records-in-index-but-i-just-want-to-sync-the-data/175354)
**Author:** [@Sumit\_Kumar1](https://discuss.elastic.co/u/Sumit_Kumar1)\ **Replies:** 13\ **Last updated:** [April 11, 2019, 9:57am UTC](https://discuss.elastic.co/t/updating-data-in-csv-logstash-is-pushing-the-entire-csv-file-again-with-updated-data-which-duplicates-my-records-in-index-but-i-just-want-to-sync-the-data/175354 "2019-04-11T09:57:02Z")
input{ file { path =\> "/home/elastic/elk/logstash-6.4.3/csv\_data/attrition\_dump\_v3.csv" start\_position =\> "beginning" sincedb\_path =\> "/dev/null" } file { path =\> "/home/elastic/elk/logstash-6.4.3/csv\_data/workday\_dump… --- ## [Regarding timestamp index from log file name](https://discuss.elastic.co/t/regarding-timestamp-index-from-log-file-name/176184)
**Author:** [@Kalyan\_MB](https://discuss.elastic.co/u/Kalyan_MB)\ **Replies:** 5\ **Last updated:** [April 11, 2019, 9:38am UTC](https://discuss.elastic.co/t/regarding-timestamp-index-from-log-file-name/176184 "2019-04-11T09:38:10Z")
Hi, Problem: need to read time stamp from the input file name (filebeat ) from logstash filter. input file name name format: -rw-r--r-- 1 root root 204 Apr 3 15:22 free\_data\_09\_09\_13\_20.txt -rw-r--r-- 1 root root 20… --- ## [6.7.1: JRuby temp directory problem?](https://discuss.elastic.co/t/6-7-1-jruby-temp-directory-problem/176348)
**Author:** [@Bergante](https://discuss.elastic.co/u/Bergante)\ **Replies:** 0\ **Last updated:** [April 11, 2019, 8:08am UTC](https://discuss.elastic.co/t/6-7-1-jruby-temp-directory-problem/176348 "2019-04-11T08:08:29Z")
Hi, I was running 6.6.1 on FreeBSD without problems. When updating to 6.7.1 I was unable to start logstash as the logstash user (I never start it as root). I tried as root out of curiosity and it worked. The complaint … --- ## [Logstash error "Could not execute action: PipelineAction"](https://discuss.elastic.co/t/logstash-error-could-not-execute-action-pipelineaction/176344)
**Author:** [@levi\_munthe](https://discuss.elastic.co/u/levi_munthe)\ **Replies:** 0\ **Last updated:** [April 11, 2019, 7:49am UTC](https://discuss.elastic.co/t/logstash-error-could-not-execute-action-pipelineaction/176344 "2019-04-11T07:49:05Z")
i want to input log message into elasticsearch got an error. \[2019-04-11T14:35:45,839\]\[INFO \]\[logstash.pipeline \] Starting pipeline {:pipeline\_id=\>"kibanalogtest", "pipeline.workers"=\>4, "pipeline.batch.size"=\>12… --- ## [Logstash started but not fetching data](https://discuss.elastic.co/t/logstash-started-but-not-fetching-data/176313)
**Author:** [@smdfilebeat](https://discuss.elastic.co/u/smdfilebeat)\ **Replies:** 2\ **Last updated:** [April 11, 2019, 7:13am UTC](https://discuss.elastic.co/t/logstash-started-but-not-fetching-data/176313 "2019-04-11T07:13:06Z")
I successfully started my logstash, but its not getting any data from filebeat. Filebeat is running with no problem. Steps I did so far: Delete registry Restart filebeat Can someone please help me to debug? I tried g… --- ## [Logstash JDBC scheduling handling](https://discuss.elastic.co/t/logstash-jdbc-scheduling-handling/176220)
**Author:** [@henriot-gricad](https://discuss.elastic.co/u/henriot-gricad)\ **Replies:** 2\ **Last updated:** [April 11, 2019, 7:07am UTC](https://discuss.elastic.co/t/logstash-jdbc-scheduling-handling/176220 "2019-04-11T07:07:27Z")
Hi, I'm using a multiple statement JDBC input with a 10 minute schedule like this: input { jdbc { # DB connexion jdbc\_connection\_string =\> "jdbc:postgresql://myhost.fr:5432/icat" jdbc\_user =\> "db\_ro" … --- ## [Logstash file is missing in /etc/init.d after installing logstash](https://discuss.elastic.co/t/logstash-file-is-missing-in-etc-init-d-after-installing-logstash/176200)
**Author:** [@SManorathna](https://discuss.elastic.co/u/SManorathna)\ **Replies:** 2\ **Last updated:** [April 11, 2019, 6:58am UTC](https://discuss.elastic.co/t/logstash-file-is-missing-in-etc-init-d-after-installing-logstash/176200 "2019-04-11T06:58:59Z")
I am installing logstash 6.3.0 in Ubuntu with the following commands curl -L -O https://artifacts.elastic.co/downloads/logstash/logstash-6.3.0.deb sudo dpkg -i logstash-6.3.0.deb Although the installation is completed… --- ## [\_grokparsefailure for Apache access logs, working fine without fluentbit, works fine on grokdebugger](https://discuss.elastic.co/t/grokparsefailure-for-apache-access-logs-working-fine-without-fluentbit-works-fine-on-grokdebugger/176019)
**Author:** [@vaibhavz](https://discuss.elastic.co/u/vaibhavz)\ **Replies:** 5\ **Last updated:** [April 11, 2019, 6:37am UTC](https://discuss.elastic.co/t/grokparsefailure-for-apache-access-logs-working-fine-without-fluentbit-works-fine-on-grokdebugger/176019 "2019-04-11T06:37:11Z")
Hello, I'm getting \_grokparsefailure while parsing access logs using FluentBit, I'm able to parse it usign grok debugger. sample logs 22.244.133.97 : 22.244.133.97 - - \[09/Apr/2019:11:04:15 +0000\] GET /app/include/sty… --- ## [Not able to get data using tracking\_column as not found in data\_set](https://discuss.elastic.co/t/not-able-to-get-data-using-tracking-column-as-not-found-in-data-set/176323)
**Author:** [@virendra\_oswal](https://discuss.elastic.co/u/virendra_oswal)\ **Replies:** 1\ **Last updated:** [April 11, 2019, 6:24am UTC](https://discuss.elastic.co/t/not-able-to-get-data-using-tracking-column-as-not-found-in-data-set/176323 "2019-04-11T06:24:09Z")
Getting below tracking\_column not found in dataset. {:tracking\_column=\>"LOG\_ID"} Here is my JDBC Input: jdbc { jdbc\_driver\_library =\> "/deploy/libs/ojdbc6-11.2.0.4.0.jar" jdbc\_driver\_class =\> "Java::oracle.jd… --- ## [Logstash JDBC query using joins](https://discuss.elastic.co/t/logstash-jdbc-query-using-joins/176136)
**Author:** [@pyerunka](https://discuss.elastic.co/u/pyerunka)\ **Replies:** 10\ **Last updated:** [April 11, 2019, 5:14am UTC](https://discuss.elastic.co/t/logstash-jdbc-query-using-joins/176136 "2019-04-11T05:14:19Z")
Hello @dadoonet, I have the ELK stack installed and all three are communicating correctly. I'm trying to get logstash to query a 'mysql' database. My query contains joins from multiple tables. Can we used joins using “s… --- ## [Append field to json](https://discuss.elastic.co/t/append-field-to-json/175857)
**Author:** [@arp220](https://discuss.elastic.co/u/arp220)\ **Replies:** 6\ **Last updated:** [April 11, 2019, 3:54am UTC](https://discuss.elastic.co/t/append-field-to-json/175857 "2019-04-11T03:54:25Z")
Hi, i have this json message: {id:123456,time:20190225,color:black,result:1} i want append filed to this json and send to kafka. What i want: {id:123456,time:20190225,color:black,result:1,tag:server1} How should I do… --- ## [Logstash - update CSV content changes into ES](https://discuss.elastic.co/t/logstash-update-csv-content-changes-into-es/175984)
**Author:** [@cheriemilk](https://discuss.elastic.co/u/cheriemilk)\ **Replies:** 8\ **Last updated:** [April 10, 2019, 10:18pm UTC](https://discuss.elastic.co/t/logstash-update-csv-content-changes-into-es/175984 "2019-04-10T22:18:04Z")
Hi Team. I have a user scenario that to maintain automation test cases in elk. Step1(No Issue) - my colleges will send me first CSV file which containing below columns. And I use CSV filter to output them in to ES. 1… --- ## [Non recoverable exception while writing to InfluxDB ERROR](https://discuss.elastic.co/t/non-recoverable-exception-while-writing-to-influxdb-error/176093)
**Author:** [@naveenrt23](https://discuss.elastic.co/u/naveenrt23)\ **Replies:** 4\ **Last updated:** [April 10, 2019, 9:16pm UTC](https://discuss.elastic.co/t/non-recoverable-exception-while-writing-to-influxdb-error/176093 "2019-04-10T21:16:44Z")
Hi, I am trying to send data from logstash to influxdb through logstash\_output\_influxdb plugin. the plugin version is 5.0.5 and logstash version is 6.2.3. I see the below error : \[2019-04-09T15:09:33,095\]\[WARN \]\[log… --- ## [Using HTTP output plugin and mapping tag](https://discuss.elastic.co/t/using-http-output-plugin-and-mapping-tag/176278)
**Author:** [@cyxop](https://discuss.elastic.co/u/cyxop)\ **Replies:** 2\ **Last updated:** [April 10, 2019, 8:21pm UTC](https://discuss.elastic.co/t/using-http-output-plugin-and-mapping-tag/176278 "2019-04-10T20:21:16Z")
I am adding fields to event using mutate { add\_field } When I try to ship it adding event in mapping, I get an error: http { http\_method =\> "post" url =\> "http://endpoint" mapping =\> { "event" =\> "%{event}" "so… --- ## [Buffering logs in logstash to wait for entire trace of logs](https://discuss.elastic.co/t/buffering-logs-in-logstash-to-wait-for-entire-trace-of-logs/176293)
**Author:** [@Mudit\_Mehta](https://discuss.elastic.co/u/Mudit_Mehta)\ **Replies:** 0\ **Last updated:** [April 10, 2019, 8:09pm UTC](https://discuss.elastic.co/t/buffering-logs-in-logstash-to-wait-for-entire-trace-of-logs/176293 "2019-04-10T20:09:51Z")
I have configured logstash on multiple AWS instances. These logstash instances receive thousand of logs and I want a way to wait for the entire trace of logs in logstash and take action on the batch instead of a single l… --- ## [Logstash Error Notification System](https://discuss.elastic.co/t/logstash-error-notification-system/176178)
**Author:** [@muratcelebiler](https://discuss.elastic.co/u/muratcelebiler)\ **Replies:** 4\ **Last updated:** [April 10, 2019, 3:22pm UTC](https://discuss.elastic.co/t/logstash-error-notification-system/176178 "2019-04-10T15:22:34Z")
Hello Guys; I have to sending data with RabbitMQ-Logstash-Elasticsearch tree. I would like watch to the logstash's errors and if logstash has a error then I want send notification to me (slack or etc.). What do I config… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=566) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=568)