# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1217 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1218 --- ## [New fields resolve as expected in stdout output but not in elasticsearch output](https://discuss.elastic.co/t/new-fields-resolve-as-expected-in-stdout-output-but-not-in-elasticsearch-output/35206)
**Author:** [@vtst2412](https://discuss.elastic.co/u/vtst2412)\ **Replies:** 2\ **Last updated:** [November 24, 2015, 9:35pm UTC](https://discuss.elastic.co/t/new-fields-resolve-as-expected-in-stdout-output-but-not-in-elasticsearch-output/35206 "2015-11-24T21:35:04Z")
input { file { type =\> "json" path =\> "/home/user/json/test.json" #codec =\> json start\_position =\> "beginning" sincedb\_path =\> "/dev/null" } } filter { json { source =\> "message"… --- ## [Want to extract data from a file path, i have the path in mysql ? possible in logstash?](https://discuss.elastic.co/t/want-to-extract-data-from-a-file-path-i-have-the-path-in-mysql-possible-in-logstash/35240)
**Author:** [@Hack\_Twentfour](https://discuss.elastic.co/u/Hack_Twentfour)\ **Replies:** 3\ **Last updated:** [November 24, 2015, 9:06pm UTC](https://discuss.elastic.co/t/want-to-extract-data-from-a-file-path-i-have-the-path-in-mysql-possible-in-logstash/35240 "2015-11-24T21:06:35Z")
Hi, I am very new to logstah. my requirement is read the log file path from mysql field and extract file for elasticseacrh. something like this: Mysql database:testDB-\>fileTable-\>fileLocation :"/path/to/the/file.log" … --- ## [ElasticSearch indexing events from previous date into current date logstash-\* folder](https://discuss.elastic.co/t/elasticsearch-indexing-events-from-previous-date-into-current-date-logstash-folder/35177)
**Author:** [@raghu\_ae](https://discuss.elastic.co/u/raghu_ae)\ **Replies:** 5\ **Last updated:** [November 24, 2015, 8:08pm UTC](https://discuss.elastic.co/t/elasticsearch-indexing-events-from-previous-date-into-current-date-logstash-folder/35177 "2015-11-24T20:08:31Z")
Elastic search is indexing the events from November 19th 2015, 18:00:00.000 into logstash-2015.11.20. Could you please let us know why events from previous date are getting indexed into current date? --- ## [Ruby errors after updating logstash-input-s3 to 2.0.3](https://discuss.elastic.co/t/ruby-errors-after-updating-logstash-input-s3-to-2-0-3/35107)
**Author:** [@plonka2000](https://discuss.elastic.co/u/plonka2000)\ **Replies:** 1\ **Last updated:** [November 24, 2015, 3:05pm UTC](https://discuss.elastic.co/t/ruby-errors-after-updating-logstash-input-s3-to-2-0-3/35107 "2015-11-24T15:05:21Z")
Today I updated the plugin logstash-input-s3 to version 2.0.3 using this command: /opt/logstash/bin/plugin update logstash-input-s3 Simple enough, but now I get a bunch of Ruby errors: RuntimeError: Logstash expects… --- ## [LS2 'stuck'?](https://discuss.elastic.co/t/ls2-stuck/35270)
**Author:** [@mad\_min](https://discuss.elastic.co/u/mad_min)\ **Replies:** 6\ **Last updated:** [November 24, 2015, 1:59pm UTC](https://discuss.elastic.co/t/ls2-stuck/35270 "2015-11-24T13:59:35Z")
Hi group, I have a LS2 setup taking input from Redis and sending output to remote ES2. LS2 has multiple multiline filters and two outputs: debug and ES2. LS2 seems to be 'stuck'/'frozen' after going through several e… --- ## [Date filter issues](https://discuss.elastic.co/t/date-filter-issues/35455)
**Author:** [@Kryten](https://discuss.elastic.co/u/Kryten)\ **Replies:** 2\ **Last updated:** [November 24, 2015, 1:49pm UTC](https://discuss.elastic.co/t/date-filter-issues/35455 "2015-11-24T13:49:59Z")
Having problems updating the @timestamp value for my messages. Would appreciate any insight into what might be going wrong here. Sample log line:- 2015-11-24 11:44:55,888 INFO \[com.blandio.ct.avd.dao.CSDAO: 238\] - Dis… --- ## [Logstash stops without opening file to read](https://discuss.elastic.co/t/logstash-stops-without-opening-file-to-read/35336)
**Author:** [@fares](https://discuss.elastic.co/u/fares)\ **Replies:** 7\ **Last updated:** [November 24, 2015, 1:22pm UTC](https://discuss.elastic.co/t/logstash-stops-without-opening-file-to-read/35336 "2015-11-24T13:22:40Z")
Hi, Here's my config file : input{file { path =\> "/data/ttr.csv" sincedb\_path =\> "/dev/null" start\_position =\> "beginning" }} output{ stdout{ } } When i start logstash with debug mode: Reading config file… --- ## [How can I convert to geo\_point a field extracted with translate filter?](https://discuss.elastic.co/t/how-can-i-convert-to-geo-point-a-field-extracted-with-translate-filter/35141)
**Author:** [@adellarocca](https://discuss.elastic.co/u/adellarocca)\ **Replies:** 8\ **Last updated:** [November 24, 2015, 1:19pm UTC](https://discuss.elastic.co/t/how-can-i-convert-to-geo-point-a-field-extracted-with-translate-filter/35141 "2015-11-24T13:19:43Z")
Hi! I have extracted geo information with translate filter using a dictionary file YAML as follow location1: 42.1497, -74.9384 location2: 34.5677, -32.8393 ......................................... The logstash conf fil… --- ## [Cannot listen file in logstash with absolute path in windows](https://discuss.elastic.co/t/cannot-listen-file-in-logstash-with-absolute-path-in-windows/35444)
**Author:** [@djvidov](https://discuss.elastic.co/u/djvidov)\ **Replies:** 2\ **Last updated:** [November 24, 2015, 12:36pm UTC](https://discuss.elastic.co/t/cannot-listen-file-in-logstash-with-absolute-path-in-windows/35444 "2015-11-24T12:36:47Z")
I have logstash installed on path: D:\\WORK\\ElasticSearch\\logstash-2.0.0\\bin and my apps write logs on path: D:\\Logs When I use logstash path to read files from "D:\\WORK\\ElasticSearch\\logstash-2.0.0\\bin" with this scrip… --- ## [@timestamp not set when parsing nginx logs](https://discuss.elastic.co/t/timestamp-not-set-when-parsing-nginx-logs/35432)
**Author:** [@jon\_black](https://discuss.elastic.co/u/jon_black)\ **Replies:** 2\ **Last updated:** [November 24, 2015, 12:35pm UTC](https://discuss.elastic.co/t/timestamp-not-set-when-parsing-nginx-logs/35432 "2015-11-24T12:35:53Z")
For a long time I assumed @timestamp was being parsed correctly; the time in the kibana Time field matches the @timestamp field, which matched the data being logged; however, logstash crashed over the weekend so I starte… --- ## [How do I set the pubblish\_address in logstash?](https://discuss.elastic.co/t/how-do-i-set-the-pubblish-address-in-logstash/34935)
**Author:** [@danilo\_gimenez\_ramos](https://discuss.elastic.co/u/danilo_gimenez_ramos)\ **Replies:** 4\ **Last updated:** [November 24, 2015, 11:26am UTC](https://discuss.elastic.co/t/how-do-i-set-the-pubblish-address-in-logstash/34935 "2015-11-24T11:26:51Z")
Hi, I see a lot of discussions on how to set the publish address on logstash, but all answers refers to config the bind\_host on elasticsearch server rather than logstash. My problem is, I have many servers where I ins… --- ## [Install logstash 2.0 in windows server 2008 R2](https://discuss.elastic.co/t/install-logstash-2-0-in-windows-server-2008-r2/35433)
**Author:** [@supermario](https://discuss.elastic.co/u/supermario)\ **Replies:** 0\ **Last updated:** [November 24, 2015, 10:20am UTC](https://discuss.elastic.co/t/install-logstash-2-0-in-windows-server-2008-r2/35433 "2015-11-24T10:20:01Z")
when i install logstash 2.0 in windows server 2008 R2 , i get a error . i already install jdk 1.8 and add environment path. when i run logstash -f logstash.conf(contains input and out setting). i get error unable to… --- ## [Logstash running extremely slow](https://discuss.elastic.co/t/logstash-running-extremely-slow/35373)
**Author:** [@brashendeavours](https://discuss.elastic.co/u/brashendeavours)\ **Replies:** 2\ **Last updated:** [November 24, 2015, 7:47am UTC](https://discuss.elastic.co/t/logstash-running-extremely-slow/35373 "2015-11-24T07:47:07Z")
So I'm running Bro + ELK on a Ubuntu 14.04 LTS box and I'm getting less than one request per second fed into elasticsearch... I'm running Bro 2.4.1 Logstash 2.0.0 Elasticsearch 2.0.0 Kibana 4.2.1 Bro peels through my… --- ## [Logstash forwarder question](https://discuss.elastic.co/t/logstash-forwarder-question/34955)
**Author:** [@Ariel\_Assaraf](https://discuss.elastic.co/u/Ariel_Assaraf)\ **Replies:** 1\ **Last updated:** [November 24, 2015, 7:16am UTC](https://discuss.elastic.co/t/logstash-forwarder-question/34955 "2015-11-24T07:16:09Z")
Hi all, we at Coralogix are working on a logstash forwarder to our Log Analytics application. Since we need the log data structured we need the user to do some pre-definitions. The question is: will you as Logstash u… --- ## [Logstash Grep and Drop](https://discuss.elastic.co/t/logstash-grep-and-drop/35313)
**Author:** [@Jackal9301](https://discuss.elastic.co/u/Jackal9301)\ **Replies:** 3\ **Last updated:** [November 24, 2015, 6:52am UTC](https://discuss.elastic.co/t/logstash-grep-and-drop/35313 "2015-11-24T06:52:25Z")
I have an old conf file my predecessor made with grep in it for logstash. Under that is the Drop command. What can I use as the equivalent to make it still work the same way? if \[SourceModuleName\] == 'eftxferlogs' { … --- ## [How can two Logstash istances communicate to each other?](https://discuss.elastic.co/t/how-can-two-logstash-istances-communicate-to-each-other/35306)
**Author:** [@sampasei](https://discuss.elastic.co/u/sampasei)\ **Replies:** 3\ **Last updated:** [November 24, 2015, 6:47am UTC](https://discuss.elastic.co/t/how-can-two-logstash-istances-communicate-to-each-other/35306 "2015-11-24T06:47:07Z")
Hi, i have two logstash istances (not forwarder, not filebet) on two different servers: which input/output plugins should i use to permit the data transfer to each other? Thanks --- ## [Join data from an ES index with data from logstash](https://discuss.elastic.co/t/join-data-from-an-es-index-with-data-from-logstash/35330)
**Author:** [@OlivierD](https://discuss.elastic.co/u/OlivierD)\ **Replies:** 1\ **Last updated:** [November 24, 2015, 6:43am UTC](https://discuss.elastic.co/t/join-data-from-an-es-index-with-data-from-logstash/35330 "2015-11-24T06:43:47Z")
I have indexed data from a referential (a list of providers, with their full name) from a file. With logstash, I index data from a DB that contains also the ID of providers. I want, when reading data from DB with logst… --- ## [Logstash jdbc plugin refresh new data](https://discuss.elastic.co/t/logstash-jdbc-plugin-refresh-new-data/35320)
**Author:** [@Rishik\_Bahri](https://discuss.elastic.co/u/Rishik_Bahri)\ **Replies:** 1\ **Last updated:** [November 24, 2015, 3:57am UTC](https://discuss.elastic.co/t/logstash-jdbc-plugin-refresh-new-data/35320 "2015-11-24T03:57:45Z")
Hi, I am suing elasticsearch 1.7 with logstash 1.3 and trying to use Logstash to add new data added from an oracle database to a elasticsearch index. For this we are using the jdbc plugin but the refresh job doesnt see… --- ## [Custom template is getting ignored by latest LS2.0](https://discuss.elastic.co/t/custom-template-is-getting-ignored-by-latest-ls2-0/35363)
**Author:** [@vikas\_gopal](https://discuss.elastic.co/u/vikas_gopal)\ **Replies:** 1\ **Last updated:** [November 23, 2015, 10:26pm UTC](https://discuss.elastic.co/t/custom-template-is-getting-ignored-by-latest-ls2-0/35363 "2015-11-23T22:26:58Z")
Hi Experts , I am trying to implement my custom mapping which is in "elasticsearch-2.0.0\\config\\templates" folder. Actually I copied my old template from LS1.5.4. It was working absolutely fine in the older version of … --- ## [Using LS 2.0, can't get split and/or geoip working - trying to split my xforwardfor field and pass into geoip](https://discuss.elastic.co/t/using-ls-2-0-cant-get-split-and-or-geoip-working-trying-to-split-my-xforwardfor-field-and-pass-into-geoip/35354)
**Author:** [@scumola](https://discuss.elastic.co/u/scumola)\ **Replies:** 2\ **Last updated:** [November 23, 2015, 9:22pm UTC](https://discuss.elastic.co/t/using-ls-2-0-cant-get-split-and-or-geoip-working-trying-to-split-my-xforwardfor-field-and-pass-into-geoip/35354 "2015-11-23T21:22:59Z")
My LS filter config line: split { field =\> "xforwardfor" terminator =\> "," } but in kibana, I still see the field as one string "referer": "http://www.mysite.com/", "xforwardfor": "1.2.3.4, 5.6.7.8", "hostname": … --- ## [Transforming data WHILE indexing](https://discuss.elastic.co/t/transforming-data-while-indexing/35292)
**Author:** [@KavyaS](https://discuss.elastic.co/u/KavyaS)\ **Replies:** 8\ **Last updated:** [November 23, 2015, 7:10pm UTC](https://discuss.elastic.co/t/transforming-data-while-indexing/35292 "2015-11-23T19:10:42Z")
How can we transform data while it gets indexed? Some examples please. --- ## [Logstash 2.0 change configuration without restart?](https://discuss.elastic.co/t/logstash-2-0-change-configuration-without-restart/35191)
**Author:** [@Deseao](https://discuss.elastic.co/u/Deseao)\ **Replies:** 2\ **Last updated:** [November 23, 2015, 5:15pm UTC](https://discuss.elastic.co/t/logstash-2-0-change-configuration-without-restart/35191 "2015-11-23T17:15:01Z")
I was under the impression in the webinar about Elasticsearch 2.0, Kibana 4.2.1, and Logstash 2.0 that it would now be possible to change the configuration without bringing down the logstash instance. Is this not the cas… --- ## [Logstash crash on windows under high volume](https://discuss.elastic.co/t/logstash-crash-on-windows-under-high-volume/35076)
**Author:** [@arizonawayfarer](https://discuss.elastic.co/u/arizonawayfarer)\ **Replies:** 7\ **Last updated:** [November 23, 2015, 4:00pm UTC](https://discuss.elastic.co/t/logstash-crash-on-windows-under-high-volume/35076 "2015-11-23T16:00:06Z")
When logstash has eventlog inputs configured on systems that have a high volume of eventlog traffic it crashes after a relatively short period of time with the following dump: Peak volume appears to be about 150 eventlo… --- ## [Remove nested dynamic fields](https://discuss.elastic.co/t/remove-nested-dynamic-fields/35073)
**Author:** [@masoman22](https://discuss.elastic.co/u/masoman22)\ **Replies:** 2\ **Last updated:** [November 23, 2015, 3:40pm UTC](https://discuss.elastic.co/t/remove-nested-dynamic-fields/35073 "2015-11-23T15:40:22Z")
Hey, Was hoping someone can help me. I'm trying to reduce the amount of fields that are being created in elasticsearch. I'm using the kafka input for logstash 2.0 and elasticsearch 2.0. I would like to remove the \[bid… --- ## [CouchDB plugin set dynamic type for Elasticsearch](https://discuss.elastic.co/t/couchdb-plugin-set-dynamic-type-for-elasticsearch/2293)
**Author:** [@JuHwon](https://discuss.elastic.co/u/JuHwon)\ **Replies:** 21\ **Last updated:** [November 23, 2015, 3:34pm UTC](https://discuss.elastic.co/t/couchdb-plugin-set-dynamic-type-for-elasticsearch/2293 "2015-11-23T15:34:15Z")
Hello I try to set up couchdb with logstash and elasticsearch. I already managed to get the couchdb. My documents in the couchDB do have a type attribute and i want this type to be the index type in my elasticsearch. … --- ## [\[solved\] \[accuracy\] randomly missing events](https://discuss.elastic.co/t/solved-accuracy-randomly-missing-events/34813)
**Author:** [@vgm](https://discuss.elastic.co/u/vgm)\ **Replies:** 3\ **Last updated:** [November 23, 2015, 9:46am UTC](https://discuss.elastic.co/t/solved-accuracy-randomly-missing-events/34813 "2015-11-23T09:46:19Z")
Hi all, I'm wondering whether I can use logstash for ensuring accuracy during log collection (i.e. that no events will be skipped during input or output). For instance, let's consider the following configuration: inpu… --- ## [Logstash produces duplicates](https://discuss.elastic.co/t/logstash-produces-duplicates/35216)
**Author:** [@vladmiller](https://discuss.elastic.co/u/vladmiller)\ **Replies:** 2\ **Last updated:** [November 23, 2015, 8:38am UTC](https://discuss.elastic.co/t/logstash-produces-duplicates/35216 "2015-11-23T08:38:18Z")
Hello, So the goal is to import existing mysql table which has about 2 million records in the ES index. However, ES index after a while has much more data. I also try to generate unique sha1 fingerprint of each message… --- ## [Odd Logstash Message](https://discuss.elastic.co/t/odd-logstash-message/35254)
**Author:** [@dsmhood](https://discuss.elastic.co/u/dsmhood)\ **Replies:** 4\ **Last updated:** [November 23, 2015, 5:04am UTC](https://discuss.elastic.co/t/odd-logstash-message/35254 "2015-11-23T05:04:23Z")
Hi all, I'm trying to use Logstash for the first time, I've setup my configuration files and it was working perfectly but suddenly now it's stopped. I think I've made a change but I don't 100% remember what change woul… --- ## [Logstash 2.0 with https protocol to elasticsearch](https://discuss.elastic.co/t/logstash-2-0-with-https-protocol-to-elasticsearch/35256)
**Author:** [@sasha\_meltser](https://discuss.elastic.co/u/sasha_meltser)\ **Replies:** 1\ **Last updated:** [November 23, 2015, 1:46am UTC](https://discuss.elastic.co/t/logstash-2-0-with-https-protocol-to-elasticsearch/35256 "2015-11-23T01:46:30Z")
I have install Elasticsearch 2.0, Kibana 4.2, Logstash 2.0 and Shield. After configuring logstash with input file, filter and output I am unable to send logs to elasticsearch error message is {:timestamp=\>"2015-… --- ## [Logstash and ~8000 files in ~40 file inputs?](https://discuss.elastic.co/t/logstash-and-8000-files-in-40-file-inputs/35169)
**Author:** [@jsosic](https://discuss.elastic.co/u/jsosic)\ **Replies:** 3\ **Last updated:** [November 22, 2015, 11:36pm UTC](https://discuss.elastic.co/t/logstash-and-8000-files-in-40-file-inputs/35169 "2015-11-22T23:36:02Z")
Hi guys, I have a specific situation where I want to index ~8000 files from a network share. Those are IIS logs that are rotated regularly. Is this something that one 2.0.0 agent on Linux can handle, or is it maybe s… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1216) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1218)