# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1149 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1150 --- ## [(Problem) ELK + Amazon S3 + Apache Access Logs](https://discuss.elastic.co/t/problem-elk-amazon-s3-apache-access-logs/51085)
**Author:** [@jnunez](https://discuss.elastic.co/u/jnunez)\ **Replies:** 3\ **Last updated:** [May 28, 2016, 12:53am UTC](https://discuss.elastic.co/t/problem-elk-amazon-s3-apache-access-logs/51085 "2016-05-28T00:53:10Z")
Hi friends, I'm implementing an ELK with apache access logs stores on Amazon S3, but I have a problem when I try to view the graph on Kibana, the dates in the graph doesn't coincide with the dates in logs files. Here a … --- ## [Use filter mutate gsub emoticons (emoji) in unicode values](https://discuss.elastic.co/t/use-filter-mutate-gsub-emoticons-emoji-in-unicode-values/51194)
**Author:** [@Gautam\_S\_Thakur](https://discuss.elastic.co/u/Gautam_S_Thakur)\ **Replies:** 1\ **Last updated:** [May 27, 2016, 9:08pm UTC](https://discuss.elastic.co/t/use-filter-mutate-gsub-emoticons-emoji-in-unicode-values/51194 "2016-05-27T21:08:27Z")
I would like to use mutate (gsub) on "text" fieldname to replace unicode value of emoticons to respective string values. An example - "\\ud83d\\ude27" replaced by "emoticon\_anguished" I tried to escape single backslas… --- ## [Error logsatsh](https://discuss.elastic.co/t/error-logsatsh/51180)
**Author:** [@Jose\_Campos](https://discuss.elastic.co/u/Jose_Campos)\ **Replies:** 1\ **Last updated:** [May 27, 2016, 8:04pm UTC](https://discuss.elastic.co/t/error-logsatsh/51180 "2016-05-27T20:04:16Z")
Hi there I found this issue Trouble parsing csv {:source=\>"message", :raw=\>"", :exception=\>#, :level=\>:warn} ^CSIGINT received. Shutting down the pipeline. {:level=\>:warn} ^CSIGINT received. Terminating immediately.… --- ## [Grok - Skipping Line](https://discuss.elastic.co/t/grok-skipping-line/50605)
**Author:** [@anil\_varghese](https://discuss.elastic.co/u/anil_varghese)\ **Replies:** 8\ **Last updated:** [May 27, 2016, 7:42pm UTC](https://discuss.elastic.co/t/grok-skipping-line/50605 "2016-05-27T19:42:30Z")
Good Evening Everyone, I am a beginner in logstash. I was trying to parse a log file which have 159 lines in it. I am interested only in the first ,second and last line. Is there any way to skip or omit these lines wit… --- ## [Facing problem while installing Logstash,Elasticsearch and Kibana on CentOS 7](https://discuss.elastic.co/t/facing-problem-while-installing-logstash-elasticsearch-and-kibana-on-centos-7/51135)
**Author:** [@its\_joe](https://discuss.elastic.co/u/its_joe)\ **Replies:** 1\ **Last updated:** [May 27, 2016, 7:24pm UTC](https://discuss.elastic.co/t/facing-problem-while-installing-logstash-elasticsearch-and-kibana-on-centos-7/51135 "2016-05-27T19:24:50Z")
Hello All, I am trying to install Logstash,Elasticsearch and Kibana on centos7 (64-bit) using latest rpm of all 3 packages. But each time I face problem while installing. I don't want Nginx with that. I want to run it on… --- ## [Logstash fail to start,it shows translation missing: en.logstash.runner.configuration.file-not-found](https://discuss.elastic.co/t/logstash-fail-to-start-it-shows-translation-missing-en-logstash-runner-configuration-file-not-found/51189)
**Author:** [@Hobbit](https://discuss.elastic.co/u/Hobbit)\ **Replies:** 2\ **Last updated:** [May 27, 2016, 6:57pm UTC](https://discuss.elastic.co/t/logstash-fail-to-start-it-shows-translation-missing-en-logstash-runner-configuration-file-not-found/51189 "2016-05-27T18:57:06Z")
Hi ! I'm following the doc to set up the first-pipeline-conf,but after I ran bin/logstash -f first-pipeline.conf --configtest ,this error shows: An unexpected error occurred! {:error=\>#\ **Author:** [@naga1](https://discuss.elastic.co/u/naga1)\ **Replies:** 2\ **Last updated:** [May 27, 2016, 5:47pm UTC](https://discuss.elastic.co/t/logstash-with-multiple-input-plugins/51173 "2016-05-27T17:47:52Z") Hi, I am trying to have multiple input plugins one is tcp and other is filebeat on the same logstash instance, but I see logs are only coming from the tcp port not from beats servers, logs . Below is the input configur… --- ## [Flushing an incomplete line to the output](https://discuss.elastic.co/t/flushing-an-incomplete-line-to-the-output/51171)
**Author:** [@111121](https://discuss.elastic.co/u/111121)\ **Replies:** 0\ **Last updated:** [May 27, 2016, 3:20pm UTC](https://discuss.elastic.co/t/flushing-an-incomplete-line-to-the-output/51171 "2016-05-27T15:20:34Z")
Hello, I am currently trying to integrate logstash into my company's logging system. The issue is following: I cant change how the files are created No files end with EOL (issue!) Most of the files are one liners … --- ## [Grok parse exception](https://discuss.elastic.co/t/grok-parse-exception/51039)
**Author:** [@Sandhya\_Rathinapandi](https://discuss.elastic.co/u/Sandhya_Rathinapandi)\ **Replies:** 5\ **Last updated:** [May 27, 2016, 11:19am UTC](https://discuss.elastic.co/t/grok-parse-exception/51039 "2016-05-27T11:19:58Z")
Hi I am trying to convert the response time in seconds using the below ruby code in the filter segment. code =\> "event\['responsetime'\] = (event\['responsetime'\].to\_f / 1000000) This was throwing grok parse … --- ## [Index Mapping Templates and sustainable practices](https://discuss.elastic.co/t/index-mapping-templates-and-sustainable-practices/50863)
**Author:** [@whyapenny](https://discuss.elastic.co/u/whyapenny)\ **Replies:** 3\ **Last updated:** [May 27, 2016, 11:18am UTC](https://discuss.elastic.co/t/index-mapping-templates-and-sustainable-practices/50863 "2016-05-27T11:18:37Z")
Wasn't sure whether to post this question in Logstash forum or ElasticSearch since it can technically be in either. Scenario: Multiple applications logging to a single ELK stack. Each application's logs will differ i… --- ## [ConditionalUpdate](https://discuss.elastic.co/t/conditionalupdate/51037)
**Author:** [@Laurens\_Raven](https://discuss.elastic.co/u/Laurens_Raven)\ **Replies:** 1\ **Last updated:** [May 27, 2016, 12:46am UTC](https://discuss.elastic.co/t/conditionalupdate/51037 "2016-05-27T00:46:11Z")
Dear Logstash/ElasticSearch, With logstash I'm reading an event file. Each event has a timestamp with it, and several events are relating to one document in elastic search. Typically events are 1° "started" , … --- ## [Best practice for two type of logs of same application(Ex. Apache)](https://discuss.elastic.co/t/best-practice-for-two-type-of-logs-of-same-application-ex-apache/51064)
**Author:** [@sukhada369](https://discuss.elastic.co/u/sukhada369)\ **Replies:** 1\ **Last updated:** [May 26, 2016, 11:18pm UTC](https://discuss.elastic.co/t/best-practice-for-two-type-of-logs-of-same-application-ex-apache/51064 "2016-05-26T23:18:48Z")
I wanted to know what has to be done in following situation: Scenario: Considering Apache Logs. It has two types of logs with difference in it's fields as access.log and error.log. I wanted to ask if these logs should b… --- ## [Logstash redis input plugin](https://discuss.elastic.co/t/logstash-redis-input-plugin/51099)
**Author:** [@divya\_hc](https://discuss.elastic.co/u/divya_hc)\ **Replies:** 0\ **Last updated:** [May 26, 2016, 10:18pm UTC](https://discuss.elastic.co/t/logstash-redis-input-plugin/51099 "2016-05-26T22:18:15Z")
My logstash indexer is trying to read to from redis server. I am adding redis hash sets. Eg: hgetall user:1000 1) "username" 2) "antirez" 3) "birthyear" 4) "1977" 5) "verified" 6) "1" The logstash indexer doesn't seem… --- ## [Logstash: http\_poller: first URL request's response should be input to second URL's request param](https://discuss.elastic.co/t/logstash-http-poller-first-url-requests-response-should-be-input-to-second-urls-request-param/51057)
**Author:** [@amitvi](https://discuss.elastic.co/u/amitvi)\ **Replies:** 1\ **Last updated:** [May 26, 2016, 10:05pm UTC](https://discuss.elastic.co/t/logstash-http-poller-first-url-requests-response-should-be-input-to-second-urls-request-param/51057 "2016-05-26T22:05:27Z")
I have two URLs (due to security concern i will explain by using dummy) a\> https://xyz.company.com/ui/api/token b\> https://xyz.company.com/request/transaction?date=2016-01-21&token= When you hit url mentioned in po… --- ## [Kv & new line (\\n) errors](https://discuss.elastic.co/t/kv-new-line-n-errors/51098)
**Author:** [@Romey-Rome](https://discuss.elastic.co/u/Romey-Rome)\ **Replies:** 0\ **Last updated:** [May 26, 2016, 9:57pm UTC](https://discuss.elastic.co/t/kv-new-line-n-errors/51098 "2016-05-26T21:57:34Z")
So ran into this problem that is flooding the logs. Doesn't seem to affect functionality & actual indexing. Just 80gb of logstash logs a day. {:timestamp=\>"2016-05-26T14:17:13.352000-0700", :message=\>"Failed action. ",… --- ## [How to use mutate in elasticsearch Input plugin?](https://discuss.elastic.co/t/how-to-use-mutate-in-elasticsearch-input-plugin/51046)
**Author:** [@vikas\_gopal](https://discuss.elastic.co/u/vikas_gopal)\ **Replies:** 1\ **Last updated:** [May 26, 2016, 9:46pm UTC](https://discuss.elastic.co/t/how-to-use-mutate-in-elasticsearch-input-plugin/51046 "2016-05-26T21:46:36Z")
Hi Experts , Requirement :- Remove fields from an existing Index. I am using LS to reindex my data . I am using input { elasticsearch { hosts =\> \["localhost"\] port =\> "9200" index =\> "cef-2016-04-01" scroll =\> "2… --- ## [How to convert a unix\_timestamp field to a new date field](https://discuss.elastic.co/t/how-to-convert-a-unix-timestamp-field-to-a-new-date-field/50966)
**Author:** [@changxy](https://discuss.elastic.co/u/changxy)\ **Replies:** 1\ **Last updated:** [May 26, 2016, 9:45pm UTC](https://discuss.elastic.co/t/how-to-convert-a-unix-timestamp-field-to-a-new-date-field/50966 "2016-05-26T21:45:32Z")
my json log is : { "fileid": 203936363, "timestamp": 1464187170, "pubTime": 1464172574, "reads": 111 } my current filter is: filter{ date { match =\> \[ "timestamp", "UNIX" \] } } now i wish copy the "pubTime… --- ## [Is there a limit on number of redis servers a logstash can read from?](https://discuss.elastic.co/t/is-there-a-limit-on-number-of-redis-servers-a-logstash-can-read-from/50996)
**Author:** [@divya\_hc](https://discuss.elastic.co/u/divya_hc)\ **Replies:** 1\ **Last updated:** [May 26, 2016, 9:43pm UTC](https://discuss.elastic.co/t/is-there-a-limit-on-number-of-redis-servers-a-logstash-can-read-from/50996 "2016-05-26T21:43:28Z")
My usecase is, we have client servers which stores events data in redis. We would like to store these centrally in the elastic search and analyse the log data. What is the limit on the number of external redis servers … --- ## [Replace multiple patterns with gsub mutate filter](https://discuss.elastic.co/t/replace-multiple-patterns-with-gsub-mutate-filter/51070)
**Author:** [@auerbecktj](https://discuss.elastic.co/u/auerbecktj)\ **Replies:** 1\ **Last updated:** [May 26, 2016, 9:07pm UTC](https://discuss.elastic.co/t/replace-multiple-patterns-with-gsub-mutate-filter/51070 "2016-05-26T21:07:44Z")
I'm trying to write a mutate filter that will replace a certain set of symbols as well as a string in a certain field that we've defined. Currently we're trying to do something like :slight\_smile :slight\_smile : mutat… --- ## [How to grok something out of the middle of a line?](https://discuss.elastic.co/t/how-to-grok-something-out-of-the-middle-of-a-line/50998)
**Author:** [@jerrac](https://discuss.elastic.co/u/jerrac)\ **Replies:** 3\ **Last updated:** [May 26, 2016, 8:56pm UTC](https://discuss.elastic.co/t/how-to-grok-something-out-of-the-middle-of-a-line/50998 "2016-05-26T20:56:32Z")
If I have a line like: the quick brown fox How can I grok "brown" out of that line? (?\(brown)) Does not work. If I do something like (?\.\*(brown)), the color field ends up with "the quick brown". My… --- ## [Logstash + CSVs Data](https://discuss.elastic.co/t/logstash-csvs-data/51079)
**Author:** [@sarthak2016](https://discuss.elastic.co/u/sarthak2016)\ **Replies:** 0\ **Last updated:** [May 26, 2016, 4:03pm UTC](https://discuss.elastic.co/t/logstash-csvs-data/51079 "2016-05-26T16:03:37Z")
Hello All, Here is Elastic Search mapping, "properties": { "Runtime": { "type": "date", "format": "epoch\_millis||MM/dd/yyyy HH:mm" }, "Cstring1": { "type": "string", "index": "not\_analyzed" }, "Cstring2"… --- ## [No tengo conexión de logstash a elastisearch](https://discuss.elastic.co/t/no-tengo-conexion-de-logstash-a-elastisearch/50337)
**Author:** [@Diego\_Gomez](https://discuss.elastic.co/u/Diego_Gomez)\ **Replies:** 14\ **Last updated:** [May 26, 2016, 3:23pm UTC](https://discuss.elastic.co/t/no-tengo-conexion-de-logstash-a-elastisearch/50337 "2016-05-26T15:23:29Z")
Hola a todos tengo un problema no puedo conectar el el logstash a elastisearch aunque me dice el logstash que la configuración esta ok y se inicia perfecto, pero no lo veo reflejado ni en elasticsearch ni en kibana, acá … --- ## [Jopenssl/load error on Logstash start](https://discuss.elastic.co/t/jopenssl-load-error-on-logstash-start/37759)
**Author:** [@Nitz](https://discuss.elastic.co/u/Nitz)\ **Replies:** 3\ **Last updated:** [May 26, 2016, 3:01pm UTC](https://discuss.elastic.co/t/jopenssl-load-error-on-logstash-start/37759 "2016-05-26T15:01:51Z")
Hi everyone, I installed Logstash 2.1.1 on Ubuntu 14.04. I'm trying to work with it, but when I start the service - I got the following stacktrace error: LoadError: load error: jopenssl/load -- java.lang.VerifyError:… --- ## [Remove part of a field](https://discuss.elastic.co/t/remove-part-of-a-field/51043)
**Author:** [@Karl\_Trasschaert](https://discuss.elastic.co/u/Karl_Trasschaert)\ **Replies:** 3\ **Last updated:** [May 26, 2016, 12:58pm UTC](https://discuss.elastic.co/t/remove-part-of-a-field/51043 "2016-05-26T12:58:34Z")
Hi, I'm working with phone numbers, i would like to save the 2 or 3 first digit of a phone number ( international code) . Actually i have a field "called\_number", how can i save the 2 or 3 first digit in another fie… --- ## [Exception while starting Logstash \[LoadError: load error: jopenssl/load -- java.lang.VerifyError: class org.bouncycastle.asn1.ASN1Primitive overrides final method equals\]](https://discuss.elastic.co/t/exception-while-starting-logstash-loaderror-load-error-jopenssl-load-java-lang-verifyerror-class-org-bouncycastle-asn1-asn1primitive-overrides-final-method-equals/51024)
**Author:** [@ravitandur](https://discuss.elastic.co/u/ravitandur)\ **Replies:** 1\ **Last updated:** [May 26, 2016, 11:32am UTC](https://discuss.elastic.co/t/exception-while-starting-logstash-loaderror-load-error-jopenssl-load-java-lang-verifyerror-class-org-bouncycastle-asn1-asn1primitive-overrides-final-method-equals/51024 "2016-05-26T11:32:04Z")
I am getting below exception while starting logstash \[logstash-1.5.3\]. Can any one please help me. LoadError: load error: jopenssl/load -- java.lang.VerifyError: class org.bouncycastle.asn1.ASN1Primitive overrides final … --- ## [Different ssyslog source](https://discuss.elastic.co/t/different-ssyslog-source/50843)
**Author:** [@Karl\_Trasschaert](https://discuss.elastic.co/u/Karl_Trasschaert)\ **Replies:** 2\ **Last updated:** [May 26, 2016, 10:27am UTC](https://discuss.elastic.co/t/different-ssyslog-source/50843 "2016-05-26T10:27:13Z")
Hi, How can i apply different filter on different source. Actually, all my devices send logs to the same port 514 natted to 1514. i need to apply a different filter for one of the devices, how can i change the typ… --- ## [Can't get rsyslog+ELK working](https://discuss.elastic.co/t/cant-get-rsyslog-elk-working/51009)
**Author:** [@kevinthierry](https://discuss.elastic.co/u/kevinthierry)\ **Replies:** 3\ **Last updated:** [May 26, 2016, 8:51am UTC](https://discuss.elastic.co/t/cant-get-rsyslog-elk-working/51009 "2016-05-26T08:51:59Z")
Hello, I am trying to setup a rsyslog+ELK stack following this tutorial without success: https://www.elastic.co/blog/how-to-centralize-logs-with-rsyslog-logstash-and-elasticsearch-on-ubuntu-14-04 My configuration is … --- ## [Elasticsearch output plugin: "workers" setting](https://discuss.elastic.co/t/elasticsearch-output-plugin-workers-setting/43644)
**Author:** [@mguiwin](https://discuss.elastic.co/u/mguiwin)\ **Replies:** 1\ **Last updated:** [May 26, 2016, 8:49am UTC](https://discuss.elastic.co/t/elasticsearch-output-plugin-workers-setting/43644 "2016-05-26T08:49:41Z")
Hello, If I understand well what is described in the Logstash Processing Pipeline article , it might be that the number of ES output plugin workers (as defined by the 'workers' setting of the ES output plugin) equals … --- ## [File input does not work (even when stdin does!)](https://discuss.elastic.co/t/file-input-does-not-work-even-when-stdin-does/51007)
**Author:** [@jpatani](https://discuss.elastic.co/u/jpatani)\ **Replies:** 2\ **Last updated:** [May 26, 2016, 4:03am UTC](https://discuss.elastic.co/t/file-input-does-not-work-even-when-stdin-does/51007 "2016-05-26T04:03:08Z")
This has been asked before but not sure if it has been resolved. My logstash config file as a simple file input (specifying a single log file). When executing, it doesn't read the log file. It does however work when … --- ## [Logstash error with redis and elasticsearch](https://discuss.elastic.co/t/logstash-error-with-redis-and-elasticsearch/50970)
**Author:** [@pvssraju](https://discuss.elastic.co/u/pvssraju)\ **Replies:** 1\ **Last updated:** [May 26, 2016, 1:43am UTC](https://discuss.elastic.co/t/logstash-error-with-redis-and-elasticsearch/50970 "2016-05-26T01:43:53Z")
HI Team, I am getting below error when I am trying to start logstash service for redis {:timestamp=\>"2016-05-25T12:27:35.665000-0400", :message=\>"Pipeline aborted due to error", :exception=\>#https://discuss.elastic.c… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1148) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1150)