# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=998 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 999 --- ## [@timestamp with \[13-Apr-2017 16:36:22 America/New\_York\]](https://discuss.elastic.co/t/timestamp-with-13-apr-2017-16-36-22-america-new-york/82441)
**Author:** [@mikemi](https://discuss.elastic.co/u/mikemi)\ **Replies:** 2\ **Last updated:** [April 19, 2017, 3:22am UTC](https://discuss.elastic.co/t/timestamp-with-13-apr-2017-16-36-22-america-new-york/82441 "2017-04-19T03:22:41Z")
I have a PHP log I'm trying to get times from. Logstash is just setting @timestamp as the current time the log is parsed. The log starts off with this: \[13-Apr-2017 16:36:22 America/New\_York\] The relevant part of m… --- ## [Multiple Logstash 2.2 on single machine](https://discuss.elastic.co/t/multiple-logstash-2-2-on-single-machine/41612)
**Author:** [@Kryten](https://discuss.elastic.co/u/Kryten)\ **Replies:** 2\ **Last updated:** [April 19, 2017, 1:49am UTC](https://discuss.elastic.co/t/multiple-logstash-2-2-on-single-machine/41612 "2017-04-19T01:49:44Z")
Hi, I'm sure I asked this before and was never able to get it working properly. With Logstash 2.2 is it possible to run two or more completely independent instances of Logstash at the same time? if so, are there any … --- ## [Unable to parse a simple log file using logstash](https://discuss.elastic.co/t/unable-to-parse-a-simple-log-file-using-logstash/82582)
**Author:** [@ItsMeSurajR](https://discuss.elastic.co/u/ItsMeSurajR)\ **Replies:** 3\ **Last updated:** [April 18, 2017, 10:10pm UTC](https://discuss.elastic.co/t/unable-to-parse-a-simple-log-file-using-logstash/82582 "2017-04-18T22:10:46Z")
I'm a beginner and I could able to parse a log file successfully before. But all after a sudden, even a simple conf isn't working. Can anyone tell me what's wrong here? There are no errors, everything looks ok for me but… --- ## [Logstash performance degrades as I add in a filter and output](https://discuss.elastic.co/t/logstash-performance-degrades-as-i-add-in-a-filter-and-output/82764)
**Author:** [@k1ng87](https://discuss.elastic.co/u/k1ng87)\ **Replies:** 0\ **Last updated:** [April 18, 2017, 6:45pm UTC](https://discuss.elastic.co/t/logstash-performance-degrades-as-i-add-in-a-filter-and-output/82764 "2017-04-18T18:45:11Z")
I'm noticing something very odd in performance in logstash. If I have my config file set like this: input { kafka { topics =\> \["kafka-jmx"\] bootstrap\_servers =\> "kafka1.com:909… --- ## [MySQL fields not visible in Kibana](https://discuss.elastic.co/t/mysql-fields-not-visible-in-kibana/82574)
**Author:** [@Ferdi](https://discuss.elastic.co/u/Ferdi)\ **Replies:** 7\ **Last updated:** [April 18, 2017, 6:06pm UTC](https://discuss.elastic.co/t/mysql-fields-not-visible-in-kibana/82574 "2017-04-18T18:06:24Z")
I have loaded data into Elastic Seach using logstash and a JDBC MysQL driver. However, in Kibana, I see stuff like: cluster\_uuid:xxxxxxxxxxxxxxxxx timestamp:April 11th 2017, 05:37:39.580 source\_node.uuid:yyyyyyyyyyyy… --- ## [Check if an array contains some value](https://discuss.elastic.co/t/check-if-an-array-contains-some-value/82669)
**Author:** [@Mohamed](https://discuss.elastic.co/u/Mohamed)\ **Replies:** 1\ **Last updated:** [April 18, 2017, 4:45pm UTC](https://discuss.elastic.co/t/check-if-an-array-contains-some-value/82669 "2017-04-18T16:45:48Z")
Hello, Let's say I have created a field like this: "market": \[ { "name": \["name1","name2","name3","name4","name5" \] }, {"place":\["P1","P2","P3"\]} \] No… --- ## [Error when i try to crowle a secured mail box with IMAP plugin in logstash](https://discuss.elastic.co/t/error-when-i-try-to-crowle-a-secured-mail-box-with-imap-plugin-in-logstash/82733)
**Author:** [@amine](https://discuss.elastic.co/u/amine)\ **Replies:** 0\ **Last updated:** [April 18, 2017, 2:19pm UTC](https://discuss.elastic.co/t/error-when-i-try-to-crowle-a-secured-mail-box-with-imap-plugin-in-logstash/82733 "2017-04-18T14:19:31Z")
Hi everyone; I try to crowle a secured mail box with IMAP plugin in logstash with this configuration : imap { host =\> "mydom.mydomain.com" user =\> "test-search@mydomain.com" password =\> "myPas… --- ## [Re clowling a mail box with IMAP logstash plugin](https://discuss.elastic.co/t/re-clowling-a-mail-box-with-imap-logstash-plugin/82730)
**Author:** [@amine](https://discuss.elastic.co/u/amine)\ **Replies:** 0\ **Last updated:** [April 18, 2017, 2:08pm UTC](https://discuss.elastic.co/t/re-clowling-a-mail-box-with-imap-logstash-plugin/82730 "2017-04-18T14:08:37Z")
Hi everyone; I'm trying to index Email box with logstash IMAP plugin, but i want to restart crawling Everytime i restart my Logstash. You can help me to do it? thank you in advance. --- ## [Input jdbc to gelf output](https://discuss.elastic.co/t/input-jdbc-to-gelf-output/82717)
**Author:** [@skoruppa](https://discuss.elastic.co/u/skoruppa)\ **Replies:** 0\ **Last updated:** [April 18, 2017, 1:17pm UTC](https://discuss.elastic.co/t/input-jdbc-to-gelf-output/82717 "2017-04-18T13:17:39Z")
Hello I try to parse logs from my database to gelf using logstash, with my current configuration input { jdbc { jdbc\_driver\_library =\> "/home/ubuntu/sqljdbc\_6.0/enu/jre8/sqljdbc42.jar" jdbc\_driver\_class =\> "… --- ## [Logstash not forwarding logs to ES via Kafka](https://discuss.elastic.co/t/logstash-not-forwarding-logs-to-es-via-kafka/82602)
**Author:** [@luciferdude](https://discuss.elastic.co/u/luciferdude)\ **Replies:** 5\ **Last updated:** [April 18, 2017, 1:09pm UTC](https://discuss.elastic.co/t/logstash-not-forwarding-logs-to-es-via-kafka/82602 "2017-04-18T13:09:31Z")
I'm using ELK 5.0.1 and Kafka 0.10.1.0 . I'm not sure why my logs aren't forwarding I installed Kafkacat and was successfully able to Produce and Consume logs from all the 3 servers where Kafka cluster is installed. … --- ## [ERROR\]\[logstash.outputs.elasticsearch\] Failed to install template. {:message=\>"Unexpected character ('\<' (code 60))](https://discuss.elastic.co/t/error-logstash-outputs-elasticsearch-failed-to-install-template-message-unexpected-character-code-60/82049)
**Author:** [@yoshua](https://discuss.elastic.co/u/yoshua)\ **Replies:** 6\ **Last updated:** [April 18, 2017, 12:38pm UTC](https://discuss.elastic.co/t/error-logstash-outputs-elasticsearch-failed-to-install-template-message-unexpected-character-code-60/82049 "2017-04-18T12:38:14Z")
Hi, In elastic 5.2.2 and localhost the attach .conf and .json srcipts tun ok But in elastic cloud trial 5.3 Don't work Any suggestion Thanks Luis Logstash output { elasticsearch { user =\> "elastic" … --- ## [Check whether a given file in input section is processed and give the status like file processed to output section](https://discuss.elastic.co/t/check-whether-a-given-file-in-input-section-is-processed-and-give-the-status-like-file-processed-to-output-section/82391)
**Author:** [@anil\_kumar\_Samantula](https://discuss.elastic.co/u/anil_kumar_Samantula)\ **Replies:** 1\ **Last updated:** [April 18, 2017, 11:31am UTC](https://discuss.elastic.co/t/check-whether-a-given-file-in-input-section-is-processed-and-give-the-status-like-file-processed-to-output-section/82391 "2017-04-18T11:31:08Z")
Hi , Can anyone pls help me out .Logstash continuously listens to input section and pumps the data to Output Section by processing data in filter section ,but problem is! if i gave file in input section it keeps on … --- ## [Logstash with kafka input - refresh topic list](https://discuss.elastic.co/t/logstash-with-kafka-input-refresh-topic-list/82701)
**Author:** [@devorin](https://discuss.elastic.co/u/devorin)\ **Replies:** 0\ **Last updated:** [April 18, 2017, 11:15am UTC](https://discuss.elastic.co/t/logstash-with-kafka-input-refresh-topic-list/82701 "2017-04-18T11:15:15Z")
I am useing kafka input plugin to consume data from single topic. It works fine, but I have new usecase to do. I have to read from many topics. I have tried to use topics\_pattern and it works well if the list of topics d… --- ## [Logstash mutate replace field](https://discuss.elastic.co/t/logstash-mutate-replace-field/82530)
**Author:** [@Jonesthomas](https://discuss.elastic.co/u/Jonesthomas)\ **Replies:** 6\ **Last updated:** [April 18, 2017, 9:45am UTC](https://discuss.elastic.co/t/logstash-mutate-replace-field/82530 "2017-04-18T09:45:54Z")
I'm trying to replace the field "message" with value . My example input message is: \[{ "logID": 1, "organizationID": 1, "userID": 1001, "appID": 123, "moduleID": 1, "screenID": 213, "timeStamp": "2017-03-15 10:06:… --- ## [Logstash GROK filter query](https://discuss.elastic.co/t/logstash-grok-filter-query/81627)
**Author:** [@cyberwhiz](https://discuss.elastic.co/u/cyberwhiz)\ **Replies:** 7\ **Last updated:** [April 18, 2017, 9:23am UTC](https://discuss.elastic.co/t/logstash-grok-filter-query/81627 "2017-04-18T09:23:17Z")
Hi All, I'm trying to visualise the statusCode in Kibana and try and get it as a filter. So I originally tried setting filters in logstash with json and mutate however that didn't get me the desired filter. Finally I j… --- ## [Mutate - replace source value](https://discuss.elastic.co/t/mutate-replace-source-value/672)
**Author:** [@achechen](https://discuss.elastic.co/u/achechen)\ **Replies:** 7\ **Last updated:** [April 18, 2017, 9:16am UTC](https://discuss.elastic.co/t/mutate-replace-source-value/672 "2017-04-18T09:16:35Z")
I am trying to change the value of the @source field in my syslogs using mutate - replace. logstash configuration: \[/opt/logstash/bin\] # cat /etc/logstash/conf.d/logstash.conf input { tcp { port =\> 5000 type =\> syslog… --- ## [RabbitMQ input plugin: add\_field doesn't write @metadata](https://discuss.elastic.co/t/rabbitmq-input-plugin-add-field-doesnt-write-metadata/82152)
**Author:** [@gertvv](https://discuss.elastic.co/u/gertvv)\ **Replies:** 4\ **Last updated:** [April 18, 2017, 8:33am UTC](https://discuss.elastic.co/t/rabbitmq-input-plugin-add-field-doesnt-write-metadata/82152 "2017-04-18T08:33:13Z")
This is using logstash 5.2.0 and the version of the RabbitMQ plugin that comes with it. input { rabbitmq { ... add\_field =\> { "\[@metadata\]\[indexType\]" =\> "myindex" } } output { elasticsearch { … --- ## [Beginners ELK design doubts](https://discuss.elastic.co/t/beginners-elk-design-doubts/82596)
**Author:** [@arpitwanchoo](https://discuss.elastic.co/u/arpitwanchoo)\ **Replies:** 3\ **Last updated:** [April 18, 2017, 8:33am UTC](https://discuss.elastic.co/t/beginners-elk-design-doubts/82596 "2017-04-18T08:33:04Z")
Hi I am exploring the ELK framework for creating a centralized logging system across all our application(java,ruby,php) running on ec2 ubuntu servers . After going through some readings I have few doubts : 1) How Is … --- ## [Please help with Grok pattern splitting out a filename](https://discuss.elastic.co/t/please-help-with-grok-pattern-splitting-out-a-filename/82591)
**Author:** [@ZillaG](https://discuss.elastic.co/u/ZillaG)\ **Replies:** 3\ **Last updated:** [April 18, 2017, 7:34am UTC](https://discuss.elastic.co/t/please-help-with-grok-pattern-splitting-out-a-filename/82591 "2017-04-18T07:34:37Z")
One of these days I'll learn regex. I have the following filename PE-run1000hbgmm3f1-job1000hbgmm3dt-Output-Workflow-1000hbgmm3fb-22.07.17.log I'm able to get this to work so... (?\\[^-\]+)-(?\\[^-\]+)… --- ## [Can't get text on a START\_OBJECT](https://discuss.elastic.co/t/cant-get-text-on-a-start-object/80408)
**Author:** [@cwt](https://discuss.elastic.co/u/cwt)\ **Replies:** 10\ **Last updated:** [April 18, 2017, 5:43am UTC](https://discuss.elastic.co/t/cant-get-text-on-a-start-object/80408 "2017-04-18T05:43:55Z")
Hello, I have many errors in my logstash logs: "error"=\>{"type"=\>"mapper\_parsing\_exception", "reason"=\>"failed to parse \[datetime\]", "caused\_by"=\>{"type"=\>"illegal\_state\_exception", "reason"=\>"Can't get text on a STAR… --- ## [How to set @timestamp with my timezone?](https://discuss.elastic.co/t/how-to-set-timestamp-with-my-timezone/82457)
**Author:** [@Bazum](https://discuss.elastic.co/u/Bazum)\ **Replies:** 2\ **Last updated:** [April 18, 2017, 5:39am UTC](https://discuss.elastic.co/t/how-to-set-timestamp-with-my-timezone/82457 "2017-04-18T05:39:50Z")
Hi, Hey guys I woud like to know if it is possible to change the @timezone from logstash to my timezone (Brazil/East). I need to change the timezone because I am using the -%{+YYYY-MM-dd} to create index with its proce… --- ## [Multiple Fields substitution doesn't work in logstash](https://discuss.elastic.co/t/multiple-fields-substitution-doesnt-work-in-logstash/82119)
**Author:** [@vramakrishnan](https://discuss.elastic.co/u/vramakrishnan)\ **Replies:** 8\ **Last updated:** [April 18, 2017, 5:31am UTC](https://discuss.elastic.co/t/multiple-fields-substitution-doesnt-work-in-logstash/82119 "2017-04-18T05:31:57Z")
I have annotated filebeat to add these fields. "fields" =\> { "provider" =\> "foobar", "resourcetype" =\> "ipmi", "customer" =\> "customer001" }, And in logstash conf file, I want … --- ## [Ingest Relational DB data with outer join relationships to Nested objects in Elasticsearch through Logstash](https://discuss.elastic.co/t/ingest-relational-db-data-with-outer-join-relationships-to-nested-objects-in-elasticsearch-through-logstash/81164)
**Author:** [@NSK](https://discuss.elastic.co/u/NSK)\ **Replies:** 4\ **Last updated:** [April 18, 2017, 5:26am UTC](https://discuss.elastic.co/t/ingest-relational-db-data-with-outer-join-relationships-to-nested-objects-in-elasticsearch-through-logstash/81164 "2017-04-18T05:26:41Z")
Hello All - I am trying to ingest Oracle Data into Elasticsearch. Here is the data structure number number\_type Comments 01 S abc 01 F … --- ## [Filebeat -\> logstash, wrapping message and extra fields problems!](https://discuss.elastic.co/t/filebeat-logstash-wrapping-message-and-extra-fields-problems/82494)
**Author:** [@Ilia](https://discuss.elastic.co/u/Ilia)\ **Replies:** 6\ **Last updated:** [April 18, 2017, 5:24am UTC](https://discuss.elastic.co/t/filebeat-logstash-wrapping-message-and-extra-fields-problems/82494 "2017-04-18T05:24:33Z")
I'm using the following configuration as my ELK stack architecture: (filebeat)\* -\> (kafka -\> logstash -\> elasticsearch -\> kibana) My problem is that the events are augmented with some unnecessary fields when they reach… --- ## [Add custom encoder for kafka output plugin](https://discuss.elastic.co/t/add-custom-encoder-for-kafka-output-plugin/82655)
**Author:** [@codenamelxl](https://discuss.elastic.co/u/codenamelxl)\ **Replies:** 0\ **Last updated:** [April 18, 2017, 4:24am UTC](https://discuss.elastic.co/t/add-custom-encoder-for-kafka-output-plugin/82655 "2017-04-18T04:24:10Z")
After searching around and reading source of kafka output plugin. I think that i can support avro encoding with Confluent schema registry format by adding some config to the plugin and use io.confluent.kafka.serializers.… --- ## [Help! Multiple inputs -\>filters-\>outputs?](https://discuss.elastic.co/t/help-multiple-inputs-filters-outputs/82553)
**Author:** [@reo7189](https://discuss.elastic.co/u/reo7189)\ **Replies:** 2\ **Last updated:** [April 17, 2017, 10:43pm UTC](https://discuss.elastic.co/t/help-multiple-inputs-filters-outputs/82553 "2017-04-17T22:43:39Z")
I'm testing logstash, and my test.conf is as below: input { exec { command =\> "tail /var/log/testx.log" interval =\> 5 type =\> "testx" } exec { … --- ## [Logstash S3 Output Canned ACL Issue](https://discuss.elastic.co/t/logstash-s3-output-canned-acl-issue/82626)
**Author:** [@aresen](https://discuss.elastic.co/u/aresen)\ **Replies:** 0\ **Last updated:** [April 17, 2017, 8:07pm UTC](https://discuss.elastic.co/t/logstash-s3-output-canned-acl-issue/82626 "2017-04-17T20:07:40Z")
Hi, I'm using the s3 output plugin to output to a private S3 bucket on AWS. Using the option canned\_acl =\> "private" works fine but setting it to "authenticated\_read" results in errors such as this: \[ERROR\]\[logstash… --- ## [How to parse the log using grok to get the key value pairs from the log, and have the key as a new field](https://discuss.elastic.co/t/how-to-parse-the-log-using-grok-to-get-the-key-value-pairs-from-the-log-and-have-the-key-as-a-new-field/82623)
**Author:** [@sanju090](https://discuss.elastic.co/u/sanju090)\ **Replies:** 1\ **Last updated:** [April 17, 2017, 7:53pm UTC](https://discuss.elastic.co/t/how-to-parse-the-log-using-grok-to-get-the-key-value-pairs-from-the-log-and-have-the-key-as-a-new-field/82623 "2017-04-17T19:53:35Z")
{"apiResponse":"{\\"errors\\":\[{\\"code\\":\\"APP-1007\\",\\"message\\":\\"Session has Expired.\\",\\"fields\\":null}\],\\"warnings\\":null}","requestUrl":"http://localhost:8082/checkSessionValidity"} This is how the message looks li… --- ## [Any plugin available to process Informatica session logs?](https://discuss.elastic.co/t/any-plugin-available-to-process-informatica-session-logs/82475)
**Author:** [@armharish](https://discuss.elastic.co/u/armharish)\ **Replies:** 1\ **Last updated:** [April 17, 2017, 4:57am UTC](https://discuss.elastic.co/t/any-plugin-available-to-process-informatica-session-logs/82475 "2017-04-17T04:57:51Z")
Hi, I want to analyze my Informatica session logs with ELK. Is there any input plugin available? --- ## [Logstash unable to convert date to string correctly](https://discuss.elastic.co/t/logstash-unable-to-convert-date-to-string-correctly/82499)
**Author:** [@niels1189](https://discuss.elastic.co/u/niels1189)\ **Replies:** 1\ **Last updated:** [April 17, 2017, 4:50am UTC](https://discuss.elastic.co/t/logstash-unable-to-convert-date-to-string-correctly/82499 "2017-04-17T04:50:43Z")
Elasticsearch version: 5.3.0 Plugins installed: N/A - (Elasticsearch, Kibana & Logstash setup) JVM version: 1.8.0\_121 OS version: Ubuntu 16.04.2 LTS Description of the problem including expected versus actual beh… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=997) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=999)