# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=912 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 913 --- ## [HTTP\_poller input plugin](https://discuss.elastic.co/t/http-poller-input-plugin/99817)
**Author:** [@Sharivari](https://discuss.elastic.co/u/Sharivari)\ **Replies:** 2\ **Last updated:** [September 8, 2017, 9:01am UTC](https://discuss.elastic.co/t/http-poller-input-plugin/99817 "2017-09-08T09:01:23Z")
Hi, I am running the sebp/elk Container for Docker on a Virtual Machine. My boss told me to install the http\_poller input plugin. I managed to do that. But I have no clue how to use this. I read the doc (https://www.… --- ## [Efficient way of doing a complex conditional compare](https://discuss.elastic.co/t/efficient-way-of-doing-a-complex-conditional-compare/99808)
**Author:** [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Replies:** 3\ **Last updated:** [September 8, 2017, 9:00am UTC](https://discuss.elastic.co/t/efficient-way-of-doing-a-complex-conditional-compare/99808 "2017-09-08T09:00:56Z")
I have a range of ranges I want to check an integer field against and I want to do something like if "value" \> 100 and "value" \< "300" and "value" \> "600" and "value" \< "700". And repeat that across 6 sets of ranges. I … --- ## [How to make an own timestamp from old log file](https://discuss.elastic.co/t/how-to-make-an-own-timestamp-from-old-log-file/99543)
**Author:** [@Muhammad\_Rizky](https://discuss.elastic.co/u/Muhammad_Rizky)\ **Replies:** 5\ **Last updated:** [September 8, 2017, 8:46am UTC](https://discuss.elastic.co/t/how-to-make-an-own-timestamp-from-old-log-file/99543 "2017-09-08T08:46:53Z")
Hello, i want to store my own log file to logstash, then i use grok to mapping the date from log file to a field. But, in kibana, that field cannot read as a timestamp, so i can not use the real time stamp in timelion. … --- ## [Aggregat about data](https://discuss.elastic.co/t/aggregat-about-data/99690)
**Author:** [@Beuhlet\_Reseau](https://discuss.elastic.co/u/Beuhlet_Reseau)\ **Replies:** 7\ **Last updated:** [September 8, 2017, 8:42am UTC](https://discuss.elastic.co/t/aggregat-about-data/99690 "2017-09-08T08:42:57Z")
Hello, I search to put aggregat on my data. Here a party of my data : TYPE Usage FREEUSER 345 PREMIUM\_USER 8653 FREEUSER 1369 FREEUSER 87654 PREMIUM\_USER 43678 FREEUSER 8654 PREMIUM\_USER 2387 FREEUSER 98723 FREEUSER 4… --- ## [Help me parse this message field](https://discuss.elastic.co/t/help-me-parse-this-message-field/99807)
**Author:** [@inhinyera16](https://discuss.elastic.co/u/inhinyera16)\ **Replies:** 0\ **Last updated:** [September 8, 2017, 7:14am UTC](https://discuss.elastic.co/t/help-me-parse-this-message-field/99807 "2017-09-08T07:14:11Z")
Hello, I have a problem parsing this logs ####################################################################### START of Response ####################################################################### Log message: … --- ## [Need some support to create custom grok filter to parse the log file](https://discuss.elastic.co/t/need-some-support-to-create-custom-grok-filter-to-parse-the-log-file/99196)
**Author:** [@viuha8](https://discuss.elastic.co/u/viuha8)\ **Replies:** 3\ **Last updated:** [September 8, 2017, 6:08am UTC](https://discuss.elastic.co/t/need-some-support-to-create-custom-grok-filter-to-parse-the-log-file/99196 "2017-09-08T06:08:08Z")
Need some advice to create custom grok filter inorder to parse application specific log. Our log format looks like 2017-08-29 14:49:28 INFO Logger:? - ‘ApplicationName’:string, ‘EventStatus’:string, ‘SeverityLevel’:st… --- ## [Read and filter 1million xml files](https://discuss.elastic.co/t/read-and-filter-1million-xml-files/99658)
**Author:** [@AntonQUT](https://discuss.elastic.co/u/AntonQUT)\ **Replies:** 5\ **Last updated:** [September 8, 2017, 6:04am UTC](https://discuss.elastic.co/t/read-and-filter-1million-xml-files/99658 "2017-09-08T06:04:53Z")
I have never used Logstash, but want to. I need to read approximately 1 million xml files, extract 2 specific fields (title and abstract) and load them into a named elasticsearch index with corresponding fields. I look… --- ## [Loading Fixed legth CSV File with Logstash](https://discuss.elastic.co/t/loading-fixed-legth-csv-file-with-logstash/99365)
**Author:** [@KIRAN1205](https://discuss.elastic.co/u/KIRAN1205)\ **Replies:** 5\ **Last updated:** [September 8, 2017, 6:03am UTC](https://discuss.elastic.co/t/loading-fixed-legth-csv-file-with-logstash/99365 "2017-09-08T06:03:22Z")
Hi, I’m new to elasticsearch, i install elasticsearch, logstash now i need to import a fixed legth CSV file and process data to elastic search. please can help me in configuration. thanks in advance --- ## [How to parse log and do look up where we have multiple values for a particular field](https://discuss.elastic.co/t/how-to-parse-log-and-do-look-up-where-we-have-multiple-values-for-a-particular-field/99377)
**Author:** [@gchakkalakkal](https://discuss.elastic.co/u/gchakkalakkal)\ **Replies:** 2\ **Last updated:** [September 8, 2017, 5:42am UTC](https://discuss.elastic.co/t/how-to-parse-log-and-do-look-up-where-we-have-multiple-values-for-a-particular-field/99377 "2017-09-08T05:42:55Z")
Sample log key: system":\[{“systemCode”:“abc”},{“systemCode”:“def”},{“systemCode”:“EOS”}\] Here we also need to do look of values mentioned in systemCode --- ## [Deserializing avro file in logstash](https://discuss.elastic.co/t/deserializing-avro-file-in-logstash/99793)
**Author:** [@Pratheek](https://discuss.elastic.co/u/Pratheek)\ **Replies:** 0\ **Last updated:** [September 8, 2017, 5:26am UTC](https://discuss.elastic.co/t/deserializing-avro-file-in-logstash/99793 "2017-09-08T05:26:08Z")
Hi, Its mentioned in the documentation for avro codec plugin that there is 'an unique method for deserializing an avro file'.Can anyone please provide me a sample snippet on how to process it in logstash using file inpu… --- ## [How can I reach real-time index updating using ES + Logstash + MySQL when data was updated, added or deleted?](https://discuss.elastic.co/t/how-can-i-reach-real-time-index-updating-using-es-logstash-mysql-when-data-was-updated-added-or-deleted/99704)
**Author:** [@ramgarrot](https://discuss.elastic.co/u/ramgarrot)\ **Replies:** 4\ **Last updated:** [September 8, 2017, 4:44am UTC](https://discuss.elastic.co/t/how-can-i-reach-real-time-index-updating-using-es-logstash-mysql-when-data-was-updated-added-or-deleted/99704 "2017-09-08T04:44:53Z")
Hi there, I have MySQL DB with one table without joins with ~100 millions of rows. These items can be updated or removed and also new records are saved into MySQL DB e.g. every minutes. How can I reach updating indexes i… --- ## [TCP Input and Multiline Codec Question](https://discuss.elastic.co/t/tcp-input-and-multiline-codec-question/99778)
**Author:** [@ptinson](https://discuss.elastic.co/u/ptinson)\ **Replies:** 0\ **Last updated:** [September 8, 2017, 2:30am UTC](https://discuss.elastic.co/t/tcp-input-and-multiline-codec-question/99778 "2017-09-08T02:30:48Z")
Hi There, I have been trying to use the multiline codec with a TCP input. So far with no luck. Reading through other posts it seems to be that the multiline codec with a TCP input isn't really ideal. I would use file … --- ## [Logstash start up issues](https://discuss.elastic.co/t/logstash-start-up-issues/98734)
**Author:** [@hburnswell](https://discuss.elastic.co/u/hburnswell)\ **Replies:** 15\ **Last updated:** [September 7, 2017, 9:03pm UTC](https://discuss.elastic.co/t/logstash-start-up-issues/98734 "2017-09-07T21:03:49Z")
Hi All, I just set up a 3 node Elasticsearch and Logstash environment using the repos on RHEL 7: logstash-5.5.2-1.noarch elasticsearch-5.5.2-1.noarch I initially installed the X-Pack for ES and Kibana but then remove… --- ## [Logstash.yml file is missing on Linux Redhat distribution](https://discuss.elastic.co/t/logstash-yml-file-is-missing-on-linux-redhat-distribution/99344)
**Author:** [@amruth](https://discuss.elastic.co/u/amruth)\ **Replies:** 5\ **Last updated:** [September 7, 2017, 7:00pm UTC](https://discuss.elastic.co/t/logstash-yml-file-is-missing-on-linux-redhat-distribution/99344 "2017-09-07T19:00:00Z")
Hi, I’ve installed Logstash 2.4 on Redhat based system using rpm install. From the documentation, logstash.yml should be at /etc/logstash/ but I can’t find any of the files there except conf.d directory. Can someone pl… --- ## [Logstash sql\_last\_value to eastern timestamp](https://discuss.elastic.co/t/logstash-sql-last-value-to-eastern-timestamp/99758)
**Author:** [@swaplogstash](https://discuss.elastic.co/u/swaplogstash)\ **Replies:** 0\ **Last updated:** [September 7, 2017, 6:55pm UTC](https://discuss.elastic.co/t/logstash-sql-last-value-to-eastern-timestamp/99758 "2017-09-07T18:55:33Z")
Our database has date field with datatype as TIMESTAMP and its eastern time (ex:05-SEP-17 10.00.00.209000000 AM) and logstash sql\_last\_value has UTC(ex: 2017-09-07 18:40:00.205000000 Z). How can we convert sql\_last\_va… --- ## [Join CSV content with logs](https://discuss.elastic.co/t/join-csv-content-with-logs/99741)
**Author:** [@meryem](https://discuss.elastic.co/u/meryem)\ **Replies:** 1\ **Last updated:** [September 7, 2017, 5:50pm UTC](https://discuss.elastic.co/t/join-csv-content-with-logs/99741 "2017-09-07T17:50:44Z")
Hi, I want to join and map a CSV content to existent logs in elasticsearch : My CSV contain : ClientName, ClientID My logs contain : ClientID, ClientMessage… I wonder if it’s possible to join the CSV content so that … --- ## [Logstash 5x output plugin for kairosdb](https://discuss.elastic.co/t/logstash-5x-output-plugin-for-kairosdb/99515)
**Author:** [@stecino](https://discuss.elastic.co/u/stecino)\ **Replies:** 2\ **Last updated:** [September 7, 2017, 4:55pm UTC](https://discuss.elastic.co/t/logstash-5x-output-plugin-for-kairosdb/99515 "2017-09-07T16:55:12Z")
Hello, Does this plugin exist? Thanks --- ## [Logstash-plugin fails with java errors on a clean install](https://discuss.elastic.co/t/logstash-plugin-fails-with-java-errors-on-a-clean-install/99744)
**Author:** [@rrue](https://discuss.elastic.co/u/rrue)\ **Replies:** 5\ **Last updated:** [September 7, 2017, 4:25pm UTC](https://discuss.elastic.co/t/logstash-plugin-fails-with-java-errors-on-a-clean-install/99744 "2017-09-07T16:25:51Z")
Hi All, Forgive what is likely a newbie question, I’m pretty green around ELK and especially ruby and gems. Just installed LS 5.5.2 via RPM to openSUSE LEAP 42.2 and if I cd to /usr/share/logstash and try to run any ca… --- ## [Timestamp field has a 5 minutes delay](https://discuss.elastic.co/t/timestamp-field-has-a-5-minutes-delay/99679)
**Author:** [@h\_foxit](https://discuss.elastic.co/u/h_foxit)\ **Replies:** 12\ **Last updated:** [September 7, 2017, 3:02pm UTC](https://discuss.elastic.co/t/timestamp-field-has-a-5-minutes-delay/99679 "2017-09-07T15:02:21Z")
hello, still a newbie here I installed the 5.5 version of ElasticStack I have a time problem 5 min difference with the local time I do receive ossec logs 2017 Sep 06 00:05:03 WinEvtLog: Security: AUDIT\_SUCCESS(4624) … --- ## [Logstash date filter in UNIX\_MS from two fields](https://discuss.elastic.co/t/logstash-date-filter-in-unix-ms-from-two-fields/99580)
**Author:** [@Frances\_Buontempo](https://discuss.elastic.co/u/Frances_Buontempo)\ **Replies:** 9\ **Last updated:** [September 7, 2017, 12:59pm UTC](https://discuss.elastic.co/t/logstash-date-filter-in-unix-ms-from-two-fields/99580 "2017-09-07T12:59:30Z")
I have some data from snort which has two subfields, in an !event¬ fields: “event-microsecond” =\> 289367, “event-second” =\> 1493741082, I am trying to compose these into a float and then tell Elasticsearch it’s the ti… --- ## [Timestamp and @timestamp not properly mapping](https://discuss.elastic.co/t/timestamp-and-timestamp-not-properly-mapping/99636)
**Author:** [@Blake\_2112](https://discuss.elastic.co/u/Blake_2112)\ **Replies:** 4\ **Last updated:** [September 7, 2017, 12:34pm UTC](https://discuss.elastic.co/t/timestamp-and-timestamp-not-properly-mapping/99636 "2017-09-07T12:34:45Z")
I am sure that this has been asked a number of times and from my review of the previous posts nothing seems to be helping me solve my problem. My JSON has separate date and time fields. I’m creating an ‘eventTime’ fiel… --- ## [Configure CloudWatch log group and log stream in CloudWatch Output Plugin](https://discuss.elastic.co/t/configure-cloudwatch-log-group-and-log-stream-in-cloudwatch-output-plugin/99721)
**Author:** [@sukeshss](https://discuss.elastic.co/u/sukeshss)\ **Replies:** 0\ **Last updated:** [September 7, 2017, 12:28pm UTC](https://discuss.elastic.co/t/configure-cloudwatch-log-group-and-log-stream-in-cloudwatch-output-plugin/99721 "2017-09-07T12:28:29Z")
We are trying to configure CloudWatch Output Plugin. We are able to add "access\_key\_id" =\> “” “secret\_access\_key” =\> “” “region” =\> "" in logstash configuration file. Could you please guide me how I can give CloudWa… --- ## [Add timestamp of the ELK server](https://discuss.elastic.co/t/add-timestamp-of-the-elk-server/99707)
**Author:** [@h\_foxit](https://discuss.elastic.co/u/h_foxit)\ **Replies:** 1\ **Last updated:** [September 7, 2017, 11:43am UTC](https://discuss.elastic.co/t/add-timestamp-of-the-elk-server/99707 "2017-09-07T11:43:03Z")
I want to delete the timestamp field send by filebeat and replaced by the date of the ELK server Any idea how to do that thx --- ## [Issue with kafka input plugin](https://discuss.elastic.co/t/issue-with-kafka-input-plugin/97451)
**Author:** [@axelfelix](https://discuss.elastic.co/u/axelfelix)\ **Replies:** 2\ **Last updated:** [September 7, 2017, 10:12am UTC](https://discuss.elastic.co/t/issue-with-kafka-input-plugin/97451 "2017-09-07T10:12:47Z")
Hi all, I am on my way to upgrade all my ELK cluster. The last point is the logtsash upgrade (before 2.3, future 5.5). Kafka version: 0.11.0.0 With logstash kafka input plugin version 2.0.6, I do not have any problem … --- ## [Problem with creation indexes based on custom field](https://discuss.elastic.co/t/problem-with-creation-indexes-based-on-custom-field/99621)
**Author:** [@alex\_gorb](https://discuss.elastic.co/u/alex_gorb)\ **Replies:** 6\ **Last updated:** [September 7, 2017, 10:02am UTC](https://discuss.elastic.co/t/problem-with-creation-indexes-based-on-custom-field/99621 "2017-09-07T10:02:19Z")
Hi, I am using ELK 5.5.2. I have a problem with creation indexes. On filebeat config I have a following setup input\_type: log document\_type: prod paths: c:\\somepath\*.log fields: my\_log\_type: app On logstas… --- ## [Unable to install plugin After Logstash upgrade 5.5.1 - logstash-input-mongodb](https://discuss.elastic.co/t/unable-to-install-plugin-after-logstash-upgrade-5-5-1-logstash-input-mongodb/99701)
**Author:** [@bkumbar](https://discuss.elastic.co/u/bkumbar)\ **Replies:** 0\ **Last updated:** [September 7, 2017, 9:31am UTC](https://discuss.elastic.co/t/unable-to-install-plugin-after-logstash-upgrade-5-5-1-logstash-input-mongodb/99701 "2017-09-07T09:31:36Z")
Receiving below error while installing Mongodb plugin Compressed indicies: 0.032s Bundler::GemNotFound: You have requested: logstash-codec-plain = 3.0.2 The bundle currently has logstash-codec-plain locked at 3.0.3. … --- ## [Logstash with x-pack authentication problems without Authentication Header](https://discuss.elastic.co/t/logstash-with-x-pack-authentication-problems-without-authentication-header/99692)
**Author:** [@Metehan\_Selvi](https://discuss.elastic.co/u/Metehan_Selvi)\ **Replies:** 3\ **Last updated:** [September 7, 2017, 9:17am UTC](https://discuss.elastic.co/t/logstash-with-x-pack-authentication-problems-without-authentication-header/99692 "2017-09-07T09:17:53Z")
Hello, as stated in Logstash with x-pack authentication problems I have the same problems after I have updated ELK-Stack with x-pack. \[2017-09-07T08:44:37,664\]\[ERROR\]\[logstash.outputs.elasticsearch\] Got a bad response… --- ## [Config file for multiple multiline patterns](https://discuss.elastic.co/t/config-file-for-multiple-multiline-patterns/98079)
**Author:** [@iamsaurabhrk](https://discuss.elastic.co/u/iamsaurabhrk)\ **Replies:** 8\ **Last updated:** [September 7, 2017, 9:00am UTC](https://discuss.elastic.co/t/config-file-for-multiple-multiline-patterns/98079 "2017-09-07T09:00:15Z")
I have installed logstash v5.5.0. I want to parse logs which have multiple multiline formats. I have logs in the following format: Fri 3/3/17 14:31:30: AbCdEueuq\_11111111111.PqrThr-4: TRACE : xx.xxx.xxx Administrator - … --- ## [Error parsing csv - NoMethodError](https://discuss.elastic.co/t/error-parsing-csv-nomethoderror/98226)
**Author:** [@Beuhlet\_Reseau](https://discuss.elastic.co/u/Beuhlet_Reseau)\ **Replies:** 2\ **Last updated:** [September 7, 2017, 8:35am UTC](https://discuss.elastic.co/t/error-parsing-csv-nomethoderror/98226 "2017-09-07T08:35:05Z")
Hello, I have lots of following message in logstash’s logs : \[WARN \]\[logstash.filters.csv \] Error parsing csv {:field=\>"message", :source=\>"", :exception=\>#\ **Author:** [@jogoinar10](https://discuss.elastic.co/u/jogoinar10)\ **Replies:** 3\ **Last updated:** [September 7, 2017, 8:26am UTC](https://discuss.elastic.co/t/logstash-close-the-port-after-transfer/98051 "2017-09-07T08:26:49Z") Does logstash closes the port after transfer? I frequently encountering connection refused. I have 3 server. 2 server was sending data to another server which compiled the data. The size of the data was huge and it mu… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=911) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=913)