# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=994 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 995 --- ## [Logstash → InfluxDB wrong output](https://discuss.elastic.co/t/logstash-influxdb-wrong-output/82413)
**Author:** [@artobstrel](https://discuss.elastic.co/u/artobstrel)\ **Replies:** 1\ **Last updated:** [April 26, 2017, 8:12am UTC](https://discuss.elastic.co/t/logstash-influxdb-wrong-output/82413 "2017-04-26T08:12:53Z")
Hello, guys! I'm sending сyrillic text in Elastisearch and InfluxDB. With the Elastisearch, there are no problems, but in InfluxDB text encoding is broken. Can anybody help? Logstash configinput { generator { … --- ## [How to collect logs files from different machines?](https://discuss.elastic.co/t/how-to-collect-logs-files-from-different-machines/83658)
**Author:** [@ginzan](https://discuss.elastic.co/u/ginzan)\ **Replies:** 3\ **Last updated:** [April 26, 2017, 7:48am UTC](https://discuss.elastic.co/t/how-to-collect-logs-files-from-different-machines/83658 "2017-04-26T07:48:05Z")
Hi everyone, As the topic what i mentioned, a few machines produce bussiness logs files which is 'xx.log' alike ,and i have to use logstash to output to a detination which aquire NETWORK IP,but unfortunately those machin… --- ## [Match field based on content via regex](https://discuss.elastic.co/t/match-field-based-on-content-via-regex/83660)
**Author:** [@evlonden](https://discuss.elastic.co/u/evlonden)\ **Replies:** 1\ **Last updated:** [April 26, 2017, 7:28am UTC](https://discuss.elastic.co/t/match-field-based-on-content-via-regex/83660 "2017-04-26T07:28:13Z")
Hello, Loglines often contains a severity field containing INFO, WARNING etc. I tried to use a pattern like BREVTCLASS (?:INFO?|WARNING?|ERROR?|CRITICAL?) which I pick up in grok with %{BREVTCLASS:syslog\_evtclass} b… --- ## [Extract UNIX\_MS using 'grok' regex - use it as date using 'date'](https://discuss.elastic.co/t/extract-unix-ms-using-grok-regex-use-it-as-date-using-date/83536)
**Author:** [@uda\_kay](https://discuss.elastic.co/u/uda_kay)\ **Replies:** 4\ **Last updated:** [April 26, 2017, 7:19am UTC](https://discuss.elastic.co/t/extract-unix-ms-using-grok-regex-use-it-as-date-using-date/83536 "2017-04-26T07:19:27Z")
Hello i have a field named "message", it contains the string: "GET1: +992545989898 GET2: 365981022808151-821eaec1d4b27cc974c97e11a2343f705c0c4612-1492310893103" The field contains Phone-number, ID, HEX-data, and epoch (… --- ## [Logstash pipeline](https://discuss.elastic.co/t/logstash-pipeline/83632)
**Author:** [@alfredo.deluca](https://discuss.elastic.co/u/alfredo.deluca)\ **Replies:** 2\ **Last updated:** [April 26, 2017, 5:48am UTC](https://discuss.elastic.co/t/logstash-pipeline/83632 "2017-04-26T05:48:48Z")
hi all. just a quick question. As far as I understand the logstash pipeline is basically....(https://www.elastic.co/guide/en/logstash/current/execution-model.html) Input \> filter \> output and if I have multiple filte… --- ## [Custom index with field from xpath](https://discuss.elastic.co/t/custom-index-with-field-from-xpath/83478)
**Author:** [@daian.martinho](https://discuss.elastic.co/u/daian.martinho)\ **Replies:** 8\ **Last updated:** [April 26, 2017, 5:08am UTC](https://discuss.elastic.co/t/custom-index-with-field-from-xpath/83478 "2017-04-26T05:08:46Z")
Filebeat is sending xml files to logstash. input { beats { port =\> "5044" } } filter { ## interpret the message as XML if \[type\] == "nessus-report" { xml… --- ## [Error after upgrading 5.2 -\> 5.3 : "Unknown setting 'trim' for kv"](https://discuss.elastic.co/t/error-after-upgrading-5-2-5-3-unknown-setting-trim-for-kv/83616)
**Author:** [@dfinn](https://discuss.elastic.co/u/dfinn)\ **Replies:** 2\ **Last updated:** [April 25, 2017, 9:04pm UTC](https://discuss.elastic.co/t/error-after-upgrading-5-2-5-3-unknown-setting-trim-for-kv/83616 "2017-04-25T21:04:49Z")
Looks like I'm hitting this bug: Here's the pertinent part of my config that uses trim: # process key-value data is it exists if \[postfix\_keyvalue\_data\] { kv { source =\> "postfix\_keyvalue\_data" … --- ## [Unable to connect Logstash with Elasticsearch](https://discuss.elastic.co/t/unable-to-connect-logstash-with-elasticsearch/82355)
**Author:** [@likhita](https://discuss.elastic.co/u/likhita)\ **Replies:** 19\ **Last updated:** [April 25, 2017, 8:02pm UTC](https://discuss.elastic.co/t/unable-to-connect-logstash-with-elasticsearch/82355 "2017-04-25T20:02:25Z")
Hello, I have installed Logstash, Elasticsearch and Kibana on my CentOS 7 machine. I was able to do the installations successfully. But when I try to access the apache access logs, it doesn't get the output. here is wh… --- ## [Drop events older than three days](https://discuss.elastic.co/t/drop-events-older-than-three-days/83423)
**Author:** [@seth.yes](https://discuss.elastic.co/u/seth.yes)\ **Replies:** 5\ **Last updated:** [April 25, 2017, 7:16pm UTC](https://discuss.elastic.co/t/drop-events-older-than-three-days/83423 "2017-04-25T19:16:08Z")
I'm using Logstash v5.3 on Ubuntu 14.04. I'm looking to drop events sent to logstash that have a timestamp which is three days old or more. I'm parsing the date off the events and using the 'date' plugin to index of… --- ## [New to it all. Would like a rotating 7 indexes from logstash to Elastisearch](https://discuss.elastic.co/t/new-to-it-all-would-like-a-rotating-7-indexes-from-logstash-to-elastisearch/82434)
**Author:** [@runatyr](https://discuss.elastic.co/u/runatyr)\ **Replies:** 2\ **Last updated:** [April 25, 2017, 5:47pm UTC](https://discuss.elastic.co/t/new-to-it-all-would-like-a-rotating-7-indexes-from-logstash-to-elastisearch/82434 "2017-04-25T17:47:34Z")
Hello: First off... Wow... These products are amazing! I have a single logstash source and a single elasticsearch/kibana source set up. All on the same server. (Yes Still learning it all :slight\_smile: ) I would lik… --- ## [Parsing different messages using different filters](https://discuss.elastic.co/t/parsing-different-messages-using-different-filters/83570)
**Author:** [@ibrahimsharaf](https://discuss.elastic.co/u/ibrahimsharaf)\ **Replies:** 1\ **Last updated:** [April 25, 2017, 5:19pm UTC](https://discuss.elastic.co/t/parsing-different-messages-using-different-filters/83570 "2017-04-25T17:19:33Z")
Hello, I have the following log-file: 2017-01-01 07:53:44 \[connectionpool.py\] DEBUG: "POST /api/v1/crawledproducts/merchant/ariika/new\_crawl\_iteration/ HTTP/1.1" 200 None 2017-01-01 07:53:44 \[monitor\_utils.py\] INFO: Cr… --- ## [Add extra timestamp field for local time](https://discuss.elastic.co/t/add-extra-timestamp-field-for-local-time/83510)
**Author:** [@kernelpanic](https://discuss.elastic.co/u/kernelpanic)\ **Replies:** 1\ **Last updated:** [April 25, 2017, 5:14pm UTC](https://discuss.elastic.co/t/add-extra-timestamp-field-for-local-time/83510 "2017-04-25T17:14:40Z")
Hello, I understand that Elasticsearch stores timestamp in UTC format internally, which are an hour behind my local time (UK time). I would like to add a field that shows the correct local time, am I right in thinking I … --- ## [Logstash filter not working for Nagios](https://discuss.elastic.co/t/logstash-filter-not-working-for-nagios/82892)
**Author:** [@rijinmp](https://discuss.elastic.co/u/rijinmp)\ **Replies:** 4\ **Last updated:** [April 25, 2017, 4:39pm UTC](https://discuss.elastic.co/t/logstash-filter-not-working-for-nagios/82892 "2017-04-25T16:39:24Z")
My Logstash Conf input { beats { port =\> "5043" type =\> "nagios" } udp { port =\> 9010 type =\> "ossec" } } filter { if \[type\] == "ossec" { grok { match =\> { "message"… --- ## [Logstash do not create an output to elasticsearch](https://discuss.elastic.co/t/logstash-do-not-create-an-output-to-elasticsearch/82422)
**Author:** [@Marcus\_Muller](https://discuss.elastic.co/u/Marcus_Muller)\ **Replies:** 6\ **Last updated:** [April 25, 2017, 4:35pm UTC](https://discuss.elastic.co/t/logstash-do-not-create-an-output-to-elasticsearch/82422 "2017-04-25T16:35:14Z")
Hi all, I'm using the latest ELK-Stack (5.3.0) and installed ES, Kibana, and Logstash on the same machine (Win 2016 Server). In addition to that, I installed Filebeat on another machine (my notebook \[Win 8.2\]) that is … --- ## [Grok patterns : creating new field](https://discuss.elastic.co/t/grok-patterns-creating-new-field/83576)
**Author:** [@Sylfaen](https://discuss.elastic.co/u/Sylfaen)\ **Replies:** 4\ **Last updated:** [April 25, 2017, 2:58pm UTC](https://discuss.elastic.co/t/grok-patterns-creating-new-field/83576 "2017-04-25T14:58:57Z")
Hello ! I was wondering if it is possible to embed the content of a field to a new "structured field" ? For example, from a JDBC entry, I've got a column named message which contains informations with the following for… --- ## [Indexing new fields](https://discuss.elastic.co/t/indexing-new-fields/83583)
**Author:** [@jfarcher](https://discuss.elastic.co/u/jfarcher)\ **Replies:** 1\ **Last updated:** [April 25, 2017, 2:48pm UTC](https://discuss.elastic.co/t/indexing-new-fields/83583 "2017-04-25T14:48:44Z")
Hi all, apologies if this has already been covered. I've just gone through the process of getting a grok match written for a syslog entry which comes from a firewall. I've got the relevant information being seen as fi… --- ## [NMAP codec - HTTP gives a 500 status when using the codec on an http input but 200 status without the codec](https://discuss.elastic.co/t/nmap-codec-http-gives-a-500-status-when-using-the-codec-on-an-http-input-but-200-status-without-the-codec/79818)
**Author:** [@TheZeke](https://discuss.elastic.co/u/TheZeke)\ **Replies:** 1\ **Last updated:** [April 25, 2017, 2:26pm UTC](https://discuss.elastic.co/t/nmap-codec-http-gives-a-500-status-when-using-the-codec-on-an-http-input-but-200-status-without-the-codec/79818 "2017-04-25T14:26:07Z")
I've been following this tutorial: - https://www.elastic.co/blog/using-nmap-logstash-to-gain-insight-into-your-network Does anyone know why I might be seeing this in my logs and the codec not working? (getting a 500 er… --- ## [Grokparsefailure although test pages show a match](https://discuss.elastic.co/t/grokparsefailure-although-test-pages-show-a-match/83582)
**Author:** [@jahlives](https://discuss.elastic.co/u/jahlives)\ **Replies:** 0\ **Last updated:** [April 25, 2017, 2:19pm UTC](https://discuss.elastic.co/t/grokparsefailure-although-test-pages-show-a-match/83582 "2017-04-25T14:19:39Z")
I have troubles with a grok pattern. Running logstash 2.3.4 on a Centos (CentOS Linux release 7.3.1611 (Core)) with latest updates. A pattern that I have does not match the line, although the pages https://grokdebug.her… --- ## [Logstash hanging after a few polls to jmx](https://discuss.elastic.co/t/logstash-hanging-after-a-few-polls-to-jmx/83571)
**Author:** [@k1ng87](https://discuss.elastic.co/u/k1ng87)\ **Replies:** 0\ **Last updated:** [April 25, 2017, 1:28pm UTC](https://discuss.elastic.co/t/logstash-hanging-after-a-few-polls-to-jmx/83571 "2017-04-25T13:28:10Z")
I've tried running both logstash 5.3 and 5.2 and I'm noticing the same behavior. In the log output I see this at the end continuing to repeat over and over... \[2017-04-25T08:30:00,222\]\[DEBUG\]\[logstash.inputs.jmx … --- ## [Basic logstash example for CSV](https://discuss.elastic.co/t/basic-logstash-example-for-csv/83554)
**Author:** [@hgpit](https://discuss.elastic.co/u/hgpit)\ **Replies:** 5\ **Last updated:** [April 25, 2017, 1:28pm UTC](https://discuss.elastic.co/t/basic-logstash-example-for-csv/83554 "2017-04-25T13:28:06Z")
Hi I am trying to get my head around the ELK stack "thing" but being defeated! I have installed all the latest v5.x components (ELK) onto CentOS 7. Now, I have tonnes of 100mb files being FTP'd to the server, which I… --- ## [JDBC duplicate records sending to SumoLogic](https://discuss.elastic.co/t/jdbc-duplicate-records-sending-to-sumologic/82950)
**Author:** [@duxy007](https://discuss.elastic.co/u/duxy007)\ **Replies:** 1\ **Last updated:** [April 25, 2017, 1:24pm UTC](https://discuss.elastic.co/t/jdbc-duplicate-records-sending-to-sumologic/82950 "2017-04-25T13:24:55Z")
Hi, I have been struggling for several days to understand and prevent getting duplicate records sent from JDBC(Oracle) to SumoLogic using the logstash-output-sumologic plugin. I have been searching the forums but all … --- ## [Problem in grok pattern](https://discuss.elastic.co/t/problem-in-grok-pattern/83552)
**Author:** [@ibrahimsharaf](https://discuss.elastic.co/u/ibrahimsharaf)\ **Replies:** 1\ **Last updated:** [April 25, 2017, 12:23pm UTC](https://discuss.elastic.co/t/problem-in-grok-pattern/83552 "2017-04-25T12:23:45Z")
Hello, I have this snippet of a log-file 2017-01-01 07:53:44 \[connectionpool.py\] DEBUG: "POST /api/v1/crawledproducts/merchant/ariika/new\_crawl\_iteration/ HTTP/1.1" 200 None 2017-01-01 07:53:44 \[monitor\_utils.py\] INFO:… --- ## [Strange log parsing](https://discuss.elastic.co/t/strange-log-parsing/83529)
**Author:** [@Darya\_Semenova](https://discuss.elastic.co/u/Darya_Semenova)\ **Replies:** 0\ **Last updated:** [April 25, 2017, 9:18am UTC](https://discuss.elastic.co/t/strange-log-parsing/83529 "2017-04-25T09:18:01Z")
Hello, I am in the process of learning logstash for parsing particular logs, and for some reason not all of my logs are parsed as they should. I would be very grateful for the help with getting what is done wrong. With… --- ## [Is there a way to parse precisely xcat logs?](https://discuss.elastic.co/t/is-there-a-way-to-parse-precisely-xcat-logs/83540)
**Author:** [@Darya\_Semenova](https://discuss.elastic.co/u/Darya_Semenova)\ **Replies:** 0\ **Last updated:** [April 25, 2017, 10:20am UTC](https://discuss.elastic.co/t/is-there-a-way-to-parse-precisely-xcat-logs/83540 "2017-04-25T10:20:55Z")
Hello! I'd like to ask if there is a pattern specially for xcat logs or maybe a plugin for them? --- ## [How to convert Apache timestamp(HTTPDATE) to ISO8601 format log](https://discuss.elastic.co/t/how-to-convert-apache-timestamp-httpdate-to-iso8601-format-log/83146)
**Author:** [@yokawasa](https://discuss.elastic.co/u/yokawasa)\ **Replies:** 6\ **Last updated:** [April 25, 2017, 10:12am UTC](https://discuss.elastic.co/t/how-to-convert-apache-timestamp-httpdate-to-iso8601-format-log/83146 "2017-04-25T10:12:04Z")
Hi experts, Currently I'm forwarding apache access log to output target with the configuration, and I'd like to convert current timestamp which is Apache timestamp format (HTTPDATE) to ISO8601 format and add it to out… --- ## [Logstash2.4 Failed action. status 403](https://discuss.elastic.co/t/logstash2-4-failed-action-status-403/83521)
**Author:** [@wongchao](https://discuss.elastic.co/u/wongchao)\ **Replies:** 0\ **Last updated:** [April 25, 2017, 8:29am UTC](https://discuss.elastic.co/t/logstash2-4-failed-action-status-403/83521 "2017-04-25T08:29:19Z")
elasticsearch { index =\> "%{type}-%{+YYYY.MM.dd}" codec =\> "json" hosts =\> \["172.16.19.28"\] flush\_size =\> 5000 } Failed action. {:status=\>403, :action=\>\["index", {:id=\>nil, :index=\>"ca… --- ## [Csv file not reading last row from file input](https://discuss.elastic.co/t/csv-file-not-reading-last-row-from-file-input/83452)
**Author:** [@rkhapre](https://discuss.elastic.co/u/rkhapre)\ **Replies:** 4\ **Last updated:** [April 25, 2017, 8:20am UTC](https://discuss.elastic.co/t/csv-file-not-reading-last-row-from-file-input/83452 "2017-04-25T08:20:16Z")
Hi All i am using csv file as input, using file plugin. After couple of trials i realized, it is not reading only the last line Any idea how this can be fixed? Thanks! --- ## [Netflow from Nokia router flowset id 258](https://discuss.elastic.co/t/netflow-from-nokia-router-flowset-id-258/83498)
**Author:** [@adankert](https://discuss.elastic.co/u/adankert)\ **Replies:** 0\ **Last updated:** [April 25, 2017, 6:59am UTC](https://discuss.elastic.co/t/netflow-from-nokia-router-flowset-id-258/83498 "2017-04-25T06:59:43Z")
I have set up a Nokia router to send Netflow to Logstash. The Netflow I receive has id 258. From a pcap I can see that relevant information is sent and received by the Logstash server. (See attachment). According to … --- ## [Insert data depending on match and](https://discuss.elastic.co/t/insert-data-depending-on-match-and/83457)
**Author:** [@bob\_monty](https://discuss.elastic.co/u/bob_monty)\ **Replies:** 1\ **Last updated:** [April 25, 2017, 7:23am UTC](https://discuss.elastic.co/t/insert-data-depending-on-match-and/83457 "2017-04-25T07:23:36Z")
in my logstash config, in the output section i have the following section which works fine: if \[@measurement\] =~ /(CAR\_.\*)/ { influxdb { host =\> "hardbor001:8086" db =\> "April" retention\_policy =\> "30days" ... i… --- ## [How do i aggregate the same event in the message field](https://discuss.elastic.co/t/how-do-i-aggregate-the-same-event-in-the-message-field/83499)
**Author:** [@Jonesthomas](https://discuss.elastic.co/u/Jonesthomas)\ **Replies:** 0\ **Last updated:** [April 25, 2017, 7:02am UTC](https://discuss.elastic.co/t/how-do-i-aggregate-the-same-event-in-the-message-field/83499 "2017-04-25T07:02:22Z")
this is my json log file,I want to group the error logged event from the message field, and one more thing what is the pattern of the message field \[{ "logID": 1, "organizationID": 1, "userID": 1001, "appID": 123, … [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=993) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=995)