# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=545 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 546 --- ## [我想用正则重命名字段](https://discuss.elastic.co/t/topic/181757)
**Author:** [@Layoomielie](https://discuss.elastic.co/u/Layoomielie)\ **Replies:** 0\ **Last updated:** [May 20, 2019, 7:43am UTC](https://discuss.elastic.co/t/topic/181757 "2019-05-20T07:43:15Z")
这是从mysql中同步的数据，我想通过类似正则的方式去修改字段名而不是一一匹配修改 例如c\_id字段我想修改为id c\_ms字段我想修改为ms 请告诉我怎么做 谢谢 --- ## [Greedydata not working for part of the log record](https://discuss.elastic.co/t/greedydata-not-working-for-part-of-the-log-record/181582)
**Author:** [@shwetha\_v](https://discuss.elastic.co/u/shwetha_v)\ **Replies:** 2\ **Last updated:** [May 20, 2019, 7:17am UTC](https://discuss.elastic.co/t/greedydata-not-working-for-part-of-the-log-record/181582 "2019-05-20T07:17:14Z")
I've a log record for which I've written the grok pattern (both pasted below), 120-155473611354900075^|^UPC^|^Optimus^|^N1VL-TA-NPI03.india.airtel.itm^|^2019-05-15 19:06:47.233356^|^9560041297^|^syniverse^|^Read^|^1.233… --- ## [Receiving logs for only one of the index](https://discuss.elastic.co/t/receiving-logs-for-only-one-of-the-index/181462)
**Author:** [@Godson\_Raju](https://discuss.elastic.co/u/Godson_Raju)\ **Replies:** 1\ **Last updated:** [May 20, 2019, 7:01am UTC](https://discuss.elastic.co/t/receiving-logs-for-only-one-of-the-index/181462 "2019-05-20T07:01:12Z")
I am using linux centos. This is my filebeat.yml file. #=========================== Filebeat inputs ============================= filebeat.inputs: type: log enabled: true paths: /ip/logs/ip fields: { log\_type:… --- ## [Twitter plugin not working](https://discuss.elastic.co/t/twitter-plugin-not-working/179908)
**Author:** [@pathri](https://discuss.elastic.co/u/pathri)\ **Replies:** 7\ **Last updated:** [May 20, 2019, 6:36am UTC](https://discuss.elastic.co/t/twitter-plugin-not-working/179908 "2019-05-20T06:36:53Z")
i have shipped the data from twitter using a config file but facing below issue when trying running same config file in logstash 6.7 on another system \[ERROR\]\[logstash.plugins.registry\] Tried to load a plugin's code, bu… --- ## [How to parse dynamic REST endpoint and create one constant endpoint for them](https://discuss.elastic.co/t/how-to-parse-dynamic-rest-endpoint-and-create-one-constant-endpoint-for-them/181511)
**Author:** [@Arjun\_Dandagi](https://discuss.elastic.co/u/Arjun_Dandagi)\ **Replies:** 2\ **Last updated:** [May 20, 2019, 5:53am UTC](https://discuss.elastic.co/t/how-to-parse-dynamic-rest-endpoint-and-create-one-constant-endpoint-for-them/181511 "2019-05-20T05:53:06Z")
So, I am trying to parse the dynamically incoming rest API calls to make them look one constant to view them in kibana so currently i am using GSUB to match the string and replace them using a constant value, I mean fo… --- ## [Logstashのfilterについて mutateの記載数 を減らしたい、デフォルトをintegerにする方法](https://discuss.elastic.co/t/logstash-filter-mutate-integer/181508)
**Author:** [@tomomr](https://discuss.elastic.co/u/tomomr)\ **Replies:** 2\ **Last updated:** [May 20, 2019, 4:54am UTC](https://discuss.elastic.co/t/logstash-filter-mutate-integer/181508 "2019-05-20T04:54:17Z")
はじめまして Logstashについて、ぜひコメントをお願いします。 フィルタで以下のようにすると、price\_eurの属性をintegerとすることは 理解できたのですが、 filter { csv { separator =\> "," columns =\> \[ "price\_eur" \] } mutate {convert =\> \["price\_eur", "integer"\] } } 50個ほどあるデ… --- ## [Elastic input does NOT support aggs?](https://discuss.elastic.co/t/elastic-input-does-not-support-aggs/181546)
**Author:** [@Sjaak01](https://discuss.elastic.co/u/Sjaak01)\ **Replies:** 2\ **Last updated:** [May 19, 2019, 11:50pm UTC](https://discuss.elastic.co/t/elastic-input-does-not-support-aggs/181546 "2019-05-19T23:50:25Z")
Hi, Can somebody confirm whether Logstash 6.6 (or higher) and the elasticsearch input support aggs? I'm on 6.6 and while normal queries work, aggs do not. Other topic over the years talk about the same issue and in all… --- ## [Can not run Logstash on my server](https://discuss.elastic.co/t/can-not-run-logstash-on-my-server/181715)
**Author:** [@zenimagine](https://discuss.elastic.co/u/zenimagine)\ **Replies:** 8\ **Last updated:** [May 19, 2019, 11:46pm UTC](https://discuss.elastic.co/t/can-not-run-logstash-on-my-server/181715 "2019-05-19T23:46:30Z")
I have 2 servers (Ubuntu 18.04) : monitoring.example.com with ELK (ELK server) www.example.com with my Drupal site and Filebeat (Client server) I generated an SSL certificate on ELK and sent it to the client server. … --- ## [XX:+HeapDumpOnOutOfMemoryError -Djava.security.egd=file:/dev/urandom](https://discuss.elastic.co/t/xx-heapdumponoutofmemoryerror-djava-security-egd-file-dev-urandom/181723)
**Author:** [@lowdmt](https://discuss.elastic.co/u/lowdmt)\ **Replies:** 1\ **Last updated:** [May 19, 2019, 10:02pm UTC](https://discuss.elastic.co/t/xx-heapdumponoutofmemoryerror-djava-security-egd-file-dev-urandom/181723 "2019-05-19T22:02:36Z")
Can anyone help me with this error when trying to start logstash Main PID: 53686 (java) CGroup: /system.slice/logstash.service └─53686 /bin/java -Xms1g -Xmx1g -XX:+UseConcMarkSweepGC -XX:CMSInitiatingO… --- ## [LS error with lumberjack input](https://discuss.elastic.co/t/ls-error-with-lumberjack-input/181521)
**Author:** [@sunilmchaudhari](https://discuss.elastic.co/u/sunilmchaudhari)\ **Replies:** 0\ **Last updated:** [May 17, 2019, 6:48am UTC](https://discuss.elastic.co/t/ls-error-with-lumberjack-input/181521 "2019-05-17T06:48:17Z")
Hi, What we did: Migrated all Filters/input/output from LS-2.4 to LS-6.3. Lumberjack input receives logs from AIX and Solaris clients using LSF. All filters/plugins work well in LS-2.4. Having problems when we run s… --- ## [Default grok patterns for clamav/clamd](https://discuss.elastic.co/t/default-grok-patterns-for-clamav-clamd/181538)
**Author:** [@nousees](https://discuss.elastic.co/u/nousees)\ **Replies:** 0\ **Last updated:** [May 17, 2019, 7:52am UTC](https://discuss.elastic.co/t/default-grok-patterns-for-clamav-clamd/181538 "2019-05-17T07:52:41Z")
Hello, I've found some default patterns on https://github.com/logstash-plugins/logstash-patterns-core/tree/master/patterns but I'm missing clamav/clamd pattern. Do you know if is there any pattern for clamav please? Th… --- ## [Input from sqlite with sqlcipher](https://discuss.elastic.co/t/input-from-sqlite-with-sqlcipher/181708)
**Author:** [@alisedighiankashi](https://discuss.elastic.co/u/alisedighiankashi)\ **Replies:** 0\ **Last updated:** [May 19, 2019, 11:21am UTC](https://discuss.elastic.co/t/input-from-sqlite-with-sqlcipher/181708 "2019-05-19T11:21:38Z")
Hello, I want to ingest data from a sqlite file that is encrypted with sqlcipher. but there is not any option in default jdbc driver for my purpose. i found this fork of driver but it's not working either: sqlite-jdb… --- ## [Logstash - split array into individual events](https://discuss.elastic.co/t/logstash-split-array-into-individual-events/181422)
**Author:** [@RonGros](https://discuss.elastic.co/u/RonGros)\ **Replies:** 6\ **Last updated:** [May 19, 2019, 6:17am UTC](https://discuss.elastic.co/t/logstash-split-array-into-individual-events/181422 "2019-05-19T06:17:45Z")
Hi! One of our logs contains lines of arrays of json objects.... I want to take each one of the items in the array and make it a single even. I read some posts on how to do it : https://discuss.elastic.co/t/split-nest… --- ## [Recursive depth of Logstash - not finding my files?](https://discuss.elastic.co/t/recursive-depth-of-logstash-not-finding-my-files/181602)
**Author:** [@sgreszcz](https://discuss.elastic.co/u/sgreszcz)\ **Replies:** 9\ **Last updated:** [May 18, 2019, 7:59pm UTC](https://discuss.elastic.co/t/recursive-depth-of-logstash-not-finding-my-files/181602 "2019-05-18T19:59:55Z")
I'm really having problems finding the csv files I need to parse with Logstash. I'm wondering if it is my configuration or there is a limit to the recursion depth. test@host-001:/data/tracelogger/tracelogger\_data# find… --- ## [How to add/rename fields recursively instead using mutate or instead doing one by one?](https://discuss.elastic.co/t/how-to-add-rename-fields-recursively-instead-using-mutate-or-instead-doing-one-by-one/181688)
**Author:** [@rkhapre](https://discuss.elastic.co/u/rkhapre)\ **Replies:** 2\ **Last updated:** [May 18, 2019, 5:07pm UTC](https://discuss.elastic.co/t/how-to-add-rename-fields-recursively-instead-using-mutate-or-instead-doing-one-by-one/181688 "2019-05-18T17:07:14Z")
Hi All How to add/rename fields recursively, instead doing one by one Here is my dataset with message value as \[{ "Row ID" : 201, "Order ID" : "CA-2017-105074", "Order Date" : "6/24/2017", "Segment" : "Consumer", … --- ## [Logstash 7.0.0 File Input on windows 10 issue](https://discuss.elastic.co/t/logstash-7-0-0-file-input-on-windows-10-issue/179070)
**Author:** [@bytecode](https://discuss.elastic.co/u/bytecode)\ **Replies:** 2\ **Last updated:** [May 18, 2019, 4:55pm UTC](https://discuss.elastic.co/t/logstash-7-0-0-file-input-on-windows-10-issue/179070 "2019-05-18T16:55:13Z")
Hello folks, This looks like the same old issue of logstash on windows. Input file is not being picked up and doesn't give out anything on console output. I've checked various posts on this and tried most of the soluti… --- ## [Mutate add\_field gives sprint f format for nil values. How to solve this?](https://discuss.elastic.co/t/mutate-add-field-gives-sprint-f-format-for-nil-values-how-to-solve-this/181473)
**Author:** [@rkhapre](https://discuss.elastic.co/u/rkhapre)\ **Replies:** 4\ **Last updated:** [May 18, 2019, 1:01pm UTC](https://discuss.elastic.co/t/mutate-add-field-gives-sprint-f-format-for-nil-values-how-to-solve-this/181473 "2019-05-18T13:01:58Z")
Hi All I am using mutate add\_field in my confile mutate { add\_field =\> {"CloserName" =\> "%{\[Bugs\]CloserName}"} } The value for many CloserName is null but some has value Which ever record is having null values, i… --- ## [Best way to right grok filters](https://discuss.elastic.co/t/best-way-to-right-grok-filters/181676)
**Author:** [@nuwancs](https://discuss.elastic.co/u/nuwancs)\ **Replies:** 1\ **Last updated:** [May 18, 2019, 11:05am UTC](https://discuss.elastic.co/t/best-way-to-right-grok-filters/181676 "2019-05-18T11:05:44Z")
Hi all, We can use two approaches to right a grok filter as follows. My concern is what is the best approach performance wise? 1st Approach Having a single grok for all the variables grok { match =\> { "message" =\> … --- ## [Error: com.mysql.jdbc.Driver not loaded](https://discuss.elastic.co/t/error-com-mysql-jdbc-driver-not-loaded/181542)
**Author:** [@prabhat](https://discuss.elastic.co/u/prabhat)\ **Replies:** 2\ **Last updated:** [May 18, 2019, 8:39am UTC](https://discuss.elastic.co/t/error-com-mysql-jdbc-driver-not-loaded/181542 "2019-05-18T08:39:35Z")
i am getting this error \[2019-05-17T12:49:58,687\]\[ERROR\]\[logstash.inputs.jdbc \] Failed to load C:\\Users\\shiwani\\Downloads\\ELK\\mysql-connector-java-5.1.47\\mysql-connector-java-5.1.47-bin.jar {:exception=\>#\ **Author:** [@Badger](https://discuss.elastic.co/u/Badger)\ **Replies:** 0\ **Last updated:** [May 17, 2019, 9:39pm UTC](https://discuss.elastic.co/t/aggregate-filter-does-not-decorate-events-that-it-generates/181662 "2019-05-17T21:39:06Z") Due to the noError variable, an aggregate filter will only decorate an event (implementing common options like add\_tag) for events that pass through it. Events that it generates (timeouts, final flush) are not decorated.… --- ## [Tags does not work with aggregate filter and elasticsearch output plugins](https://discuss.elastic.co/t/tags-does-not-work-with-aggregate-filter-and-elasticsearch-output-plugins/181489)
**Author:** [@sachi\_mr](https://discuss.elastic.co/u/sachi_mr)\ **Replies:** 10\ **Last updated:** [May 17, 2019, 9:25pm UTC](https://discuss.elastic.co/t/tags-does-not-work-with-aggregate-filter-and-elasticsearch-output-plugins/181489 "2019-05-17T21:25:29Z")
I am trying to get the tags to work with filter and output and so far no luck with it . The tags are required to import data from multiple tables conditionally. when the tags are applied to aggregate and output plugins t… --- ## [Message field split into multiple entries from CSV input](https://discuss.elastic.co/t/message-field-split-into-multiple-entries-from-csv-input/181626)
**Author:** [@JeremyP](https://discuss.elastic.co/u/JeremyP)\ **Replies:** 4\ **Last updated:** [May 17, 2019, 9:03pm UTC](https://discuss.elastic.co/t/message-field-split-into-multiple-entries-from-csv-input/181626 "2019-05-17T21:03:57Z")
I'm running into an issue where my logstash message field is being split into multiple messages. I have it ingesting a CSV. There are a couple of gsub filters happening to remove "," delimiters and replace them with "^" … --- ## [Grok error while loading](https://discuss.elastic.co/t/grok-error-while-loading/181515)
**Author:** [@pathri](https://discuss.elastic.co/u/pathri)\ **Replies:** 13\ **Last updated:** [May 17, 2019, 6:51pm UTC](https://discuss.elastic.co/t/grok-error-while-loading/181515 "2019-05-17T18:51:04Z")
i am running a logstash for the log file with below a sample entry Sample entry : 72.14.199.105 - - \[30/Apr/2019:06:21:26 +0000\] "GET /catalog/view/javascript/font-awesome/css/font-awesome.min.css HTTP/1.1" 200 4748 "h… --- ## [Change CSV Delimiter in the Source data](https://discuss.elastic.co/t/change-csv-delimiter-in-the-source-data/181607)
**Author:** [@JeremyP](https://discuss.elastic.co/u/JeremyP)\ **Replies:** 6\ **Last updated:** [May 17, 2019, 4:46pm UTC](https://discuss.elastic.co/t/change-csv-delimiter-in-the-source-data/181607 "2019-05-17T16:46:37Z")
My source CSV contains "," delimiters but has field data which contain this delimiter within double quotes. I originally started the approach to remove the single "," within the double quoted strings since the csv{} fil… --- ## [Multiple csv filter in logstash not displaying header in kibana](https://discuss.elastic.co/t/multiple-csv-filter-in-logstash-not-displaying-header-in-kibana/181053)
**Author:** [@damisco](https://discuss.elastic.co/u/damisco)\ **Replies:** 7\ **Last updated:** [May 17, 2019, 4:16pm UTC](https://discuss.elastic.co/t/multiple-csv-filter-in-logstash-not-displaying-header-in-kibana/181053 "2019-05-17T16:16:31Z")
Hi all, I'm just getting into using elastic services and as a proof of concept am trying to get csv data from two files that will have two different sets of headers to kibana. The data gets there just fine but I have no… --- ## [Get current time](https://discuss.elastic.co/t/get-current-time/181609)
**Author:** [@Android\_Games](https://discuss.elastic.co/u/Android_Games)\ **Replies:** 2\ **Last updated:** [May 17, 2019, 3:10pm UTC](https://discuss.elastic.co/t/get-current-time/181609 "2019-05-17T15:10:26Z")
i would like to get the current timestamp from logstash, currently i'm using the following code to get the current time. input { stdin { codec =\> json } # Supports "cron", "every", "at" and "in" schedules by ru… --- ## [Can't update or replace with a string value](https://discuss.elastic.co/t/cant-update-or-replace-with-a-string-value/181408)
**Author:** [@Arnolin](https://discuss.elastic.co/u/Arnolin)\ **Replies:** 8\ **Last updated:** [May 17, 2019, 2:50pm UTC](https://discuss.elastic.co/t/cant-update-or-replace-with-a-string-value/181408 "2019-05-17T14:50:04Z")
Hello, I have a problem, as said in the title i can't mutate a value from int to string. I have the following line of log : name,24/02/19,12345,123456 12345 is a number of hour and i want to convert it in Day(s), Mon… --- ## [Need to compare a field with other field using logstash](https://discuss.elastic.co/t/need-to-compare-a-field-with-other-field-using-logstash/181578)
**Author:** [@Ganesh2303](https://discuss.elastic.co/u/Ganesh2303)\ **Replies:** 2\ **Last updated:** [May 17, 2019, 2:23pm UTC](https://discuss.elastic.co/t/need-to-compare-a-field-with-other-field-using-logstash/181578 "2019-05-17T14:23:30Z")
HI Team, I have one scenario can anyone suggest me solution for that, My scenario is i have one sample which contain some dcn id and some more fields like this 2019-05-17 \[DCN 001\] - started 2019-05-17 \[DCN 001\] - pr… --- ## [Common filter code, called from another filter?](https://discuss.elastic.co/t/common-filter-code-called-from-another-filter/181601)
**Author:** [@ghost3h](https://discuss.elastic.co/u/ghost3h)\ **Replies:** 2\ **Last updated:** [May 17, 2019, 2:22pm UTC](https://discuss.elastic.co/t/common-filter-code-called-from-another-filter/181601 "2019-05-17T14:22:29Z")
I have 2 different filters that come in on different ports. Each one gets send several input types, 3 of these are common across the two files. Current processing I have "if" blocks looking at tags, and then blocks of c… --- ## [Squid pipeline returns 'RegexpError: unmatched close parenthesis'](https://discuss.elastic.co/t/squid-pipeline-returns-regexperror-unmatched-close-parenthesis/181612)
**Author:** [@netoeuler](https://discuss.elastic.co/u/netoeuler)\ **Replies:** 0\ **Last updated:** [May 17, 2019, 2:21pm UTC](https://discuss.elastic.co/t/squid-pipeline-returns-regexperror-unmatched-close-parenthesis/181612 "2019-05-17T14:21:27Z")
Hi! I'm trying to use the ELK 7.0 to monitor squid traffic. The connection is ok but I'm sending the traffic directly to Elasticsearch because I'm having problem to create a grok filter to it. The pipeline looks like t… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=544) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=546)