# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=399 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 400 --- ## [Dealing with string date](https://discuss.elastic.co/t/dealing-with-string-date/220789)
**Author:** [@AdamJade](https://discuss.elastic.co/u/AdamJade)\ **Replies:** 4\ **Last updated:** [February 26, 2020, 8:24am UTC](https://discuss.elastic.co/t/dealing-with-string-date/220789 "2020-02-26T08:24:20Z")
Hi there I have an issue with this log line: \[2020-02-19 09:42:01.514\] \[Value1\] \[value2\] \[value3\] \[Value5\] \[Value6\] \[Value7\] grok { match =\> { "message" =\> "\[%{TIMESTAMP\_ISO8601:logDate}\] \[%{DATA:core}\] \[%{WORD:serve… --- ## [EPOCH Parsing | Logstash](https://discuss.elastic.co/t/epoch-parsing-logstash/220132)
**Author:** [@deeshu](https://discuss.elastic.co/u/deeshu)\ **Replies:** 6\ **Last updated:** [February 26, 2020, 3:05am UTC](https://discuss.elastic.co/t/epoch-parsing-logstash/220132 "2020-02-26T03:05:30Z")
Hi techs.... Does anyone have grok pattern available to convert epoch timings to date time format. If yes, could any one do the noble cause and share here.. Have seen couple of posts as below for this where Grok UNIX\_M… --- ## [How to remove "message" field added by Logstash](https://discuss.elastic.co/t/how-to-remove-message-field-added-by-logstash/220899)
**Author:** [@welin](https://discuss.elastic.co/u/welin)\ **Replies:** 6\ **Last updated:** [February 25, 2020, 11:23pm UTC](https://discuss.elastic.co/t/how-to-remove-message-field-added-by-logstash/220899 "2020-02-25T23:23:29Z")
I want to store json object {'customerId': '123', 'customerName': 'abc'} to Elastisearch via Logstash, but the json object is stored as message="{'customerId': '123', 'customerName': 'abc'}" in Elasticsearch. How can I r… --- ## [Ignore the entire message if one field is nil](https://discuss.elastic.co/t/ignore-the-entire-message-if-one-field-is-nil/220872)
**Author:** [@Incauto](https://discuss.elastic.co/u/Incauto)\ **Replies:** 2\ **Last updated:** [February 25, 2020, 7:12pm UTC](https://discuss.elastic.co/t/ignore-the-entire-message-if-one-field-is-nil/220872 "2020-02-25T19:12:38Z")
Hi, Im filtering some CSV files,and Im getting some nil fields, and if some field is nil the entire message is worthless, so how can I ignore the entire message if one of the fields value is nil? Thanks! --- ## [Logstash not able to run multi pipeline when running logstash as a service. It is running fine if i run the logstash from cli using /usr/share/logstash/bin/logstash --path.settings /etc/logstash/](https://discuss.elastic.co/t/logstash-not-able-to-run-multi-pipeline-when-running-logstash-as-a-service-it-is-running-fine-if-i-run-the-logstash-from-cli-using-usr-share-logstash-bin-logstash-path-settings-etc-logstash/220714)
**Author:** [@raghuveera](https://discuss.elastic.co/u/raghuveera)\ **Replies:** 14\ **Last updated:** [February 25, 2020, 4:57pm UTC](https://discuss.elastic.co/t/logstash-not-able-to-run-multi-pipeline-when-running-logstash-as-a-service-it-is-running-fine-if-i-run-the-logstash-from-cli-using-usr-share-logstash-bin-logstash-path-settings-etc-logstash/220714 "2020-02-25T16:57:18Z")
Logstash not able to run multi pipeline when running logstash as a service. It is running fine if i run the logstash from cli using /usr/share/logstash/bin/logstash --path.settings /etc/logstash/. logstash.service file … --- ## [Logs in duplicate in kibana](https://discuss.elastic.co/t/logs-in-duplicate-in-kibana/220192)
**Author:** [@Robin\_Coda](https://discuss.elastic.co/u/Robin_Coda)\ **Replies:** 5\ **Last updated:** [February 25, 2020, 3:26pm UTC](https://discuss.elastic.co/t/logs-in-duplicate-in-kibana/220192 "2020-02-25T15:26:56Z")
Hello, In Kibana i see different logs in duplicate, I don't know for what. logs are generated via winlogbeat and logstash, duplicate logs are: Can you explain me this please. Thank's --- ## [Load multiple CSV to multiple index in a single conf file](https://discuss.elastic.co/t/load-multiple-csv-to-multiple-index-in-a-single-conf-file/220299)
**Author:** [@rp12](https://discuss.elastic.co/u/rp12)\ **Replies:** 1\ **Last updated:** [February 25, 2020, 2:47pm UTC](https://discuss.elastic.co/t/load-multiple-csv-to-multiple-index-in-a-single-conf-file/220299 "2020-02-25T14:47:58Z")
What is the best way for loading multiple csv files to different indexes from one config file ? --- ## [How can I parse filebeat data with logstash and send it to kibana?](https://discuss.elastic.co/t/how-can-i-parse-filebeat-data-with-logstash-and-send-it-to-kibana/220810)
**Author:** [@nihi](https://discuss.elastic.co/u/nihi)\ **Replies:** 2\ **Last updated:** [February 25, 2020, 1:20pm UTC](https://discuss.elastic.co/t/how-can-i-parse-filebeat-data-with-logstash-and-send-it-to-kibana/220810 "2020-02-25T13:20:45Z")
Hi, 3 days on elk stack and i'm kinda stuck. What i want to do ? I got an apache server with classic access.log and errors.log and i would like to pull thoses logs with some parsing. What i've understood so far ? I … --- ## [Unable to push data from local LOGSTASH to AWS elasticsearch service](https://discuss.elastic.co/t/unable-to-push-data-from-local-logstash-to-aws-elasticsearch-service/220860)
**Author:** [@vidhyadhara](https://discuss.elastic.co/u/vidhyadhara)\ **Replies:** 0\ **Last updated:** [February 25, 2020, 1:07pm UTC](https://discuss.elastic.co/t/unable-to-push-data-from-local-logstash-to-aws-elasticsearch-service/220860 "2020-02-25T13:07:15Z")
Hello Folks, Below is my logstash configuration file i've in my local system, I'm trying to push data from my Server DB to AWS Elasticsearch endpoint but getting this error------ Attempted to resurrect connection to de… --- ## [Read mode input plugin and sincedb](https://discuss.elastic.co/t/read-mode-input-plugin-and-sincedb/220843)
**Author:** [@akwoprosper](https://discuss.elastic.co/u/akwoprosper)\ **Replies:** 0\ **Last updated:** [February 25, 2020, 11:54am UTC](https://discuss.elastic.co/t/read-mode-input-plugin-and-sincedb/220843 "2020-02-25T11:54:10Z")
Hello everyone, I am presently using logstash to ingest billions of records into elasticsearch. I am using the file input plugin to read the files with the input mode set to read since all the data is already present. … --- ## [Unable to print column names in the csv file using csv filter plugin](https://discuss.elastic.co/t/unable-to-print-column-names-in-the-csv-file-using-csv-filter-plugin/220840)
**Author:** [@shwetha\_v](https://discuss.elastic.co/u/shwetha_v)\ **Replies:** 1\ **Last updated:** [February 25, 2020, 11:54am UTC](https://discuss.elastic.co/t/unable-to-print-column-names-in-the-csv-file-using-csv-filter-plugin/220840 "2020-02-25T11:54:21Z")
filter{ csv { columns =\> \["plmn", "state", "leakage\_value", "MetricRSRP\_values"\] separator =\> "," autodetect\_column\_names =\> true autogenerate\_column\_names =\> true }} I have tried running the above logsta… --- ## [I want to add a date column to my logstash with a date i choose not a timestamp hellpp](https://discuss.elastic.co/t/i-want-to-add-a-date-column-to-my-logstash-with-a-date-i-choose-not-a-timestamp-hellpp/220627)
**Author:** [@samargh](https://discuss.elastic.co/u/samargh)\ **Replies:** 3\ **Last updated:** [February 25, 2020, 11:25am UTC](https://discuss.elastic.co/t/i-want-to-add-a-date-column-to-my-logstash-with-a-date-i-choose-not-a-timestamp-hellpp/220627 "2020-02-25T11:25:47Z")
i need to add a new column to my logstash file i have to add a column that shows a date in all my lines exmpl : 2020-01-01 i don't get it i tried this but it's not working date { match =\> \[ "DATE", "2020-01-01" \] t… --- ## [Logstash not printing stdout](https://discuss.elastic.co/t/logstash-not-printing-stdout/220688)
**Author:** [@volcano](https://discuss.elastic.co/u/volcano)\ **Replies:** 6\ **Last updated:** [February 25, 2020, 11:03am UTC](https://discuss.elastic.co/t/logstash-not-printing-stdout/220688 "2020-02-25T11:03:13Z")
Here is my logstash.conf and here is the logstash log file in /var/log/logstash issue is , application logs are not printing in logstash STDOUT. ... I dont see any error also. I am using logstash 7.4 in AWS E… --- ## [Logstash service not reading application log file](https://discuss.elastic.co/t/logstash-service-not-reading-application-log-file/220779)
**Author:** [@volcano](https://discuss.elastic.co/u/volcano)\ **Replies:** 0\ **Last updated:** [February 25, 2020, 6:56am UTC](https://discuss.elastic.co/t/logstash-service-not-reading-application-log-file/220779 "2020-02-25T06:56:37Z")
I am using Logstash 7.4 in AWS EC2 (Amazon Linux 2) following command works #/usr/share/logstash/bin/logstash -f /etc/logstash/conf.d/logstash.conf --debug // this is reading application log file But this command i… --- ## [Parser un fichier text avec logstash](https://discuss.elastic.co/t/parser-un-fichier-text-avec-logstash/220676)
**Author:** [@fatfall](https://discuss.elastic.co/u/fatfall)\ **Replies:** 2\ **Last updated:** [February 25, 2020, 9:12am UTC](https://discuss.elastic.co/t/parser-un-fichier-text-avec-logstash/220676 "2020-02-25T09:12:07Z")
bonjour, j'ai un fichier .txt que je dois ingérer dans logstash pour l'intégrer sur elasticsearch; voici un extait du fichier: {"\_index":"logs-2010.11.27","\_type":"doc","\_id":"xcA","\_score":null,"\_source":{"app\_name":… --- ## [Is it possible to use a field from one filter to the next filter](https://discuss.elastic.co/t/is-it-possible-to-use-a-field-from-one-filter-to-the-next-filter/220701)
**Author:** [@Raj\_K](https://discuss.elastic.co/u/Raj_K)\ **Replies:** 1\ **Last updated:** [February 25, 2020, 9:08am UTC](https://discuss.elastic.co/t/is-it-possible-to-use-a-field-from-one-filter-to-the-next-filter/220701 "2020-02-25T09:08:04Z")
I am using multiple filters in my logstash. I want to use one field from a filter to my next filter as a query parameter. Is it possible? --- ## [What is the supported jdbc driver to connect sybase 16.0 from logstash 7.20](https://discuss.elastic.co/t/what-is-the-supported-jdbc-driver-to-connect-sybase-16-0-from-logstash-7-20/220792)
**Author:** [@sai\_kadiveti](https://discuss.elastic.co/u/sai_kadiveti)\ **Replies:** 0\ **Last updated:** [February 25, 2020, 8:37am UTC](https://discuss.elastic.co/t/what-is-the-supported-jdbc-driver-to-connect-sybase-16-0-from-logstash-7-20/220792 "2020-02-25T08:37:06Z")
Hi, Iam trying to connect to sybase database by using below mentioned details in conf file but it is giving error as "\[ERROR\]\[logstash.inputs.jdbc \] Unable to connect to database. Tried 1 times {:error\_message=\>"Ja… --- ## [How to output data if field value equal to "string"](https://discuss.elastic.co/t/how-to-output-data-if-field-value-equal-to-string/220541)
**Author:** [@saidjon](https://discuss.elastic.co/u/saidjon)\ **Replies:** 1\ **Last updated:** [February 25, 2020, 7:22am UTC](https://discuss.elastic.co/t/how-to-output-data-if-field-value-equal-to-string/220541 "2020-02-25T07:22:24Z")
Hi there , I'm getting data from csv file using this conf filter{ csv{ columns=\>\['geonameid','name','asciiname','alternatenames','latitude','longitude','feature class','feature code','country code','cc2','admin1 … --- ## [Problem with date filter](https://discuss.elastic.co/t/problem-with-date-filter/220758)
**Author:** [@alxfernando](https://discuss.elastic.co/u/alxfernando)\ **Replies:** 1\ **Last updated:** [February 25, 2020, 4:33am UTC](https://discuss.elastic.co/t/problem-with-date-filter/220758 "2020-02-25T04:33:41Z")
how can I load two dates of an event (event started - event ended) using logstash --- ## [Filtering log message from Mulesoft app with grok](https://discuss.elastic.co/t/filtering-log-message-from-mulesoft-app-with-grok/220760)
**Author:** [@aparasxidis](https://discuss.elastic.co/u/aparasxidis)\ **Replies:** 0\ **Last updated:** [February 25, 2020, 4:09am UTC](https://discuss.elastic.co/t/filtering-log-message-from-mulesoft-app-with-grok/220760 "2020-02-25T04:09:23Z")
Hi, I have the next message from a Mulesoft app "message": "event:1072eae0-5723-11ea-b982-0ad52e78f90f correlationId=ab7ab3a6-32b7-4529-872f-af0321c3f418" I want to filter and get the "event" and "correlationId" in sep… --- ## [Help with processing metadata at start of file](https://discuss.elastic.co/t/help-with-processing-metadata-at-start-of-file/220740)
**Author:** [@uppaljs](https://discuss.elastic.co/u/uppaljs)\ **Replies:** 1\ **Last updated:** [February 25, 2020, 12:20am UTC](https://discuss.elastic.co/t/help-with-processing-metadata-at-start-of-file/220740 "2020-02-25T00:20:22Z")
Hi, I am trying to process a file which has the following format CreationDate;Application;ReportType;FromDate;UntilDate 2020-01-01 03:15:00;VOPS;System Report;2019-12-01 00:00:00;2020-01-01 00:00:10 SerialNo;Value;Fi… --- ## [Date Parsing Error After Index Rotation - Any Ideas?](https://discuss.elastic.co/t/date-parsing-error-after-index-rotation-any-ideas/220270)
**Author:** [@strikerx22](https://discuss.elastic.co/u/strikerx22)\ **Replies:** 3\ **Last updated:** [February 25, 2020, 12:14am UTC](https://discuss.elastic.co/t/date-parsing-error-after-index-rotation-any-ideas/220270 "2020-02-25T00:14:53Z")
Hello Everyone, I have been parsing TACACS logs for a few months now without error and all of a sudden, on 2/1/2020, the logs stopped appearing in Kibana. I mention this because my current index name is logstash-2020.02… --- ## [What is the purpose of running logstash as a service?](https://discuss.elastic.co/t/what-is-the-purpose-of-running-logstash-as-a-service/220739)
**Author:** [@Incauto](https://discuss.elastic.co/u/Incauto)\ **Replies:** 0\ **Last updated:** [February 24, 2020, 11:29pm UTC](https://discuss.elastic.co/t/what-is-the-purpose-of-running-logstash-as-a-service/220739 "2020-02-24T23:29:22Z")
If I can run pipelines and multiple pipelines without running logstash as a service? --- ## [Logstash + pfsense issues](https://discuss.elastic.co/t/logstash-pfsense-issues/220723)
**Author:** [@artson](https://discuss.elastic.co/u/artson)\ **Replies:** 0\ **Last updated:** [February 24, 2020, 8:46pm UTC](https://discuss.elastic.co/t/logstash-pfsense-issues/220723 "2020-02-24T20:46:23Z")
hi everyone i try to see the logs from pfsense in kibana but i think i miss something with the configuration files, i cant find the index logstash-\* on kibana maybe someone can help with that? this is the first file : #… --- ## [java.lang.IllegalStateException: Logstash stopped processing because of an error: (LoadError) load error: ffi/ffi -- java.lang.NullPointerException: null](https://discuss.elastic.co/t/java-lang-illegalstateexception-logstash-stopped-processing-because-of-an-error-loaderror-load-error-ffi-ffi-java-lang-nullpointerexception-null/220706)
**Author:** [@eng.lucianamendes](https://discuss.elastic.co/u/eng.lucianamendes)\ **Replies:** 0\ **Last updated:** [February 24, 2020, 5:48pm UTC](https://discuss.elastic.co/t/java-lang-illegalstateexception-logstash-stopped-processing-because-of-an-error-loaderror-load-error-ffi-ffi-java-lang-nullpointerexception-null/220706 "2020-02-24T17:48:04Z")
I install Logstash 7.6.0 in Raspiberry Pi with OS Rasbian. When I execute a command line: ./logstash -e 'input {stdin {}} output {stdout {}}', the Logstash stopped work. --- ## [Add field date problem](https://discuss.elastic.co/t/add-field-date-problem/220665)
**Author:** [@samargh](https://discuss.elastic.co/u/samargh)\ **Replies:** 3\ **Last updated:** [February 24, 2020, 4:37pm UTC](https://discuss.elastic.co/t/add-field-date-problem/220665 "2020-02-24T16:37:31Z")
i used this code to add a new column with a specific date mutate { add\_field =\> {"D"=\> \["2020-01-01"\]} } it gives me the date but its adding minutes and hours Jan 1, 2020 @ 01:00:00.000 so is it normal ?? but i wan… --- ## [Visualizing maps on Kibana in logstash originated data, when I already HAVE the geo information](https://discuss.elastic.co/t/visualizing-maps-on-kibana-in-logstash-originated-data-when-i-already-have-the-geo-information/220677)
**Author:** [@GenXJason](https://discuss.elastic.co/u/GenXJason)\ **Replies:** 1\ **Last updated:** [February 24, 2020, 4:36pm UTC](https://discuss.elastic.co/t/visualizing-maps-on-kibana-in-logstash-originated-data-when-i-already-have-the-geo-information/220677 "2020-02-24T16:36:23Z")
Hey there, I have a Django implementation that logs into Logstash. I already send the geographical information to logstash. I do not need it to convert IP addresses to geo-information. Here's a sample of what I send (… --- ## [Rsyslog input : Any way to disable severity and severity label?](https://discuss.elastic.co/t/rsyslog-input-any-way-to-disable-severity-and-severity-label/220480)
**Author:** [@Travis](https://discuss.elastic.co/u/Travis)\ **Replies:** 5\ **Last updated:** [February 24, 2020, 4:06pm UTC](https://discuss.elastic.co/t/rsyslog-input-any-way-to-disable-severity-and-severity-label/220480 "2020-02-24T16:06:29Z")
Hello guys, I noticed that some logs I collect have a field "severity" (ips logs). This is in conflict with the field "severity" which is related to my syslog input : severity of syslog is long and severity of ips logs … --- ## [Prevent logstash from reading file from beginning when restarted](https://discuss.elastic.co/t/prevent-logstash-from-reading-file-from-beginning-when-restarted/220682)
**Author:** [@akwoprosper](https://discuss.elastic.co/u/akwoprosper)\ **Replies:** 0\ **Last updated:** [February 24, 2020, 3:13pm UTC](https://discuss.elastic.co/t/prevent-logstash-from-reading-file-from-beginning-when-restarted/220682 "2020-02-24T15:13:14Z")
Hi everyone, I don't understand why Logstash does not continue reading a file from where it left off when restarted. Following is my input plugin configuration: input { file{ path=\>"/data/backup/recovery-data/QUICKS… --- ## [Logstash Failed to execute {:action=\>LogStash::PipelineAction}](https://discuss.elastic.co/t/logstash-failed-to-execute-action-logstash-pipelineaction/220675)
**Author:** [@nikoscsq](https://discuss.elastic.co/u/nikoscsq)\ **Replies:** 0\ **Last updated:** [February 24, 2020, 2:26pm UTC](https://discuss.elastic.co/t/logstash-failed-to-execute-action-logstash-pipelineaction/220675 "2020-02-24T14:26:21Z")
Hello engineers, could you please help me? i start the logstash: .\\logstash -f DAP.conf but i get: Starting Logstash {"logstash.version"=\>"7.6.0"} \[2020-02-24T16:04:55,330\]\[ERROR\]\[logstash.agent \] Failed t… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=398) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=400)