# Source file path transfer to Logstash

**URL:** <https://discuss.elastic.co/t/source-file-path-transfer-to-logstash/74708>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [February 10, 2017, 7:56pm UTC](https://discuss.elastic.co/t/source-file-path-transfer-to-logstash/74708 "2017-02-10T19:56:52Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![DenysK](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/denysk/32/16309_2.png) [@DenysK](https://discuss.elastic.co/u/DenysK)\
**Post date:** [February 10, 2017, 7:56pm UTC](https://discuss.elastic.co/t/source-file-path-transfer-to-logstash/74708/1 "2017-02-10T19:56:52Z")

</div>

Hello,

I'm not new with ELK stuck already. But I did not use all functionality of ELK.  
i have a question for community about Filebeat v1.3 configuration.

I need to transfer Source file location(path) as a field to Logstash for after treatment.  
I have read about "fields" option [FileBeat v1.3 Field\_option](https://www.elastic.co/guide/en/beats/filebeat/1.3/configuration-filebeat-options.html#configuration-fields), but I'm not able to figure out how to do this.

Here is the configuration of filebeat:

```
  paths:
      - /var/www/dbreports/DAILY/reportdb.direct.com/2017-01-14/Active_users_for_all_companies.csv
  input_type: report
  fields_under_root: true
  ignore_older: 24h
  document_type: daily_reports
  tail_files: false

```

Many thanks for help and + 100 to Karma )

---

<div class="post-metadata">

**Author:** ![DenysK](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/denysk/32/16309_2.png) [@DenysK](https://discuss.elastic.co/u/DenysK)\
**Post date:** [February 13, 2017, 11:48am UTC](https://discuss.elastic.co/t/source-file-path-transfer-to-logstash/74708/2 "2017-02-13T11:48:58Z")

</div>

I find out that Filebeat is passing "source" field. But I'm not able to get in in Logstash or Elasticsearch.

I have the following architecture:

FilbeBeat -\> Logstash(shipper) -\> RabbitMQ -\> Logstash(processor) -\> Elasticsearch

---

<div class="post-metadata">

**Author:** ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)\
**Post date:** [February 13, 2017, 3:41pm UTC](https://discuss.elastic.co/t/source-file-path-transfer-to-logstash/74708/3 "2017-02-13T15:41:51Z")

</div>

what you mean by, you don't get the source in Logstash? Is your Logstash shipper filtering/removing some event fields?

---

<div class="post-metadata">

**Author:** ![DenysK](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/denysk/32/16309_2.png) [@DenysK](https://discuss.elastic.co/u/DenysK)\
**Post date:** [February 14, 2017, 10:08am UTC](https://discuss.elastic.co/t/source-file-path-transfer-to-logstash/74708/4 "2017-02-14T10:08:08Z")

</div>

I mean, that In Elasticsearch the source field does not contain "path" for source file.  
I did not apply any parsing, just sending message directly to Elasticsearch through FilbeBeat -\> Logstash(shipper) -\> RabbitMQ -\> Logstash(processor) -\> Elasticsearch pipeline

---

<div class="post-metadata">

**Author:** ![tudor](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tudor/32/3753_2.png) [@tudor](https://discuss.elastic.co/u/tudor)\
**Post date:** [February 14, 2017, 10:57am UTC](https://discuss.elastic.co/t/source-file-path-transfer-to-logstash/74708/5 "2017-02-14T10:57:09Z")

</div>

We store the file path in the `source` [field](https://www.elastic.co/guide/en/beats/filebeat/current/exported-fields-log.html). Perhaps the `fields_under_root` option is causing that field to be overwritten? Just a guess..

---

<div class="post-metadata">

**Author:** ![DenysK](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/denysk/32/16309_2.png) [@DenysK](https://discuss.elastic.co/u/DenysK)\
**Post date:** [February 14, 2017, 12:06pm UTC](https://discuss.elastic.co/t/source-file-path-transfer-to-logstash/74708/6 "2017-02-14T12:06:57Z")

</div>

Thanks for the advise.  
I have tried to set it "false", but I got the same result.

---

<div class="post-metadata">

**Author:** ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)\
**Post date:** [February 14, 2017, 1:49pm UTC](https://discuss.elastic.co/t/source-file-path-transfer-to-logstash/74708/7 "2017-02-14T13:49:03Z")

</div>

did you try to debug in logstash with stdout output? Can you share your full filebeat config? Can you share your logstash config? It's the first time I hear the source field being removed... which normally is only the case with filters.

---

<div class="post-metadata">

**Author:** ![DenysK](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/denysk/32/16309_2.png) [@DenysK](https://discuss.elastic.co/u/DenysK)\
**Post date:** [March 13, 2017, 8:45pm UTC](https://discuss.elastic.co/t/source-file-path-transfer-to-logstash/74708/8 "2017-03-13T20:45:55Z")

</div>

Smart People on ElasticON conference show me the problem and a fix.  
The root cause is that I'm sending to Message Queue plain text, but I should use json.

Thanks you, Smart Guys.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [April 10, 2017, 8:46pm UTC](https://discuss.elastic.co/t/source-file-path-transfer-to-logstash/74708/9 "2017-04-10T20:46:15Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.