# Read source filename from beats and redirect output to different Elastic Search index

**URL:** <https://discuss.elastic.co/t/read-source-filename-from-beats-and-redirect-output-to-different-elastic-search-index/127987>\
**Category:** Logstash\
**Created:** [April 13, 2018, 1:45pm UTC](https://discuss.elastic.co/t/read-source-filename-from-beats-and-redirect-output-to-different-elastic-search-index/127987 "2018-04-13T13:45:15Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![pavanssetty](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/pavanssetty/32/25314_2.png) [@pavanssetty](https://discuss.elastic.co/u/pavanssetty)\
**Post date:** [April 13, 2018, 1:45pm UTC](https://discuss.elastic.co/t/read-source-filename-from-beats-and-redirect-output-to-different-elastic-search-index/127987/1 "2018-04-13T13:45:15Z")

</div>

I want to read the source of the csv data and redirect the output to a different ElasticSearch index.  
I tried as below (if condition with source in output) by following various discussions but it did not work. It is always going to else part.

Here is my logstash config:

```
input {
	beats {
		port => "5044"
	}
}
filter {
	csv {
		autodetect_column_names => true
		separator => ","
		remove_field => ["message"]
	}
}

output {
	**if [source] == 'E:\\test_ingest\\test.csv'** {
		elasticsearch {
			hosts => "localhost:9200"
			manage_template => false
			index => "test_index"
			document_type => "nasuni_log"
		}
	} else {
              elasticsearch {
			hosts => "localhost:9200"
			manage_template => false
			index => "b_index"
			document_type => "nasuni_log"
		}
	}
}
```

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [April 13, 2018, 3:09pm UTC](https://discuss.elastic.co/t/read-source-filename-from-beats-and-redirect-output-to-different-elastic-search-index/127987/2 "2018-04-13T15:09:31Z")

</div>

The approach seems reasonable. What does one of the events look like? Copy and paste from the JSON tab in Kibana Discover.

---

<div class="post-metadata">

**Author:** ![pavanssetty](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/pavanssetty/32/25314_2.png) [@pavanssetty](https://discuss.elastic.co/u/pavanssetty)\
**Post date:** [April 16, 2018, 9:07am UTC](https://discuss.elastic.co/t/read-source-filename-from-beats-and-redirect-output-to-different-elastic-search-index/127987/3 "2018-04-16T09:07:42Z")

</div>

Here is the input example from csv with header:

> timestamp(UTC),category,event type,path/from,new path/to,user,group,sid,share/export name,volume type,client IP,snapshot timestamp(UTC),shared link  
> 2018-04-12 12:16:57.846484,Write,Truncate File,/Design/CCH/t3est/review comments.docx,,EMEA\kandrgan,EMEA\domain users,S-1-5-21-1409082233-362288127-725345543-319468,Design,CIFS,10.123.3.58,,

Here is the json from Kibana:  
I have only hidden the business related values

```
{
  "_index": "b_index",
  "_type": "nasuni_log",
  "_id": "AWK_97z0iUaCMb2pr7XB",
  "_version": 2,
  "_score": null,
  "_source": {
    "shared link": null,
    "new path/to": null,
    "offset": 363965,
    "volume type": "CIFS",
    "path/from": "/Design/CCH/t3est/review comments.docx",
    "input_type": "log",
    "source": "E:\\test_ingest\\test1.csv",
    "type": "log",
    "tags": [
      "beats_input_codec_plain_applied",
      "_dateparsefailure"
    ],
    "sid": "S-1-5-21-1409082233-362288127-725345543-319468",
    "@timestamp": "2018-04-13T17:05:01.399Z",
    "event type": "Truncate File",
    "share/export name": "Design",
    "@version": "1",
    "beat": {
      "hostname": "DEHEREMAP9769",
      "name": "DEHEREMAP9769",
      "version": "5.4.0"
    },
    "host": "DEHEREMAP9769",
    "timestamp(UTC)": "2018-04-12 19:27:32.456324",
    "snapshot timestamp(UTC)": null,
    "category": "Write",
    "client IP": "10.143.11.26",
    "user": "EMEA\\kandrgan",
    "group": "EMEA\\domain users"
  },
  "fields": {
    "@timestamp": [
      1523639101399
    ]
  },
  "sort": [
    1523639101399
  ]
}

```

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [April 16, 2018, 11:41am UTC](https://discuss.elastic.co/t/read-source-filename-from-beats-and-redirect-output-to-different-elastic-search-index/127987/4 "2018-04-16T11:41:02Z")

</div>

If that is what you see in Kibana then I believe this, with single backslashes, will compare true:

```auto
if [source] == 'E:\test_ingest\test.csv'

```

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [May 14, 2018, 11:41am UTC](https://discuss.elastic.co/t/read-source-filename-from-beats-and-redirect-output-to-different-elastic-search-index/127987/5 "2018-05-14T11:41:02Z")

</div>

Closed on May 14, 2018, 11:41am UTC

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.