# Logstash with CSV input

**URL:** <https://discuss.elastic.co/t/logstash-with-csv-input/142044>\
**Category:** Logstash\
**Created:** [July 28, 2018, 5:52pm UTC](https://discuss.elastic.co/t/logstash-with-csv-input/142044 "2018-07-28T17:52:25Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![Vincent\_Maury](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vincent_maury/32/59973_2.png) [@Vincent\_Maury](https://discuss.elastic.co/u/Vincent_Maury)\
**Post date:** [July 28, 2018, 5:52pm UTC](https://discuss.elastic.co/t/logstash-with-csv-input/142044/1 "2018-07-28T17:52:25Z")

</div>

Hi,  
I'm trying to read a CSV file and insert in ES, all in 6.3.2. Should be easy right?  
I created an index template:

```auto
PUT _template/mytemplate
{
  "index_patterns": ["test*"],
  "settings": {
    "index.mapping.total_fields.limit": 10000,
    "number_of_shards": 1
  },
  "mappings": {
    "doc": {
      "properties": {
				"sex": { "type": "text" },
				"country": { "type": "text" }
			}
		}
	}
}

```

Then I'm using this logstash.conf:

```auto
input {
  file {
    path => "C:\Users\Vincent\Documents\tech\logstash-6.3.2\data\wholedf_100_colon_small.csv"
    start_position => "beginning"
    close_older => 60
    sincedb_path => "/dev/null"
  }
}

filter {
  csv {
    separator => ","
    autodetect_column_names => true
    autogenerate_column_names => true
    remove_field => ["path", "message", "host", "@timestamp", "@version"]
  }
}

output {
  elasticsearch {
    hosts => ["https://ec_id.europe-west1.gcp.cloud.es.io:9243"]
    user => "elastic"
    password => "mypassword"
    index => "test"
    manage_template => false
    template_name => "mytemplate"
  }
}

```

And this sample CSV

```auto
"sex","country"
"M","FR"
"F","UK"

```

And after logstash execution, I run a `GET test/_search` and I get:

```auto
...
  "hits": {
    "total": 2,
    "max_score": 1,
    "hits": [
      {
        "_index": "test",
        "_type": "doc",
        "_id": "14b84WQBmdGmeuM4UGwp",
        "_score": 1,
        "_source": {
          "country": "FR",
          "sex": "M"
        }
      },
      {
        "_index": "test",
        "_type": "doc",
        "_id": "2Ib84WQBmdGmeuM4UGwp",
        "_score": 1,
        "_source": {
          "country": "UK",
          "sex": "F"
        }
      }
    ]
  }

```

What should I do to get the fields as fields and not in the \_source field?  
Thanks a lot in advance for your help!

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [July 28, 2018, 7:03pm UTC](https://discuss.elastic.co/t/logstash-with-csv-input/142044/2 "2018-07-28T19:03:58Z")

</div>

Everything looks fine. What's under `_source` is the actual contents of the stored document. `_source`, `_index`, `_type` et al are document metadata.

---

<div class="post-metadata">

**Author:** ![Vincent\_Maury](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vincent_maury/32/59973_2.png) [@Vincent\_Maury](https://discuss.elastic.co/u/Vincent_Maury)\
**Post date:** [July 30, 2018, 3:24pm UTC](https://discuss.elastic.co/t/logstash-with-csv-input/142044/3 "2018-07-30T15:24:10Z")

</div>

Thanks you very much Magnus for your answer.  
Weirdly, when I disable the \_source field, I can still search like `_search?q=sex:M` but when I look at the doc I don't see the fields anymore!

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [August 27, 2018, 3:24pm UTC](https://discuss.elastic.co/t/logstash-with-csv-input/142044/4 "2018-08-27T15:24:11Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.