# Index fields concatenated together? **URL:** \ **Category:** Logstash\ **Created:** [August 28, 2017, 8:17pm UTC](https://discuss.elastic.co/t/index-fields-concatenated-together/98604 "2017-08-28T20:17:04Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![bulsac](https://avatars.discourse-cdn.com/v4/letter/b/50afbb/32.png) [@bulsac](https://discuss.elastic.co/u/bulsac)\ **Post date:** [August 28, 2017, 8:17pm UTC](https://discuss.elastic.co/t/index-fields-concatenated-together/98604/1 "2017-08-28T20:17:04Z")
I recently added 2 new .CONF files to my /etc/logstash/conf.d/ directory to ingest data into 2 new indices, in addition to 1 existing index. When I view the 2 new indices in Kibana, it appears that the 2 new indices contain all columns from all indices. This creates obvious problems when I try to import data: data from index 2 does not map to index 3, much less indices 1 and 3 simultaneously. I've checked the CONF files for obvious mistakes like missed braces, parentheses, commas. I have also searched these forums and not found any similar topics. Any advice on how to troubleshoot this issue? Thanks, Cyrus ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [August 29, 2017, 5:26am UTC](https://discuss.elastic.co/t/index-fields-concatenated-together/98604/2 "2017-08-29T05:26:40Z")
> I recently added 2 new .CONF files to my /etc/logstash/conf.d/ directory to ingest data into 2 new indices, in addition to 1 existing index. What do the elasticsearch outputs in these two files look like? > When I view the 2 new indices in Kibana, it appears that the 2 new indices contain all columns from all indices. Not sure exactly what you mean by this. ---
**Author:** ![bulsac](https://avatars.discourse-cdn.com/v4/letter/b/50afbb/32.png) [@bulsac](https://discuss.elastic.co/u/bulsac)\ **Post date:** [August 29, 2017, 7:59pm UTC](https://discuss.elastic.co/t/index-fields-concatenated-together/98604/3 "2017-08-29T19:59:29Z")
CONF files are pasted below. Let me try to explain the issue better. I have 3 indices specified in separate CONF files, A.CONF B.CONF C.CONF. Each of these indices has a unique set of columns, and ingests different sources of data from different directories. The problem is that indices B and C contain all columns from indices A, B, and C when I view the index in Kibana, rather than just the columns I specified in the CONF file. I hope this explanation is more clear. ``` **A.CONF** input { file { path => "/var/elk/csv/sep/*.csv" start_position => "beginning" } } filter { csv { separator => "," columns => ["Computer_Name","Measure_Date","Pattern_Date", "Operating_System","Client_Version","Policy_Serial", "HI_Status","Status","Auto_Protect_On","Worst_Detection", "Last_Scan_Time","Antivirus_engine_On","Download_Insight_On", "SONAR_On","Tamper_Protection_On","Intrusion_Prevention_On", "IE_Browser_Protection_On","Firefox_Browser_Protection_On", "Early_Launch_Antimalware_On","Server_Name","MAC_Address1", "cmdb_name","cmdb_friendly","cmdb_model_id","cmdb_serial","found"] } } output { elasticsearch { hosts => "http://localhost:9200" index => "sep-index" } } **B.CONF** input { file { path => "/var/elk/csv/tasks/*.csv" start_position => "beginning" } } filter { csv { separator => "," columns => ["number","priority","state","assignment_group", "short_description","sys_class_name", "sys_created_on","closed_at","duration"] } } output { elasticsearch { hosts => "http://localhost:9200" index => "tasks-index" } } **C.CONF** input { file { path => "/var/elk/csv/incidents/*.csv" start_position => "beginning" } } filter { csv { separator => "," columns => ["number","opened_at","short_description", "priority","state","subcategory","closed_at", "close_notes","current_impact", "future_impact","calendar_duration"] } } output { elasticsearch { hosts => "http://localhost:9200" index => "incidents-index" } } ``` ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [August 30, 2017, 3:31am UTC](https://discuss.elastic.co/t/index-fields-concatenated-together/98604/4 "2017-08-30T03:31:25Z")
Unless you explicitly define multiple pipelines (see [https://www.elastic.co/guide/en/logstash/master/multiple-pipelines.html](https://www.elastic.co/guide/en/logstash/master/multiple-pipelines.html)), Logstash effectively concatenates all configuration files so that all events from all inputs are passed through all filters and reach all outputs. If you don't want this you need to wrap filters and outputs in conditionals. ---
**Author:** ![bulsac](https://avatars.discourse-cdn.com/v4/letter/b/50afbb/32.png) [@bulsac](https://discuss.elastic.co/u/bulsac)\ **Post date:** [September 1, 2017, 10:31pm UTC](https://discuss.elastic.co/t/index-fields-concatenated-together/98604/5 "2017-09-01T22:31:00Z")
Thank you Magnus for your help. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [September 29, 2017, 10:31pm UTC](https://discuss.elastic.co/t/index-fields-concatenated-together/98604/6 "2017-09-29T22:31:04Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.