# Elastic Dissect

**URL:** <https://discuss.elastic.co/t/elastic-dissect/329117>\
**Category:** Logstash\
**Created:** [April 2, 2023, 9:39am UTC](https://discuss.elastic.co/t/elastic-dissect/329117 "2023-04-02T09:39:13Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![oleksiiorel](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/oleksiiorel/32/118205_2.png) [@oleksiiorel](https://discuss.elastic.co/u/oleksiiorel)\
**Post date:** [April 2, 2023, 9:39am UTC](https://discuss.elastic.co/t/elastic-dissect/329117/1 "2023-04-02T09:39:13Z")

</div>

Hi everyone 🙂  
How to split a single cell with a different number of strings in CVS file into separate cells (fields).  
Assign the names of the fields from the string itself. If I manually write the names of the fields and their number(like in the example below), it will work only if all strings are contained in each cell but in reality, the number of strings in a cell is different.  
example

Data cvs.file

```auto
ID,Name,Category
1,Desk Chair,"Furniture,Hard furnishings,categoryE7E7163,Hendi,"
2,Sofa,"Furniture,Soft furnishings, Bartscher"

```

```auto
input{
  file {
	path => "E:/test.csv"  
    start_position => "beginning"
    sincedb_path => "NULL"
  }	  
}

filter {
  dissect {
    mapping => {
      "message" => '%{product_id},%{name},"%{category},%{sub_category},%{alt_category},%{brand}"'
      # it's okay for id=1 (Desk Chair) but not right for id=2 (Sofa)
    }
  }
}
output{
  elasticsearch {
    hosts => "http://localhost:9200"
    index => "test_products"
	user => "elastic"
    password => "**" 
  }
stdout{}
}

```

What I need to be stored in ELK

```auto
"_source": {
"ID": 1,
"Name": "Desk Chair",
"Category": "Furniture",
"Hard furnishings": "Hard furnishings",
"categoryE7E7163": "categoryE7E7163",
"Hendi": "Hendi",
}

"_source": {
"ID": 2,
"Name": "Sofa",
"Category": "Furniture",
"Soft furnishings": "Soft furnishings",
"categoryE7E7163": "categoryE7E7163",
"Bartscher": "Bartscher",
}

```

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [April 2, 2023, 10:14am UTC](https://discuss.elastic.co/t/elastic-dissect/329117/2 "2023-04-02T10:14:42Z")

</div>

What do you mean by:

> it's okay for id=1 (Desk Chair) but not right for id=2 (Sofa)

What does this produce?

BTW I'm moving your question to #Logstash

---

<div class="post-metadata">

**Author:** ![oleksiiorel](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/oleksiiorel/32/118205_2.png) [@oleksiiorel](https://discuss.elastic.co/u/oleksiiorel)\
**Post date:** [April 2, 2023, 10:52am UTC](https://discuss.elastic.co/t/elastic-dissect/329117/3 "2023-04-02T10:52:57Z")

</div>

Hi, David!  
At the moment it produces all "\_dissectfailure" strings  
Let me try to rephrase.  
I need to write a universal entry in 'message' which would separate strings into separate fields regardless of their number. The names of the fields must match the value of the string in the cell. Here is the working code:  
Data

```auto
ID,Name,Category
1,Sofa,"Furniture,Soft furnishings,TestString"
2,Desk Chair,"Furniture,Hard furnishings,TestString2"

```

```auto
input{
  file {
	path => "E:/test.csv"  
    start_position => "beginning"
    sincedb_path => "NULL"
  }	  
}

filter {
  dissect {
    mapping => {
      "message" => '%{product_id},%{name},"%{category},%{sub_category},%{alt_category}"'
    }
  }
}
output{
  elasticsearch {
    hosts => "http://localhost:9200"
    index => "test_products5"
	user => "elastic"
    password => "**" 
  }
stdout{}
}

```

The problem is that the Category for each row can be only 3 strings or 6 strings and the names are different. So I decided that it's better to take the field names from the string itself  
In the example it works because in the cvs file I put the same number of string in the Category, but in the real database there is a different number of string for each row

Real Data Example

```auto
ID,Name,Category
1,Sofa,"1_brands,1_series,1_smallwares,1_storage_and_transport,2_brand_contacto,2_food_storage_supplies,2_food_storage_supplies_2,2_series_501_1000,3_gastronorm_containers,3_gastronorm_containers_2,3_series_contacto_gn_container_melamin,categoryE7E7163"
2,Desk Chair,"1_brands,1_series,1_smallchair"

```

I found ''. dot key solution'' which might help me but don't know how to implement it. It should look like this I guess:

```auto
filter {
  dissect {
    mapping => {
      "message" => '%{product_id},%{name},"%{.},%{.},%{.}%{.},%{.},%{.}"'
    }
  }

```

but it doesn't work also I don't know how to deal with different numbers of Category strings  
To put it simply, I need to somehow save the data from the Category into separate json fields  
}

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [April 2, 2023, 3:13pm UTC](https://discuss.elastic.co/t/elastic-dissect/329117/4 "2023-04-02T15:13:02Z")

</div>

I would not use dissect at all try

```
    csv { autodetect_column_names => true }
    ruby {
        code => '
            c = event.get("Category")
            if c
                a = c.split(",")
                event.set("Category", a.shift)
                a.each { |x| event.set(x.strip, x.strip) }
            end
        '
    }

```

you will need pipeline.ordered to evaluate to true and pipeline.workers to be 1.

In your file input `sincedb_path => "NULL"` will result in the in-memory sincedb being persisted across restarts in a file called NULL in logstash's working directory. If you do not want it persisted then use `sincedb_path => "NUL"`.

---

<div class="post-metadata">

**Author:** ![oleksiiorel](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/oleksiiorel/32/118205_2.png) [@oleksiiorel](https://discuss.elastic.co/u/oleksiiorel)\
**Post date:** [April 2, 2023, 3:57pm UTC](https://discuss.elastic.co/t/elastic-dissect/329117/5 "2023-04-02T15:57:31Z")

</div>

Works perfectly! Thank you very much!

---

<div class="post-metadata">

**Author:** ![oleksiiorel](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/oleksiiorel/32/118205_2.png) [@oleksiiorel](https://discuss.elastic.co/u/oleksiiorel)\
**Post date:** [April 3, 2023, 8:25am UTC](https://discuss.elastic.co/t/elastic-dissect/329117/6 "2023-04-03T08:25:13Z")

</div>

Hi man!  
Do you have any idea why when I put your code only 6873 objects are imported into elastic instead of 53k. When I remove the ruby code, all the elements could be stored. I don't get any errors.

Data

```auto
SKU;Kategoria;Rodzina;Marka;Kod u dostawcy;Kod u dostawcy 2;Nazwa towaru
00000290;1_brands,1_restaurant_equipment,2_brand_rm_gastro,2_food_holding_and_warming_equipment,3_steam_heaters_and_buffets,4_bain_marie_heaters,categoryE7E7163;bemary grzewcze;rm gastro;00000290;BMPD 2120;Bemar elektryczny 2-komorowy, GN 1/1, jezdny z kranem spustowym, 1,4 kW;

```

code

```auto
input{
  file {
	path => "C:/db.csv" 
    start_position => "beginning"
    sincedb_path => "NULL"
  }	  
}

filter {
   csv { 
	separator => ";"
        skip_empty_columns => true
	columns => ["SKU","Kategoria","Rodzina","Marka","Kod u dostawcy","Kod u dostawcy2", "Nazwa towaru"]
	}
	
	ruby {
        code => '
            c = event.get("Kategoria")
            if c
                a = c.split(",")
                event.set("Kategoria", a.shift)
                a.each { |x| event.set(x.strip, x.strip) }
            end
        '
    }
}
output{
  elasticsearch {
    hosts => "http://localhost:9200"
    index => "products"
	user => "elastic"
    password => "**" 
  }
stdout{}
}

I replaced 'autodetect_column_names => true' - because this parameter caused incorrect placement of the columns and their contents

```

when I comment this part of code I'm able to import all text from csv:

```auto
	ruby {
        code => '
            c = event.get("Kategoria")
            if c
                a = c.split(",")
                event.set("Kategoria", a.shift)
                a.each { |x| event.set(x.strip, x.strip) }
            end
        '
    }

```

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [May 1, 2023, 8:25am UTC](https://discuss.elastic.co/t/elastic-dissect/329117/7 "2023-05-01T08:25:37Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.