# Parse string data with ; separator in logstash

**URL:** https://discuss.elastic.co/t/parse-string-data-with-separator-in-logstash/87524
**Category:** Logstash
**Created:** [May 30, 2017, 8:42am UTC](https://discuss.elastic.co/t/parse-string-data-with-separator-in-logstash/87524 "2017-05-30T08:42:30Z")
**Posts on this page:** 7
**Page:** 1

<div class="post-metadata">

### Author: ![Charlotte\_EGM](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/charlotte_egm/32/13503_2.png) [@Charlotte\_EGM](https://discuss.elastic.co/u/Charlotte_EGM)
#### Post date: [May 30, 2017, 8:42am UTC](https://discuss.elastic.co/t/parse-string-data-with-separator-in-logstash/87524/1 "2017-05-30T08:42:30Z")

</div>

Hi,

I'm trying to parse data that looks like this :

“timestamp1;value1; timestamp2;value2;… timestampN;valueN;”

I'm trying csv filter (as they originally comes from csv), but it is not working well,as it doesn't understand that there is only 2 features.

I know that grok filter can do a lot of things, but i don't know how to configure it so that logstash understand that there is 2 repeated features only.

Any help ?

Thanks

---

<div class="post-metadata">

### Author: ![Sylfaen](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sylfaen/32/17653_2.png) [@Sylfaen](https://discuss.elastic.co/u/Sylfaen)
#### Post date: [May 30, 2017, 9:02am UTC](https://discuss.elastic.co/t/parse-string-data-with-separator-in-logstash/87524/2 "2017-05-30T09:02:19Z")

</div>

Hi

You can try [mutate split](https://www.elastic.co/guide/en/logstash/current/plugins-filters-mutate.html#plugins-filters-mutate-split). It'll create an array [timestamp1,value1,timestamp2...] then you can remove the even indexes which match with the timestampX field's name and have [value1,value2,...]. But what about your output ?

---

<div class="post-metadata">

### Author: ![Charlotte\_EGM](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/charlotte_egm/32/13503_2.png) [@Charlotte\_EGM](https://discuss.elastic.co/u/Charlotte_EGM)
#### Post date: [May 30, 2017, 1:24pm UTC](https://discuss.elastic.co/t/parse-string-data-with-separator-in-logstash/87524/3 "2017-05-30T13:24:54Z")

</div>

Thanks,

I can change the input format for “timestamp1,value1; timestamp2,value2;… timestampN,valueN;”

Then i tried to split on ";" but csv filter does not recognize the array then.

So i tried implementing a new line instead of ";" with "gsub" filter. Here is my config :

```
 filter {
    mutate{
        gsub => ["message",";","^M"]
    }

   csv {
     separator => ","
     columns => ["Time","Temp"]
     convert => {"Temp" => "float"}
   }
}

```

So I get the following message before csv filter :  
“timestamp1,value1  
timestamp2,value2  
…  
timestampN,valueN;”

That's what I wanted, to look like a csv file...

But after csv filter, i get only the first line parsed...  
How can I parse all the lines ?

---

<div class="post-metadata">

### Author: ![Sylfaen](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sylfaen/32/17653_2.png) [@Sylfaen](https://discuss.elastic.co/u/Sylfaen)
#### Post date: [May 30, 2017, 1:51pm UTC](https://discuss.elastic.co/t/parse-string-data-with-separator-in-logstash/87524/4 "2017-05-30T13:51:37Z")

</div>

> [@Charlotte\_EGM](#):
>
> timestamp1,value1  
> timestamp2,value2  
> …  
> timestampN,valueN;

It's already like a CSV file. If you create a .csv file with those datas, you'll have a CSV file if you choose **,** as separator no ? So you don't need the csv filter plugin exceptf if you want to add the columns.

What do you want is to transform a string with a pattern to a CSV file right ? So your mutate clause is sufficient to do what you want or I miss something ^^'

---

<div class="post-metadata">

### Author: ![Charlotte\_EGM](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/charlotte_egm/32/13503_2.png) [@Charlotte\_EGM](https://discuss.elastic.co/u/Charlotte_EGM)
#### Post date: [May 30, 2017, 2:13pm UTC](https://discuss.elastic.co/t/parse-string-data-with-separator-in-logstash/87524/5 "2017-05-30T14:13:25Z")

</div>

I don't want to create a csv file, I just want to parse the data as if they were a csv file, because I have 2 repeated features.  
If they is another way than csv, i would be happy to try it.

But the output is into elasticsearch, and i need to do it for streaming events that will come into logstash.

---

<div class="post-metadata">

### Author: ![Charlotte\_EGM](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/charlotte_egm/32/13503_2.png) [@Charlotte\_EGM](https://discuss.elastic.co/u/Charlotte_EGM)
#### Post date: [May 30, 2017, 3:02pm UTC](https://discuss.elastic.co/t/parse-string-data-with-separator-in-logstash/87524/6 "2017-05-30T15:02:39Z")

</div>

I succeed to obtain what I wanted by adding

```
  split {
     terminator => "^M"
  }

```

after mutate{} and before csv {}

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [June 27, 2017, 3:03pm UTC](https://discuss.elastic.co/t/parse-string-data-with-separator-in-logstash/87524/7 "2017-06-27T15:03:33Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
