Adding incremental column based on values of another column

[Felicien_Ihirwe]

I want to create a logstash filter to come from table 1 to table 2:

col1
in
in
out
in
out
..
.

I want to add a new column that will contain incremental values and the data will look like
col1 | col 2
in | 1
in | 1
out | 1
in | 2
out | 2
.. | 3
.

any help?

[Badger]

logstash is not a good fit for this task. You would have to write a ruby filter to keep track of the values to be added in the second column. You would also need a single worker thread (pipeline.workers 1) and make sure that pipeline.ordered evaluates to true.

[Felicien_Ihirwe]

Thank you for your answer.. it works and I was able to solve it. I used the in and out values to differentiate the start and the end of the aggregation tasks.. but now I have another issue:

what if the left column is not a preknown value? for instance a continuous number like this:

col1
2.3
2.3
2.3
5.7
5.7
6.1
6.1
....
..
.

I want to achieve this:
col1 | col2
2.3 | 1
2.3 |1
2.3 |1
5.7 |2
5.7 |2
6.1 |3
6.1 |3
....
..
.