# \[Resolved\]Logstash - data is not aggregrated in chronological order of a httpsession and output file looks like in Json format
**URL:** https://discuss.elastic.co/t/resolved-logstash-data-is-not-aggregrated-in-chronological-order-of-a-httpsession-and-output-file-looks-like-in-json-format/178835
**Category:** Logstash
**Created:** [April 29, 2019, 4:40am UTC](https://discuss.elastic.co/t/resolved-logstash-data-is-not-aggregrated-in-chronological-order-of-a-httpsession-and-output-file-looks-like-in-json-format/178835 "2019-04-29T04:40:54Z")
**Posts on this page:** 8
**Page:** 1
### Author:  [@cheriemilk](https://discuss.elastic.co/u/cheriemilk)
#### Post date: [April 29, 2019, 4:40am UTC](https://discuss.elastic.co/t/resolved-logstash-data-is-not-aggregrated-in-chronological-order-of-a-httpsession-and-output-file-looks-like-in-json-format/178835/1 "2019-04-29T04:40:54Z")
Hi Team,
Logstash configuraiton run as below
input { file{ path =\> "C:/elkstack/elasticsearch-6.5.1/logs/userscenario.csv"
start\_position =\> "beginning"
sincedb\_path =\> "C:/elkstack/elasticsearch-6.5.1/sincedb/sincedb.txt" }}
```
filter {
csv { columns => [ "when",
"httpsessionId",
"module",
"page",
"userId",
"actionType"]
separator => ","
skip_header => "true"}
aggregate { task_id => "%{httpsessionId}"
code => ' map["userscenario"] ||= ""
map["userscenario"] += event.get("actionType") + "->"
map["userId"] = event.get("userId")
event.cancel '
push_map_as_event_on_timeout => true
timeout_task_id_field => "httpsessionId"
timeout => 3600
timeout_code => ' event.set("userscenario", event.get("userscenario").chomp("->")) ' }}
output { file { path => "C:/elkstack/elasticsearch-6.5.1/logs/agguserscenario.csv" }
stdout { codec => rubydebug }
}
```
Content of the source file "userscenario.csv"
```
when,httpsessionId,module,page,userId,actionType
2019-02-13 10:01:30,sid1,succession,talentsearch,cgrant1,scm.ts.list_saved_search
2019-02-13 10:01:31,sid3,calibration,ManageCalibrationTemplates,hr1,cal.mct.create
2019-02-13 10:01:31,sid1,succession,talentsearch,cgrant1,scm.ts.start_over
2019-02-13 10:01:33,sid1,succession,talentsearch,cgrant1,scm.ts.delete_saved_search
2019-02-13 10:01:33,sid3,calibration,ManageCalibrationTemplates,hr1,cal.mct.edit
2019-02-13 10:01:30,sid2,succession,talentsearch,lokamoto1,scm.ts.list_saved_search
2019-02-13 10:01:33,sid2,succession,talentsearch,lokamoto1,scm.ts.search
2019-02-13 10:01:35,sid2,succession,talentsearch,lokamoto1,scm.ts.nominate
2019-02-13 10:01:35,sid3,calibration,ManageCalibrationTemplates,hr1,cal.mct.delete
```
\*\*After logstash finished the processing, I open the CSV file created by output file filter, and the file content is not created as expected. \*\*
**1. All content is in one column, looks like it's 3 json object**
**2. the final the string of userscenario field is not concatenating by time ascendly(The when field)**
```
{"httpsessionId":"sid1","userId":"cgrant1","@version":"1","userscenario":"scm.ts.start_over->scm.ts.delete_saved_search->scm.ts.list_saved_search","@timestamp":"2019-04-29T07:08:23.971Z"}
{"httpsessionId":"sid3","userId":"hr1","@version":"1","userscenario":"cal.mct.create->cal.mct.edit->cal.mct.delete","@timestamp":"2019-04-29T07:08:23.990Z"}
{"httpsessionId":"sid2","userId":"lokamoto1","@version":"1","userscenario":"scm.ts.nominate->scm.ts.list_saved_search->scm.ts.search","@timestamp":"2019-04-29T07:08:23.991Z"}
```
Expected output file
```
httpsessionId,userId,userscenario
sid1,cgrant1,scm.ts.list_saved_search->scm.ts.start_over->scm.ts.delete_saved_search
sid2,hr1,cal.mct.create->cal.mct.edit->cal.mct.delete
sid3,lokamoto1,scm.ts.list_saved_search->scm.ts.search->scm.ts.nominate
```
How to correct?
---
### Author:  [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [April 29, 2019, 2:16pm UTC](https://discuss.elastic.co/t/resolved-logstash-data-is-not-aggregrated-in-chronological-order-of-a-httpsession-and-output-file-looks-like-in-json-format/178835/2 "2019-04-29T14:16:10Z")
> [@cheriemilk](#):
>
> All content is in one column, looks like it's 3 json object
You are using a file output, which defaults to a json\_lines codec. Try using a csv output.
---
### Author:  [@cheriemilk](https://discuss.elastic.co/u/cheriemilk)
#### Post date: [April 29, 2019, 10:32pm UTC](https://discuss.elastic.co/t/resolved-logstash-data-is-not-aggregrated-in-chronological-order-of-a-httpsession-and-output-file-looks-like-in-json-format/178835/3 "2019-04-29T22:32:58Z")
Ok. Thx.Trying...
Another question: how to make the data aggregated in chronological order??
The userscenario for cgrant1 is aggregated to "userscenario":"scm.ts.start\_over-\>scm.ts.delete\_saved\_search-\>scm.ts.list\_saved\_search"
But the expected aggregated sequence should be below according to the “when” field which means when the actionType happened on WebUI.
"userscenario":"scm.ts.list\_saved\_search-\>scum.ts.start\_over-\>scm.ts.delete\_saved\_search”
---
### Author:  [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [April 29, 2019, 11:49pm UTC](https://discuss.elastic.co/t/resolved-logstash-data-is-not-aggregrated-in-chronological-order-of-a-httpsession-and-output-file-looks-like-in-json-format/178835/4 "2019-04-29T23:49:38Z")
Do you have pipeline.java\_execution enabled? Do you have "--pipeline.workers 1" set?
---
### Author:  [@cheriemilk](https://discuss.elastic.co/u/cheriemilk)
#### Post date: [April 30, 2019, 2:24am UTC](https://discuss.elastic.co/t/resolved-logstash-data-is-not-aggregrated-in-chronological-order-of-a-httpsession-and-output-file-looks-like-in-json-format/178835/5 "2019-04-30T02:24:02Z")
[2019-04-29T15:46:34,297][DEBUG][logstash.runner] pipeline.java\_execution: false
[2019-04-29T15:46:34,296][DEBUG][logstash.runner] pipeline.workers: 4
Do you have pipeline.java\_execution enabled?
- No
Do you have "--pipeline.workers 1" set?
- No
What does this two mean??
---
### Author:  [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [April 30, 2019, 1:27pm UTC](https://discuss.elastic.co/t/resolved-logstash-data-is-not-aggregrated-in-chronological-order-of-a-httpsession-and-output-file-looks-like-in-json-format/178835/6 "2019-04-30T13:27:02Z")
The execution engine that logstash runs on is being rewritten into Java. At one point I was seeing ordering problems with that.
Your issue is much more likely to be that you have multiple worker threads. That produces race conditions because the threads may process events out of order. Set "--pipeline.workers 1". That means logstash will only use 1 CPU, but it is a requirement of the aggregate filter.
---
### Author:  [@cheriemilk](https://discuss.elastic.co/u/cheriemilk)
#### Post date: [May 2, 2019, 6:02am UTC](https://discuss.elastic.co/t/resolved-logstash-data-is-not-aggregrated-in-chronological-order-of-a-httpsession-and-output-file-looks-like-in-json-format/178835/7 "2019-05-02T06:02:12Z")
Thank you~
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [May 30, 2019, 6:02am UTC](https://discuss.elastic.co/t/resolved-logstash-data-is-not-aggregrated-in-chronological-order-of-a-httpsession-and-output-file-looks-like-in-json-format/178835/8 "2019-05-30T06:02:13Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.