# Unable to output csv file from Logstash

**URL:** <https://discuss.elastic.co/t/unable-to-output-csv-file-from-logstash/49588>\
**Category:** Logstash\
**Created:** [May 9, 2016, 9:12pm UTC](https://discuss.elastic.co/t/unable-to-output-csv-file-from-logstash/49588 "2016-05-09T21:12:03Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![Dark\_Light](https://avatars.discourse-cdn.com/v4/letter/d/7ba0ec/32.png) [@Dark\_Light](https://discuss.elastic.co/u/Dark_Light)\
**Post date:** [May 9, 2016, 9:12pm UTC](https://discuss.elastic.co/t/unable-to-output-csv-file-from-logstash/49588/1 "2016-05-09T21:12:03Z")

</div>

I am unable to figure out how to generate \*.csv files from Logstash. My goal is to generate csv files from multiple inputs i.e. winlogbeat, filebeat (from Mac Syslogs) and syslog (from our router). I have these 3 inputs working and producing output to Elasticsearch and availabe to view in Kibana. Keeping the config file simple for trouble shooting I have the following logstash conf file working and producing output to stdout and Kibana. I just cannot seem to create a csv file.

Thank you for any assistance and education you can provide.  
Rick

input {  
beats {  
port =\> 5044  
}  
}

filter {  
csv {}  
}

output {  
file {  
path =\> "/home/user/winlogbeat\_out.csv"  
codec =\> plain  
}  
csv {  
fields =\> ["1","2","3","4","5","6"]  
path =\> "/home/user/winlogbeat1\_out.csv"  
}  
stdout {  
codec =\> rubydebug  
}   
elasticsearch {  
hosts =\> ["localhost:9200"]  
index =\> "%{[@metadata][beat]}-%{+YYYY.MM.dd}"  
document\_type =\> "%{[@metadata][type]}"  
}  
}

Here is a sample of the stdout data

{  
"message" =\> "An account was successfully logged on.\n\nSubject:\n\tSecurity ID:\t\tS-1-0-0\n\tAccount Name:\t\t-\n\tAccount Domain:\t\t-\n\tLogon ID:\t\t0x0\n\nLogon Type:\t\t\t3\n\nImpersonation Level:\t\tDelegation\n\nNew Logon:\n\tSecurity ID:\t\tS-1-5-21-96102576-1364680283-2145283710-1121\n\tAccount Name:\t\tNEBBIOLO$\n\tAccount Domain:\t\tCTCI\n\tLogon ID:\t\t0x2770E0EF\n\tLogon GUID:\t\t{983A4897-CD3E-62D0-5626-02B60DF03A42}\n\nProcess Information:\n\tProcess ID:\t\t0x0\n\tProcess Name:\t\t-\n\nNetwork Information:\n\tWorkstation Name:\t\n\tSource Network Address:\t192.168.5.151\n\tSource Port:\t\t54675\n\nDetailed Authentication Information:\n\tLogon Process:\t\tKerberos\n\tAuthentication Package:\tKerberos\n\tTransited Services:\t-\n\tPackage Name (NTLM only):\t-\n\tKey Length:\t\t0\n\nThis event is generated when a logon session is created. It is generated on the computer that was accessed.\n\nThe subject fields indicate the account on the local system which requested the logon. This is most commonly a service such as the Server service, or a local process such as Winlogon.exe or Services.exe.\n\nThe logon type field indicates the kind of logon that occurred. The most common types are 2 (interactive) and 3 (network).\n\nThe New Logon fields indicate the account for whom the new logon was created, i.e. the account that was logged on.\n\nThe network fields indicate where a remote logon request originated. Workstation name is not always available and may be left blank in some cases.\n\nThe impersonation level field indicates the extent to which a process in the logon session can impersonate.\n\nThe authentication information fields provide detailed information about this specific logon request.\n\t- Logon GUID is a unique identifier that can be used to correlate this event with a KDC event.\n\t- Transited services indicate which intermediate services have participated in this logon request.\n\t- Package name indicates which sub-protocol was used among the NTLM protocols.\n\t- Key length indicates the length of the generated session key. This will be 0 if no session key was requested.",  
"@version" =\> "1",  
"@timestamp" =\> "2016-05-09T20:37:21.430Z",  
"beat" =\> {  
"hostname" =\> "ctci-01",  
"name" =\> "ctci-01"  
},  
"category" =\> "Logon",  
"computer\_name" =\> "ctci-01.CTCI.local",  
"count" =\> 1,  
"event\_id" =\> 4624,  
"level" =\> "Information",  
"log\_name" =\> "Security",  
"record\_number" =\> "43672042",  
"source\_name" =\> "Microsoft-Windows-Security-Auditing",  
"type" =\> "wineventlog",  
"host" =\> "ctci-01",  
"tags" =\> [  
[0] "beats\_input\_codec\_plain\_applied"  
],  
"column1" =\> "An account was successfully logged on."  
}

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [May 9, 2016, 9:15pm UTC](https://discuss.elastic.co/t/unable-to-output-csv-file-from-logstash/49588/2 "2016-05-09T21:15:53Z")

</div>

> [@Dark\_Light](#):
>
> fields =\> ["1","2","3","4","5","6"]

You do not seem to have any fields with these names in the event.

---

<div class="post-metadata">

**Author:** ![Dark\_Light](https://avatars.discourse-cdn.com/v4/letter/d/7ba0ec/32.png) [@Dark\_Light](https://discuss.elastic.co/u/Dark_Light)\
**Post date:** [May 9, 2016, 9:34pm UTC](https://discuss.elastic.co/t/unable-to-output-csv-file-from-logstash/49588/3 "2016-05-09T21:34:52Z")

</div>

Thank you for the education, I incorrectly assumed that data in colum1 would be put into field1, etc.

For future reference is there a way to dynamically create these field names as the content will change with various event logs.

I have updated my conf file with the following field names.  
fields =\> ["message","@version","@timestamp","beat","hostname","name","category","computer\_name","count","event\_id","level","log\_name","record\_number","source\_name","type","host","tags","column1"]

However a csv file is still being created.  
Rick

---

<div class="post-metadata">

**Author:** ![Dark\_Light](https://avatars.discourse-cdn.com/v4/letter/d/7ba0ec/32.png) [@Dark\_Light](https://discuss.elastic.co/u/Dark_Light)\
**Post date:** [May 9, 2016, 9:35pm UTC](https://discuss.elastic.co/t/unable-to-output-csv-file-from-logstash/49588/4 "2016-05-09T21:35:53Z")

</div>

correction, still NOT being created.

---

<div class="post-metadata">

**Author:** ![Dark\_Light](https://avatars.discourse-cdn.com/v4/letter/d/7ba0ec/32.png) [@Dark\_Light](https://discuss.elastic.co/u/Dark_Light)\
**Post date:** [May 9, 2016, 11:55pm UTC](https://discuss.elastic.co/t/unable-to-output-csv-file-from-logstash/49588/5 "2016-05-09T23:55:37Z")

</div>

I found the problem with my conf file. Logstash did not have permissions to write to /home/user.  
Rick

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 4:58am UTC](https://discuss.elastic.co/t/unable-to-output-csv-file-from-logstash/49588/6 "2017-07-06T04:58:27Z")

</div>


