# Not able to load text files into elastic search **URL:** \ **Category:** Logstash\ **Created:** [October 19, 2018, 7:28pm UTC](https://discuss.elastic.co/t/not-able-to-load-text-files-into-elastic-search/153220 "2018-10-19T19:28:53Z")\ **Posts on this page:** 10\ **Page:** 1
**Author:** ![ruban7483](https://avatars.discourse-cdn.com/v4/letter/r/df788c/32.png) [@ruban7483](https://discuss.elastic.co/u/ruban7483)\ **Post date:** [October 19, 2018, 7:28pm UTC](https://discuss.elastic.co/t/not-able-to-load-text-files-into-elastic-search/153220/1 "2018-10-19T19:28:53Z")
I am new to elastic search. I have a code to load text files into elastic search. The text file has two columns and its not working. But the same code is working fine for single column. Please some one help me to fix this issue Here is my code input { file { path =\> "/ruban/data/logs/sample" start\_position =\> "beginning" type =\> "md5" } } filter { csv { columns =\> ["name","age"] separator =\> " ," remove\_field =\> ["host", "message", "path"] } } output { elasticsearch { hosts =\> "[http://localhost:9200](http://localhost:9200)" index =\> "employee" } stdout { codec =\> rubydebug } } ---
**Author:** ![rijinmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rijinmp/32/24634_2.png) [@rijinmp](https://discuss.elastic.co/u/rijinmp)\ **Post date:** [October 19, 2018, 9:06pm UTC](https://discuss.elastic.co/t/not-able-to-load-text-files-into-elastic-search/153220/2 "2018-10-19T21:06:31Z")
Can you please share a single line sample text ? What is its output now in logstash ? ---
**Author:** ![ruban7483](https://avatars.discourse-cdn.com/v4/letter/r/df788c/32.png) [@ruban7483](https://discuss.elastic.co/u/ruban7483)\ **Post date:** [October 19, 2018, 10:04pm UTC](https://discuss.elastic.co/t/not-able-to-load-text-files-into-elastic-search/153220/3 "2018-10-19T22:04:05Z")
Here is the sample input { file { path =\> "/ruban/data/logs/sample" start\_position =\> "beginning" type =\> "md5" } } filter { csv { columns =\> ["name","age"] separator =\> " ," remove\_field =\> ["host", "message", "path"] } } output { elasticsearch { hosts =\> "[http://localhost:9200](http://localhost:9200)" user =\> logfileload password =\> digital index =\> "employee" } stdout { codec =\> rubydebug } } Logstash output [2018-10-19T14:42:45,678][INFO][logstash.outputs.elasticsearch] Attempting to install template {:manage\_template=\>{"template"=\>"logstash-_", "version"=\>60001, "settings"=\>{"index.refresh\_interval"=\>"5s"}, "mappings"=\>{"default"=\>{"dynamic\_templates"=\>[{"message\_field"=\>{"path\_match"=\>"message", "match\_mapping\_type"=\>"string", "mapping"=\>{"type"=\>"text", "norms"=\>false}}}, {"string\_fields"=\>{"match"=\>"_", "match\_mapping\_type"=\>"string", "mapping"=\>{"type"=\>"text", "norms"=\>false, "fields"=\>{"keyword"=\>{"type"=\>"keyword", "ignore\_above"=\>256}}}}}], "properties"=\>{"@timestamp"=\>{"type"=\>"date"}, "@version"=\>{"type"=\>"keyword"}, "geoip"=\>{"dynamic"=\>true, "properties"=\>{"ip"=\>{"type"=\>"ip"}, "location"=\>{"type"=\>"geo\_point"}, "latitude"=\>{"type"=\>"half\_float"}, "longitude"=\>{"type"=\>"half\_float"}}}}}}}} [2018-10-19T14:42:46,420][INFO][logstash.inputs.file] No sincedb\_path set, generating one based on the "path" setting {:sincedb\_path=\>"C:/Ruban/ELK/logstash-6.4.1/data/plugins/inputs/file/.sincedb\_dad80e46eb98251c2e14f6bb22f93c17", :path=\>["/ruban/data/logs/sample"]} [2018-10-19T14:42:46,486][INFO][logstash.pipeline] Pipeline started successfully {:pipeline\_id=\>"main", :thread=\>"#\"} [2018-10-19T14:42:46,616][INFO][logstash.agent] Pipelines running {:count=\>1, :running\_pipelines=\>[:main], :non\_running\_pipelines=\>[]} [2018-10-19T14:42:46,619][INFO][filewatch.observingtail] START, creating Discoverer, Watch with file and sincedb collections [2018-10-19T14:42:47,375][INFO][logstash.agent] Successfully started Logstash API endpoint {:port=\>9600} ---
**Author:** ![rijinmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rijinmp/32/24634_2.png) [@rijinmp](https://discuss.elastic.co/u/rijinmp)\ **Post date:** [October 21, 2018, 4:22pm UTC](https://discuss.elastic.co/t/not-able-to-load-text-files-into-elastic-search/153220/4 "2018-10-21T16:22:50Z")
Hi I am asking about the sample log . Can you provide the contend in "/ruban/data/logs/sample" file. Only some sample lines ---
**Author:** ![rijinmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rijinmp/32/24634_2.png) [@rijinmp](https://discuss.elastic.co/u/rijinmp)\ **Post date:** [October 21, 2018, 4:26pm UTC](https://discuss.elastic.co/t/not-able-to-load-text-files-into-elastic-search/153220/5 "2018-10-21T16:26:40Z")
You are applying CSV filter for a log/text file . I am not sure about its working . Normally CSV filter is applying to CSV files. One example: input{ ``` file { path => "/home/elastic/elk/samplelog/cpu.csv" start_position => "beginning" } } ``` filter { ``` csv { separator => "," columns => [ "table_id", "sampletime", "CPU_Utilization", "samplestdev", "samplerate", "samplemax", "tz_offset" ] ``` } ---
**Author:** ![rijinmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rijinmp/32/24634_2.png) [@rijinmp](https://discuss.elastic.co/u/rijinmp)\ **Post date:** [October 21, 2018, 5:36pm UTC](https://discuss.elastic.co/t/not-able-to-load-text-files-into-elastic-search/153220/6 "2018-10-21T17:36:34Z")
For a text/log file grok filter is suitable . Please check it [https://www.elastic.co/guide/en/logstash/current/plugins-filters-grok.html](https://www.elastic.co/guide/en/logstash/current/plugins-filters-grok.html) ---
**Author:** ![ruban7483](https://avatars.discourse-cdn.com/v4/letter/r/df788c/32.png) [@ruban7483](https://discuss.elastic.co/u/ruban7483)\ **Post date:** [October 22, 2018, 1:55am UTC](https://discuss.elastic.co/t/not-able-to-load-text-files-into-elastic-search/153220/7 "2018-10-22T01:55:55Z")
Thanks Rijin The above code is working fine for csv file, I need the same in .txt file version. ---
**Author:** ![rijinmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rijinmp/32/24634_2.png) [@rijinmp](https://discuss.elastic.co/u/rijinmp)\ **Post date:** [October 30, 2018, 9:58am UTC](https://discuss.elastic.co/t/not-able-to-load-text-files-into-elastic-search/153220/8 "2018-10-30T09:58:53Z")
If you give me the sample log ( one line) , i will try to crate a filter for you by grok ---
**Author:** ![Mustapha\_MJ](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mustapha_mj/32/37140_2.png) [@Mustapha\_MJ](https://discuss.elastic.co/u/Mustapha_MJ)\ **Post date:** [October 31, 2018, 10:17am UTC](https://discuss.elastic.co/t/not-able-to-load-text-files-into-elastic-search/153220/9 "2018-10-31T10:17:19Z")
Hi Mr @rijinmp i have the same question, i need to know the grok filter for MS SQL server ErrorLog file it is a text file, in the following is a piece of this text file: 2018-10-23 12:27:47.93 spid54 Using 'xpstar.dll' version '2014.120.2000' to execute extended stored procedure 'xp\_instance\_regread'. This is an informational message only; no user action is required. 2018-10-23 12:29:32.49 spid54 Attempting to load library 'xplog70.dll' into memory. This is an informational message only. No user action is required. 2018-10-23 12:29:32.52 spid54 Using 'xplog70.dll' version '2014.120.2000' to execute extended stored procedure 'xp\_msver'. This is an informational message only; no user action is required. 2018-10-23 13:45:21.71 Logon Error: 18456, Severity: 14, State: 7. 2018-10-23 13:45:21.71 Logon Login failed for user 'sa'. Reason: An error occurred while evaluating the password. [CLIENT:] 2018-10-23 13:46:54.70 Logon Error: 18470, Severity: 14, State: 1. 2018-10-23 13:46:54.70 Logon Login failed for user 'sa'. Reason: The account is disabled. [CLIENT:] ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [November 28, 2018, 10:31am UTC](https://discuss.elastic.co/t/not-able-to-load-text-files-into-elastic-search/153220/10 "2018-11-28T10:31:57Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.