How to change delimiter with json_lines


(Henri) #1

Hi here is my conf file

input {
  elasticsearch {
	hosts => "http://localhost:9200"
	index => "sirene"
	query => '{"query": {"query_string" : {"query": "((DEPET:54 AND provider:sp_mairie) OR (DEPET:55 AND provider:sp_mairie) OR (DEPET:60 AND provider:sp_mairie))"}}}'
  }
}
output {
  csv {
	fields => ["nom", "street", "cp","nomcommune","nom_maire","siteweb","email"]
	path => "/home/2803/public_html/datas/dt_0d83a4d79454b856ab249ca6496b17b1.csv"
  }
  file {
	codec => "json_lines"
	path => "/home/da/public_html/datas/dt_0d83a4d79454b856ab249ca6496b17b1.json"  
  }
}

How can change the line demiliter in order to build a valid json file?

thanks


(Magnus Bäck) #2

How about the delimiter option, available since 2.0.4 of the json_lines plugin?


(Henri) #3

I try this

output {
  file {
	codec => "json_lines"
	delimiter => ","
	path => "/home/data-prospection/public_html/jsondata/57cae745456ab85d4ff76a83ef3f1f0e/dp_577b539512dfa520df2c0a18ffb8482f.json"
  }
}

But i have this error

08:54:27.251 [LogStash::Runner] ERROR logstash.outputs.file - Unknown setting 'delimiter' for file

Thanks for your help!


(Magnus Bäck) #4

The option is for the codec and not the file input. Do this instead:

codec => json_lines {
  delimiter => ","
}

(Henri) #5

Damned indeed you're right! Thanks

The strange thing is the delimiter also add a "," at the end of the file...


(Magnus Bäck) #6

The strange thing is the delimiter also add a "," at the end of the file...

Hmm, well, that actually makes total sense with the default delimiter. Fixing this problem would probably complicate the code quite a bit.


(Henri) #7

Yep I understand but for the output way of this plugin, this complicate a lot the production of valid json files!


(Magnus Bäck) #8

Sure, but I wouldn't call either

{...},{...}

or

{...},{...},

valid JSON files.


(Henri) #9

yes I need some brackets around to make the file valid


(system) #10

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.