Message_format in otput "file"

hello,
i have in input json file like this
{
"_index": "null",
"_type": "logs",
"_id": "Wh7cLM6uT0mmSLnqOrWxQw",
"_score": null,
"_source": {
"host": "null",
"@timestamp": "2015-10-12T12:09:12.338Z",
"event": "2015-10-12 15:08:50,154",
"session_id": "null",
"module": "null",
"LEVEL": "INFO",
"DT": "20151012 15:08:49.727",
"TYPE": "INREQ",
"REF": "null",
"EXTATTR": {
"THD": "http-8082-31",
"MDL": "test",
"STATE": "OK",
"MTD": "doTest"
},
"DURATION": 419,
"REQUEST_URI": "/scheme/list",
"RESPONSE_CODE": "200",
"RESPONSE_BODY": "<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"yes\"?><c id="0000"/>",
"RESPONSE_HEADERS": {
"Access-Control-Allow-Origin": "*",
"Access-Control-Allow-Methods": "POST, GET, OPTIONS, DELETE",
"Access-Control-Allow-Credentials": "true",
"Access-Control-Allow-Headers": "Origin, X-Requested-With, Content-Type, Accept, sessionID",
"reqID": "null"
},
"REQUEST_HEADERS": {
"x-real-ip": "127.0.0.1",
"host": "localhost",
"connection": "close",
"content-type": "application/xml",
"x-forwarded-for": "127.0.0.1",
"sessionid": "6475ggfdtrhhjdiurt076jg9",
"ssl": "YES"
},
"REQUEST_TYPE": "GET"
},
}
in output i need to get something like this :

I have this line is formed by "message_format"

T0]|[null]|[/scheme/list]|[20151012 15:08:49.727]|[419]|[%{STATE}]|[200]|[%{MTD}]|[]|[]
|[]|[]|[%{x-real-ip}]|[null]|[]|[]|[]|[]|[]|[GET

with elements of the first level there is no problem, but when I try to substitute the embedded value ("EXTATTR": {"THD": "http-8082-31", "MDL": "test", "STATE": "OK", "MTD ":" doTest "}), AutoCorrect regular expression to the value occurs.

I have tried different ways to substitute the regular season, but nothing happens. Can someone tell me the solution?

I'm not really sure what you're asking, but it seems you have problems with subfields. Use the [field][subfield] notation for that, e.g. like this:

message_format = "... %{[EXTATTR][MTD]} ..."

Documented here: https://www.elastic.co/guide/en/logstash/current/event-dependent-configuration.html#logstash-config-field-references

GREAT!!! that's what it was I needed !!! thank

Hello Magnus!
i have another question about "message_format" parameter in Output file. My config is

if [TYPE] == "OUTREQ" and ("T1" in [EXTATTR][L]){
file {
path => "/tmp/iq.out"
message_format => 'T1]|[%{session_id}]|[%{[EXTATTR][INM]}]|[%{DT_iq_format}]|[%{DURATION}]|[%{[EXTATTR][EST]}]|[%{RESPONSE_CODE}]|[%{[EXTATTR][MDL]}]|[%{[EXTATTR][MTD]}]|[%{REQUEST_URI}]|[]|[]|[%{[EXTATTR][CIP]}]|[%{[EXTATTR][SNM]}]|[]|[]|[]|[]|[]|[%{[EXTATTR][ORT]}'
}
}
But in log i have something dynamic field, I mean that some of the values listed in the output may be missing in the log. In this case, the conclusion is substituted the name of a regular expression.
as an example:
instead of
T1]|[kas01_k1.99b29f5acee000]|[searchPS]|[2016.03.02 10:47:10.183]|[5]|[OK]|[200]|[ex-furby]|[execute]|[http/127.0.0.1/furby/trunk/session/check]|[]|[]|[127.0.0.1, 127.0.0.1]|[kas01_k1]|[]|[]|[]|[]|[]|[WEBT0]|[kas01_k1.f42538a7587af9]|[searchPS]|[2016.03.02 10:47:09.590]|[7]|[OK]|[200]|[cashier]|[]|[]|[]|[]|[127.0.0.1, 127.0.0.1]|[kas01_k1]|[]|[]|[]|[2369832]|[]|[
i have
T1]|[kas01_k1.949b2641ab6da3]|[selectPA]|[2016.03.02 10:47:10.076]|[0]|[OK]|[%{RESPONSE_CODE}]|[ex-errors]|[getMsg]|[%{REQUEST_URI}]|[]|[]|[127.0.0.1, 127.0.0.1]|[kas01_k1]|[]|[]|[]|[]|[]|[LIB
because the log entry does not contain fields %{REQUEST_URI}, %{RESPONSE_CODE}
The ideal would be to field, do not have values, were substituted not, or were zero

can i do this in logstash? because describe all cases it is not possible

Then you'll have to build up that string in another way. Perhaps something like this:

filter {
  if [EXTATTR][SNM] {
    mutate {
      replace => { "message" => "%{message}|[%{[EXTATTR][SNM]}]" }
    }
  }
}

A ruby filter would also work.

I'm afraid it's not quite what I need. output string must be clearly defined. and it does not use all of the field log. I think 10 out of 30 - 40. The string must be exactly such as then loaded using the loader to SQL db

As I said, look at the ruby filter. If it gets too complicated for a ruby filter consider writing your custom filter. Logstash's stock plugins aren't very capable when it comes to string formatting.

maybe someone will be useful, I have decided as follows:
Check whether the field exists, if not then create it with an empty value
example:

if (![RESPONSE_CODE]) {
mutate {
<------>add_field => {"RESPONSE_CODE" => ""}
}
}
if (![EXTATTR][MDL]) {
mutate {
<------>add_field => {"[EXTATTR][MDL]" => ""}
}
}
After that, quietly form a message based on the regex