### Author:  [@neolithik](https://discuss.elastic.co/u/neolithik)
#### Post date: [October 12, 2015, 12:30pm UTC](https://discuss.elastic.co/t/message-format-in-otput-file/32006/1 "2015-10-12T12:30:44Z")
hello,
i have in input json file like this
{
"\_index": "null",
"\_type": "logs",
"\_id": "Wh7cLM6uT0mmSLnqOrWxQw",
"\_score": null,
"\_source": {
"host": "null",
"@timestamp": "2015-10-12T12:09:12.338Z",
"event": "2015-10-12 15:08:50,154",
"session\_id": "null",
"module": "null",
"LEVEL": "INFO",
"DT": "20151012 15:08:49.727",
"TYPE": "INREQ",
"REF": "null",
"EXTATTR": {
"THD": "http-8082-31",
"MDL": "test",
"STATE": "OK",
"MTD": "doTest"
},
"DURATION": 419,
"REQUEST\_URI": "/scheme/list",
"RESPONSE\_CODE": "200",
"RESPONSE\_BODY": "\\",
"RESPONSE\_HEADERS": {
"Access-Control-Allow-Origin": "\*",
"Access-Control-Allow-Methods": "POST, GET, OPTIONS, DELETE",
"Access-Control-Allow-Credentials": "true",
"Access-Control-Allow-Headers": "Origin, X-Requested-With, Content-Type, Accept, sessionID",
"reqID": "null"
},
"REQUEST\_HEADERS": {
"x-real-ip": "127.0.0.1",
"host": "localhost",
"connection": "close",
"content-type": "application/xml",
"x-forwarded-for": "127.0.0.1",
"sessionid": "6475ggfdtrhhjdiurt076jg9",
"ssl": "YES"
},
"REQUEST\_TYPE": "GET"
},
}
in output i need to get something like this :
I have this line is formed by "message\_format"
T0]|[null]|[/scheme/list]|[20151012 15:08:49.727]|[419]|[%{STATE}]|[200]|[%{MTD}]|[]|[]
|[]|[]|[%{x-real-ip}]|[null]|[]|[]|[]|[]|[]|[GET
with elements of the first level there is no problem, but when I try to substitute the embedded value ("EXTATTR": {"THD": "http-8082-31", "MDL": "test", "STATE": "OK", "MTD ":" doTest "}), AutoCorrect regular expression to the value occurs.
I have tried different ways to substitute the regular season, but nothing happens. Can someone tell me the solution?
---
### Author:  [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [October 12, 2015, 12:34pm UTC](https://discuss.elastic.co/t/message-format-in-otput-file/32006/2 "2015-10-12T12:34:10Z")
I'm not really sure what you're asking, but it seems you have problems with subfields. Use the `[field][subfield]` notation for that, e.g. like this:
```
message_format = "... %{[EXTATTR][MTD]} ..."
```
Documented here: [https://www.elastic.co/guide/en/logstash/current/event-dependent-configuration.html#logstash-config-field-references](https://www.elastic.co/guide/en/logstash/current/event-dependent-configuration.html#logstash-config-field-references)
---
### Author:  [@neolithik](https://discuss.elastic.co/u/neolithik)
#### Post date: [October 12, 2015, 12:57pm UTC](https://discuss.elastic.co/t/message-format-in-otput-file/32006/3 "2015-10-12T12:57:52Z")
GREAT!!! that's what it was I needed !!! thank
---
### Author:  [@neolithik](https://discuss.elastic.co/u/neolithik)
#### Post date: [March 9, 2016, 12:13pm UTC](https://discuss.elastic.co/t/message-format-in-otput-file/32006/4 "2016-03-09T12:13:55Z")
Hello Magnus!
i have another question about "message\_format" parameter in Output file. My config is
if [TYPE] == "OUTREQ" and ("T1" in [EXTATTR][L]){
file {
path =\> "/tmp/iq.out"
message\_format =\> 'T1]|[%{session\_id}]|[%{[EXTATTR][INM]}]|[%{DT\_iq\_format}]|[%{DURATION}]|[%{[EXTATTR][EST]}]|[%{RESPONSE\_CODE}]|[%{[EXTATTR][MDL]}]|[%{[EXTATTR][MTD]}]|[%{REQUEST\_URI}]|[]|[]|[%{[EXTATTR][CIP]}]|[%{[EXTATTR][SNM]}]|[]|[]|[]|[]|[]|[%{[EXTATTR][ORT]}'
}
}
But in log i have something dynamic field, I mean that some of the values listed in the output may be missing in the log. In this case, the conclusion is substituted the name of a regular expression.
as an example:
instead of
T1]|[kas01\_k1.99b29f5acee000]|[searchPS]|[2016.03.02 10:47:10.183]|[5]|[OK]|[200]|[ex-furby]|[execute]|[http/127.0.0.1/furby/trunk/session/check]|[]|[]|[127.0.0.1, 127.0.0.1]|[kas01\_k1]|[]|[]|[]|[]|[]|[WEBT0]|[kas01\_k1.f42538a7587af9]|[searchPS]|[2016.03.02 10:47:09.590]|[7]|[OK]|[200]|[cashier]|[]|[]|[]|[]|[127.0.0.1, 127.0.0.1]|[kas01\_k1]|[]|[]|[]|[2369832]|[]|[
i have
T1]|[kas01\_k1.949b2641ab6da3]|[selectPA]|[2016.03.02 10:47:10.076]|[0]|[OK]|[%{RESPONSE\_CODE}]|[ex-errors]|[getMsg]|[%{REQUEST\_URI}]|[]|[]|[127.0.0.1, 127.0.0.1]|[kas01\_k1]|[]|[]|[]|[]|[]|[LIB
because the log entry does not contain fields %{REQUEST\_URI}, %{RESPONSE\_CODE}
The ideal would be to field, do not have values, were substituted not, or were zero
can i do this in logstash? because describe all cases it is not possible
---
### Author:  [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [March 9, 2016, 12:20pm UTC](https://discuss.elastic.co/t/message-format-in-otput-file/32006/5 "2016-03-09T12:20:41Z")
Then you'll have to build up that string in another way. Perhaps something like this:
```nohighlight
filter {
if [EXTATTR][SNM] {
mutate {
replace => { "message" => "%{message}|[%{[EXTATTR][SNM]}]" }
}
}
}
```
A ruby filter would also work.
---
### Author:  [@neolithik](https://discuss.elastic.co/u/neolithik)
#### Post date: [March 9, 2016, 12:25pm UTC](https://discuss.elastic.co/t/message-format-in-otput-file/32006/6 "2016-03-09T12:25:59Z")
I'm afraid it's not quite what I need. output string must be clearly defined. and it does not use all of the field log. I think 10 out of 30 - 40. The string must be exactly such as then loaded using the loader to SQL db
---
### Author:  [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [March 9, 2016, 12:50pm UTC](https://discuss.elastic.co/t/message-format-in-otput-file/32006/7 "2016-03-09T12:50:28Z")
As I said, look at the ruby filter. If it gets too complicated for a ruby filter consider writing your custom filter. Logstash's stock plugins aren't very capable when it comes to string formatting.
---
### Author:  [@neolithik](https://discuss.elastic.co/u/neolithik)
#### Post date: [March 24, 2016, 7:13am UTC](https://discuss.elastic.co/t/message-format-in-otput-file/32006/8 "2016-03-24T07:13:02Z")
maybe someone will be useful, I have decided as follows:
Check whether the field exists, if not then create it with an empty value
example:
if (![RESPONSE\_CODE]) {
mutate {
\<------\>add\_field =\> {"RESPONSE\_CODE" =\> ""}
}
}
if (![EXTATTR][MDL]) {
mutate {
\<------\>add\_field =\> {"[EXTATTR][MDL]" =\> ""}
}
}
After that, quietly form a message based on the regex
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 6, 2017, 5:05am UTC](https://discuss.elastic.co/t/message-format-in-otput-file/32006/9 "2017-07-06T05:05:35Z")