How to filter nested JSON Data in Logstash

Elastic Stack Logstash
1

hey all,
i am trying to get some specific fileds(filter them) from a big nested JSON Data, i have like example this JSON Data:
{
"agent": "Mozilla/5.0 (compatible; MSIE 9.0)",
"ip": "192.168.24.44",
"request": "/index.html",
"response": {"status": 200,"bytes": 52353},
"ua": {"os": "Windows 7"},
"name": ["first","secound"]
}

and i want just to see the "ip" and the "ua" fields as result from all this Data.
JSON filter help to do that just when the Data is writting in one Line(in a row), otherwise it does not help.
Can anyone please help me to solve this Questin, or to write me just the Filter.
Best regards

2

Exactly what does the source data look like? Multiline JSON strings like in your example above?

3

my source data ist like this, the problem is, when i use JSON filter, it does not do anything helpfull, cause the data is Multiline JSON strings with spaces, so even grock does not help me in this case, here some of my Source Data:
{
"filter_level" => "low",
"retweeted" => false,
"source" => "<a href="http://twitter.com" rel="nofollow">Twitter Web Client",
"possibly_sensitive" => false,
"in_reply_to_user_id_str" => nil,
"place" => nil,
"id" => 1009785371482361856,
"@version" => "1",
"token" => "",
"@timestamp" => 2018-06-21T13:09:28.000Z,
"in_reply_to_screen_name" => nil,
"created_at" => "Thu Jun 21 13:09:28 +0000 2018",
"coordinates" => nil,
"contributors" => nil,
"in_reply_to_user_id" => nil,
"id_str" => "1009785371482361856",
"retweet_count" => 0,
"reply_count" => 0,
"favorite_count" => 0,
"favorited" => false,
"is_quote_status" => false,
"in_reply_to_status_id_str" => nil,
"quote_count" => 0,
"truncated" => false,
"text" => "elasticsearch is all fun and games until None=null!="" and ["field"] != "field".\nhttps://t.co/GIQCoa6uLD\nSince 2015-07-21.",
"timestamp_ms" => "1529586568123",
"in_reply_to_status_id" => nil,
"entities" => {
"hashtags" => [],
"symbols" => [],
"user_mentions" => [],
"urls" => [
[0] {
"indices" => [
[0] 81,
[1] 104
],
"display_url" => "github.com/elastic/elasti?",
"expanded_url" => "https://github.com/elastic/elasticsearch/issues/12366",
"url" => "https://t.co/GIQCoa6uLD"
}
]
},
"lang" => "en",
"geo" => nil,
"user" => {
"friends_count" => 640,
"default_profile" => false,
"name" => "GLOBAL UUID DATABASE",
"profile_sidebar_fill_color" => "000000",
"profile_background_image_url_https" => "https://abs.twimg.com/images/themes/theme1/bg.png",
"id" => 748900936479891456,
"listed_count" => 22,
"contributors_enabled" => false,
"profile_use_background_image" => false,
"profile_image_url" => "http://pbs.twimg.com/profile_images/849003789587820548/CClfLHE7_normal.jpg",
"profile_image_url_https" => "https://pbs.twimg.com/profile_images/849003789587820548/CClfLHE7_normal.jpg",
"protected" => false,
"followers_count" => 520,
"created_at" => "Fri Jul 01 15:27:51 +0000 2016",
"url" => "https://uuid.pirate-server.com",
"id_str" => "748900936479891456",
"verified" => false,
"utc_offset" => nil,
"profile_banner_url" => "https://pbs.twimg.com/profile_banners/748900936479891456/1518164058",
"default_profile_image" => false,
"notifications" => nil,
"follow_request_sent" => nil,
"translator_type" => "none",
"profile_background_image_url" => "http://abs.twimg.com/images/themes/theme1/bg.png",
"time_zone" => nil,
"description" => "120% legitimate pentest. not purple team nor research lead. my timeline is half memes half UUIDs half frenchness half halves.",
"is_translator" => false,
"location" => "wmic process get name /format:"https://uuid.pirate-server.com/c.xsl"",
"profile_link_color" => "000000",
"profile_sidebar_border_color" => "000000",
"following" => nil,
"profile_background_color" => "000000",
"profile_background_tile" => false,
"favourites_count" => 5649,
"screen_name" => "582a1cb9",
"profile_text_color" => "000000",
"lang" => "en",
"geo_enabled" => false,
"statuses_count" => 1791
}
}

how can i get specific field from this Multiline JSON Data?
thank you in advance.

4

That's not JSON data. It looks like output from Logstash's rubydebug codec.

5

Magnus Bäck, whatever the Typ of the Data is. i want to get some specific Fields from it.
Do you have any Idea how to write a Filter for one of those two examples to get a specific Field from it??
best regards.

6

Yes, but unless it's clear exactly what the data looks like and what you want to do with it I won't be able to give the specific advice that you're asking for.

7

Please do not answer my Question with another Question.
My Question was from the Beginning clear, i gave an JSON Multiline Example, and asked to get two Fields from it. That is it. i have tried to write the json filter, but it works for JSON data ,which is written in a row(in Lines) not under each other with spaces(Multilines) like the first Example.
So Please if you can answer my First Question with the First Example what i wrote , it will be very good.

8

Okay. Good luck finding someone who wants to help you when that's your attitude.

9

if you really wanted to help , you would have answered my first Question in my first Example very easy, but from the beginn seems to be you wanted just to confuse me during asking unnecessary Questions, even though my Question above and my first example were very clear.

10

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.