Conditional search in Lucene or Query DSL

[Eric_Barreto]

Hi guys,

I'm trying to do a search via Lucene or Query DSL that does the following:

I have a log that contains several fields, among them the sourceIp and resourceKey fields. I need to create a survey that shows me all the logs where the same sourceIP brings more than 1 resourceKey value, the problem is that I don't know where to start.

Thanks

[spinscale (Alexander Reelsen)]

hey,

can you share a sample document, and mapping, that would make it easier to come up with a proper query.

Thank you!

--Alex

[Eric_Barreto]

Thanks for answering @spinscale. Below is a sample of the log.
What I need is to bring the sourceIPs that have + than x resourceKey values.

So it would be, if 1 IP connects to 1 or more resourceKey brings the result.

@timestamp
Jun 18, 2021 @ 18:28:43.040

---

@version
1

---

_id
X5gGIXoBknzImTVfq8np

---

_index
xxxxxxxxxxxx

---

_score
** - **

---

_type
_doc

---

apiKey
xxxxxxxxxxxxx

---

application
xxxxxxxxxxxxx

---

brand
xxxxxxxxxxx

---

category
xxxxxxxxxx

---

class
xxxxxxxxxxx

---

cluster_name
team-checkout-pf

---

date
Jun 18, 2021 @ 18:28:42.000

---

environment
aws

---

file
BaseExceptionHandlerControllerAdvice.java

---

headers.content_length
114051

---

headers.content_type
application/json

---

headers.http_accept
** - **

---

headers.http_host
xxxxxxxxxxxxxxxxxx

---

headers.http_user_agent
Fluent-Bit

---

headers.http_version
HTTP/1.1

---

headers.request_method
POST

---

headers.request_path
/

---

headers.x_amzn_trace_id
xxxxxxxxxxxxx

---

headers.x_forwarded_for
xxxxxxxxxxxxx

---

headers.x_forwarded_port
443

---

headers.x_forwarded_proto
https

---

host
xxxxxxxxxxx

---

hostname
xxxxxxxxxxxxxxxx

---

httpCode
403

---

httpMethod
POST

---

kubernetes.annotations.cattle.io/timestamp
Aug 21, 2020 @ 01:35:58.000

---

kubernetes.annotations.checksum/config
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855

---

kubernetes.annotations.checksum/secret
dd043c802fd10d3a603b89e83c2e8b6c8b55b5e464be1c7303d0ac30e3016da3

---

kubernetes.annotations.kubernetes.io/psp
eks.privileged

---

kubernetes.annotations.sidecar.istio.io/inject
false

---

kubernetes.container_hash
xxxxxxxxxxxx

---

kubernetes.container_image
xxxxxxxxxxxx

---

kubernetes.container_name
customer-token

---

kubernetes.docker_id
e60d2a0c6496b01eb848104ebf7ef2ad0461629f9966219ee46599c266af2715

---

kubernetes.host
xxxxxxxxxxx

---

kubernetes.labels.app
customer-token-v1

---

kubernetes.labels.brand
xxxxxxxxx

---

kubernetes.labels.chart
deployment-1.26

---

kubernetes.labels.elbv2.k8s.aws/pod-readiness-gate-inject
enabled

---

kubernetes.labels.flow
pf

---

kubernetes.labels.heritage
Tiller

---

kubernetes.labels.name
xxxxxxxxx

---

kubernetes.labels.pod-template-hash
5bdb6769b

---

kubernetes.labels.release
xxxxxxxxxxxx

---

kubernetes.namespace_name
team-customer

---

kubernetes.pod_id
xxxxxxxxxxx

---

kubernetes.pod_name
xxxxxxxxxxx

---

line_number
113

---

log_level
ERROR

---

log_message
Nao foi possivel autenticar o cliente com o nome de usuario e senha informados. Login: garrastazuxxxx@xxxxxx.com.br

---

method
handleServCorpException

---

referer
UNKNOWN_REFERER

---

resourceKey
garrastazuxxxx@xxxxxx.com.br

---

service.name
customer_token

---

sourceIp
222.XXX.92.XXX

---

thread_name
http-nio-8080-exec-4

---

throwable.exception_class
xxxxx

---

throwable.exception_message
Nao foi possivel autenticar o cliente com o nome de usuario e senha informados. Login: garrastazuxxx@xxxxx.com.br

[spinscale (Alexander Reelsen)]

Please take some more time to properly write up a problem statement and a minimal example. There is no index provided, no mapping, and no sample document in JSON format that others could use for testing. A multi page long sample document, where only 2 columns are needed out of 20 is also not helpful to understand the full context.

Also

What I need is to bring the sourceIPs that have + than x resourceKey values.

is something that does not make too much sense to me. Coming up with a sample query (even if verbalized) plus a document that matched and also explaining when a document does not match would help a lot.

I know this require some more work up front, but otherwise getting help is a lot harder and community members will much more likely ignore your problem.

Thank you!