Get search with json request

[razs (ricardo)]

Hi, how can i get a search with json and python?
0

Been reading this Run Elasticsearch API requests | Kibana Guide [7.15] | Elastic and im having doubst, can anyone help me create a json request to run from python to do this?

this is the link that i whant to convert to json request:
https://log-viewer.dev/app/logtrail#/q="var1"%20AND%20"var2"&h=All&t=Now&i=filebeat-*&_g=()

Been playing with this but i get the html of the page not the response

import requests

url = "https://log-viewer.dev/app/logtrail#/?i=filebeat-*&_g=()&q="

payload={}
headers = {
        'text': 'flow',
        'date': '2021-10-25'
}

response = requests.request("GET", url, headers=headers, data=payload, timeout=55)

print(response.text)

Im getting

> {
>     "statusCode": 404,
>     "error": "Not Found",
>     "message": "Not Found"
> }

[warkolm (Mark Walkom)]

Welcome to our community!

It's not clear why you are querying logtrail here, and not Elasticsearch directly. Can you elaborate why you are taking this approach?

[razs (ricardo)]

its the first time im doing this, so maybe im not doing it the right way.

I have a python program and i need to get the result from the search above, i thought ib doing it thru a json request, do you advice doing this in a diferent way?

[warkolm (Mark Walkom)]

I don't know logtrail but does it have an API? If not, then you will need to query Elasticsearch directly.

[razs (ricardo)]

yes, it can be done, i have a link in my post, just having dificultys setting it up, thats why i decide to ask for some help.

Am i in the wrong section?

[warkolm (Mark Walkom)]

I am not sure what sort of logtrail experience people here have, so it's unlikely we will be unable to help with that aspect. You will need to ask the logtrail devs/community if you really want to go that route.

However, you can definitely request Elasticsearch directly with Python. eg from Python Elasticsearch Client — Elasticsearch 7.15.1 documentation and making a super basic example;

from datetime import datetime
from elasticsearch import Elasticsearch
es = Elasticsearch()

res = es.search(index="filebeat-*", query={"match_all": {}})
print("Got %d Hits:" % res['hits']['total']['value'])
for hit in res['hits']['hits']:
    print("%(timestamp)s %(author)s: %(text)s" % hit["_source"])

[razs (ricardo)]

@warkolm

from datetime import datetime
from elasticsearch import Elasticsearch
es = Elasticsearch()

res = es.search(index="filebeat-*", query={"match_all": {}})
print("Got %d Hits:" % res['hits']['total']['value'])
for hit in res['hits']['hits']:
    print("%(timestamp)s %(author)s: %(text)s" % hit["_source"])

Thank you

do i need to set user and password?

i got

elasticsearch.exceptions.ConnectionError: ConnectionError(<urllib3.connection.HTTPConnection object at 0x7fe1a2c3c940>: Failed to establish a new connection: [Errno 61] Connection refused) caused by: NewConnectionError(<urllib3.connection.HTTPConnection object at 0x7fe1a2c3c940>: Failed to establish a new connection: [Errno 61] Connection refused)

[warkolm (Mark Walkom)]

Yep, use something like;

        'http://user:secret@localhost:9200/',

[razs (ricardo)]

no need to add user or pass

es = Elasticsearch('https://log-viewer.dev/app/logtrail')
print(es) 

res = es.search(index="filebeat-*", query={"match_all": {}})
print("Got %d Hits:" % res['hits']['total']['value'])
for hit in res['hits']['hits']:
    print("%(timestamp)s %(author)s: %(text)s" % hit["_source"])

<Elasticsearch([{'host': 'log-viewer.dev', 'port': 443, 'use_ssl': True, 'url_prefix': '/app/logtrail'}])>

raise HTTP_EXCEPTIONS.get(status_code, TransportError)(
elasticsearch.exceptions.TransportError: TransportError(301, '')

Something is missing :s

[warkolm (Mark Walkom)]

The examples above are not for logtrail, they are for talking directly to Elasticsearch.

[razs (ricardo)]

do you know a way to connect to logtrail then? this is just a log i only need to get the data