Run Logstash config file from a web app

Elastic Stack Logstash
1

Hello,

I build a web app with flask. Through a button the user gonna choose a csv file from his desktop and when he clicks to submit a LOGSTASH config file gonna run to import that file (the user previously chosen) into ELASTICSEARCH. So I got a difficulty to code that with flask, precisely how the config file takes the path of the csv file like an input? and how i be able to run the config file?

This is my try:

1- file.conf

input{
	file{
	    path => 'new_path'      #Here it should be a dynamic variable 'new_path'
                               #and it should take the path of the csv file like
                              # an input ???
      start_position => "beginning"
      sincedb_path => "NUL"
      codec => plain { charset => "CP1252" }
	}
}
filter{
	csv {
        separator => ";"
        
          
        convert => {
            "longitude" => "float"
            "latitude" => "float"
            
            
        } 
        
    }
    date { match => [ "time", "dd MMM yy HH:mm:ss" ] }

    mutate{ add_field => { "location" => "%{latitude},%{longitude}" } }
  
}

output{
	elasticsearch { 
	    action => "index"
        hosts => ["http://localhost:9200/"] 
        index => "data"
        document_type => "doc"
  }
  stdout { codec => rubydebug }
}

2- profile.html

{% extends "base.html" %}strong text
{% block content %}
    <h1 class="title"> Welcome, {{ name }}! </h1>
    <h3>You can import your data here </h3>
    <form enctype="multipart/form-data" class="" action="data" method="post">
    	<input type="file" name="csvfilename" value="">
    	<input type="submit" name="" value="submit">
    </form>
{% endblock %}

3- main.py

#That is the function that is responsible for the import of data into ES:  
@main.route('/data', methods=['GET', 'POST'])
def data():
	if request.method == 'POST':
		# Here I want to take the path of the file 
		new_path = os.path.abspath('csvfilename')
		# Here I should run the conf file ???
		 main.config.from_object('file', 'new_path')
	return render_template('data.html')

I will be grateful if anyone could help me. :slight_smile:

2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.