Flask mqtt esp32 for dummies

Random or Off Topic
1

In this case… I’m the dummy :slight_smile:

@Jeffb

Since PM doesn’t work, I thought I’d start this thread to get some help. We were straying off topic on a separate thread.

What I’m trying to do is implement a simple flask app to send configuration updates to my esp32 via mqtt. My esp32 devices send current metrics to mqtt which then get dumped to influxdb for viewing via grafana.

I’m assuming the easiest thing to do would use sqlite to store my configuration values and then use the flask app to display current settings and send the mqtt updates on change. Does that sound like the simplest method of tracking configured values?

The esp32 will then read the config changes off of mqtt, update the values (micropython) and then write them to a config.json for the case where the device gets rebooted.

I haven’t programmed anything in flask yet, but it looks like I’ll have a basic index.html using jinja that will get populated with some tables for each of the devices. Clicking Submit will look for any values that were changed and update the local database store and then push the value via mqtt.

 

2

I think you might have meant @jeffeb3, but I found it anyway.

If it were me, I would leave sql out of it. But you can store the settings wherever you like.

Here’s the steps I see:

  1. ESP32 needs to subscribe to the settings topics, and be able to read them when they change.
  2. You should be able to write to those topics in python. You can do this without flask or anything, just using mosquitto or some other mqtt library
  3. Store the settings to a file, and read them from the file in python. I wouldn’t use sql for this, I would just use a json txt file.
  4. Use flask to format a page to show the settings, make buttons, things like that from a list of settings.
  5. Make the buttons POST back to flask when the settings change. This can be a single POST when the form is filled.

So if you look at this:
http://flask.pocoo.org/docs/0.12/quickstart/#routing

You can see you can put any python into those functions, and when the browser asks for the page, that code will be called.

If you look at this:
http://flask.pocoo.org/docs/0.12/quickstart/#the-request-object

You can see that one of those routes can accept a POST request and the request object will be populated with information from the webpage. That’s how you’ll do something like get the set temperature from the webpage. The post in the webpage will fill in that information and python gets it from the POST and publishes the new temperature to mqtt, and saves it in the settings.

It’s kind of a mess (I wrote this years ago), but you might be able to see how I called a POST from a button on my thermostat:

3

Yeah. I could do that.

I’ll have multiple different devices. The flask app could either have 1 config.json with all the devices’ data, or I could even break it up so that each device config is in a different file.

Thank-you for the links to the examples. I’ll take a look at those later today.

Totally had the wrong username :slight_smile: I thought about double checking, but was being lazy… hope whoever jeffb is isn’t too upset.

4

If you need per device settings (messaging) then each device has to subscribe topic like

/devices/{device-id}/config

5

Yeah. If you start with just one device (but being aware of multiples in the future) you’ll end up with a good prototype on how to expand for many devices or structures. The first one, with some simple set interactions is going to be a challenge enough, without trying to generalize it to solve everything at once.

This. I have (or had) a few esps connected to temperature sensors, and they had mqtt namespaces like:

/evilhouse/livingroom/temperature
/evilhouse/bedroom/temperature
/evilhouse/basement/temperature

And I had a python script that was reading /evilhouse/*/temperature and publishing /evilhouse/avgTemperature for the thermostat.

You can have a similar structure to the json file, as long as you’re careful about the structure (lists don’t translate well). So the json might be:

[scode]
{
thermostat: {
enable: true,
setTemp: 70.0,
},
sprinlers: {
zone1: true,
zone2: true,
zone3: false,
},
coffee: {
ontime: 0700,
notify: true,
},
}
[/scode]

which would turn into the mqtt:
[scode]
/house/thermostat/enable true
/house/thermostat/setTemp 70.0
/house/sprinklers/zone1 true
…
[/scode]

6

Ok. It sounds like I’m on the right page so far.

So far I have

metrics/temperature/air

metrics/temperature/tank

metrics/relay/heater

metrics/relay/fan

metrics/relay/chiller


I’m working on the following:

config/temperature/heater_on

config/temperature/fan_on

config/temperature/chiller_on

config/temperature/delta_temp

config/temperature/tank_calibrate

config/temperature/air_calibrate


This is just for the temperature controller. Once I have that device working, I’ll be adding more devices. Some of the other devices won’t even have configurable values as they’re more of a ‘if this then that’ type of an operation.

There will be some relays that I want to control from the app. I figured those will look like this:

control/relay/X/value : where value will be on/off and X will be 0-<number of relays>
Each relay box will have 4 discreet outlets on it. This should allow me to easily add more outlets in the future by just increasing X in flask and adding a second box.

One thing I recently read is you don’t need the leading / on your MQTT subscriptions. Adding a leading / basically adds a 0/ to the front.

7

It’s a linux thing for me. Paths that don’t start with ‘/’ are relative in my head, and I can’t convince myself otherwise :).

That all looks good. If you get stuck, throw it on github and I’ll take a look.

8

Made some good progress.

I now have flask running in a container. I have a basic index page created that will list out all the json files on the filesystem. Clicking on a json file will take you to the edit page for that file. You can then make changes to the values on the page. Clicking submit will re-render the edit page, update the config file with the new values, and print a list of values that changed.

Next is to take the changed values and push them to mqtt.

My dictionary/list manipulation capabilities aren’t that great, so it took me quite some time just to get this far.

I’m sure there’s a cleaner way to do this, but here’s the code I’m using for the file manipulation:

@app.route(’/edit/’, methods=(‘GET’, ‘POST’))
def edit_page(config_name):
“”" opens a config file and provides document for editing it “”“app.logger.info(“Test Print”)with open(config_name) as file:
data = json.load(file)if request.method == ‘POST’:
app.logger.info(“Got Post Data”)form_data = request.form
changed_data = {}for key in form_data.keys():
for value in form_data.getlist(key):
debug_print = “%s : %s” % (key, value)
app.logger.info(debug_print)
app.logger.info(data[key])if float(data[key]) != float(value):
app.logger.info(”%s is different", key)
changed_data[key] = value
data[key] = valuewith open(config_name, ‘w’) as file:
json.dump(data, file)return render_template(‘edit.html’, data=data, config_name=config_name, changed_data=changed_data)

9

forget it… the code will just have to stay ugly.

10

It’s a pain. I wrote about the scode tag in updates, IIRC. But you basically wrap it in left square bracket scode right square bracket and then close with left square bracket slash scode right square bracket. It’s really hard to describe since it aggressively uses the code if I just write it here :).

Anyways, depending on what’s in form_data, you can probably do:

[scode lang=“python”]
for (key, value) in form_data.items():
# do something here

[/scode]

Also, I can’t tell, but hopefully you aren’t writing to the file you already have open.

11

heh. Believe it or not… I have email alerts turned on. The email shows everything in plain text.

so

[scode lang=“python”]
with open(file) as file:
do_something
[/scode]

if you open a file using the ‘with’ function, then it automagically closes it after it’s through with ‘do_something’.

This does mean that with the way I wrote my edit function, if values are being changed I’m opening/closing the file twice.

BTW… are you running your code under gunicorn or something? Right now I’m just running the straight python file, but I’m not so sure that’s very smart.

12

Right, but I can’t tell if you’re closing the first open first. Because there is no scope in the text.

Meh, I run it on pis scattered around my house.

13

Made some great progress this morning.

flask is pushing the changes to MQTT and the device is grabbing the change, implementing it, and then saving a new local config file.

I’ve also setup some more data going to influxdb and changed the way my series were being created to make them more dynamic and easier for the python to decipher.

My grafana now shows the current temperatures, all the states of the relays, and I added a table that shows the most recent values set for the settings, so I have a full loop I can compare to what flask says should be configured on the devices.

Next up will be designing a housing for the electronics and a PCB for some of the hardware.

14

Quick work!