Executing code after request has ben responded to

[mirko]

Is your feature request related to a problem? Please describe.

This is slightly related to #105

In a request handler I drop the layer2 connection (wifi on esp32).
That leaves the webbrowser hanging, eventually timing out.

Describe the solution you'd like

In my little world, a decorator alike after_request would solve the issue, but after sending out the response - let's name it afterafter_request.

So, a request handler could respond and complete the HTTP request (e.g. by sending 200/OK), and then disconnect from wifi.

Describe alternatives you've considered
Initially I figured the after_request-decorator would maybe do what I want. But its decorated function is being called before the HTTP client is being responded to.

Also, e.g.:

    try:
        print("OOO", "Responding to client")
        return "OK!"
    finally:
        print("OOO", "Disconnecting from wifi")
        wifi.disconnect()

does not appear to send the response before disconnecting from wifi, despite the print() statements being executed in the correct order.

[miguelgrinberg]

You can subclass Microdot and override the handle_request() method. In the override you can implement the request handling logic that you need, either by calling the parent class and then adding your own logic, or if that isn't sufficient by copying what Microdot does and adapting it to your needs.

But in any case, disconnecting from wifi while the web server is running is not something I would recommend. You can shutdown the web server and then disconnect from wifi and then you would be doing things in a graceful way.

[mirko]

Thanks a lot for the incredibly fast response!

You can subclass Microdot and override the handle_request() method.

While that sounds sound at first, I don't immediately figure, how to execute code only after one specific request was handled.

But in any case, disconnecting from wifi while the web server is running is not something I would recommend. You can shutdown the web server and then disconnect from wifi and then you would be doing things in a graceful way.

I remember and partially get this sentiment. But especially using a webserver on embedded devices, which quite often involve on-boarding processes, where the device starts in WiFi Access Point mode and you then configure it to switch to WiFi STA / client mode, there should be a defined way.

PS: Switching from AP to STA mode is the easy path, at least on ESP32, as it can operate in AP and STA mode simultaneously. It's way more difficult switching from one network to another (both STA in mode) - and providing proper feedback to the HTTP client having re- and potentially misconfigured the platform's WiFi.

[miguelgrinberg]

Can you explain why the resistance to doing a graceful shutdown, then doing what you need to do? This is the supported way to do what you want, if I understand you correctly. The process is:

• start the web server
• in your endpoint, do what you need to do, then call shutdown()
• your run() or start_server() will return gracefully.
• reconfigure your network, or anything else you need
• start the server again under the new configuration

This is the supported way.

[mirko]

This is the supported way.

If that's the case so be it - maybe an example / an informal hint in the docs would help.

My "resistance" might come from this being a very typical use case and it would help tremendously if we could - e.g. by passing a flag to run()/start_server() - just rely on the webserver software doing whatever it takes to keep listening on the address (e.g. 0.0.0.0) originally specified and the webserver was initially started with.

[miguelgrinberg]

Graceful shutdown example: https://github.com/miguelgrinberg/microdot/blob/main/examples/hello/hello.py

this being a very typical use case

Any working examples of this elsewhere you'd like to point me at? My gut feeling is that you are relying on undocumented behavior on the hardware that you are using, and not on a well established practice.

[mirko]

Any working examples of this elsewhere you'd like to point me at? My gut feeling is that you are relying on undocumented behavior on the hardware that you are using, and not on a well established practice.

By typical I meant the use-case.

I was thinking quite a bit about (my) assumptions and fact is, that the sample rate of me being familair with WiFi enabled SoCs and their underlying SDKs/OS'es is rather limited.

Thinking of e.g. Linux, If I start a service on a specific physical network interface (e.g. "eth0") and that interface is gone: a crash (and no consecutive attempts of whatever) seems the exact right thing I'd expect.

If I however start tcpdump with -i any it should just listen on any interfaces.

The same I feel like if starting a service binding to "0.0.0.0" and all physical networking is abstracted to me as the developer / user of the service's software.
Me, at that point not knowing anything about the layer2 state, its interface(s) (wifi, ethernet, both?!), I feel like I shouldn't need to care about layer2 disconnects, once I started a service á la "listening on everything".

[mirko]

Another scenario which I don't see why it should be different:

Unintentional network disconnects. WiFi signal/stability issues, Access Point the microdot running platform is connected to is going down, ethernet cable being unplugged.

I don't have any control over such events - and listening and reacting to such events by shutting down the (microdot-)service and (re-)starting it again just doesn't feel like it should be the user's responsibility.

I see the technical challenges, but from the user's point of view I feel rather confident there should be mechanisms.

Looking at MQTT libs for micropython and hence also running on the ESP32, they all provide more or less transparent ways of attempting to reconnect, if disconnect / outage was discovered, following the assumption: Once a connect was triggered, the connection should stay established or tried to be re-established by all means until explicitly called to shut down.

[mirko]

Either way, I allowed myself to create a PR - introducing a @after_completed_request-decorator: #290

I'm fine with it not being merged, but at least it's there and might help others.

[miguelgrinberg]

You claim that it is possible to disconnect an unhealthy wifi and reconnect it while the server is listening for connections. You say this is a common use case, and I need to see examples of how this is done. Have you tested this and confirmed it works? And if you have, have you tested it on other devices? Is this a documented way of handling outages or is this something you just randomly tested and found it to work?

And in particular with your proposed solution of the after request extension, what happens if there are multiple requests being handled when the network goes down? Which one of them does the reconnection work? How do you prevent race conditions between them? I have to insist that I believe you haven't thought of all the possible complications with your idea.

Thinking of e.g. Linux, If I start a service on a specific physical network interface (e.g. "eth0") and that interface is gone: a crash (and no consecutive attempts of whatever) seems the exact right thing I'd expect.

I understand that this is your expectation, but does this work this way? I don't think it does.

The same I feel like if starting a service binding to "0.0.0.0" and all physical networking is abstracted to me as the developer / user of the service's software.
Me, at that point not knowing anything about the layer2 state, its interface(s) (wifi, ethernet, both?!), I feel like I shouldn't need to care about layer2 disconnects, once I started a service á la "listening on everything".

I have no disagreement here, but Microdot does not provide networking. This is something that the networking subsystem must do for you. Microdot is a web framework, it has nothing to do with low level networking.

Looking at MQTT libs for micropython and hence also running on the ESP32, they all provide more or less transparent ways of attempting to reconnect

MQTT on MicroPython is a client, right? For a client it's a easy to do this, and in fact it is common for reconnection features to be implemented in clients. I don't believe it is the same for servers, but feel free to correct me if you think I'm wrong. A server cannot reconnect since it doesn't connect to anything, it just waits for connections. Not the same thing.

In any case, I invite you to show me a complete fully working solution of your idea if you think I'm missing something. Feel free to add it to your PR as an example. Maybe that will help me have a better picture of the solution you have in mind.

[mirko]

Fair enough and understood.

MQTT on MicroPython is a client, right?

Correct

I don't believe it is the same for servers [..] A server cannot reconnect since it doesn't connect to anything, it just waits for connections. Not the same thing.

Reviving lost/closed/reset TCP connections was not what I tried to propose by showing this analogy. This is not about #105. As a matter of fact I'm not talking about client-server connections at all, but am solely focusing on the server listening, bound to 0.0.0.0 - even/again when the layer 2 network changed.

Apart from those what I think are/were misunderstandings, I'm with you that this needs more research (from my side).

[miguelgrinberg]

but am solely focusing on the server listening, bound to 0.0.0.0 - even/again when the layer 2 network changed.

The only argument I've heard so far against stopping the server, waiting for the network to reconfigure and restart, and then start the server again is that you'd like a different solution.

I think we've discussed this in the past in other issues. If the network stack fails, then the expected outcome is that the run() or start_server() function that you used to start the server will raise an exception. If you want to attempt a recovery then you have two options:

1. wrap your server start command in a while loop that attempts a network recovery and then restarts the server.
2. use a network stack that handles hiccups transparently so that applications don't have to.

Just to make sure this is clear, a pseudocode example of the first option would be something like this:

while True:
    wifi.connect()
    try:
        app.run()
    except Exception as e:
        print('Server error:', str(e))

Is there any argument against this, except that you don't like it?