I'm still thinking about the different ways to make a clean and nice looking enclosure for the 4.4" display. Meanwhile I made a very simple aluminium frame, which is not so bad in my opinion.

I have put some unconnected plated holes on the PCB is order to fix it to a future mount, then some small copper plates do the job fairly well:

The frame is just an aluminium plate, it probably needs a bit more of polishing :

With these 3 AAA batteries and a 2 minutes refresh period, it lasts around a week. Some code tweaking is probably still possible on the ESP8266 side, and I'm sure I can adjust the refresh rate according to the time of the day. (do I need a 2 minutes refresh rate at 03:00 am ?)