🚀 Intelligent Coffee Watcher App with Power Apps, Power Automate and AI Builder.
I’ve built Power Apps Canvas App that captures photos of our Turku office coffee maker every minute. These images are then sent to Power Automate, which leverages an AI model from the AI Hub. The model identifies whether the coffee pot contains coffee and informs Turku office chat whenever there are new coffee.
The solution offers also up-to-date picture in Teams channel, so you can see your selves how much there is. Updating adaptive card every minute does not create notification of new message. When there is new coffee meaning the pot is full after brewing, it sends message to chat which is active.
This solution is experimental and it is not in production since it does not fulfill Digia’s security requirements. It is not possible to leave device unlocked that has account logged in it. The safest or best way of developing this would get a WiFi camera, that takes picture every minute and sends it in email to some address.
AI assisted solution architecture
For getting the information when coffee pot is full, I create custom AI model in Power Platform AI hub and taught it to recognize it when the coffee pot is in progress, when it is empty, full or half-full.
It was very fun and learning experimentation to use different pictures or the same target combined with Power Apps and Power Automate. I had in some point the object detection control in Canvas App as well but then I did not want to put all logic in the app which is running in the mobile. So there was architectural design as well.
I encounter multiple trigger issues when calling Power Automate from Power Apps. The most difficult was that there usually was not error message or it happened only in my mobile and not in the browser or maker portal where I could debug better. At the heat of the development during breaks and other meetings I totally forgot about Monitor tool but figured out anyway the trigger problems. I tackled them with a jump flow (like jump server).
There are detailed blog posts for each phase in the project, please stay tuned for the upcoming blog posts about this topic.
- Solution presentation “Is there and coffee left”
- Teaching custom AI model for object detection
- Power Apps Canvas to capture picture every minute
- Solution architecture and trigger errors
- Flows to use Object Detection and informing of fresh Coffee
- Adaptive card updating every minute with thumbnail picture
At start it supposed to be one hour quick demo but then it turned out to be a bit more complex. When welcoming the idea last Sunday in LinkedIn it got more than 3k impressions, I thought that I will do this what ever it takes. This is maybe more important than lunch menu in Intranet 😉
