Coffee Watcher solution is based on image recognition, so let’s create the heart of the solution first. You need to have at least 15 pictures of each object you are doing to teach for your model.
1. Go to Maker Portal and to navigate to the AI hub. Please pin the AI hub so that it is easier to find there in future. Click the AI models (small brain icon) to select the model you want to create.
2. Select Images and then Detect custom objects in images and press button Create custom model
3. Select the model’s domain and give name for your model on the right top corner.
I choose common objects for my domain since the other two did match in my coffee pot scenario. Click Next.
4. Select the objects you want to detect.
I recognized that I want to detect when coffee is brewing, when the pot is empty, half-full or full. After you are done, click Next.
5. Add pictures of the object.
I had pictures of Moccamaster machine from Internet but then I also went to take pictures of the actual machine in different angels and phases of the brewing. If you know your image detection object, it is best to use the actual images of the object.
After you are done, click Next.
6. Start tagging the pictures
You need to select picture by picture and draw rectangles in the picture where you see object you want to detect. Once the rectangle is there, select the object name so that it stays in the rectangle title like below picture. Sometimes I had to draw them multiple times since it was hard or there was some error with mouse.
Once you are done, click on the top right corner Done tagging button.
After you are done, click Next.
7. Check the summary and train your model.
When you are done, click Train. Then you can close the popup and click the button to return tot he model view.
8. Publish the model
You can see the status in top right corner. Training takes some minute in my case with 100 pictures. When it is trained publish the model. Now you can use the model in Power Automate or Power Apps.
Other posts in this solution
- Solution presentation “Is there and coffee left”
- Teaching custom AI model for object detection
- Power Apps Canvas to capture picture every minute
- Solution architecture and trigger errors
- Flows to use Object Detection and informing of fresh Coffee
- Adaptive card updating every minute with thumbnail picture