The Liebherr Domestic Appliances division is working closely with Microsoft as part of its digitalisation initiative. Microsoft is a strong partner and perfectly complements Liebherr’s expertise in refrigeration and freezing technologies, which will facilitate the integration of the very latest digital services in our appliances.
Imagine that you no longer have to keep track of your foods’ shelf life religiously and spend countless trips to the grocery store. How, you might ask. Well, it will soon be possible. Thanks to the latest development together with Microsoft. Introducing the Liebherr smart refrigerator that is able to recognise the items stored inside your refrigerator, determine the best conditions for food storage and voice control capabilities.
This cooperation has resulted in the launch of the new generation SmartDeviceBox, whose powerful hardware and Windows 10 IoT Core operating system upgrades appliance and equip consumers for future digital scenarios. The system is designed to be future-proof: modular units can be integrated into existing SmartDevice-compatible appliances and upgraded at any time. The open standards make it possible to adapt to future products and services. Liebherr currently offers the widest range of intelligent refrigerators on the market.
Voice module add foods to a shopping list
In the future, Liebherr Domestic Alliances will introduce intelligent food management that will help consumers with their shopping and meal planning. Stuff stored in the refrigerator will be captured by cameras and recorded/categorised using an object recognition algorithm. The contents of the refrigerator will be captured and stored items will be identified individually. The information will then be automatically added to an inventory, which will allow Liebherr customers to check the contents of their refrigerator even whilst out. The SmartDeviceBox’s voice module will also be used to add food items to a shopping list. IOS, Android and Windows platforms will all be supported.
Within the scope of this collaboration, Microsoft is supplying the necessary object recognition technology (computer vision), amongst other services. Microsoft is a leading innovator in the field of deep learning algorithms for image analysis and won the ImageNet Object Challenge in 2015. Deep learning technologies are being increasingly incorporated into Microsoft products, e.g. Microsoft Cognitive Services Computer Vision API, which can identify a high number of objects in images. The deep learning algorithms on which this technology is based are part of the open source Computational Network Toolkit (CNTK) and can be used to create unique models for new applications.
Object recognition is raring to go
Based on the deep learning algorithms in the CNTK, Microsoft data scientists have worked with Liebherr to develop a new image analysis system that enables pre-defined food items inside the refrigerator to be identified. Building on a general object recognition model that has learned from millions of images, the system can learn new types of objects, such as milk cartons, jars of gherkins and bottles of ketchup. When a new image of the contents of a refrigerator is sent to the trained model, it can identify individual objects that it has already learned. The image above shows an example of the result of this kind of evaluation.
For this specific project, Microsoft implemented Fast R-CNN in its Computational Network Toolkit. Fast R-CNN is an ultra-modern object recognition technology that won the Pascal VOC Object Detection Challenge – the greatest challenge in the field of computer vision. The new implementation enables the base algorithm to use the calculatory benefits of CNTK. Seamless integration into the GPU processors can also occur. This is the first time that these new CNTK capabilities have been implemented in practice and the results have proven extremely positive.
The current system is still a prototype yet, all the time, deep learning technologies are quickly maturing and becoming ever more powerful. Liebherr and Microsoft are working together to improve these technologies to make refrigerators truly intelligent, so that not only will they refrigerate food but they will also help to interactively manage it.
Just have a look to understand our vision of a Smart Home !!
If you have questions or other concerns please comment below or join the discussions on our Facebook!
