Machine learning approaches promise accurate results in many societal sectors, ranging from health to industrial applications. However, they require vast amounts of data, which contradicts the ever growing privacy regulations. Federated learning is a novel technique to learn across data silos while preserving privacy. In this article, we explore how federated learning can revolutionize robot picking.
Why privacy matters
Thanks to the ever-increasing technological progress, industrial logistics gets more and more digitized. Robots grasping objects self-sufficiently to support the automated intra-logistics show the interaction of different digitized technologies during the age of the fourth industrial revolution.
In picking cells, robotic arms are provided with cameras to get a visual perspective on the items in front of them. With the camera image, the robot arms automatically detect the different items and select an appropriate grasping procedure. Due to the variety of items in an industrial warehouse, this is a complicated task and large amounts of data are required to achieve reasonable results. Creating such large amounts of data is time consuming, as images need to be labeled manually and picking cells need assistance during their initial tries. With data gathered from picking cells across different organizations, the grasp prediction for picking cells could be improved – but who wants to distribute their own sensitive data?
Sensitive data and privacy internationally
Data privacy needs to be ensured for a secure distribution of sensitive data. Countries all over the world have recognized the challenges and adopted privacy and data protection laws for individuals that ensure rights of person, transparency, and security of personal data, like the European General Data Protection Regulation (GDPR) or the Canadian Personal Information Protection and Electronics Documents Act (PIPEDA) [1][2]. As privacy laws and data infrastructures differ across jurisdictions, data aggregation from different sites (across or within countries) for analysis and collaborative projects is a lengthy, bureaucratic process..
In the industrial context, competitor risk further emphasizes the need to protect the data. While cooperation with related businesses in corresponding tasks is essential and beneficial, sharing large amounts of data is not. In the context of picking cells, data can provide insights on very sensitive information like the frequency of picked items or the shape of detected objects. This would result in a scenario not unlike mexican standoffs – whoever includes newly developed objects in the data set first, would lose this benefit immediately.
Federated learning as the enabler of privacy
To enable privacy-preserving cooperation, federated learning, a new paradigm for machine learning was developed: instead of sending training data to a central machine learning server to train the model there like in traditional collaborative machine learning problems, the training is performed at many different sites where the data is created initially, like picking cells. The locally trained models are then sent to the central machine learning server resulting in the highly sensitive training data not leaving the data provider. Still, federated learning enables learning across data silos by aggregating the distributed models, ultimately enabling a highly accurate and data-driven prediction on the object detection and the consecutive grasp. A general process of federated learning is illustrated in Figure 2.
Figure 2: The five steps of the federated learning process [3]. The global model is broadcast to the selected clients. Each of the clients then trains the model locally and sends the updated model back to the origin, where it is aggregated (i.e., the average of the distributed models is taken). The aggregated model can then be broadcast to all clients in the network to start the next iteration.
The contribution of the FLAIROP project
In the context of the FLAIROP project the consortial partners aim to contribute to this problem. We develop a federated learning infrastructure which connects multiple picking cells of different organizations. Each picking cell has its unique data set and own objects it excels in detecting and predicting. With the implemented infrastructure we aim to improve the grasping success across all picking cells. Especially if a picking cell is confronted with objects only seen by other cells, the grasp should be as precise. Since the data itself is not exchanged, FLAIROP contributes to the establishment of trustful and benevolent cooperation across company borders.
References
[1] Data protection | European Commission (europa.eu)
[3] Kairouz, P., McMahan, H. B., Avent, B., Bellet, A., Bennis, M., Bhagoji, A. N., … & Zhao, S. (2019). Advances and open problems in federated learning. arXiv preprint arXiv:1912.04977.