Optical multimodal soft sensing

The biological skin enables animals to sense various stimuli. Extensive efforts have recently been made to develop smart skin-like sensors to extend the capabilities of biological skins; however, simultaneous sensing of several types of stimuli in a large area remains challenging because this requires large-scale sensor integration with numerous wire connections. We developed a simple, highly sensitive, and multimodal sensing approach that does not require the integration of multiple sensors. The proposed approach is based on an optical interference technique that can encode the information of various stimuli in a spatial pattern. In contrast to the existing approach, the proposed approach combined with a deep neural network enables us to freely select the sensing mode based on our needs. Simultaneous sensing of different physical quantities, such as the contact force, contact location, and temperature, can be achieved using a single soft material without requiring complex integration. Another unique property of the proposed approach is spatially continuous sensing with an ultrahigh resolution of a few tens of micrometers, enabling identification of the shape of the object in contact. We aim to pioneer a new science and technology for high-performance smart skin-like sensors for human-machine interfaces.