Dr Shan Luo granted the EPSRC New Investigator Award

Published on

Shan Luo - ViTac: Visual-Tactile Synergy for Handling Flexible Materials

Dr Shan Luo, from the Department of Computer Science, has recently been granted the EPSRC New Investigator Award. Details of the grant can be found below:

Title: “ViTac: Visual-Tactile Synergy for Handling Flexible Materials”

Funder: EPSRC

Amount: £536k (£402k funded by the EPSRC)

Funding scheme: EPSRC New Investigator Award

Duration: 3 years

Industrial partners: Unilever and Shadow Robot Company

Details of the grant: https://gow.epsrc.ukri.org/NGBOViewGrant.aspx?GrantRef=EP/T033517/1

The grant was ranked the first in the panel:



Motivation of the project

Handling flexible materials is common in industrial, domestic and retail applications. Such tasks have been highly dependent on human labour and are still challenging for autonomous systems (i.e., robots) due to the complex dynamics of flexible materials.

Some example cases for use:

Case 1: Using robots for evaluating new fabric products in the fashion industry or the effects of new softener and conditioners products for washing clothes. For example, Unilever relies on Sensory Panels, i.e., a group of human testers, to evaluate the effects of their newly produced softeners and conditioners on fabrics after washing.

Case of 'twist' robot 

Example image: A robot evaluates a piece of fabric by twisting it using the two robot arms.


Case 2: Domestic robots being used for sorting clothes at home.

Case of 'flattening' robot

Example image: The robot flattens a piece of clothes by two robotic hands.


Case 3: Robots being used for replenishing shelves in a clothing store.


The challenges to address

The flexible materials demonstrate very complex dynamics in the above operations, e.g., fabric evaluation (slipping), clothes sorting and shelves replenishing. For example, their shape, poses, centre of mass etc. may change any time in these operations. This makes it challenging to track the states of the flexible materials and therefore makes the flexible materials hard to be handled by the robots.

The aim of the proposal

The proposal aims to develop a new visuo-tactile integration mechanism for estimating the dynamic states and properties of flexible materials while they are being manipulated by robot hands. The grasp points of the robot at the flexible materials will be tracked continuously using both tactile sensing and vision, facilitating the grasping of the flexible materials.

Impact of the project

This technique offers the potential to revolutionise the autonomous systems for handling flexible materials, allowing inclusion of their automated handling in larger automated production processes and process management systems. While the initial system to be developed in this work is for handling the textiles, the same technology would have the potential to be applied in handling other flexible materials including fragile products in the food industry, flexible objects in manufacturing and hazardous materials (e.g., wound dressings and face masks) in healthcare.