Multimodal Deep Learning for Resilient and Robust Remote Sensing Semantic Segmentation

About this opportunity

This project is part of a 4-year Dual PhD degree programme between the National Tsing Hua University (NTHU) in Taiwan and the University of Liverpool in England. Such a project offers a unique global research experience, granting 2 PhD awards from world-leading institutions, enabling international and cultural experience, providing access to large-scale national research facilities, and offering the opportunity to build a worldwide network of contacts.

Sustainable industrial growth and resilient infrastructure are central to global development, yet both depend on accurate and fine-grained environmental and land-use mapping. Conducting such mapping through traditional fieldwork is impractical due to its high cost, time requirements, and limited scalability. To address this, researchers have started to combine remote sensing imagery with artificial intelligence in a task commonly known as semantic segmentation – a powerful technique that allows large-scale, automated mapping at reduced cost and effort. Despite the success, most current approaches for remote sensing semantic segmentation still face major limitations, including: (i) reliance on single-source imagery, which hampers their ability to capture the complex interactions among environmental and industrial factors, especially under rapid urbanization and climate change, (ii) class imbalance, where some land-cover types are underrepresented, and (iii) missing data, which often occur in multimodal applications, leading to information loss.

This PhD project aims to develop a next-generation AI-based framework for fine-grained semantic segmentation that leverages multimodal remote sensing data (such as optical, hyperspectral, radar, thermal imagery, and so on) while ensuring robustness and resilience to class imbalance and missing data through the use of cutting-edge Vision-Language Foundation Models and Multimodal Large Language Models. This pioneering project will be the first to jointly address multimodal fusion, missing data, and class imbalance within a unified semantic segmentation framework for remote sensing, thus bringing such a relevant task significantly closer to the real world, where such challenges are extremely common, enabling resilient multimodal fusion under realistic data constraints.

The advances resulting from this PhD project have strong potential for impact across multiple application domains and directly contributes to UN Sustainable Development Goal (SDG) 9. Specifically, the outcomes of this project will support data-driven decision-making for infrastructure resilience, sustainable urban growth, and resource-efficient industrial development. The proposed methods will enable authorities and industry stakeholders to identify infrastructure deterioration, monitor pollution and emissions, assess construction-related impacts, track progress towards sustainable land-use practices, and so on. Finally, by advancing the state of the art in AI for remote sensing, the project will strengthen the innovation ecosystem underpinning UN SDG 9, linking digital technologies with sustainable infrastructure planning and environmental management.

Towards this, this 4-year doctoral program will benefit from a unique international partnership through the dual-degree programme between National Tsing Hua University (NTHU) and the University of Liverpool. In this context, the student will spend the first 2 years at NTHU under the supervision of Professor Shang-Hong Lai, followed by 2 years at the University of Liverpool under the supervision of Dr Keiller Nogueira. Upon successful completion of the programme, the student will be awarded two PhD degrees, one from each world-leading institution.

This opportunity is open to both home (UK) and international students. If you have any questions, please email keiller.nogueira@liverpool.ac.uk

Who is this for?

Candidates will have, or be due to obtain, a Master’s Degree or equivalent in a relevant subject

This PhD project aims to develop a next-generation semantic segmentation framework for multimodal remote sensing data, with a strong focus on robustness to class imbalance and missing data using state-of-the-art deep learning techniques. Overall, the outcomes of this project will bring semantic segmentation closer to the real world, with significant potential across a range of application domains.

This project is part of a 4-year Dual PhD degree programme between the National Tsing Hua University (NTHU) in Taiwan and the University of Liverpool in England. Such a project offers a unique global research experience, granting 2 PhD awards from world-leading institutions, enabling international and cultural experience, providing access to large-scale national research facilities, and offering the opportunity to build a worldwide network of contacts.

Sustainable industrial growth and resilient infrastructure are central to global development, yet both depend on accurate and fine-grained environmental and land-use mapping. Conducting such mapping through traditional fieldwork is impractical due to its high cost, time requirements, and limited scalability. To address this, researchers have started to combine remote sensing imagery with artificial intelligence in a task commonly known as semantic segmentation – a powerful technique that allows large-scale, automated mapping at reduced cost and effort. Despite the success, most current approaches for remote sensing semantic segmentation still face major limitations, including: (i) reliance on single-source imagery, which hampers their ability to capture the complex interactions among environmental and industrial factors, especially under rapid urbanization and climate change, (ii) class imbalance, where some land-cover types are underrepresented, and (iii) missing data, which often occur in multimodal applications, leading to information loss.

This PhD project aims to develop a next-generation AI-based framework for fine-grained semantic segmentation that leverages multimodal remote sensing data (such as optical, hyperspectral, radar, thermal imagery, and so on) while ensuring robustness and resilience to class imbalance and missing data through the use of cutting-edge Vision-Language Foundation Models and Multimodal Large Language Models. This pioneering project will be the first to jointly address multimodal fusion, missing data, and class imbalance within a unified semantic segmentation framework for remote sensing, thus bringing such a relevant task significantly closer to the real world, where such challenges are extremely common, enabling resilient multimodal fusion under realistic data constraints.

The advances resulting from this PhD project have strong potential for impact across multiple application domains and directly contributes to UN Sustainable Development Goal (SDG) 9. Specifically, the outcomes of this project will support data-driven decision-making for infrastructure resilience, sustainable urban growth, and resource-efficient industrial development. The proposed methods will enable authorities and industry stakeholders to identify infrastructure deterioration, monitor pollution and emissions, assess construction-related impacts, track progress towards sustainable land-use practices, and so on. Finally, by advancing the state of the art in AI for remote sensing, the project will strengthen the innovation ecosystem underpinning UN SDG 9, linking digital technologies with sustainable infrastructure planning and environmental management.

Towards this, this 4-year doctoral program will benefit from a unique international partnership through the dual-degree programme between National Tsing Hua University (NTHU) and the University of Liverpool. In this context, the student will spend the first 2 years at NTHU under the supervision of Professor Shang-Hong Lai, followed by 2 years at the University of Liverpool under the supervision of Dr Keiller Nogueira. Upon successful completion of the programme, the student will be awarded two PhD degrees, one from each world-leading institution.

This opportunity is open to both home (UK) and international students. If you have any questions, please email keiller.nogueira@liverpool.ac.uk

Funding your PhD

This project is a part of a 4-year dual PhD programme between National Tsing Hua University (NTHU) in Taiwan and the University of Liverpool in England. It is planned that students will spend 2 years at NTHU, followed by 2 years at the University of Liverpool.

Both the University of Liverpool and NTHU have agreed to waive the tuition fees for the duration of the project and provide a maintenance stipend to support living costs. During the 2 years based in Taiwan, students will receive TWD 15,233/month as a contribution to living costs. During the 2 years based in Liverpool, students will receive a stipend at the standard UKRI Studentship rate, for 2025-26 this is £20,780 pa and this rises with inflation each year.

This Studentship also comes with access to additional funding in the form of a Research Training Support Grant to fund consumables, conference attendance, etc.

These Studentships are available to any prospective student wishing to apply including both home and international students. A limited number of scholarships will be available to support outstanding international students.

We want all of our Staff and Students to feel that Liverpool is an inclusive and welcoming environment that actively celebrates and encourages diversity. We are committed to working with students to make all reasonable project adaptations including supporting those with caring responsibilities, disabilities or other personal circumstances. For example, if you have a disability, you may be entitled to a Disabled Students Allowance on top of your studentship to help cover the costs of any additional support that a person studying for a doctorate might need as a result. We believe everyone deserves an excellent education and encourage students from all backgrounds and personal circumstances to apply.

Contact us

Have a question about this research opportunity or studying a PhD with us? Please get in touch with us, using the contact details below, and we’ll be happy to assist you.