InSAI

Publication

IEEE T-PAMI 2026

OneBEV++: Towards Unifying Bird’s-Eye-View Semantic Mapping with Panoramas

PDF

BMVC 2026

X²Localizer: Cross-grained Alignment for Progressive Cross-view Video Geo-localization

PDFWebsiteCode

ICML 2026

Position: Assistive Agents Need Accessibility Alignment

PDF

2026

OneBEV++: Towards Unifying Bird’s-Eye-View Semantic Mapping with Panoramas

OneBEV++: Towards Unifying Bird’s-Eye-View Semantic Mapping with Panoramas

J. Wei, Z. Teng, F. Teng, J. Zheng, R. Liu, Y. Chen, J. Hu, K. Yang, J. Zhang, and R. Stiefelhagen

IEEE T-PAMI 2026

2026

[PDF]
X²Localizer: Cross-grained Alignment for Progressive Cross-view Video Geo-localization

X²Localizer: Cross-grained Alignment for Progressive Cross-view Video Geo-localization

Z. Zeng, W. Fan, Y. Chen, JM Goo, J. Zheng†, R. Liu, K. Peng, J. Zhang†, R. Stiefelhagen, J. Boehm

BMVC 2026(Oral)🏆

2026

[PDF][Website][Code]
Position: Assistive Agents Need Accessibility Alignment

Position: Assistive Agents Need Accessibility Alignment

Jie Hu, Changyuan Yan, Yu Zheng, Ziqian Wang, Jiaming Zhang

ICML 2026(Spotlight)🏆

2026

[PDF]
More than the Sum: Panorama-Language Models for Adverse Omni-Scenes

More than the Sum: Panorama-Language Models for Adverse Omni-Scenes

W. Fan, R. Liu, J. Wei, Y. Chen, J. Zheng†, Z. Zeng, J. Zhang†, Q. Li, L. Shen, R. Stiefelhagen

CVPR 2026

2026

[PDF][Website][Code]
RHO: Robust Holistic OSM-Based Metric Cross-View Geo-Localization

RHO: Robust Holistic OSM-Based Metric Cross-View Geo-Localization

J. Zheng*†, R. Dai*, R. Liu, Z. Zeng, Y. Chen, F. Wang, K. Peng, K. Yang, J. Zhang†, R. Stiefelhagen

CVPR 2026

2026

[PDF][Website][Code]
HybriDLA: Hybrid Generation for Document Layout Analysis

HybriDLA: Hybrid Generation for Document Layout Analysis

Y. Chen, O. Moured, R. Liu, J. Zheng, K. Peng, J. Zhang*, R. Stiefelhagen

AAAI 2026(Oral)🏆

2026

[PDF][Website][Code]

2025

Situat3DChange: Situated 3D Change Understanding Dataset for Multimodal Large Language Model

Situat3DChange: Situated 3D Change Understanding Dataset for Multimodal Large Language Model

R. Liu, J. Zheng, Y. Chen, Z. Wang, K. Peng, K. Yang, J. Zhang*, M. Pollefeys, R. Stiefelhagen

NeurIPS 2025

2025

[PDF][Website][Code]
mmWalk: Towards Multi-modal Multi-view Walking Assistance

mmWalk: Towards Multi-modal Multi-view Walking Assistance

K. Ying*, R. Liu*†, C. Chen, M. Tao, H. Shi, K. Yang, J. Zhang‡, R. Stiefelhagen

NeurIPS 2025

2025

[PDF][Website][Code]
Unlocking Constraints: Source-Free Occlusion-Aware Seamless Segmentation

Unlocking Constraints: Source-Free Occlusion-Aware Seamless Segmentation

Y. Cao*, J. Zhang*, X. Zheng, H. Shi, K. Peng, H. Liu, K. Yang, H. Zhang

ICCV 2025

2025

[PDF][Code]
RefChartQA: Grounding Visual Answer on Chart Images

RefChartQA: Grounding Visual Answer on Chart Images

A. Vogel, O. Moured, Y. Chen, J. Zhang†, R. Stiefelhagen

ICDAR 2025

2025

[PDF][Website][Code]
SFDLA: Source-Free Document Layout Analysis

SFDLA: Source-Free Document Layout Analysis

S. Tewes, Y. Chen, O. Moured, J. Zhang†, and R. Stiefelhagen

ICDAR 2025(Oral)🏆

2025

[PDF][Code]
SAMBLE: Shape-Specific Point Cloud Sampling for an Optimal Trade-Off Between Local Detail and Global Uniformity

SAMBLE: Shape-Specific Point Cloud Sampling for an Optimal Trade-Off Between Local Detail and Global Uniformity

C. Wu, Y. Wan*, H. Fu*, J. Pfrommer, Z. Zhong, J. Zheng†, J. Zhang, J. Beyerer

CVPR 2025

2025

[PDF][Website][Code]
Scene-agnostic Pose Regression for Visual Localization

Scene-agnostic Pose Regression for Visual Localization

J. Zheng, R. Liu, Y. Chen, Z. Chen, K. Yang, J. Zhang†, R. Stiefelhagen

CVPR 2025

2025

[PDF][Website][Code]
OccFiner: Offboard Occupancy Refinement with Hybrid Propagation for Autonomous Driving

OccFiner: Offboard Occupancy Refinement with Hybrid Propagation for Autonomous Driving

H. Shi*, S. Wang*, J. Zhang, X. Yin, Z. Wang, Z. Zhao, G. Wang, J. Zhu, K. Yang, and K. Wang

IEEE Transactions on Intelligent Transportation Systems (T-ITS) 2025

2025

[PDF][Code]
GraphDoc: A Graph-based Document Structure Analysis

GraphDoc: A Graph-based Document Structure Analysis

Y. Chen, R. Liu, J. Zheng, D. Wen, K. Peng, J. Zhang†, R. Stiefelhagen

ICLR 2025

2025

[PDF][Website][Code]
@Bench: Benchmarking Vision-Language Models for Human-centered Assistive Technology

@Bench: Benchmarking Vision-Language Models for Human-centered Assistive Technology

X. Jiang*, J. Zheng*, R. Liu, J. Li, J. Zhang†, S. Matthiesen, R. Stiefelhagen

WACV 2025

2025

[PDF][Website][Code]

2024

OneBEV: Using One Panoramic Image for Bird's-Eye-View Semantic Mapping

OneBEV: Using One Panoramic Image for Bird's-Eye-View Semantic Mapping

J. Wei, J. Zheng, R. Liu, J. Hu, J. Zhang†, R. Stiefelhagen

ACCV 2024(Best paper finalist)🏆

2024

[PDF][Code]
Rethinking Attention Module Design for Point Cloud Analysis

Rethinking Attention Module Design for Point Cloud Analysis

C. Wu, K. Wang, Z. Zhong, H. Fu, J. Zheng, J. Zhang, J. Pfrommer, and J. Beyerer

ICPR 2024

2024

[PDF]
LF Tracy: A Unified Single-Pipeline Approach for Salient Object Detection in Light Field Cameras

LF Tracy: A Unified Single-Pipeline Approach for Salient Object Detection in Light Field Cameras

F. Teng*, J. Zhang*, J. Liu, K. Peng, X. Cheng, Z. Li, and K. Yang

ICPR 2024

2024

[PDF][Code]
OAFuser: Towards Omni-Aperture Fusion for Light Field Semantic Segmentation

OAFuser: Towards Omni-Aperture Fusion for Light Field Semantic Segmentation

F. Teng*, J. Zhang*, K. Peng, K. Yang, Y. Wang, and R. Stiefelhagen

IEEE Transactions on Artificial Intelligence (TAI) 2024

2024

[PDF][Code]
Towards Video-based Activated Muscle Group Estimation in the Wild

Towards Video-based Activated Muscle Group Estimation in the Wild

K. Peng, D. Schneider, A. Roitberg, K. Yang, J. Zhang, M. S. Sarfraz, and R. Stiefelhagen

ACMMM 2024

2024

[PDF][Code]
Skeleton-Based Human Action Recognition with Noisy Labels

Skeleton-Based Human Action Recognition with Noisy Labels

Y. Xu, K. Peng, D. Wen, R. Liu, J. Zheng, Y. Chen, J. Zhang, A. Roitberg, K. Yang, and R. Stiefelhagen

IROS 2024

2024

[PDF][Code]
Occlusion-Aware Seamless Segmentation

Occlusion-Aware Seamless Segmentation

Y. Cao*, J. Zhang*, H. Shi, K. Peng, Y. Zhang, H. Zhang, R. Stiefelhagen, K. Yang

ECCV 2024

2024

[PDF][Code]
Open Panoramic Segmentation

Open Panoramic Segmentation

J. Zheng, R. Liu, Y. Chen, K. Peng, C. Wu, K. Yang, J. Zhang†, R. Stiefelhagen

ECCV 2024

2024

[PDF][Website][Code]
Referring Atomic Video Action Recognition

Referring Atomic Video Action Recognition

K. Peng, J. Fu, K. Yang, D. Wen, Y. Chen, R. Liu, J. Zheng, J. Zhang, S. Sarfraz, R. Stiefelhagen, A. Roitberg

ECCV 2024

2024

[PDF][Code]
Comb, Prune, Distill: Towards Unified Pruning for Vision Model Compression

Comb, Prune, Distill: Towards Unified Pruning for Vision Model Compression

J. Schmitt*, R. Liu*, J. Zheng, J. Zhang†, Rainer Stiefelhagen

ITSC 2024(Oral)🏆

2024

[PDF][Code]
CoBEV: Elevating Roadside 3D Object Detection with Depth and Height Complementarity

CoBEV: Elevating Roadside 3D Object Detection with Depth and Height Complementarity

H. Shi*, C. Peng*, J. Zhang*, K. Yang, Y. Wu, H. Ni, Y, Lin, R. Stiefelhagen, K. Wang

T-IP 2024

2024

[PDF][Code]
TransKD: Transformer Knowledge Distillation for Efficient Semantic Segmentation

TransKD: Transformer Knowledge Distillation for Efficient Semantic Segmentation

R. Liu, K. Yang, A. Roitberg, J. Zhang, K. Peng, H. Liu, and R. Stiefelhagen

IEEE Transactions on Artificial Intelligence (TAI) 2024

2024

[PDF][Code]
AltChart: Enhancing VLM-based Chart Summarization Through Multi-Pretext Tasks

AltChart: Enhancing VLM-based Chart Summarization Through Multi-Pretext Tasks

O. Moured, J. Zhang†, M. S. Sarfraz, and R. Stiefelhagen

ICDAR 2024

2024

[PDF][Code]
RoDLA: Benchmarking the Robustness of Document Layout Analysis Models

RoDLA: Benchmarking the Robustness of Document Layout Analysis Models

Y. Chen, J. Zhang†, K. Peng, J. Zheng, R. Liu, P. Torr, R. Stiefelhagen

CVPR 2024

2024

[PDF][Website][Code]
Behind Every Domain There is a Shift: Adapting Distortion-aware Vision Transformers for Panoramic Semantic Segmentation

Behind Every Domain There is a Shift: Adapting Distortion-aware Vision Transformers for Panoramic Semantic Segmentation

J. Zhang, K. Yang, H. Shi, S. Reiß, K. Peng, C. Ma, H. Fu, P. Torr, K. Wang, R. Stiefelhagen

T-PAMI 2024

2024

[PDF][Code]
MateRobot: Material Recognition in Wearable Robotics for People with Visual Impairments

MateRobot: Material Recognition in Wearable Robotics for People with Visual Impairments

J. Zheng*, J. Zhang*, K. Yang, K. Peng, R. Stiefelhagen

ICRA 2024(Best paper finalist on HRI)🏆

2024

[PDF][Website][Code]
Elevating Skeleton-Based Action Recognition with Efficient Multi-Modality Self-Supervision

Elevating Skeleton-Based Action Recognition with Efficient Multi-Modality Self-Supervision

Y. Wei, K. Peng, A. Roitberg, J. Zhang, J. Zheng, R. Liu, Y. Chen, K. Yang, and R. Stiefelhagen

ICASSP 2024

2024

[PDF][Code]
Fourier Prompt Tuning for Modality-Incomplete Scene SegmentationIV

Fourier Prompt Tuning for Modality-Incomplete Scene SegmentationIV

R. Liu, J. Zhang†, K. Peng, Y. Chen, K. Cao, J. Zheng, M. S. Sarfraz, K. Yang, and R. Stiefelhagen

IEEE IV 2024

2024

[PDF][Code]
Computational Imaging for Machine Perception: Transferring Semantic Segmentation beyond Aberrations

Computational Imaging for Machine Perception: Transferring Semantic Segmentation beyond Aberrations

Q. Jiang*, H. Shi*, S. Gao, J. Zhang, K. Yang, L. Sun, H. Ni, and K. Wang

IEEE Transactions on Computational Imaging (TCI) 2024

2024

[PDF][Code]
Skeleton-Based Human Action Recognition with Noisy Labels

Skeleton-Based Human Action Recognition with Noisy Labels

K. Peng, C. Yin, J. Zheng, R. Liu, D. Schneider, J. Zhang, K. Yang, M. S. Sarfraz, R. Stiefelhagen, and A. Roitberg

AAAI 2024

2024

[PDF][Code]
360BEV: Panoramic Semantic Mapping for Indoor Bird's-Eye View

360BEV: Panoramic Semantic Mapping for Indoor Bird's-Eye View

Z. Teng*, J. Zhang*†, K. Yang, K. Peng, H. Shi, S. Reiß, K. Cao, R. Stiefelhagen

WACV 2024

2024

[PDF][Website][Code]

2023

Tightly-Coupled LiDAR-Visual SLAM Based on Geometric Features for Mobile Agents

Tightly-Coupled LiDAR-Visual SLAM Based on Geometric Features for Mobile Agents

K. Cao, R. Liu, Z. Wang, K. Peng, J. Zhang, J. Zheng, Z. Teng, K. Yang, and R. Stiefelhagen

IEEE ROBIO 2023

2023

[PDF]
Bi-Mapper: Holistic BEV Semantic Mapping for Autonomous Driving

Bi-Mapper: Holistic BEV Semantic Mapping for Autonomous Driving

S. Li, K. Yang, H. Shi, J. Zhang, J. Lin, Z. Teng, and Z. Li

IEEE Robotics and Automation Letters (RA-L) 2021

2023

[PDF][Code]
Open Scene Understanding: Grounded Situation Recognition Meets Segment Anything for Helping People with Visual Impairments

Open Scene Understanding: Grounded Situation Recognition Meets Segment Anything for Helping People with Visual Impairments

R. Liu, J. Zhang†, K. Peng, J. Zheng, K. Cao, Y. Chen, K. Yang, and R. Stiefelhagen

ICCV Workshop on ACVR 2023

2023

[PDF][Code]
CMX: Cross-Modal Fusion for RGB-X Semantic Segmentation With Transformers

CMX: Cross-Modal Fusion for RGB-X Semantic Segmentation With Transformers

J. Zhang*, H. Liu*, K. Yang*, X. Hu, R. Liu, R. Stiefelhagen

IEEE Trans. on Intelligent Transportation Systems ( T-ITS) 2023

2023

[PDF][Code]
Line Graphics Digitization: A Step Towards Full Automation

Line Graphics Digitization: A Step Towards Full Automation

O. Moured, J. Zhang, A. Roitberg, T. Schwarz, and R. Stiefelhagen

ICDAR 2023

2023

[PDF][Code]
Delivering Arbitrary-Modal Semantic Segmentation

Delivering Arbitrary-Modal Semantic Segmentation

J. Zhang*, R. Liu*, S. Hao, K. Yang, S. Reiß, K. Peng, H. Fu, K. Wang, R. Stiefelhagen

CVPR 2023

2023

[PDF][Website][Code]
Towards Video-based Activated Muscle Group Estimation in the Wild

Towards Video-based Activated Muscle Group Estimation in the Wild

H. Shi, Y. Li, K. Yang, J. Zhang, K. Peng, A. Roitberg, Y. Ye, H. Ni, K. Wang, and R. Stiefelhagen

CVPR Workshop OmniCV 2023

2023

[PDF][Code]
Delving Deep into One-Shot Skeleton-based Action Recognition with Diverse Occlusions

Delving Deep into One-Shot Skeleton-based Action Recognition with Diverse Occlusions

K. Peng, A. Roitberg, K. Yang, J. Zhang, and R. Stiefelhagen

IEEE Transactions on Multimedia (TMM) 2023

2023

[PDF][Code]
Trans4Map: Revisiting Holistic Bird's-Eye-View Mapping from Egocentric Images to Allocentric Semantics with Vision Transformers

Trans4Map: Revisiting Holistic Bird's-Eye-View Mapping from Egocentric Images to Allocentric Semantics with Vision Transformers

C. Chen, J. Zhang†, K. Yang, K. Peng, R. Stiefelhagen

WACV 2023

2023

[PDF][Code]

2022

MatchFormer: Interleaving Attention in Transformers for Feature Matching

MatchFormer: Interleaving Attention in Transformers for Feature Matching

Q. Wang*, J. Zhang*, K. Yang, K. Peng, R. Stiefelhagen

ACCV 2022

2022

[PDF][Code]
Multi-modal Depression Estimation based on Sub-attentional Fusion

Multi-modal Depression Estimation based on Sub-attentional Fusion

P.-C. Wei, K. Peng, A. Roitberg, K. Yang, J. Zhang, and R. Stiefelhagen

ECCV Workshop ACVR 2022

2022

[PDF][Code]
TransDARC: Transformer-based Driver Activity Recognition with Latent Space Feature Calibration

TransDARC: Transformer-based Driver Activity Recognition with Latent Space Feature Calibration

K. Peng, A. Roitberg, K. Yang, J. Zhang, and R. Stiefelhagen

IROS 2022

2022

[PDF][Code]
MASS: Multi-Attentional Semantic Segmentation of LiDAR Data for Dense Top-View Understanding

MASS: Multi-Attentional Semantic Segmentation of LiDAR Data for Dense Top-View Understanding

K. Peng, J. Fei, K. Yang, A. Roitberg, J. Zhang, F. Bieder, P. Heidenreich, C. Stiller, and R. Stiefelhagen

IEEE Trans. on Intelligent Transportation Systems ( T-ITS) 2022

2022

[PDF][Code]
Indoor Navigation Assistance for Visually Impaired People via Dynamic SLAM and Panoptic Segmentation with an RGB-D Sensor

Indoor Navigation Assistance for Visually Impaired People via Dynamic SLAM and Panoptic Segmentation with an RGB-D Sensor

W. Ou, J. Zhang, K. Peng, K. Yang, G. Jaworek, K. Müller, and R. Stiefelhagen

ICCHP 2022

2022

[PDF][Code]
Bending Reality: Distortion-aware Transformers for Adapting to Panoramic Semantic Segmentation

Bending Reality: Distortion-aware Transformers for Adapting to Panoramic Semantic Segmentation

J. Zhang, K. Yang, C. Ma, S. Reiß, K. Peng, R. Stiefelhagen

CVPR 2022

2022

[PDF][Code]
Towards Robust Semantic Segmentation of Accident Scenes via Multi-Source Mixed Sampling and Meta-Learning

Towards Robust Semantic Segmentation of Accident Scenes via Multi-Source Mixed Sampling and Meta-Learning

X. Luo*, J. Zhang*, K. Yang, A. Roitberg, K. Peng, and R. Stiefelhagen

CVPR Workshop WAD 2022

2022

[PDF][Code]
Should I take a walk? Estimating Energy Expenditure from Video Data

Should I take a walk? Estimating Energy Expenditure from Video Data

K. Peng, A. Roitberg, K. Yang, J. Zhang, and R. Stiefelhagen

CVPR Workshop CVPM 2022

2022

[PDF][Code]
Trans4Trans: Efficient Transformer for Transparent Object and Semantic Scene Segmentation in Real-World Navigation Assistance

Trans4Trans: Efficient Transformer for Transparent Object and Semantic Scene Segmentation in Real-World Navigation Assistance

J. Zhang, K. Yang, C. Ma, S. Reiß, K. Peng, R. Stiefelhagen

IEEE Trans. on Intelligent Transportation Systems ( T-ITS) 2022

2022

[PDF][Code]

2021

Flying Guide Dog: Walkable Path Discovery for the Visually Impaired Utilizing Drones and Transformer-based Semantic Segmentation

Flying Guide Dog: Walkable Path Discovery for the Visually Impaired Utilizing Drones and Transformer-based Semantic Segmentation

H. Tan, C. Chen, X. Luo, J. Zhang, C. Seibold, K. Yang, R. Stiefelhagen

IEEE ROBIO 2021

2021

[PDF][Code]
Exploring Event-Driven Dynamic Context for Accident Scene Segmentation

Exploring Event-Driven Dynamic Context for Accident Scene Segmentation

J. Zhang, K. Yang, R. Stiefelhagen

IEEE Trans. on Intelligent Transportation Systems ( T-ITS) 2021

2021

[PDF][Code]
Transfer beyond the Field of View: Dense Panoramic Semantic Segmentation via Unsupervised Domain Adaptation

Transfer beyond the Field of View: Dense Panoramic Semantic Segmentation via Unsupervised Domain Adaptation

J. Zhang, C. Ma, K. Yang, A. Roitberg, K. Peng, R. Stiefelhagen

IEEE Trans. on Intelligent Transportation Systems ( T-ITS) 2021

2021

[PDF][Code]
ISSAFE: Improving Semantic Segmentation in Accidents by Fusing Event-based Data

ISSAFE: Improving Semantic Segmentation in Accidents by Fusing Event-based Data

J. Zhang, K. Yang, R. Stiefelhagen

IROS 2021

2021

[PDF][Code]
DensePASS: Dense Panoramic Semantic Segmentation via Unsupervised Domain Adaptation with Attention-Augmented Context Exchange

DensePASS: Dense Panoramic Semantic Segmentation via Unsupervised Domain Adaptation with Attention-Augmented Context Exchange

C. Ma, J. Zhang, K. Yang, A. Roitberg, R. Stiefelhagen

ITSC 2021

2021

[PDF][Code]
HIDA: Towards Holistic Indoor Understanding for the Visually Impaired via Semantic Instance Segmentation with a Wearable Solid-State LiDAR Sensor

HIDA: Towards Holistic Indoor Understanding for the Visually Impaired via Semantic Instance Segmentation with a Wearable Solid-State LiDAR Sensor

H. Liu, R. Liu, K. Yang, J. Zhang, K. Peng, R. Stiefelhagen

ICCV Workshop on ACVR 2021

2021

[PDF][Code]
Trans4Trans: Efficient Transformer for Transparent Object Segmentation to Help Visually Impaired People Navigate in the Real World

Trans4Trans: Efficient Transformer for Transparent Object Segmentation to Help Visually Impaired People Navigate in the Real World

J. Zhang, K. Yang, A. Constantinescu, K. Peng, K. Müller, R. Stiefelhagen

ICCV Workshop on ACVR 2021

2021

[PDF][Code]
Pose2Drone: A Skeleton-Pose-based Framework for Human-Drone Interaction

Pose2Drone: A Skeleton-Pose-based Framework for Human-Drone Interaction

H. Liu, R. Liu, K. Yang, J. Zhang, K. Peng, R. Stiefelhagen

EUSIPCO 2021

2021

[PDF][Code]
Panoptic Lintention Network: Towards Efficient Navigational Perception for the Visually Impaired

Panoptic Lintention Network: Towards Efficient Navigational Perception for the Visually Impaired

W. Mao*, J. Zhang*, K. Yang, R. Stiefelhagen

IEEE RCAR 2021

2021

[PDF][Code]
Perception Framework through Real-Time Semantic Segmentation and Scene Recognition on a Wearable System for the Visually Impaired

Perception Framework through Real-Time Semantic Segmentation and Scene Recognition on a Wearable System for the Visually Impaired

Y. Zhang, H. Chen, K. Yang, J. Zhang, R. Stiefelhagen

IEEE RCAR 2021

2021

[PDF][Code]
Capturing Omni-Range Context for Omnidirectional Segmentation

Capturing Omni-Range Context for Omnidirectional Segmentation

K. Yang, J. Zhang, S. Reiß, X. Hu, R. Stiefelhagen

CVPR 2021

2021

[PDF][Code]