Tianhe Ren

I am currently working at The International Digital Economy Academy (IDEA) as Computer Vision Engineer, advised by Prof. Lei Zhang . In 2021, I got my bachelor's degree from MAC Lab in Xiamen University advised by associate professor Yiyi Zhou and Prof. Rongrong Ji.

Email  /  Google Scholar  /  Github  /  ZhiHu

Selected Publications

See full list at Google Scholar. (* indicates equal contribution, # indicates corresponding author)

A novel parameter-effective method for enhancing large language models' vision-language capabilities. When applied to a model named LLaMA, the resulting LaVIN demonstrates competitive performance in both single-modality and multi-modality tasks, with significant efficiency and reduced training costs.

A new operator named 3D-Deformable-Attention for 2D to 3D feature lifting, which can be used and boost the performance in a range of 3D detection models.

Addressed the unstable matching issue in DETR-based models caused by multi-path optimization, by introducing a simple and efficient loss design that uses position metrics to supervise the classification scores of positive examples.

We proposed Sparse SAM (SSAM), an efficient training scheme that improves upon the SharpnessAware Minimization (SAM) by using sparse perturbations via a binary mask, reducing computational overhead. Sparse-SAM is shown to maintain or even enhance performance while being more efficient than SAM, achieving up to 50% sparsity in perturbations.

A standardized and unified benchmarking tool for Transformer-based object detection, segmentation, pose estimation and other visual recognition tasks.

A simple and strong DETR-based framework for open-set detection, achieving zero-shot 52.5 AP on COCO (training without COCO data).

A novel framework for real-time panoptic segmentation task with competitive performance compared to state-of-the-art methods.

A plain, non-hierarchical Vision Transformer (ViT) backbone for diffusion models.

An efficient variant of SAM optimizer achieved by computing a sparse perturbation based on fisher information and dynamic sparse training.

A novel dynamic routing attention mechanism brings a consistent performance gain for a range of vision and language tasks.

Open Source Projects

* indicates project lead, # indicates directional lead

A strong vision foundation model pipeline by combining Grounding-DINO and Segment-Anything-Model which can detect and segment everything with arbitrary text prompts.

A unified and lightweight research platform for DETR-based object detection, segmentation, pose estimation and other visual recognition tasks.

A simple and efficient toolbox for the research of referring expression comprehension and segmentation, supporting large-scale pre-training and multi-task learning.

Website Template