Tianhe Ren

I am currently working at The International Digital Economy Academy (IDEA) as Computer Vision Engineer, advised by Prof. Lei Zhang . In 2021, I got my bachelor's degree from MAC Lab in Xiamen University advised by Associate Prof. Yiyi Zhou and Prof. Rongrong Ji. I'm primarily interested in researching vision foundation models, object detection and segmentation, and multi-modal learning. I'm also passionate about open-source projects in AI community. The research work and open-source projects I'm involved in have garnered almost 20.0K stars on Github.

Email  /  Google Scholar  /  Resume  /  Github  /  ZhiHu

Research

See full list at Google Scholar. (* indicates equal contribution, # indicates corresponding author). Representative papers or projects are highlighted.

An overview technical report about our Grounded-SAM project, involving its base pipeline and more applications.

LLaVA-Grounding connects Large Multimodal Model (LMM) with a grounding model to facilitate grounded visual chat.

DINOv is a visual in-context prompting framework for referring and generic segmentation tasks.

T-Rex is an object counting model that can first detect then count any objects through visual prompting.

Extending LLaVA by incorporating a large and diverse set of external tools that can be selected, composed, and activated on the fly for performing tasks.

A novel parameter-effective method for enhancing large language models' vision-language capabilities. When applied to LLaMA, our LaVIN demonstrates competitive performance in both single-modality and multi-modality tasks, with significant efficiency and reduced training costs.

A strong vision foundation model pipeline by combining Grounding-DINO and Segment-Anything-Model which can detect and segment everything with arbitrary text prompts.

A new operator named 3D-Deformable-Attention for 2D to 3D feature lifting, which can be used and boost the performance in a range of 3D detection models.

Addressed the unstable matching issue in DETR-based models caused by multi-path optimization, by introducing a simple and efficient loss design that uses position metrics to supervise the classification scores of positive examples.

A standardized and unified benchmarking tool for Transformer-based object detection, segmentation, pose estimation and other visual recognition tasks.

A simple and strong DETR-based framework for open-set detection, achieving zero-shot 52.5 AP on COCO (training without COCO data).

A novel framework for real-time panoptic segmentation task with competitive performance compared to state-of-the-art methods.

A plain, non-hierarchical Vision Transformer (ViT) backbone for diffusion models.

An efficient variant of SAM optimizer achieved by computing a sparse perturbation based on fisher information and dynamic sparse training.

A novel dynamic routing attention mechanism brings a consistent performance gain for a range of vision and language tasks.

Amazing template by Jon Barron. Big thanks!