Chengyuan (CY) Xu

I'm a PhD candidate in Media Arts and Technology and a master's student in Computer Science at UC Santa Barbara, advised by Prof. Tobias Höllerer.

I collaborated with astrophysicists to identity false astronomical events in their observations with SOTA deep-learning models and interactive tools. I helped marine scientists count ocean-bound plastic wastes intelligently. I experienced video annotators' tedious work and proposed human-in-the-loop AI-assisted workflows to make their job easier.

The real-world problems and data I tackled are distinctive, but they all shared a common goal: to improve human-AI collaboration with domain experts' knowledge, computer vision, and interactive tools.

Email  /  Github  /  CV

profile photo
Interactive Segmentation and Visualization for Tiny Objects in Multi-megapixel Images
Chengyuan Xu, Boning Dong, Noah Stier, Curtis McCully, D. Andrew Howell, Pradeep Sen, Tobias Höllerer
CVPR 2022, demo track
paper / poster / arXiv / code

An open-source software toolkit for identifying, inspecting, and editing tiny objects in multi-megapixel HDR images. These tools offer streamlined workflows for analyzing scientific images across many disciplines, such as astronomy, remote sensing, and biomedicine.

Cosmic-ConNN: A Cosmic Ray Detection Deep Learning Framework, Dataset, and Toolbox
Chengyuan Xu, Curtis McCully, Boning Dong, D. Andrew Howell, Pradeep Sen
The Astrophysical Journal (Accepted)
240th Meeting of the American Astronomical Society (Oral)
arXiv / code / dataset

Cosmic-CoNN is a generic deep-learning cosmic ray (CR) detector deployed at Las Cumbres Observatory's 24 telescopes around the world. We built a large and diverse ground-based CR dataset and proposed a novel loss function and a neural network optimized for telescope imaging data to train generic CR detection models. Our model achieves a high precision on Las Cumbres imaging data and maintains a consistent performance on new ground-based instruments never used for training.

The electron capture origin of supernova 2018zd
Daichi Hiramatsu, et al., including Chengyuan Xu
Nature Astronomy, Volume 5, Issue 9
cover story / paper / arXiv

A worldwide team led by scientists at Las Cumbres Observatory has discovered the first convincing evidence for a new type of stellar explosion -- an electron-capture supernova. While they have been theorized for 40 years, real-world examples have been elusive. I had the pleasure to provide supplementary evidence to help rule out the presence of cosmic-ray hits at or around the progenitor site to contribute to closing the 40-year-old theoretical loop.

BOI Baltimore Trash Wheel Computer Vision Model and Dataset
Chengyuan Xu, Molly Morse, Chris Lang, Ari Olivelli, Aaron Roan, et al.
dataset and code pending release

We produced a new dataset and a detection model to identify 15 types of ocean-bound river wastes like plastic bottles or bags, foam fragments, and other inorganic wastes in complex trash wheel images. The project aims to support more efficient and more accurate data collection for a greater understanding of the types and sources of river waste and to ultimately turn off the tap of plastic and other solid waste pollution into the ocean.

Coherent Video Style Transfer
Chengyuan Xu, Ekta Prashnani, Pradeep Sen

We propose a novel generative adversarial network (GAN) architecture to achieve spatially and temporally coherent video style transfers. Started in 2018, this work was one of the first deep learning-based methods for video style transfers.

motionLight, an interactive public installation
Chengyuan Xu
demo / code

motionLight is a playful interactive visual audio installation inspired by Jim Campbell’s low resolution artworks. It reads camera and microphone signals for lighting and motion changes to generate six modes of temporal and spatial interpolations.

Before I dived into CNNs, I was a video journalist covering China for CNN and BBC.

Website source code from Jon Barron.