Chengyuan (CY) Xu

I'm a PhD candidate in Media Arts and Technology and a master's student in Computer Science at UC Santa Barbara, advised by Prof. Tobias Höllerer.

Human-Computer Interaction, Human-AI Collaboration, Computer Vision, and their application to real-world data and problems.

Email  /  Github  /  CV

profile photo
Research
Free-form Conversation with Human and Symbolic Avatars in Mixed Reality
Jiarui Zhu, Radha Kumaran, Chengyuan Xu, Tobias Höllerer
IEEE ISMAR 2023 (Oral Presentation)

We found evidence that the use of virtual or augmented realities can influence conversation content. Users chatting with avatars in virtual reality made significantly more references to the location or the space, suggesting they tended to perceive conversations as occurring in the agent’s space, whereas the physical AR environment was perhaps more perceived as the user’s space. Conversations with the human avatar improve user recall of the conversation, even though there is no evidence of increased information extracted during the conversation.

Open Baby Monitor
Open source, 2023. Short demo video.

My passion project as a new parent! Rather than buying more gear, why not repurpose old tech into a baby monitor that fits my needs? It's more than just a simple monitor - it detects the baby's movements, sends alerts, tracks the baby's sleep, and performs exceptionally in low-light conditions...

Comparing Zealous and Restrained AI Recommendations in a Real-World Human-AI Collaboration Task
Chengyuan Xu, Kuo-Chin Lien, Tobias Höllerer
ACM CHI 2023 (Oral Presentation)
project page

Careful exploitation of the tradeoffs in AI precision and recall can harness the complementary strengths in the human-AI collaboration to significantly improve team performance. Naively pairing humans with an AI system designed for autonomous settings could potentially have a negative training effect on the users.

Interactive Segmentation and Visualization for Tiny Objects in Multi-megapixel Images
Chengyuan Xu, Boning Dong, Noah Stier, Curtis McCully, D. Andrew Howell, Pradeep Sen, Tobias Höllerer
CVPR 2022, demo track.
paper / poster / arXiv / code

An open-source software toolkit for identifying, inspecting, and editing tiny objects in multi-megapixel HDR images. These tools offer streamlined workflows for analyzing scientific images across many disciplines, such as astronomy, remote sensing, and biomedicine.

Cosmic-ConNN: A Cosmic Ray Detection Deep Learning Framework, Dataset, and Toolbox
Chengyuan Xu, Curtis McCully, Boning Dong, D. Andrew Howell, Pradeep Sen
The Astrophysical Journal
240th Meeting of the American Astronomical Society (Oral Presentation)
paper / code / dataset

Cosmic-CoNN is a generic deep-learning cosmic ray (CR) detector deployed at Las Cumbres Observatory's 24 telescopes around the world. We built a large and diverse ground-based CR dataset and proposed a novel loss function and a neural network optimized for telescope imaging data to train generic CR detection models. Our model achieves a high precision on Las Cumbres imaging data and maintains a consistent performance on new ground-based instruments never used for training.

The electron capture origin of supernova 2018zd
Daichi Hiramatsu, et al., including Chengyuan Xu
Nature Astronomy (Cover Story)
paper / arXiv

A worldwide team led by scientists at Las Cumbres Observatory has discovered the first convincing evidence for a new type of stellar explosion -- an electron-capture supernova. While they have been theorized for 40 years, real-world examples have been elusive. I had the pleasure to provide supplementary evidence to help rule out the presence of cosmic-ray hits at or around the progenitor site to contribute to closing the 40-year-old theoretical loop.

BOI Baltimore Trash Wheel Computer Vision Model and Dataset
Chengyuan Xu, Molly Morse, Chris Lang, Ari Olivelli, Aaron Roan, et al.
dataset and code pending release

We produced a new dataset and a detection model to identify 15 types of ocean-bound river wastes like plastic bottles or bags, foam fragments, and other inorganic wastes in complex trash wheel images. The project aims to support more efficient and more accurate data collection for a greater understanding of the types and sources of river waste and to ultimately turn off the tap of plastic and other solid waste pollution into the ocean.

Coherent Video Style Transfer
Chengyuan Xu, Ekta Prashnani, Pradeep Sen
demo

We propose a novel generative adversarial network (GAN) architecture to achieve spatially and temporally coherent video style transfers. Started in 2018, this work was one of the first deep learning-based methods for video style transfers.

motionLight, an interactive public installation
Chengyuan Xu
demo / code

motionLight is a playful interactive visual audio installation inspired by Jim Campbell’s low resolution artworks. It reads camera and microphone signals for lighting and motion changes to generate six modes of temporal and spatial interpolations.

Before I dived into convolutional neural networks (CNNs), I was a video journalist covering China for CNN and BBC.


Website source code from Jon Barron.