Vivian Zhao | UT Austin

Coding Projects

Noise Alerter Pro

An Android-based home security application that utilizes machine learning and audio signal processing to perform real-time sound analysis and categorize sounds (i.e. smoke detectors, knocking, glass breaking, doorbells, dog barking, and other noises).

Languages/Tools Used: Python, TensorFlow, Keras, Librosa, Java, Android Studio

Demonstration Video

Technical Documentation

Detecting knocks;
shows spectrogram

Detecting smoke detector;
shows FFT graph

Closeup of decibel meter and audio signal

Decibel meter (top): tracks volume in real time
Audio signal (bottom): red area means detection has been triggered

Frequency domain over time for three sets of knocks

Closeup of machine learning detection results

Right: bar heights represent likelihood of each sound; for example, 4 out of 5 predictions showed a high probability for pro-knocking
Left: length of bar represents overall likelihood of a sound

Research Project: Combining Vocal and Gesture Interactions for Robots

A research project that combines object segmentation and recognition, voice-to-text, and semantic analysis to implement the following functionalities: detect whether a user is pointing with their index finger, determine the direction of the point, record and transcribe a command spoken by the user, analyze whether they want the robot to go somewhere or identify an object, and accordingly either move in the direction the user is pointing in or output the identity of an object the user is pointing at.

Created During: UT Austin's Freshman Research Initiative

Languages/Tools Used: Python, OpenAI's Whisper, Scikit Learn, YOLOv8, MediaPipe

Presentation with Demonstration

Research Paper

Video demonstration of the "identify" command (please full-screen video and have sound on for best effect)

Research Project: Personalized Literary Machine Translation

To synthesize my passions for computer science and translation, I researched the capabilities and limitations of machine translation by creating a tool for my translation purposes. While Google Translate is quite powerful, it fails to match my translation accuracy and styles. Thus, I decided to use my own translations to fine-tune a pre-trained machine translation model. After segmenting over 2,000 sentences, I used the language pairs as training input to create a custom translation model, which was able to capture the specific terms, tenses, and sentence structures I tend to use. The BLEU score¹ increased from 18.6 (significant errors) to 35.2 (good translation quality).

Languages/Tools Used: Python, HANLP and NLTK for sentence segmentation, Google AutoML for training and evaluation

[1] a metric for evaluating machine-translated text

Example Outputs (comparison between my custom translation model and standard Google Translate)

Chinese

船只泊岸，这里的克隆人军队已不像东方大陆一般戒备森严。

Improvements:

Google Translate produces a run-on sentence; my custom model produces two grammatically correct sentences
In my translation, I chose to use the term "android" as opposed to the literal translation of "clone." The custom model picked up on this
I use a casual narration style, so I tend to use apostrophes. The custom model used "wasn't," whereas Google Translate used "was not"

English (custom model)

The ship moored ashore. The android army here wasn't as heavily guarded as it was on the eastern mainland.

English (Google Translate)

The ship berthed, the clone army here was not as heavily guarded as the Eastern Continent.

Chinese

整个大厅内顿时炸锅了，飞洛的子弹发生了大爆炸。

Improvements:

The Chinese sentence uses an idiomatic expression that literally means "exploded pot." My custom model correctly translated it as "in an uproar," whereas Google Translate used the literal translation of "exploded"
My custom model correctly translates the name "Feiluo"
Overall, my custom model produces more varied and interesting sentence structure

English (custom model)

In an instant, the entire hall was in an uproar, as Feiluo’s bullet caused a huge explosion.

English (Google Translate)

The entire hall suddenly exploded, and Fei Luo's bullets exploded.

Noise Alerter

A home security app that alerts the user of any loud noises in their home while they’re away, and allows them to listen to short clips of the noises as the app refreshes in real time.

Created During: PioneerHacks III

Awarded: Best Mobile App out of 100+ participants

Languages/Tools Used: Java, Android Studio, Firebase

Full Demo

Picture of app with list of sound alerts

Gem Rush

A 4D endless runner game created for the Cave Automatic Virtual Environment, a mixed-reality platform including three surrounding wall-sized screens and a moving floor. The player races through a mine cave while attacking enemies and collecting gems.

Created During: National High School Game Academy (Carnegie Mellon University Pre-College)

Role: Lead Programmer

Languages/Tools Used: C#, Unity

Start Screen

Player Tilting

Collecting Gem

Attacking Enemy Bat

Chinese Checkers AI

An AI that can play Chinese Checkers, a board game in which players race to get their pieces to the opposite corner. The program calculates the best move to take using locally optimized strategies.

Created During: Norcal Hacks

Awarded: First Place out of 100+ participants

Languages/Tools Used: Java

Full Demo

Gif of me playing Chinese Checkers against AI

Red = me, Green = AI

Disclaimer: Used open-source code for GUI

Schedulable

A productivity app that displays the user's schedule in a color-coded clock. The app also uses text-to-speech to notify the user when it is time to start their next task.

Created During: CodeHers Hack

Awarded: Best App out of 100+ participants

Languages/Tools Used: Java, Android Studio

Full Demo

Egg Run

A 2D running game where the player rolls as an egg and then flies as a bird while avoiding obstacles. Uses realistic physically-based motion and collision detection and includes background music and sound effects.

Created During: Superposition V Hackathon

Awarded: Best Game out of 300+ participants

Languages/Tools Used: Java, Android Studio

Full Demo

Added more features after the hackathon:

Gif of egg rolling, jumping, and turning into bird

T*Bert

A remake of the arcade game Q*bert with added features and changed theme. The player controls the green dinosaur and tries to turn all of the tiles yellow while avoiding enemies like the purple frog.

Created During: National High School Game Academy

Role: Programmer

Languages/Tools Used: C#, Unity

Notle

An ear training game inspired by Wordle but instead of guessing a word, the player guesses the notes in a melody that is played. Provides a fun and engaging way for musicians to hone their listening skills.

Created During: Sonoma Hacks

Languages/Tools Used: Java, Android Studio

Full Demo