PROJECTS
Here are a few selected projects to showcase some of my work and illustrate what types of research most interest me! (Some of my primary projects currently being considered for publication are omitted for anonymity reasons.)
Current research | Low-resource MT and ASR
I am currently working in a vein of research focused on low-resource languages. I am leading a team exploring data augmentation methods for machine translation. We are comparing established and novel augmentation techniques including variations of back-translation, multilingual source training, synonym switching, and phonological and syntactic transfer from related languages. In addition, I am exploring low-resource ASR. Two research teams and I are exploring novel data augmentation in that realm, WFST-based phonologixal finetuning, and endangered language ASR for education and revitalization. Some examples of my recent work, accepted for publication at Interspeech 2022 and LoResMT Workshop 2022, are below.
Publication | Toward Neural Programming Interfaces
Recently published and presented at NeurIPS 2020 conference proceedings. Three fellow researchers and I developed a novel algorithm to control text outputs of language generation model GPT-2. Auxiliary feed-forward networks trained on hidden layer activations from the GPT-2 latent variables classify output text and coax it towards use or avoidance of a desired word or style. See full paper below.
Publication | Improved Word Representations Via Summed Target and Context Embeddings
How do we generate better numerical representations of words for language applications? Analogy evaluations suggest we can find oft-ignored semantic information in the context embeddings produced by a skip-gram neural model. View our project recently published at IEEE's SAMI 2021.
Publication | Text Classifications Learned from Language Model Hidden Layers
Turns out in some applications the deep inner representations of a large language model can be used in place of traditional text embeddings! See results from our transfer learning method involving OpenAI's GPT-2 and text classification! See our project recently published at IEEE's SAMI 2021.
.jpg)
Project | Translation voice assistant
Ever wanted a voice assistant like Siri or Alexa that can translate for you in a way that facilitates language learning? I developed a Python interface that does just that, displaying translated text, images, and linguistic information about words and phrases. See video demo with showcased features below.
Project | Statistical Correlations in Neural Language Representations
How can we visualize what neural networks "think" about? Which "neurons" (positions in activation tensors) light up when different words are fed into a large language model? What's the neural activation pattern for "love"? What about "mother"? A logistic regression analysis shows which neurons are most responsive to different words present in inputs.
Project | Low-resource Haitian-English neural translation
Haitian is a low resource language. Here I describe a neural Transformer model I designed with PyTorch and trained with a bi-text I assembled by web-scraping translated discourses from the web. Translation results are impressive! Check out my blog post "Haitian Translation Sensation Across the Nation."
Project | Structural form in word embedding space
A statistical algorithm from 2008 finds structural forms in numerical feature data. When applied to neural word embedding vectors from a 2014 embedding algorithm, we see some really cool structures! What does this tell us about what the neural embedding model is learning? Check out my blog post, "Structural Form in Embeddings for Animals, Foods, and Cities."
Repository | Applied Computational Mathematics projects
As a bachelor's student in Applied Computational Mathematics I completed several programming projects involving mathematical algorithms such as gradient descent, Fourier analysis, Markov chains, and Metropolis-Hastings. Check out some select examples of these Python projects on my personal GitHub repository!
Repository | Deep Learning projects
As a student in Dr. David Wingate's Deep Learning class at Brigham Young University, I completed projects ranging from neural cancer detection to neural style transfer, RNN and Transformer architectures for text applications, GAN, Reinforcement Learning, and other deep learning applications. Selected examples on my GitHub repository!
