This project is the focus of my Master's research at Oregon State University.

From my NSF Research Fellowship Proposal: The objective of my research is to develop novel adaptive movement intent decoders that leverage (1) human movement models, (2) extensive prosthetic sensor data, and (3) human feedback in order to learn from the user over time, allowing naturalistic, fine-grained and intuitive control of prosthetic limbs.

This software is comprised of a library of Deep Network-based Decoders, called MINDecoders, and the corresponding GUI, called The MINDecoding-Interface, that provides the functionality to collect EMG or EEG from real subjects, train decoders with that biological data, test those decoders either online with a real subject or offline with existing datasets, and analyze those tests using a variety of result metrics.

My research is focused on label estimation methods and online learning to enable the semi-supervised adaptation of the deep networks within these decoders. I just received my M.Sc. for my thesis entitled "Adaptive Movement Intent Decoding for Intuitive Control of Neuroprostheses". You can check out the abstract here.

For my graduate Computational Theory project, fellow student Jacob Krantz and I created a Dataset Transformer that culls episodes from embodied navigation datasets. More specifically, we developed a Python tool that uses SAT Solvers to find the largest subset of episodes in existing embodied navigation datasets such that an agent can inter-navigate between them all.

I derived the reduction to the minimum vertex cover problem (MVC) and the maximum independent set problem (MIS), and derived the MAX-SAT equations necessary for solving these problems with off-the-shelf SAT Solvers. This SAT-based method proved to be even more efficient than standard MVC solvers in common Python libraries, as we show in our final paper.

For this fun project, I used my computer vision skills to create a crossword recognizer. This recognizer uses standard computer vision techniques, such as image filtering, contour and keypoint detection, and perspective transformations, to first straighten and crop the crossword. Then the individual cells of the crossword are detected using similar techniques. Lastly, I used a pretrained deep network for optical character recognition, i.e. to detect all the hints for the crossword. All of this was implemented in Python using OpenCV.

AI agents that can interact with physical environments are often trained in perceptually rich simulations before being deployed into the real world. To mimic affordances of physical environments such as unconstrained movement and vision, simulated 3D scenes can be reconstructed from real-world scenes using sparsely- captured high-resolution RGBD images. In this work, we analyze the discrepancy of visual fidelity between camera captures in reconstructed scenes and raw RGBD images of the Matterport3D dataset, home to many embodied AI tasks such as vision-and-language navigation, PointGoal navigation, and embodied question answering. We show that a classifier can be trained to discriminate between reconstructed scene viewpoints and raw RGBD images. Further, we rank scenes by reconstruction quality using a keypoint matching algorithm. Finally, we apply a domain adaptation method, “Goggles”, to improve the visual quality of agent perception used previously on the Gibson dataset.

Superlative reasoning, which requires identifying an instance from a set with the highest or lowest degree of some attribute, is often overlooked in the analysis of deep networks, despite the increased complexity of superlative tasks as compared to non-superlative tasks. We attempt to determine whether deep networks can learn superlative tasks and whether the increased complexity results in decreased performance. We first test a simple visual superlative task by teaching a CNN to identify the longest lines or largest polygons in an image. Our simple CNN learns this task easily. We further extend the problem to semantic goal navigation (SGN), where an agent must navigate an environment following language instructions. We extend the gym-miniworld virtual environment to support language grounding and superlative instructions, and train a Gated-Attention network using PPO on both superlative and non-superlative instructions. Unfortunately, results indicate that our agent was unable to learn SGN in this environment, eliminating our ability to analyze the performance of the network on superlative tasks.

For my year-long senior design project at Gonzaga, two fellow engineers and I designed, developed, and built the Smart Helmet. The Smart Helmet is a helmet for cyclists that uses vehicle-to-everything communication (V2X) to communicate with nearby vehicles and provide collision warnings to the biker.

For this project, my two teammates and I built the hardware for the helmet, including 3 revision of PCBs, designed and wrote the core firmware in C on the ultra low power Apollo 2 Blue microcontroller, and also wrote a Java desktop Traffic Simulator which provides an advanced GUI for generating and driving cars, which we used for testing the Smart Helmet. I focused on the core firmware, which uses FreeRTOS as its real time operating system. I designed and wrote the logic and advanced collision detection on the helmet and in the Traffic Simulator, and I also helped write part of the GUI for the Simulator.

As emerging smart vehicle technologies enter our roadways, Vehicle-to-Everything (V2X) communication will become essential for improving the safety of road users. However, as such technological advances are made in cars and trucks, which have ample power and space for advanced electronics, cyclists can easily be left behind. In this study, we attempt to incorporate cyclists in the growing V2X ecosystem by developing a prototype 'Smart Helmet'. This product broadcasts positional and inertial data about the cyclist to nearby cars using the predefined Dedicated Short Range Communications (DSRC) standard, and receives similar data from nearby cars in order to provide the cyclist with haptic and audible collision warnings from the helmet. We attempt to construct a custom printed circuit board for the system so that it can be easily incorporated into a bike helmet as a consumer product. Furthermore, we develop a traffic simulator program that can be easily used to test traffic scenarios with the product. Simulation results show that even with the limited processing power available, this product can adequately detect harmful collisions. Limited usability testing shows that the haptic and audible warnings could function well in a consumer product. However, the technologies involved, such as the V2X communication and advanced GPS, are still immature, and creating effective yet unobtrusive audio warnings will require much testing. Overall, this study proves that this V2X system could greatly increase cyclist safety and possibly become a common feature on roadways in the next decade as smart vehicle technologies continue to mature.

As a Software Development Intern at Open Sky Software, Inc., I spent multiple summers upgrading their flagship inventory tracking product, Skyware Inventory.

As a full stack Java web developer, I used the Spring Framework, along with Hibernate and Thymeleaf, to completely redesign the entire application. I designed and coded an advanced transaction manager in Java and MySQL that tracks inventory through time across multiple items, users, and locations. I incorporated industry standard reporting, such as LIFO, FIFO, and AVG costing. I also developed advanced front-end Javascript to handle editable tables with custom fields, ajax autocomplete, and flexible form validation.

This version 2.0 of Skyware Inventory was just released in Spring 2021! It is very slick, especially the new mobile version (which I designed and implemented almost completely by myself).

During my last semester at Gonzaga as an undergraduate, my teammate and I built an Othello playing AI for my Artificial Intelligence elective. I wrote this in Java, using Swing for the simple interface. The program allows the user to play against the AI. At the end of the class, all the teams competed in a tournament to determine the best AI, and mine won 1st!

This AI uses best first search with alpha-beta pruning to search the state space. Each move by the AI was required to complete within ten seconds, allowing this AI to search about 8-10 steps ahead in the game. The search uses an advanced heuristic value function that uses 8 different characteristics of the board to determine the value of the board's configuration. These characteristics include the score, the number of moves available, the number of corners taken or available, and the number of stable and unstable tokens. The AI also uses Zobrist Hashing to store all calculated heuristics in a hash table to increase the efficiency of the search.

As the final project for my senior year Digital Design class at Gonzaga, a fellow student and I successfully designed and programmed two FPGA dev boards in VHDL to use high speed Ethernet communication. The communication allowed the two dev boards to turn each other's LEDs on in different patterns using their switches. For this project we used Intel's Ethernet IP in Quartus to program the Ethernet hardware onto the FPGA.

This was one of my hardest projects at Gonzaga University. We were the only team in the class to successfully commplete the project according the original specifications. Completing the project required a ton of extra hours before the deadline(most of which was spent reading the Ethernet Hardware specifications), but it was definitely worth it when we were able to show off our working design to the class.