My Published Paper Project
Citation:
A. Wang, Q. Huang and Y. Chen, "A Quantized Parsimonious CNN Model for Sleep Polysomnogram Data Streams," 2024 IEEE/ACM Conference on Connected Health: Applications, Systems and Engineering Technologies (CHASE), Wilmington, DE, USA, 2024, pp. 162-163, doi: 10.1109/CHASE60773.2024.00027.
Abstract:
As sleep affects many aspects of a person’s quality of life, monitoring and evaluating sleep quality is vital for professionals to optimize their patients' sleep experiences. Thus, a parsimonious 6-layer convolutional neural network (CNN) was created from the Sleep-EDF Database Expanded (Sleep EDFX), where participants’ polysomnogram signals are stored, including an electroencephalogram, electrooculogram, electromyogram, and electrocardiogram. Unlike existing deep learning models, such as the DeepSleepNet, this proposed CNN was created from only 1D convolutional layers using training, testing, and validation sets. Quantization was included to explore the possibility of on-device CNN. By varying a range of CNN parameters, especially the number of layers, learning rate, sample size, and the number of epochs, this new CNN model without quantization was built, achieving an accuracy range of 95–99%. For the quantized CNN of different bytes, the overall accuracy decreased as expected, however, as for specific sleep stages, the new quantized CNN achieved accuracy as high as 88%. The results demonstrate this CNN's potential utility for sleep studies and promising use on edge devices.
Keywords:
Convolutional neural network - quantization - Sleep-EDF Database Expanded - deep learning model - on-device learning - polysomnogram
Please email me for full access!
Google Colab Code Link
https://colab.research.google.com/drive/1AwcmxDpJM4C66P57Q0J7ZSe6FlOPrBwD?usp=sharing
My Capstone Project
Abstract:
Millions of people worldwide suffer from some type of disability that prevents them from expressing themselves or communicating with others. Thus, a parsimonious attention-based model was created to bridge the language barrier for those reliant on American Sign Language (ASL) and address the sign language recognition issue through the creation of a novel neural network-based ASL translator based on the How2Sign dataset, consisting of a parallel corpus of more than 80 hours of sign language videos. Unlike existing projects, this proposed RNN was crafted using a sequence of recurrent layers using training, testing, validation sets, and cross-validation. Model performance evaluation metrics include accuracy, sensitivity, specificity, and F1 score. By taking RGB-D images as input and using segmentation methods, the model has succeeded in tasks like image captioning and action recognition, achieving a higher accuracy range than existing methods. The results ultimately demonstrate this RNN’s potential utility for daily use, communication, and promising use on edge devices.
Keywords:
Attention-based model - American Sign Language - How2Sign Dataset - deep learning - on-device learning - RGB-D images
Please email me with your GitHub username, so I can add you!
GitHub Link
https://github.com/annawang22/AnnaWangCapstoneProject
Stanford Course: CS193p - Developing Apps for iOS
Taking Stanford’s CS193p: Developing Apps for iOS was an exciting experience where I learned how to build apps using Swift and SwiftUI. The course started with an introduction to Swift, which is easy to learn but very powerful. I built projects like a memory card game, which taught me how to make the app’s design and data work together smoothly. I also learned how to keep my code organized using something called MVVM, which makes apps easier to update and fix. The course showed me how to use tools like CoreData to save data and Xcode Previews to quickly see how my app looks as I build it. Some parts, like fixing tricky bugs, were hard, but I learned a lot from solving them. Now I feel ready to create my own app, combining my love for technology and neuroscience, and I’m excited about what’s next!
Interviews
Interview Notes/Feedback
At the Perkins School for the Blind, there are about 55 students with varying levels of vision and hearing impairments. Some students use residual vision/hearing, but none have both entirely unusable. Oftentimes, when a student is both deaf and blind, it comes with cognitive disabilities as they grow older. Therefore, at this school, there are kids who have CHARGE syndrome along with being deaf and blind or Down syndrome along with being deaf and blind, etc. Current communication methods for these students include using spoken language, ASL, and AAC (augmentative and alternative communication) devices like iPads.
Not only are there different sign languages in different countries, but there are also different sign languages for regions. There are regional accents, which means that this project might be hindered in that way. For the students who are deaf and blind, tactile signing comes in handy as they use touch to perceive signs. At the moment, there is a company known as Tatum Robotics that is creating a robot for tactile signing, offering the potential to reduce interpreter exhaustion. I have reached out to them, and I hope to learn more about their work.
I also learned about how students attend specialized non-profit schools when local districts cannot meet their needs; district funding covers costs. Within these schools, there are high staffing ratios supporting communication styles, medical, and behavioral needs. Furthermore, I asked about these students’ post-school opportunities. Chelsea mentioned how many decide to take care of family, live in adult residential programs, go to day programs or community centers, or support volunteer work. For the majority of cases, interpreters are always with them. I found that interpreters are federally funded but sometimes require significant resources. Therefore, I believe that this “app” can help this cause.
Neha, the second woman I contacted, noted that there are many subtle facial expressions and body positions in addition to signing with one’s hands. I realized that this is an area that I need to improve in order for my model to be implemented in the real world. She also emphasized to me that many in the deaf community feel self-sufficient but acknowledge the need for specialized support. She referred me to organizations like the Massachusetts Commission for the Deaf to learn more. I have reached out to them, and I am looking forward to hearing back from them!
Ultimately, I think that this project would be impactful for those who are hard of hearing or deaf and don’t struggle with severe cognitive disabilities. In many ways, this project can provide an alternative means of understanding spoken content without relying on interpreters or lip-reading, which may not be accessible to everyone. Furthermore, if the student is in a non-specialized school, this allows kids who use ASL to communicate with non-signing peers, bridging language gaps. Moreover, as this would reduce the reliance on human interpreters, it would also offer more autonomy for the hard-of-hearing or deaf person. Normally, if one goes to a specialized school, most of the teachers know how to sign. However, if one doesn’t, this project would help teachers who don’t know ASL understand them better. Additionally, I believe that this could be beneficial in the medical field. With this model, which I hope to become an app one day, the hard-of-hearing/deaf community can have the opportunity to communicate with their doctors on their own. People might say that this might not be efficient, but with the way technology is advancing, computational time will most definitely decrease. Lastly, I feel as though this project can be very beneficial in online learning. For example, if a person was signing on Zoom, my model would automatically translate it into English for the receiver. This would only streamline communication.
Presentations
If you would like to see my presentations at each conference or specific feedback I received from professors and judges more in-depth, please reach out to my email and I would be happy to share.