Geometric Consistency-Based Self-Supervised Neural Network: A Novel Deep Learning Framework for 3D Human Shape and Motion Reconstruction

Overview

3D human motion reconstruction from a monocular video is one of the most attractive yet challenging research fields. It has the potential to enable 3D broadcasting, advance virtual and augmented reality, conduct sport analysis, deliver telepresence, etc. Existing machine learning methods for 3D reconstruction require a large number of hard-to-obtain training pairs, e.g., human images/videos and their corresponding 3D human models, and often suffer from performance degradation in practice due to appearance variations between the training and testing data. Therefore, I propose a novel geometric consistency-based self-supervised neural network (GC- SSN) for 3D human shape and motion reconstruction from a monocular video. In GC-SSN, the representation of a moving human is modeled with a geometric representation based on joints and silhouettes extracted from each frame of the video, thus avoiding the instability of appearance-based representations and constraints. During training, the joints and silhouettes of the reconstructed 3D human model are automatically extracted, rendered, and fed back to the reconstruction network to form a complete cycle. By enforcing the reconstructed 3D human model to align consistently with the extracted joints and silhouettes constraints from the input and output geometric representations in both the forward and backward directions, the generator, consisting of a feature encoder and a regressor, in GC-SSN can build the 3D human model with a high accuracy. The GC-SSN is self-supervised with automatically extracted joints and silhouettes without any manual annotations or ground truth 3D human shapes. It significantly improves the domain adaption and outperforms other state-of-the-art algorithms. Vaginal Microbiota in Recurrent, Remission, and Refractory Patients Diagnosed with Bacterial Vaginosis Mounika Katta Northville High School, Northville, MI Mentor, Dr. Robert Akins Bacterial Vaginosis, or BV, is one of the most common vaginal infections in women. It affects about 30-60% of women worldwide. In most patients, it is caused by a shift from Lactobacillus to polymicrobial flora, but the actual cause of this shift is unknown. Our hypothesis is that the abundance of specific bacteria in BV patients will determine whether treatment with oral metronidazole will be effective. In this project, BV patients treated with metronidazole were divided into three outcome groups: refractory (no recovery), recurrent (transient recovery), and remission (long-term recovery). We collected vaginal samples before and after treatment, and sequenced bacteria to determine whether compositional changes were linked to clinical outcome. We used R, Mega, and Microbiomeanalyst to analyze and graph our data. The data did not show significant differences in pre-treatment samples that could predict clinical outcome. In contrast, at post-treatment, we found certain bacteria that were significantly associated with recurrent and remission patients versus refractory patients after treatment. Future analysis of this area and data is important because it would eventually lead to clinicians being able to offer specialized treatment for BV patients.

Competition history

• JSHS 2022 Category not listed

Resources

• National JSHS abstract book ↗

Related projects

Closest projects by meaning, across every fair and year in the corpus.