ABSTRACT OF THE THESIS

Quantitative Analysis of Human Facial Expression: Moving Towards The Creation of a Virtual Patient

By

Sungah Lee

Master of Science in Oral Biology University of California Los Angeles, 2017

Background: Recent introduction of three dimensional facial images allows us to have access to more information than ever before, creating the potential for more accurate facial evaluation. In orthodontic diagnostics and treatment planning, facial soft tissue analysis has been broadly recognized as a critical factor leading to successful orthodontic treatment outcomes. Even though facial soft tissue is by nature dynamic data and facial expressions are the dynamic movement of these facial soft tissue, 2D static photos have been used in facial analysis in orthodontics. Our overall objective is to develop an innovative method to quantifies the dynamic movements of soft tissue in 3D during facial expressions, which could further not only orthodontics field but also other health care fields.

Methods: Dynamic system to quantify 3D facial soft tissue movement was explored through investigation into physics and mathematical modeling. 3dMD facial images of 29 participants at five different time points (T1, T2, T3, T4, T5) during smiling were collected from starting of each facial expression till the end. Smiling patterns were classified and only homogenous samples were finally included. 3D meshes were processed for vertex correspondence, and 28 landmarks were tracked. Data analyses were performed via MATLAB. Average smiling faces at five different time points were generated. Average displacement vectors between each time point were computed, producing the average smiling movement trajectories. Statistical p values of all landmarks in three-dimension were computed to show the significance level of displacement. Color-coded displacement vector p maps were generated for movement of each landmark over the 5 time points.

Results: 3D meshes of 10 participants at five different time points (T1, T2, T3, T4, T5) while smiling were finally included in our study. 28 landmarks were quantitatively tracked and analyzed. Average smiling faces at five time points, average displacement vectors between each time point, and statistical p values of all landmarks in 3D were generated. Average movement trajectory while smiling was generated. Corner of lip showed maximum displacement of 6.42 mm (p<~ 0.01) in upward and outward directions. Statistically Significant displacements were shown at most landmarks of oral regions (p<0.05) rather than landmarks of nasal, eye, or eyebrow regions between each time point.

Conclusion: This is the first study to demonstrate that dynamic 3D movements of facial expressions can be quantitatively tracked and analyzed, offering an added dimension to the diagnosis and treatment planning of patients. This new approach which can allow us to analyze patients’ facial expressions in three dimension would shift the diagnostic paradigms currently used in craniofacial analysis, that is, 2D static facial analysis, towards an ever progressive direction, that is, dynamic 3D facial expression analysis.

Introduction: The shape, size and volume of the bony nasal cavity is highly influenced by the width of the nasal floor, which is formed by the palatine bone and the palatal processes of the maxilla and can be orthodontically manipulated with the MSE (Maxillary Skeletal Expansion) technique. Current evaluation of changes in this area before and after expansion relay on cross-sectional areas and airflow measurements. However, these techniques are not able to truly point out the extend of bone changes and volume changes of this important structure.

Materials and Method: This retrospective study had a sample of 22 patients distributed in two groups. Group 1 was treated with Hyrax and consisted of 6 growing patients (3 females and 3 males, mean age 12y2mo), group 2 was treated with MSE and consisted of 16 non-growing patients (10 females and 6 males, mean age 20y7mo) who met inclusion criteria. Analysis of CBCT scans before and after expansion treatment was conducted in OnDemand3D software, with automated superimposition on the cranial base. Three reference planes were established, based on reliably identifiable anatomical landmarks, anterior nasal spine (ANS), posterior nasal spine (PNS) and nasion (N). The distance between ANS and PNS in the AP plane was divided into 4 sections, and the distance between ANS and N was divided into 5 sections, excluding the most superior one. Transverse linear measurements at the resulting cross-points between the vertical and horizontal sections resulted in a total of 32 measurement points along the lateral walls of the nasal cavity.

Results: The MSE group showed overall statistically significant larger movements of the lateral walls of the nasal cavity (p < 0.05). The bony structures followed the overall expansion pattern of the maxilla, though bending at the orbital rims and areas of higher bone density occured.

Conclusion: MSE produces not only larger maxillary skeletal transverse movements, but also shows more expansion effects on the lateral walls of the nasal cavity compared to tooth-borne expansion.

Introduction: The purpose of this retrospective study was to evaluate the skeleton changes induced by Maxillary Skeleton Expander (MSE) and Hyrax appliance in the Spheno-occipital Synchndroses and the vomer-bone using a Cone-Beam Computed Tomographic images (CBCT). Our hypothesis is that MSE and Hyrax result in dissimilar expansion pattern and magnitude especially in adult patients. Material and Methods: A novel methodology was developed to evaluate the changes of the SOS and vomer. Total of 23 patients were included in the study; 16 patients in the MSE group and 7 patients in the Hyrax group. CBCT scans were taken before (T0) and after treatment (T1) for every patient. The evaluation of the SOS and vomer-sphenoid joint were measured on the same CBCT cut before and after (T0:T1). The measurements of the surface area and the linear distance were recorded on both the sagittal and axial view. Dependent and independent means T tests as appropriate were used to evaluate differences in the treatment changes induced in the MSE group versus the Hyrax group. Results: A high statistical significance opening of the spheno-occipital Synchndroses were found in the MSE group despite older patients included in the treatment group. Hyrax group had younger patients and the SOS decreased. The junction between the vomer and sphenoid bone were larger in MSE and no changes were recorded in the Hyrax group.

Objectives: The recent advent of 3D photography has created the potential for comprehensive facial evaluation. However, lack of practical true 3D analysis of the information collected from 3D images has been the factor limiting widespread utilization in orthodontics. Current evaluation of 3D facial soft tissue images relies on subjective visual evaluation and 2D distances to assess facial disharmony. The objectives of this project strive to map the surface and define boundaries of 3D facial soft tissue, modify mathematical functions to average multiple 3D facial images, and mathematically average 3D facial images allowing generation of color-coded surface deviation relative to a true average.

Methods: Collaboration headed by UCLA Orthodontics with UCLA Neuroimaging was initiated to modify advanced brain mapping technology to accurately map the facial surface in 3D. 10 subjects were selected as a sample for development of the technical protocol. 3dMD photographic images were segmented, corrected using a series of topology correcting algorithms, and process to create close meshes. Shapes were mapped to a sphere using conformal and area preserving maps, and were then registered using a spherical patch mapping approach. Finally an average was created using 7-parameter procrustes alignment.

Results: Size-standardized average facial images were generated for the sample population. A single patient was then superimposed on the average and color-coded displacement maps were generated to demonstrate the clinical applicability of this protocol. Further confirmation of the methods through 3D superimposition of the initial (T0) average to the 4 week (T4) average was completed and analyzed.

Conclusions: The results of this investigation suggest that it is possible to average multiple facial images of highly variable topology. The immediate application of this research will be rapid and detailed diagnostic imaging analysis for orthodontic and surgical treatment planning. There is great potential for application to anthropometrics and genomics. This investigation resulted in establishment of a protocol for mapping the surface of the human face in three dimensions.

Objectives: Current methodologies in the field of morphometrics still employ the use of 2-dimensional images followed by a landmark-based Procrustes superimposition method to evaluate differences in shape, which can be tedious, subject to operator error, and fail to capture the true nature of shape variation between samples. The primary objective of this study is to address these limitations through the exploration and application of current methodologies used in neuroimaging and brain mapping to the field of morphometrics. By collaborating with Paul Thompson and his team at the Laboratory of Neuroimaging (LONI), we aim to:

1. Generate a 3D surface image of the mouse mandible and create an average surface for a cross sectional sample of parental strains in the Hybrid Mouse Diversity Panel (HMDP), as well as a developing longitudinal sample within a single strain.

2. Apply and modify the latest technologies used in brain mapping research to identify regional shape differences between the mandibles of these samples in 3 dimensions without the use of landmark identification, with the ultimate goal of identifying heritable quantitative traits.

3. Develop a useful, intuitive and visual method for evaluation of shape differences.

Methods: A total of fifty-three mice were studied. Ten mice, equally divided across five parental inbred strains, were obtained from the HMDP, as well as an additional 43 animals at various ages from the C57BL/6 strain. Skulls were subsequently scanned using μCT, then DICOM files transferred into a Beta version of Dolphin Imaging ® 11.7 Software. Hemi-mandibles were segmented from the entire skull and surface meshes were created using the same software. Topological correction of each hemi-mandible was performed before parametric registration and resampling of the surfaces to the same number of mesh points. Once registered to each other, a population average was then created and used as a reference template for shape comparison. Shapes were compared using the independently validated neuroimaging techniques of medial axis and tensor-based morphometrics.

Results: Population averages were created for the mandibles and four specific areas of significant shape change were identified in both the interstrain and intrastrain samples. Visual heat maps were also created to display shape differences.

Conclusions: Tensor-based morphometric evaluation of mandibular shape has far superior visualization and localization potential when compared to the current method using landmark-based analysis, while offering a more reliable solution as it eliminates the need for consistent landmark placement. Coupled with medial thickness computation, followed by additional refinement, this methodology could potentially be applied to a variety of applications concerned with evaluating shape difference in three dimensions.

Maxillary width deficiencies are a common occurrence in orthodontic patients being one of the most pervasive skeletal problems in the craniofacial region. An early orthodontic treatment can improve a transverse as well as an anteroposterior maxillary discrepancy and it consists of a true skeletal expansion or of an arch development obtained through the orthodontic wire. A pure skeletal expansion can be achieved with conventional appliances when the palatal suture is not completely fused. The introduction of TAD’s in common orthodontic practice has found several implications in the treatment of maxillary transverse deficiency. Using a micro-implants anchorage, the MSE has been proved to be a powerful tool to achieve a pure skeletal expansion even in adult subjects whose midpalatine maxillary suture is already fused.

Studies analyzing 2D measurements conducted on CBCT reported generalized changes in the skeletal structure of patients successfully treated with MSE, evidencing a more parallel expansion compared to the conventional expanders. These findings gave rise to the desire of closely studying the impact of MSE on the entire midface skeletal structure utilizing a pure three-dimensional approach.

Describing the three-dimensional movement that each bone undergoes during MSE is the major purpose of this study. No studies have previously used a pure tridimensional analysis to investigate the impact of MSE on the skeletal structures of the midface.

An innovative method for pure 3D analysis has been developed utilizing modern imaging software proving that Micro-Implant Assisted Maxillary Skeletal Expander (MSE) has a significant impact in the mid-facial bone structures.

The new analysis confirmed the previous finding obtained with 2D measurements and gave a better understanding of effect of MSE on the midfacial structures.

Introduction: The purpose of this study is to use Cone-Beam Computed Tomographic images to compare the skeletal and dental changes with MSE (Maxillary skeletal expander) and Hyrax appliance. Material and Method: 15 patients in MSE group and 8 patients in Hyrax group who met inclusion criteria were included. We used Independent-T test to compare the angulation of palatal sutures, bone bending and molar tipping between two groups. Result: In MSE group, the angulation of palatal sutures was 0.62 degree while it was 7.50 degree in the Hyrax group. (P< 0.01) The bone bending was 1.39 degrees in MSE group and it was 3.77 degrees in Hyrax group. (P<0.01) Molar angulation changes were 4.05 degrees in MSE group and 10.05 degrees in Hyrax group. (P<0.01). Estimated molar tipping in MSE group was 2.66 degrees and 6.28 degrees in Hyrax group. Conclusion: This study showed that the MSE appliance was able to produce a more parallel palatal suture opening, less bone bending and less molar tipping than the Hyrax appliance. Cone-Beam Computed Tomography is a useful tool in quantifying the skeletal and dental changes after palatal expansion.

Micro-implant assisted rapid palatal expander such as the Maxillary Skeletal Expander (MSE) have been utilized to achieve skeletal expansion as an alternative to surgically assisted expansion with some success. Previous studies show significant effects on the mid-face, including a degree of asymmetry. The aim of this study is to quantify the magnitude, parallelism, and asymmetry of expansion in non-growing patients and to explore possible factors that can predict the pattern of asymmetry of expansion.

We examined orthodontic non-growing patients (n=31) with an average age of 20.4 years old, with Cone Beam Computed Tomography (CBCT) images taken before and right after expansion using MSE. Those images were superimposed, and expansion was analyzed utilizing the Mid-Sagittal Plane (MSP) as a reference plane. Average magnitude of total expansion was 5 mm at the Anterior Nasal Spine (ANS), and 4.77 mm at the Posterior Nasal Spine (PNS) which showed statistical significance using a paired t-test with p<0.01. Expansion was parallel in the antero-posterior dimension where expansion at the PNS was 95% of that at the ANS. The sample was evaluated for asymmetry and divided into two groups; symmetric (n=15) and asymmetric (n=16), with 16 out of 31 patients exhibiting statistically significant asymmetry. The asymmetric group showed statistical significance asymmetry at p<0.05 when comparing expansion at both sides.

In order to validate any possible predicting factors, correlation of the asymmetry of expansion and multiple measurements of predicting factors was performed to both compare the direction of asymmetry, as well as the total magnitude of asymmetry.

MSE achieved highly parallel expansion in the sagittal plane in non-growing patients with an average magnitude of 5 mm at the ANS. However, transverse asymmetry of expansion was noted in 51% of the patients where one side expanded more than the other by at least 1.1 mm.

Lack of correlation was noted between the magnitude of asymmetry and both the initial asymmetry of the mid-face, as well as the difference in density values bilaterally. However, the direction of asymmetry of expansion at ANS was moderately associated with the initial asymmetry of the frontozygomatic suture.

Improving breathing for patients with obstructed airway has been heavily researched in recent years but continues to be a challenge clinically for medical practitioners including otolaryngologists, plastic surgeons, and dentists. In children with enlarged tonsils, adenotonsillectomy, partial tonsillectomy and lingual tonsillectomy are effective treatments. There are also various non-surgical treatments for obstructive sleep apnea in children including intra-nasal steroids, other anti-inflammatory medications, and oral appliances, but none of them are proven to be safe and effective.

Various forms of rapid maxillary expansion have demonstrated airway improvement but further studies are necessary to show consistent and reproducible results. The Maxillary Skeletal Expander (MSE), anchored by four palatal mini-implants, has become an increasingly more common option for patients who may not respond to traditional RME. Additionally, expansion of the maxilla by the MSE occurs more by horizontal translation and less tipping than a RME expander. The goal of the study is to investigate the role of MSE in airway improvement using objective measurements including Peak Nasal Inspiratory Flow (PNIF) and Peak Oral Inspiratory Flow (POIF) as well as subjective measures of breathing such as the Visual Analog Scale (VAS) and Nasal Obstruction Symptom Evaluation (NOSE). Furthermore, nasal septum deviation will be measured and correlated with objective measurements of breathing.

The results indicated that MSE produced improved functional breathing. Following MSE treatment, increases in total PNIF, left PNIF and right PNIF were significant at p<0.0001. Increase in POIF was significant at p<0.01. Patients reported decreased troubled breathing based on total VAS (p<0.01), left VAS (p<0.01) and right VAS (p<0.001). PNIF and VAS were correlated on each side in pre- and post- MSE groups while PNIF change and VAS change were correlated on the right. Additionally, an increase in nasal septum deviation to the left was correlated with an increase in PNIF on the right although this was not seen in the contralateral side.

If treatment with an MSE indeed raises objective and subjective measurements of airway, we may conclude that MSE is a possible treatment alternative for obstructive sleep apnea. This study has the potential to reshape the current standards for treatment of OSA creating an innovative, but less invasive cure for a large proportion of the population who suffer from OSA during sleep, have loss of daytime functionality, and have a host of co-morbid conditions.

Objectives: Current analyses of 3D cone beam computed tomography (CBCT) images used in orthodontic diagnosis and treatment planning still rely upon 2D linear and angular measures, along with subjective visual evaluation, to assess complex facial disharmony and other dentofacial characteristics. The overall objective of this investigation is to overcome these limitations by developing and testing protocol for mapping the surface of the human skull in three dimensions using CBCT images. In collaboration with the Laboratory of Neuroimaging (LONI) at UCLA, the specific goals of this project are:

-To apply and modify advanced technology used in brain mapping research to accurately and efficiently map the skull surface in 3D

-To apply and modify existing mathematical functions to find the average of multiple skull surfaces

-And to develop protocol for superimposing a sample skull on the average skull model, yielding a color-coded map of surface deviation and dysmorphology

Methods: Active patient files in the UCLA Section of Orthodontics were reviewed for the presence of a NewTom ® 3G CBCT scan. The search resulted in 67 patient scans that met the inclusion criteria for the study. CBCT files in DICOM format were first uploaded into a Beta version of Dolphin Imaging ® 11.5 Software. The surface was then segmented using a triangular mesh approach, after which the file was exported to MeshLab ® for viewing and to remove undesired nodes. Topology of each skull was corrected using a principal axis star map and smoothing of the extrapolated regions. Shapes were mapped to a sphere using conformal and area preserving maps, and were then registered using a spherical patch mapping approach. Finally an average was created using 7-parameter procrustes alignment.

Results: Size-standardized and non-size standardized average skull models were successfully created for the 67 patient sample. Color-coded displacement maps were generated for a sample patient to demonstrate the potential clinical applicability of this protocol.

Conclusions: The results of this investigation suggest that it is possible to average multiple shapes of highly variable topology such as the human skull. The most immediate application of this research will be rapid and detailed diagnostic imaging analysis for orthodontic and surgical treatment planning, particularly in complex cases involving congenital facial anomalies, facial asymmetry, and trauma. There is also great potential for application to areas outside orthodontics such as anthropometrics and genomics.