Measurement of the decay of the Higgs boson to charm quarks provides a direct probe of the Higgs coupling to second-generation quarks. Therefore, it is crucial for understanding the structure of Yukawa couplings. In this thesis, a search for the Higgs boson decaying to charm quarks with the CMS experiment is presented. The search is designed for Lorentz-boosted Higgs bosons produced in association with vector (V) bosons (W or Z bosons). A novel approach that reconstructs both quarks from the Higgs boson decay with a single large-radius jet is adopted. The charm quark pair is identified with an advanced deep learning--based algorithm. This approach leads to a highly competitive result: Using proton-proton collision data corresponding to an integrated luminosity of 35.9 fb-1, an observed (expected) upper limit on the VH cross section times the H->cc branching fraction of 71 (49) times the standard model expectation at 95% confidence level is obtained.
A detailed description of the deep learning--based boosted object identification algorithm is also presented in this thesis. It is a versatile algorithm designed to identify and classify hadronic decays of highly Lorentz-boosted top quarks and W, Z, Higgs bosons. Using deep neural networks to directly access and process the raw information of all constituent particle-flow candidates of a jet, this advanced algorithm has achieved significant performance improvements compared to traditional approaches.