Skip to main content
eScholarship
Open Access Publications from the University of California

Scholarly Works (4 results)

  • Article
  • Peer Reviewed
UC Berkeley Previously Published Works (2013)
We consider, in the modern setting of high-dimensional statistics, the classic problem of optimizing the objective function in regression using M-estimates when the error distribution is assumed to be known. We propose an algorithm to compute this optimal objective function that takes into account the dimensionality of the problem. Although optimality is achieved under assumptions on the design matrix that will not always be satisfied, our analysis reveals generally interesting families of dimension-dependent objective functions.
    • Thesis
    • Peer Reviewed
    UC Berkeley Electronic Theses and Dissertations (2014)

    Extracting relevant low-dimensional information from high-dimensional data is a common pre-processing task with an extensive history in Statistics. Dimensionality reduction can facilitate data visualization and other exploratory techniques, in an estimation setting can reduce the number of parameters to be estimated, or in hypothesis testing can reduce the number of comparisons being made. In general, dimension reduction, done in a suitable manner, can alleviate or even bypass the poor statistical outcomes associated with the so-called ``curse of dimensionality.''

    Statistical models may be specified to guide the search for relevant low-dimensional information or ``signal'' while eliminating extraneous high-dimensional ``noise.'' A plausible choice is to assume the data are a mixture of two sources: a low-dimensional signal which has a non-Gaussian distribution, and independent high-dimensional Gaussian noise. This is the Non-Gaussian Components Analysis (NGCA) model. The goal of an NGCA method, accordingly, is to project the data onto a space which contains the signal but not the noise.

    We conduct a comprehensive review of NGCA. We analyze the probabilistic features of the NGCA model and elucidate connections to similar well-known models and methods in the literature, including a hitherto-unseen and surprising connection to a set of models proposed by Cook in the context of dimension-reduction in regression. We review the literature on NGCA, catalogue existing NGCA methods, and compare them to the method proposed in Chapter 2.

    We also propose and analyze a new NGCA method based on characteristic functions called CHFNGCA. We show CHFNGCA is, under mild moment conditions on the non-Gaussian sources, consistent and asymptotically normal; the latter property has not been demonstrated for any other NGCA method in the literature. We conclude by highlighting areas for future work.

    The proof of all stated propositions, lemmas and theorems are contained in Appendices A and B.

      Cover page: Non-Gaussian Component Analysis
      • Article
      • Peer Reviewed
      UC Berkeley Previously Published Works (2013)
      We study regression M-estimates in the setting where p, the number of covariates, and n, the number of observations, are both large, but p ≤ n. We find an exact stochastic representation for the distribution of β = argmin(β∈ℝ(p)) Σ(i=1)(n) ρ(Y(i) - X(i')β) at fixed p and n under various assumptions on the objective function ρ and our statistical model. A scalar random variable whose deterministic limit rρ(κ) can be studied when p/n → κ > 0 plays a central role in this representation. We discover a nonlinear system of two deterministic equations that characterizes rρ(κ). Interestingly, the system shows that rρ(κ) depends on ρ through proximal mappings of ρ as well as various aspects of the statistical model underlying our study. Several surprising results emerge. In particular, we show that, when p/n is large enough, least squares becomes preferable to least absolute deviations for double-exponential errors.
        • Article
        • Peer Reviewed
        UC Berkeley Previously Published Works (2013)
        We demonstrate two ways in which the Fourier transforms of images that consist solely of randomly distributed electrons (shot noise) can be used to compare the relative performance of different electronic cameras. The principle is to determine how closely the Fourier transform of a given image does, or does not, approach that of an image produced by an ideal camera, i.e. one for which single-electron events are modeled as Kronecker delta functions located at the same pixels where the electrons were incident on the camera. Experimentally, the average width of the single-electron response is characterized by fitting a single Lorentzian function to the azimuthally averaged amplitude of the Fourier transform. The reciprocal of the spatial frequency at which the Lorentzian function falls to a value of 0.5 provides an estimate of the number of pixels at which the corresponding line-spread function falls to a value of 1/e. In addition, the excess noise due to stochastic variations in the magnitude of the response of the camera (for single-electron events) is characterized by the amount to which the appropriately normalized power spectrum does, or does not, exceed the total number of electrons in the image. These simple measurements provide an easy way to evaluate the relative performance of different cameras. To illustrate this point we present data for three different types of scintillator-coupled camera plus a silicon-pixel (direct detection) camera.
          Top