Constructing coexpression and association networks with omics data is crucial for studying gene-gene interactions and underlying biological mechanisms. In recent years, learning the structure of a Gaussian graphical model from high-dimensional data using L1 penalty has been well-studied and many applications in bioinformatics and computational biology have been found. However, besides the problem of biased estimators with LASSO, L1 does not always choose the true model consistently. Based on our previous work with L0 regularized regression (Liu and Li, 2014), we propose an L0 regularized sparse inverse covariance estimation (L0RICE) for structure learning with the efficient alternating direction (AD) method. The proposed method is robust and has the oracle property. The proposed method is applied to omics data including data, from next-generation sequencing technologies. Novel procedures for network construction and high-order gene-gene interaction detection with omics data are developed. Results from simulation and real omics data analysis indicate that L0 regularized structure learning can identify high-order correlation structure with lower false positive rate and outperform graphical lasso by a large margin.