We show that for an $n\times n$ random matrix $A$ with independent uniformly
anti-concentrated entries, such that $\mathbb{E} ||A||^2_{HS}\leq K n^2$, the
smallest singular value $\sigma_n(A)$ of $A$ satisfies $$ P\left(
\sigma_n(A)\leq \frac{\varepsilon}{\sqrt{n}} \right) \leq
C\varepsilon+2e^{-cn},\quad \varepsilon \ge 0. $$ This extends earlier results
of Rudelson and Vershynin, and Rebrova and Tikhomirov by removing the
assumption of mean zero and identical distribution of the entries across the
matrix, as well as the recent result of Livshyts, where the matrix was required
to have i.i.d. rows. Our model covers "inhomogeneus" matrices allowing
different variances of the entries, as long as the sum of the second moments is
of order $O(n^2)$.
In the past advances, the assumption of i.i.d. rows was required due to lack
of Littlewood--Offord--type inequalities for weighted sums of non-i.i.d. random
variables. Here, we overcome this problem by introducing the Randomized Least
Common Denominator (RLCD) which allows to study anti-concentration properties
of weighted sums of independent but not identically distributed variables. We
construct efficient nets on the sphere with lattice structure, and show that
the lattice points typically have large RLCD. This allows us to derive strong
anti-concentration properties for the distance between a fixed column of $A$
and the linear span of the remaining columns, and prove the main result.