We study the phase diagram of two-dimensional Bose-Fermi mixtures of ultracold atoms on a triangular optical lattice, in the limit when the velocity of bosonic condensate fluctuations is much larger than the Fermi velocity. We contrast this work with our previous results for a square lattice system in the work of Mathey et al. [Phys. Rev. Lett. 97, 030601 (2006)]. Using functional renormalization-group techniques we show that the phase diagrams for a triangular lattice contain exotic superconducting phases. For spin-1/2 fermions on an isotropic lattice we find a competition of s-, p-, extended d-, and f-wave symmetries, as well as antiferromagnetic order. For an anisotropic lattice, we further find an extended p-wave phase. A Bose-Fermi mixture with spinless fermions on an isotropic lattice shows a competition between p- and f-wave symmetries. These phases can be traced back to the geometric shapes of the Fermi surfaces in various regimes, as well as the intrinsic frustration of a triangular lattice.

Using a functional renormalization group approach we study the zero temperature phase diagram of two-dimensional Bose-Fermi mixtures of ultracold atoms in optical lattices, in the limit when the velocity of bosonic condensate fluctuations is much larger than the Fermi velocity. For spin-1/2 fermions we obtain a phase diagram, which shows a competition of pairing phases of various orbital symmetry (s, p, and d) and antiferromagnetic order. We determine the value of the gaps of various phases close to half filling, and identify subdominant orders as well as short-range fluctuations from the renormalization group flow. For spinless fermions we find that p-wave pairing dominates the phase diagram.