Magnetic skyrmions are topological spin-textures having immense potential for energy efficient spintronic devices. Here, we report the observation of stable skyrmions in unpatterned Ta/Co2FeAl(CFA)/MgO thin film heterostructures at room temperature in remnant state employing magnetic force microscopy. It is shown that these skyrmions consisting of ultrathin ferromagnetic CFA Heusler alloy result from strong interfacial Dzyaloshinskii-Moriya interaction (i-DMI) as evidenced by Brillouin light scattering measurements, in agreement with the results of micromagnetic simulations. We also emphasize on room temperature observation of multiple skyrmions which can be stabilized for suitable combinations of CFA layer thickness, perpendicular magnetic anisotropy, and i-DMI. These results provide a significant step towards designing of room temperature spintronic devices based on skyrmions in full Heusler alloy based thin films.