In many reinforcement learning scenarios such as many game environments or real lifesituations, the rewards are usually very limited and sparse. This kind of tasks are always difficult for agent to learn and explore. In fact, dealing with sparse reward environments has always been a challenge in reinforcement learning. In this work, we aim to study the agent driven by curiosity that can handle tasks with sparse rewards using a self-supervised method. We also want to test the possibility about agents driven by their intrinsic curiosity being able to explore the environment and generate reward by themselves. As a result, curiosity makes agents more intelligent. In the later experiments, we test two curiosity based RL methods in three different games and demonstrated that curiosity could indeed achieve very impressive performance in sparse reward game environment.