Personality research has begun to take hold in the animal kingdom as psychologists turn to animal models to investigate various aspects of personality. Similarly, behavioral ecologists and related fields have begun to explore the idea that individual variation in behavior is more than just noise around an average for a given population or group of interest. As a result, many have begun to turn to personality-related questions to explain individual differences in animal behavior. Collectively, psychologists, ecologists and related fields have created a boom in animal personality-related research. This interest has expanded to a variety of fish species, with many studies focused on an important axis of behavior in humans: the shy-bold axis. Unfortunately, there has been very little consideration for the methodology employed. We review both the experimental and statistical methodology found in a body of research on fish species, for which personality-related research has been conducted. Our aim is to shed light on many important considerations that are often overlooked in order to facilitate research concerned with the reliability and validity of the many methods used.