University Honors

This research tests whether the responses from utilizing ChatGPT can have more refined outputs compared to regular search engines when participants receive prompt-coaching. Receiving prompt-coaching means participants are provided tips and guidance that will, in theory, enhance their ChatGPT searching abilities. There will be four groups of participants–the control group, the Google search-engine users group, the ChatGPT users without prompt-coaching, and ChatGPT users with prompt-coaching. All participants will be randomly placed into a group and be asked to take a pre-survey, conduct a search activity on a randomly generated topic, complete a scored quiz on the same topic, and fill out a post-survey. Although all groups will complete a different search activity and be quizzed on different topics, the biggest differences in the searchtask will be between the control group, who will only partake in a search activity without the quiz, and the ChatGPT group with prompt-coaching, who will receive an extra section of guidance on how to enhance their inputs for improved ChatGPT answers before completing their search activity. There are three main hypotheses within the study. The first and second hypothesis states that ChatGPT users will perform better on the quiz and perceive the quiz to be easier than the control group, the Google-only users, and the ChatGPT users without prompt-coaching; the last hypothesis states that ChatGPT users will perceive a higher sense of flow than the control groups during their search compared to the control group, the Google-only users, and the ChatGPT users without prompt-coaching. Results revealed a statistically significant difference between the control group and the ChatGPT with prompt-coaching group, demonstrating that utilizing ChatGPT properly after realizing effective tips can improve participant learning and knowledge. However, no other statistically significant differences were discovered between any other remaining conditions. Additionally, the lack of statistical significance between the quiz scores of the ChatGPT users compared to the other conditions and the responses of participants on the match questions indicate they did not perceive the questions to be easier. Lastly, the absence of enjoyment and satisfaction found in the post-survey implied an inability to enter the state of flow.