Botanical carnivory is a novel feeding strategy associated with numerous physiological and morphological adaptations. However, the benefits of these novel carnivorous traits are rarely tested. We used field observations, lab experiments, and a seminatural experiment to test prey capture function of the marginal spikes on snap traps of the Venus flytrap (Dionaea muscipula). Our field and laboratory results suggested inefficient capture success: fewer than one in four prey encounters led to prey capture. Removing the marginal spikes decreased the rate of prey capture success for moderate-sized cricket prey by 90%, but this effect disappeared for larger prey. The nonlinear benefit of spikes suggests that they provide a better cage for capturing more abundant insects of moderate and small sizes, but they may also provide a foothold for rare large prey to escape. Our observations support Darwin's hypothesis that the marginal spikes form a "horrid prison" that increases prey capture success for moderate-sized prey, but the decreasing benefit for larger prey is unexpected and previously undocumented. Thus, we find surprising complexity in the adaptive landscape for one of the most wonderful evolutionary innovations among all plants. These findings enrich understanding of the evolution and diversification of novel trap morphology in carnivorous plants.