Recent network-based approaches leverage graph theoretic analyses to study individual differences in semantic networks and how they relate to other cognitive processes. However, it remains ambiguous whether individual differences captured via semantic network analyses reflect true differences in latent knowledge representations, or strategic differences in how people approach semantic relatedness tasks. To determine the robustness of content- and structure- based metrics of individual semantic networks we test their reliability across different tasks. We find both weighted and unweighted graph theoretic representations can predict individual differences in connections between semantic units across tasks. Furthermore, node centrality, a content-based metric which captures relative ‘importance’ of units within a network, is reliable across tasks, but metrics of structural properties of semantic networks, i.e. average clustering coefficient and shortest path length, are less robust. These results highlight the importance of validating graph-theoretic measures in the study of individual differences in semantic memory.