Although the Positive and Negative Syndrome Scale (PANSS) was developed for use in schizophrenia (SZ), antipsychotic drug trials use the PANSS to measure symptom change also for bipolar (BP) and schizoaffective (SA) disorder, extending beyond its original indications. If the dimensions measured by the PANSS are different across diagnoses, then the same score change for the same drug condition may have different meanings depending on which group is being studied. Here, we evaluated whether the factor structure in the PANSS was consistent across schizophrenia (n = 3647), bipolar disorder (n = 858), and schizoaffective disorder (n = 592). Along with congruency coefficients, Hancock's H, and Jaccard indices, we used target rotations and statistical tests of invariance based on confirmatory factor models. We found the five symptom dimensions measured by the 30-item PANSS did not generalize well to schizoaffective and bipolar disorders. A model based on an 18-item version of the PANSS generalized better across SZ and BP groups, but significant problems remained in generalizing some of the factors to the SA sample. Schizophrenia and bipolar disorder showed greater similarity in factor structure than did schizophrenia and schizoaffective disorder. The Anxiety/Depression factor was the most consistent across disorders, while the Positive factor was the least consistent.