Regular polysemes are sets of ambiguous words that all share the same relationship between their meanings, such as CHICKEN and LOBSTER both referring to an animal or its meat. To probe how a context embedding model, here exemplified by BERT, represents regular polysemy, we analyzed whether its embeddings support answering sense analogy questions similar to “is the mapping be- tween CHICKEN (as an animal) and CHICKEN (as a meat) the same as that which maps between LOBSTER (as an animal) to LOBSTER (as a meat)?” We found that (1) the model was sensitive to the shared structure within a regularity type; (2) the shared structure varies across regularity types, potentially reflective of a “regularity continuum;” (3) some high-order latent structure may be shared across regularity types, suggestive of a similar la- tent structure across types; and (4) there is equivocal ev- idence that the aforementioned effects are explained by meaning overlap.

We conducted a case study on how unreliable and/or unrepresentative stimuli in psycholinguistics research may impact the generalizability of experimental findings. Using the domain of lexical ambiguity as a foil, we analyzed 2033 unique words (6481 tokens) from 214 studies. Specifically, we examined how often studies agreed on the ambiguity types assigned to a word (i.e., homonymy, polysemy, and monosemy), and how well the words represented the populations underlying each ambiguity type. We observed far from perfect agreement in terms of how words are assigned to ambiguity types. We also observed that coverage of the populations is relatively poor and biased, leading to the use of a narrower set of words and associated properties. This raises concerns about the degree to which prior theoretical claims have strong empirical support, and offers targeted directions to improve research practices that are relevant to a broad set of domains.