Predicting Results from Deese
In a classic study by Deese (1959b), the goal was to predict the intrusion rates of words in free recall. Participants studied the 15 strongest associates to each of 36 critical lures while the critical lures themselves were not studied. In a free recall test, some critical lures (e.g. “sleep”) were falsely recalled about 40% of the time while other critical lures (e.g. “butterfly”) were never falsely recalled. Deese was able to predict the intrusion rates for the critical lures on the basis of the average associative strength from the studied associates to the critical lures and obtained a correlation of R=0.8. Since Deese could predict intrusion rates with word association norms, it was expected that that the WAS vector space derived from the association norms could also predict intrusion rates. The idea here is that critical items that are closely related to list words are more likely to appear as intrusions in free recall than critical items that are not closely related to list words. The average similarity was computed between each critical lure vector and list word vectors using different dimensionalities. In Figure 3, a scatter plot shows the relationship between the similarity and the intrusion rates as observed by Deese (here, the number of dimensions was 400). The obtained correlation was R=0.775. In Table 6, the correlations for other dimensionalities are listed. The correlation
3. The average similarity between critical item and list item in WAS can predict the intrusion rates for the critical item as observed by Deese (1959b).
decreases with decreasing number of dimensions. This might happen because a smaller dimensional space has less room to place 5000 words so that the resulting similarity structure does not capture as well the differences in observed intrusion rates. The table also shows the correlations when the vectors are taken from LSA. It can be seen that similarity structure of LSA does not correlate as well with the intrusion rates as WAS. Also, the effect of varying the number of dimensions does not seem to affect the correlations.
I
n extralist cued recall experiments, after studying a list of words, subjects are presented with cues that can be used to retrieve words from the study list. The cues themselves are novel words that were not presented during study and they typically are associatively related to one of the studied words. The degree to which a cue is successful in retrieving a particular target word is a measure of interest because this might be related to the associative/semantic overlap between cues and their targets. Research in this paradigm (e.g. Nelson & Schreiber, 1992; Nelson, Schreiber, & McEvoy, 1992; Nelson, McKinney, Gee, & Janczura, 1998; Nelson & Xu, 1995) has already shown that the associative strength between cue and target is one important predictor for the percentage correctly recalled targets. Therefore, we expect that the WAS similarity between cues and targets are correlated to the percentages of correct recall in these experiments. We used a database containing the percentages correct recall for 1115 cue-target pairs from over 29 extralist cued recall experiments from Doug Nelson’s laboratory (Nelson & Zhang, submitted; Nelson, personal communication). The correlations between the WAS similarity and observed recall rates for different dimensionalities are shown in Table 7.
The best result was a small but significant correlation of .36 using 400 dimensions. The correlations decreased with decreasing number of dimensions. Since a smaller number of dimensions limits the ways in which 5000 words can be placed in the space, it is possible that this factor explains the limiting effect on the correlation. The table also shows the correlations when vectors from the LSA space were taken. The correlations with the LSA vectors were less high than with WAS but were relatively close in value at 300 dimensions. This suggests that both WAS and LSA can be used as part of a process model to predict cued recall results.
D
iscussion
By a statistical analysis of a large database of free association norms, the Word Association Space (WAS) was developed. In this space, words that have similar associative structures are placed in similar regions of the space. We showed that the output order of words in free association norms is preserved to some degree in WAS: first associates in the norms are likely to be close neighbors in WAS. There are some interesting differences between the similarity structure of WAS and the associative strengths of the words in the norms. Words that are not directly associated can be close neighbors in WAS when the words are indirectly associatively related through a chain of associates. Also, in some cases, words that are directly associated in the norms are not close neighbors in WAS at all (although these are exceptions). This makes WAS not a good model for the task of predicting free association data. However, it is important to realize that WAS was not developed as a model of free association (e.g. Nelson & McEvoy, Dennis, in press) but rather as a model based on free association.
The WAS approach is an additional method available to place words in a psychological space. It differs from the LSA and HAL approaches in several ways. LSA and HAL are automatic methods and do not require any extensive data collection of ratings or free associations. With LSA and HAL, tens of thousands of words can be placed in the space, whereas in WAS, the number of words that can be placed depends on the number of words that can be normed. It took Nelson et al. (1998) more than a decade to collect the norms, highlighting the enormous human overhead of the method.
Another difference is that LSA and HAL have the potential to model the learning process a language learner goes through. For example, by feeding the LSA or HAL model successively larger chunks of text, it can be simulated what the effect learning has on the similarity structures of words in LSA or HAL. In WAS, it is in principle possible to model a language learning process by collecting free association norms for participants at different stages of the learning process. In practice however, such an approach would not easily be accomplished.
We think that the WAS, LSA, and HAL approaches to creating semantic spaces are all useful for theoretical and empirical research. It might be that the usefulness of a particular space will depend on the task it is applied to. Since the free association norms have been an integral part in predicting episodic memory phenomena (e.g. Cramer, 1968; Deese, 1965; Nelson, Schreiber, & McEvoy, 1992), it was assumed that a vector space based on free association norms would be an especially useful construct to model memory phenomena. In this research, we have already shown with simple geometric operations how the similarity structure of WAS can predict to some degree the intrusion rates observed by Deese (1959b) in his classic false memory experiment as well as the percentages of correct recall in cued recall experiments. This suggests to us that WAS forms a useful representational basis for memory models that are designed to store and retrieve words as vectors of feature values. In part II of this research, we will combine the semantic space of WAS with a process model for recognition memory. This will allow us to model the processes of recognition memory and gives us a principled way to represent words by vectors. The assumption of representing words by vectors in memory models is relatively old. However, in most memory modeling, the vectors representing words are arbitrarily chosen and are not based on or derived by some analysis of the meaning of actual words in our language. In part II, it is expected that a memory model based on these semantic vectors from WAS will be useful to make predictions about the effects of varying semantic similarity in memory experiments.
Share with your friends: |
