Download Intellectual Ability and Learning: Review of Ability-Treatment Interactions and more Exams Psychology in PDF only on Docsity!
DOCUMENT RESUME
ED029001 24 SP By-Cronbach,LeeJ.;Snow,RichardE. IndividualDifferencesinLearningAbilityasaFunctionofInstructionalVariables.FinalReport. StanfordUniv.,Calif.SchoolofEducation. SponsAgency-OfficeofEducationCHEW,Washington,D.C.BureauofElementaryandSecondaryEducation.
BureauNo-BR-6-I
PubDateMar ContractOEC-4-6-061269- Note-221p. EDRSPriceMF-SI.00HC-SI1. Descriptors-AcademicAptitude,IndividualizedInstruction,*InteractionProcessAnalysis,LearningTheories, Personality,ProgramedInstruction,ResearchDesign,ResearchMethodology,ResearchNeeds,Students Identifiers-AptitudeTreatmentInteraction,ATI ThisdocumentfocusesonhowresearchwhiChinvestigatestheinieraction between learning abilities and instructional treatments (Aptitude Treatment InteractionorAIDshouldproceed.PreviousresearchrelatedtoATIisevaluatedin thecontextoftheATIpremisethatcharacteristicsoflearnersaffecttheir attainmentofeducationalgoals(outcomesfromtreatments).Previousresearch-was foundtobeinadequatebecauseofweakmethodology,inappropriatehypotheses, andlackofreplication.Guidelinesforfutureresearch,introducedthroughoutthe document,encompassdesign, methodology,andconceptualstages(suchas vnderstandinghowgeneralabilityenters-intoapupil'slearning).InrelationtotwoATI goals(differentinstructionalmethodsfordifferentkindsofstudentsshouldbe employedtoachievethesameeducationalgoals,andpersonalitydimensionsaswell asaptitudeshouldbeacriterionforplacementratherthanforrejectionorselection inaprogram),researchevaluationuncoveredthefollowing:learningrateisafalse issue;generalabilityisrelatedtolearninginconceptualtasks;roteandmeaningful instructionmayservedifferentkindsofstudents;theprinciplesgoverningthe matchingoflearnertoindividualizedinstructionalenvironmentarenotyetknown;and thethinkingonpersonalityvariablesastheyrelatetoinstructionisinapriMitive state.A221-itemreferencelistisinduded.(LP)
Contract No. OEC 4-6-061269-12l U.S. Office of Education U.S.DEPARTMENTOFHEALTH,EDUCATION&WELFARE OfFICEOfEDUCATION
THISDOCUMENTHASBEENREPRODUCEDEXACTLYASRECEIVEDFROMNE PERSONORORGANIZATIONORIGINATINGIT. POINTSOfVIEWOROPINIONS STATEDDONOTNECESSARILYREPRESENTOFFICIALOFFICEOfEDUCATION POSITIONORPOUCY. Final Report Individual Differences in Learning Ability as a Function of Instructional Variables
Lee J. Cronbach, Director Richard E. Snow, Assoc. Director
The research reported herein was performed pursuant to a contract with the Office of Education, U. S. Department of Health, Education, and Welfare. Contractors^ under- taking such projects under Government sponsorship are encouraged to express freely their professional judgment in the conduct of the project. Points^ of^ view^ or^ opinions stated do not, therefore, necessarily represent official Office of Education position or policy.
if)
ft)
IN
0 School of Education
Stanford University
Nam
Stanford, California
liorW
March 1969
We thank the Bureau of Research for its financial support, and we thank the Stanford Center for Research and Development in Teaching, with which the project has been loosely affiliated, for unnumerable courtesies. We acknowledge the assistance of the project research assistants: (^) Nancy Bamilton Markle, Tamarra Pickford Moeller, Akimichi Omura, and Pearl Roossinck Paulson, and also a large number of other local and distant colleagues, post- doctoral trainees, and graduate students who have contributed to our work and thinking.
Stanford University M4rch 31, 1969
iii
L C
R E S
TABLE OF CONTENTS
Preliminary Statement List of Technical Reports Abstract A. (^) A Perspectiveon the ATI Problem The educational conteltt The methodological context Project aims The social and philosophical context
ii vi vii 1 1 2 6 8 B. (^) Statistical Methods and Designs (^14) Conventional analyses (^14) Description of interactionsas absolute functions (^14) Need for absolutestatements (^15) Nonlinear models (^17) Ordinality of interactions (^18) Designs (^20) Statistical analysis (^22) The Neyman-Johnsonmethod (^23) The general linear hypothesis (^23) C. (^) Learning Rate as a Variable in Educational andPsychological Research 26 Reliability of learning rate (^28) Alternative ways ofmeasuring learning (^31) Rate scores (^31) Limitations of ratemeasures (^34) What to use in place of gain scores (^36) The extended course of learning (^37) Learning rates at various stages (^38) Learning to learn (^41) Introduction to the Alvord andBunderson studies (^45) Tuning (^46) Correlations among learning measures (^49) Correlations for similar tasks (^50) Correlations across distinct tasks (^51) D. (^) The Structure of Abilities (^53) The need for parsimony (^53) The hierarchical model (^53) Importance of multitrait-multimethod designs (^56) The facet model (^57) The issue of stability (^58) Interbattery research (^59) A DAT study (^60) Alternative tests of Guilford hypotheses (^61) Cluster analyses (^62) Facet factor analysis (^64) Multidimensional scaling (^65) Further work on divergent thinking (^66) Analysis of simplex matrices (^69)
iv
List of Technical Reports
No. 1 Cronbach, Lee J.
No. (^2) Cronbach, Lge J.
No. 3 Snow, Richard E. & Salomon, Gavriel
No. 4 Alvord, Ray%
No. (^5) Milmilton, Nancy R.
No. 6 Cronbach, Lee J. & Purby, Lita
Year-to-Year Correlations of Mental Tests: A Review of the Hofstaetter Analysis Intelligence? Creativity?^ A^ Parsimonious Reinterpretation of the Wallach-Kogan Data Aptitudes and Instructional Media
Learning and Transfer in a Concept-Attainment Task: A^ Study^ of^ Individual^ Differences Differentl.al Response to Instruction Designed to Call Upon Spatial and Verbal Aptitudes How Should We Measure Change--Or Should We?
vi
Abstract
Despite the long history of research on learning and on individual differences, little progress has been made in reaching an integrated un- derstanding of the nature of aptitude or ability to learn. (^) This proiect sought to assess the present state of knowledge in this area. Specific activities and outcomes of the project, as related to five original ob. jectives, were as follows:
- (^) A careful review of the large body of relevant literature was completed. (^) One result of the review was the recognition that most of the methodology commonly used in aptitude-treatment interaction (ATI) research was weak and often wholly inappropriate for the uses intended. (^) Suggestions for methodological improvement mere formulated. Key points were illustrated using reanalyses of reported data. (^) Another clear finding reaffirmed the substantial predictive value (^) of general mental tests in instructional research. (^) Many studies support the further view that it is possible to establish treatment pairs that have high and low relation to general abil- ity. Studies of narrowly differentiated abilities or those varying program- ming treatments or content have usually failed to produce ATI. Scattered studies investigating personality and motivation variables as aptitudes were reviewed but no summary conclusions could be justified. Emphasis VAS placed on the importance of process analyses of instructional tasks as a guide to further research on ATI.
- (^) The concept of learning rate was reviewed in detail and exposed as a false issue. (^) The significance of the notion of multiple criteria for the learning.rate problem was discussed. Both rate measures and level meas- ures were judged inappropriate as representations of learning ability. Ap- proaches to reconceptualization were discussed in terms of multiple regres- sion of outcomes on aptitude information.
- (^) The meaning of "reliability" of measures of learning rate was also examined. (^) Some alternative estimation procedures were considered. The con- clusion was reached that such reliability cannot be determined. (^) Though low- er-bound estimates might be obtained by determining maximum predictability using multiple-regression techniques, the problem of deciUng whether low validity results from low reliability in a given instance is insoluble.
vii
of first grade in a local school, showed promising interactions between memory skills and both cognitive and affective criteria. Phonics instruc- tion appeared best for law ability children while whole 'word treatment served high ability children. A second study, attempting to replicate and improve upon these findings, is being carried out in the same school for the following year. The report summaries other expertments bearing on project concerns and reanalyses of pr2viously reported data. General observations on ATI research and educational policy are also included.
ix
LIST OF TABLES AND FIGURES
Table 1, Results of the Maier-Jacobs Experiment on Instructional Programs. 88 Table 2, Results of Reanalysis of Cartwright Data. (^91) Table 3, Woodruff Study Treatment Groups (^103) Figure 1, Example of Ordinal vs. Disordinal Interaction (^19) Figure 2, Learning curves for persons a and b. (^33) Figure 3, Learning curves for persons b and c. (^33) Figure 4, Nonmetric scaling for postulated order U, C, R, S, T, I, (^67) Figure 5, Correlation of concept attainment scores with "general" ability at successive stages (from data of Dunham et al.) (^77) Figure 6, Demonstratsov of reporting from American Institutes of Research timpal Report, 1965. (^99) Figure 7, Lorge-Thorndike, Verbal IQ (^105) Figure 8, Stallings and Snow Study. (^112) Figure 9, Salomon Study. (^115) Figure 10, Koran Study. (^116) Figure 11, Results of three studies projected onto Melton's model for associative learning. (^118) Figure 12, Mnitiple-regression analysis f6r Koran study, including video- modeling (VM) and written verbal modeling (WM) treatments (^120) Figure 13, Correlations of ability tests with learning trial performance for three treatments.. (^122) Figure 14, Results of Grimes-Allinsmith Study of third grade reading instruction (^154) Figure 15, Results from Leith-Bassett study of 10-year olds. (^157)
2
effect of setting different kinds of educational goals for pupils with different abilities. If,^ however,^ they^ are^ designed^ to^ move^ all^ learn- ers toward essentially the same outcomes -- so far as intellectual and personal development are concerned -- they can overcome stratification. We know that it is socially indefensible to give some children good education and some poor education. We^ have^ captured^ this^ in^ the slogan "equality of educational opportunity." But^ this^ too^ easily^ de- generates into a Lockean laissez faire which merely invites each child to compete for a place in the system, just as the Declaration of Inde- pendence affirms "the right to pursue happiness". Social^ policy^ in^ this century has turned from the passive -- guaranteeing a fair race, but putting all the burden on the individual -- to an active effort to de- sign social conditions that will help everyone run his strongest race. Jensen offers the appropriate slogan for the school: "optimal^ diversity of educational opportunity." To^ spell^ out^ just^ what^ is^ meant^ by^ ,optimal presents major tasks for the philosopher, the empirical scientist, and the practical educator. The methodological context For the empirical scientist, the problem reduces to the search for aptitude-treatment interactions (ATI). To^ discover^ and^ demonstrate these requires a style of research that has only recently become^ the conscious concern of investigators. Two^ broad^ lines^ of^ empirical^ re- search in behavioral science, the experimental and the correlational, have received extended treatment in writings on methodology and have been illustrated as the standard ways of investigating problems of^ learn- ing and aptitude. In^ the^ past^ two^ decades^ there^ has^ gradually^ emerged a realization that interaction research is a third^ variety^ whidh^ em- braces both the older types of study in a single setting, and so permits investigation of a new kind of question. Experimental research concerns itself with differences among treatments or conditions; one seeks to establish^ significant^ main^ effects, of the form, say, "Homogeneous grouping plan^ A^ works^ better^ than^ hetero- geneous grouping plan B." Correlational^ research^ concerns^ itself^ with the concurrence or covariation of distinct indicators,^ as^ in^ testing such hypotheses as "Good spellers are more successful^ in^ learning^ steno- graphy than poor spellers" or "Independent-minded students^ are^ more likely than others to drop out of engineering school."^ The^ essential
method is to compare, eitherby^ computing^ a correlation^ or^ by^ comparing means of high and low groups,the^ standing^ of^ persons^ on two^ variables.
Interactional ideas are widespread^ in^ scientific thinking.^ An
interaction is present when aneffect^ found^ for^ one^ kind^ ofsubject^ or in one kind of setting is notfound^ under^ other^ conditions.^ The^ possibi- lity of interactions isrecognized^ in^ the^ physicalscientist's^ ubiquitous qualifier "Other things being equal..."^ and^ in^ the^ socialscientist's "Can you generalize to other^ groups^ (communities,cultures,^ etc.)" Cronbach (1953), contrastingthe^ method^ of^ "correlation between^ persons" with the conventional correlation^ between^ tasks^ orsituations,^ pointed out that the whole processof^ seeking^ laws^ in^ scienceis^ to^ somehow partition a grand matrix of organisms^ and^ situations^ so^ asto^ obtain sections over which a generalization^ applies.^ That^ is,^ the^ task^ is to group subjects who aresimilar^ in^ their^ response^ to^ some selected range of situations.^ This^ kind^ of^ theory^ isespecially^ needed^ in connection with instruction: What^ characteristics^ make instructional situations "similar", in fhe sensethat^ similar^ situations^ are all beneficial for the same kind of^ pupil?^ And,^ in^ this^ context, what variables define "similar" learners,^ i.e.,^ those^ ready^ to profit^ from the same kind of instruction? There^ is^ no^ possibility of^ theory^ re- garding instruction untillearners^ and^ learning^ situations^ are character- ized in reasonably generaland^ comparable^ terms. Modern statistical methods for^ experimental^ and correlational studies derived from the work^ of^ Karl^ Pearson^ and his^ contemporaries. R. A. Fisher, in his series^ of^ impressive contributions,^ advanced methods of both these types, but he^ also^ introduced the^ possibility^ of^ system- atically testing for interactions. Some^ technicaldevelopments^ of^ the 1930's, particularly thosearising^ in^ Neyman's^ wing of^ the^ statistics department at the Universityof^ London,^ have^ been almost^ entirely neglected in behavioral science eventhough^ they^ are^ highly pertinent to the interaction problem;^ we^ shall^ return^ to them^ in^ due^ course.^ As for Fisherian methods oftesting^ interactions (e.g.,^ between^ species of wheat and effect of fertilizer),^ these^ were^ duly relayed^ to^ experimental psychologists, and it isfairly^ common^ to^ see^ reports of^ significant interactions of IQ or sex with^ an^ experimental variable.^ These^ interactions were more often regarded^ as^ nuisance than^ as^ basic^ discovery^ tobe interpreted, until they became^ the^ focus^ of special^ lines^ of^ investigation,
5
the best for another, There^ were,^ then,^ a^ number^ of^ calls^ upon psycho- logists to bring interactions squarely into the center of the^ stage. Cronbach (1957) chided "The Two Disciplines of Psychology" for,^ on^ the one hand, regarding individualdifferences^ as^ "error"^ beclouding^ ex- perimental effects and, on the other, regarding situational variance^ as uncontrollable.attenuation beclouding the prediction of individual^ success. Be urged a fusion of the two disciplines into one, which^ would^ combine correlational and experimental methodology to study interactions. Eysenc (1195 /17)^ nsisted^ that^ a^ sound^ theory of^ personality^ or^ of^ task^ perform- ance could not bedeveloped;^ that^ a^ proper^ theory^ would^ have^ to^ be^ a theory of personality and situation. Despite these stirrings, the movementtoward^ interactionist^ studies gained speed slowly. Such^ studies^ are^ relatively^ expensive,^ the method- ology for conducting them is unclear, and thetheory^ that^ would^ guide research strategy is little better than speculative. By^ 1965,^ the^ time seemed ripe for major stocktaking. A^ rather^ large^ number^ of^ scattered studies had been reported, in various contexts. The^ concern^ of^ educators for adaptation to individual differences was mounting.^ The^ problem^ be- came live in psychology, as demonstrated^ by^ the^ appearance^ of^ the^ sympo- sium edited by Gagne^ (1967) on^ learning^ andindividual^ differences^ (based on a 1965 conference),^ and^ by^ many^ references^ to^ person-situation inter- actions in the Annual Review of Psycholcv. Research is necessarily highly specific -- a study^ of^ specific^ sub- jects exposed to a specific treatment and measured^ in^ a^ specific^ way. But the main fruit of research is notthe^ microscopic^ findings^ of^ single studies. The^ main^ fruit^ is^ the^ conceptualizationof^ nature^ that^ is erected by minds reflecting on specific findings.^ While^ frequently^ re- garded as almost a by-product ofresearch,^ it^ --^ rather^ than^ the^ specific findings -- is what guides men in dealing with^ the^ world^ around^ them. A second outgrowth of the research effort^ is^ a^ method^ of^ investiga- tion, a discipline. Scientists^ are^ continually learning^ to^ investigate. Just as substantive concepts guide man in^ dealing^ with^ his^ world, method- ological concepts guide investigatorsin^ dealing^ with^ that^ world^ in^ a more systematic way. Unfortunately, the logic of research in this^ new^ comprehensive^ dis- cipline has not been clear. At^ one^ level,^ there^ havebeen^ gross^ failures to capitalize on the data in individual^ studies.^ Over^ and^ over^ investigators
6
have reported demonstrably false conclusions because they have selected inappropriate (though traditional) methods of analysis. (^) There has been far too little realization of the special statistical requirements of interactional studies. (^) We shall return later to some specific examples of the ways in which faulty Analysis has wastedmoney and research effort. Until these faults are remedied, the mix of trustworthy and untrustworthy findings in the literature provides a crumbly edificeupon which no sensible theory can be built. Beyond this level, however, we have found need for a metatheory that will provide perspective for the in- vestigator. Questions are wrongly posed because of a semantic fallacy. Such statements as "High anxiety goes with erratic behavior" seem to imply functional relations, relations of great importance. But such a state- ment is ambiguous. From the traditional viewpoint, it means that persons above the mean of a group show more erratic behavior than those below the mean. But no such relative statement is meaningful when we are dealing with interacting phenomena -- and nearly all behavioral and educational phenomena interact. It is necessary to think through the problem of formulating re- search questions so that the answers will enlarge understanding rather than obscure reality under false generalizations. (^) So far, in work directly and indirectly connected with this project, we have made some progress toward identifying blind spots in present methods. But this paper re- presents really only a beginning and much work remains. It21221. To put the ATI problem formally: Assume that a certain set of outcomes from an educational program is desired. Consider any particular instructional treatment. In^ what^ manner^ do^ the^ characteristics^ of^ learners affect the extent to which they attain the outcomes from each of the treatments that mi ht be considered? Or, considering a particular learner, which treatment is best for him? Outcomes are plural. Any^ activity^ affects^ many^ adopects^ of^ the^ person. A method optimal to attain one may have a small effect or even a detrimental effect on another. This project set out, then, with the aim of providing a survey of the work'on ATI, with particular reference to education, so as to clarify
8
The social and ziLl_.otritontext
Darwinian theory, especially^ as^ interpreted^ by^ Spencer^ and^ Galton, emphasized "survival of the fittest" as a natural law. So long as man- kind was certain to progress, the most highly evolved nation and the most highly capable within that nation became the model. There^ followed logically Galton's emphasis on selection according to merit and the con- cept of a single rank-ordering, a 1. Selection was consistent with the Darwinian emphasis on competition -- among species and against natural hazards. Selection^ by^ test^ made^ the elimination less brutal by changing it to a short, sharp shock. This was a period_of laissezlfaire; education and social status were goods for which persons'would compete, and objective tests only enabled all the likely winners to get into the competition. In^ the context of 1860, Galton's proposals were liberalizing, as they substituted merit for privilege as the basis for preferment. They^ opened^ up^ "equality of opportunityl but it was an equality of opportunity to compete. The terms of the competition were firmly fixed. It^ was^ assumed^ that^ the^ compeft titive grind of the school system was an adequate basis for finding the best. As^ Seeley^ puts^ it,^ it^ mas^ not^ a^ society^ that^ eliminated^ slavery
- but a society'where a slave could rise to be an owner of slaves. Selecting the one type of talent bestlitted to survive in the schools was in the end a conservative influence. By^ reducing^ the^ extent^ to^ which.., the schools had to deal with pupils other than the kind they handled best, selection made it less necessary for the schools to invent methods for deal- ing with other kinds of talent. Single-rank-order selection is a meritocracy, only *a shade less con- servative than the aristocratic selection it replaced. It^ fits^ only^ a talent-surplus society. A^ developed^ society^ can^ use^ trained^ persons^ in large numbers, but has almost no way to use untrained manpower. The social planner must concern himself not with running a fair^ com- petition but with running a talent-development operation that will^ bring everyone to his highest level of contribution (with^ due^ regard^ to^ dis- tributional requirements of the society). The^ complex^ technical^ society needs a high percentage of persons in advanced occupations to maintain economic growth and standards of living. Moreover,^ any^ disadvantaged segment is a source of social chaos.
9
The traditional approach of schools has been to select by attain- ment. (^) Whoever has a good school record to date is favored in the next stage, being admitted to the program that gives more status and perhaps teaches more. Thus^ educational^ careers^ tend^ to^ diverge. Early^ bloomers ate favored. Those who do not fit the school as it is are shunted to lower status at each choice point. The single-rank-order principle loses a large pool of talented persons who are in their way much more excellent (along the lines of dexterity, or leadership, or musical insight, say) than those in the top adademic quarter. But^ multivariate^ selection^ programĀ§^ using^ tests that are excellently valid for these specialized talents would simply flood the advanced school with prospective failures, so long as the educational methods have evolved to fit only persons high in verbal- academic accomplishments. Identification^ of^ the^ talented^ is^ not^ the basis problem of aptitude testing; the basic problem is to identify those who will do well in, and at the end of, a talent-development program. Something similar can be said regarding training for skilled jobs, where the training is often verbally loaded though the job is not. Development of aptitude measures and educational methods should be a mutually supporting system, with educational programs designed for the' student who does not fit the conventional school and classifica- tion procedures designed to choose the right participants for each such program. The old model says: the institution is given, pick persons who fit it. The necessary model says: design^ enough^ treatments^ so that everyone will be able to succeed in one of them. That is a different sort of "equality" entirely. The successors to Spencer, led by Ward, fostered a Social Darwinism that was essentially a program of environmental tmprovement. And^ this involved the idea that some environments are better than others, i.e., the single-rank-order concept was applied to conditions. Corwin's remarks (1950) put the contrast in views succinctly: "We are confronted with two interpretations of evolution for social application: The^ Spencerian,^ laissez-faire^ inter- pretation and the reformist interpretation. Which^ one^ was^ best warranted by the Darwinian doctrine of biological evolution? Inasmuch as Datwin centers his attention upon the struggle for existence among creatures and treats the environment in which this struggle takes place either as relatively inert or as changing in response to factors beyond human control, the answer must undoubtedly be in favor^ of^ the^ Spencerian^ inter- pretation.
