Neal Martin Kingston, Ph.D.

School of Education and Human Sciences - Educational Psychology, Center for Educ Testing and Evaluation, Psychology & Research in Education
University Distinguished Professor
Primary office:
785.864.9705
Joseph R. Pearson Hall, 605A
University of Kansas
1122 West Campus Rd
Lawrence, KS 66045-3101


Summary

Neal Kingston, Ph.D., is a University Distinguished Professor in the department of Educational Psychology at the University of Kansas, in which he also serves as the Director of Graduate Studies and coordinator of the Research, Evaluation, Measurement, and Statistics track of the Educational Psychology and Research Program and Director of the Achievement and Assessment Institute. His research focuses on large-scale assessment, with particular emphasis on how it can better support student learning through the use of learning maps and diagnostic classification models. He has served as principal investigator or co-principal investigator for over 180 research grants. Of particular note was the Dynamic Learning Maps Alternate Assessment grant from the US Department of Education, which was at that time was the largest grant in KU history and which currently serves 21 state departments of education. Other recent testing projects include the Kansas Assessment Program, Career Pathways Collaborative, and Adaptive Reading Motivation Measures.

Dr. Kingston is known internationally for his work on large-scale assessment, formative assessment, and learning maps. He has served as a consultant or advisor for organizations such as the AT&T, College Board, Department of Defense Advisory Committee on Military Personnel Testing, Edvantia, General Equivalency Diploma (GED), Kaplan, King Fahd University of Petroleum and Minerals, Merrill Lynch, National Council on Disability, Qeyas (Saudi Arabian National Center for Assessment in Higher Education), State of New Hampshire, State of Utah, and the U.S. Department of Education, and Western Governors University.

Dr. Kingston also serves as Director of the KU Achievement and Assessment Institute (AAI). Dr. Kingston is responsible for five research centers with a staff of about 300 year-round staff and about 150 temporary employees.





Education

Ph.D., Educational Measurement, Teachers College, Columbia University
M.Phil., Educational Measurement, Teachers College, Columbia University
M.Ed., Educational Measurement, Teachers College, Columbia University
M.A., Psychology in Education, Teachers College, Columbia University
B.A., Liberal Studies (concentrations in Biology & Education), State University of New York

Teaching

Currently Dr. Kingston teaches a course in Classical Test Theory in the fall of odd numbered years, Item Response Theory in the spring of even numbered years, a course on meta-analysis in the fall of even numbered years, and an advanced seminar in a topic to be announced in the spring of odd numbered years.

Prospective Students
I am seeking one or two new doctoral students (previous masters degree not required) to start in fall 2022. Students will receive a graduate research assistant position that covers their tuition and fees and provides a salary. An interest in learning and applying test theory to better support student learning is of critical importance.
A link to the Achievement and Assessment Institute graduate student handbook is included for your reference: https://aai.ku.edu/sites/aai.ku.edu/files/docs/pdfs_general/GRA_Handbook...

Teaching Interests

  • Educational Measurement theory and practice
  • Instructionally embedded assessment
  • Learning maps
  • Meta-analysis

Research

The field of education moves slowly. Theoretical improvements often take 30-50 years before wide-spread implementation. Often research-based practices are crowded out by the fad of the day. Making the challenge of improving education even greater, sub-disciplines within education far too often work in isolation. Simple solutions that do not address the complexity of individual students or the dynamics of a classroom at best have little impact and too often have a negative impact. Such has been the case in large-scale assessment where the use of assessment to drive curriculum and instruction has had numerous negative consequences.

Coming to the University of Kansas gave me the opportunity to consider the fundamental issues of education from a broader perspective. As have many others, I had long realized that thinking about curriculum, instruction, and assessment needs to be integrated. However, few researchers attempted to develop models or theories to do this. I was impressed by the eorts of some, particularly the research trajectories of Susan Embretson and Kikumi Tatsuoka, but I remained frustrated regarding how incomplete this work was and how little impact it was having on federally mandated state assessment programs. This led me to develop three conference presentations in 2009 that served to focus my thinking. The first was presented at the National Council on Measurement in Education and was entitled, "What Have We Learned about the Structure of Learning from 30 Years of Research on Integrated Cognitive-Psychometric Models? Not Much." The second was presented at the American Educational Research Association Conference – The Efficacy of Formative Assessment: A Meta-Analysis. The third, presented at the National Conference on Student Assessment, was entitled, "Large-Scale Formative Assessment: Panacea, Transitional Tool, or Oxymoron."

In 2010 an opportunity presented itself and allowed me to solidify my thinking. The US Department of Education issued a request for proposals to develop a large-scale assessment system for students with significant cognitive disabilities –the approximately one percent of students with the greatest learning challenges. It was clear to me that such an assessment system needed to do far more than measure learning – it needed to facilitate learning. I identified six features that needed to be present to do this. They are as follows.

1. Comprehensive one-grained learning maps that guide instruction and assessment
2. A subset of particularly important nodes that serve as content standards to provide an organizational structure for teachers
3. Instructionally embedded assessments that reinforce the primacy of instruction
4. Instructionally relevant testlets that model good instruction and reinforce learning
5. Accessibility by design
6. Status and growth reporting that is readily actionable

No one had ever tried to develop a learning environment in this way. Comprehensive fine-grained learning maps did not exist. The concept of instructionally relevant assessment previously was unnamed and in its infancy. Clearly much research – both basic and applied – was necessary and this has become the focus of my research.

A closely related second area of research is assessment that supports the needs of learners who face educational or assessment challenges. This includes issues of test development and universal design and which have close ties to features 3-5 in the list above. I separate it as a research focus because it is also applicable to traditional testing programs.




Research Interests

  • Large-scale assessment
  • Computer-based testing
  • Diagnostic classification modeling
  • Learning maps
  • Test development
  • Score reporting

Selected Publications

Wang, W., & Kingston, N. M. (2019). Adaptive Testing with the Hierarchical Item Response Theory Model. [Journal Articles]. Applied Psychological Measurement, 43(1), 51–67.
Heritage, M., & Kingston, N. (2019). Classroom assessment and large-scale psychometrics: shall the twain meet? (a conversation with Margaret Heritage and Neal Kingston) [Journal Articles]. Journal of Educational Measurement. Published.
Embretson, S. E., & Kingston, N. M. (2018). Automatic Item Generation: A more Efficient Process for Developing Mathematics Achievement Items. [Journal Articles]. Journal of Educational Measurement, 55(1), 112–131.
Clark, A., Nash, B., Karvonen, M., & Kingston, N. M. (2017). Condensed Mastery Profile Method for Setting Standards for Diagnostic Assessment Systems. [Journal Articles]. Educational Measurement: Issues and Practice, 36(4), 5–15.
Kingston, N. M., Karvonen, M., Thompson, J. R., Wehmeyer, M. L., & Shogren, K. A. (2017). Fostering Inclusion of Students with Significant Cognitive Disabilities by Using Learning Map Models and Map-Based Assessments. [Journal Articles]. Inclusion, 5(2), 110–120.
Kingston, N. M., & Broaddus, A. (2017). The Use of Learning Map Systems to Support Formative Assessment in Mathematics. [Journal Articles]. Inclusion, 7(41). https://doi.org/10.3390/educsci7010041
Kingston, N. M., Karvonen, M., Bechard, S., & Erickson, K. (2016). The Philosophical Underpinnings and Key Features of the Dynamic Learning Maps Alternate Assessment. [Journal Articles]. Teachers College Record, 118(14).
Popham, W. J., Berliner, D. C., Kingston, N. M., Fuhrman, S. H., Ladd, S. M., Charbonneau, J., & Chatterji, M. (2014). Can today’s standardized tests yield instructionally useful data? Challenges, promises, and the state of the art. [Journal Articles]. Quality Assurance in Education, 22(4), 300–316.
Adjei, S., Selent, D., Heffernan, N., Pardos, Z., Broaddus, A., & Kingston, N. M. (2014). Refining Learning Maps with Data Fitting Techniques: Searching for Better Fitting Learning Maps. [Conference Proceedings]. In Pardos & Stamper (Eds.) The 2014 Proceedings of International Educational Data Mining Society.
Cho, H., Wehmeyer, M., & Kingston, N. M. (2013). FACTORS THAT PREDICT ELEMENTARY EDUCATORS’ PERCEPTIONS AND PRACTICE IN TEACHING SELF-DETERMINATION: Factors That Predict Educators’ Practice
[Journal Articles]. Psychology in the Schools, 50, 770–780. https://doi.org/10.1002/PITS.21707
Gu, F., Little, T., & Kingston, N. M. (2013). Misestimation of Reliability Using Coefficient Alpha and Structural Equation Modeling when Assumptions of Tau-Equivalence and Uncorrelated Errors are Violated [Journal Articles]. Methodology, 9, 30–40.
Cho, H., & Kingston, N. M. (2013). Why IEP Teams Assign Low Performers With Mild Disabilities to the Alternate Assessment Based on Alternate Achievement Standards. [Journal Articles]. Journal of Special Education, 47, 162–174.

KU Today
One of 34 U.S. public institutions in the prestigious Association of American Universities
44 nationally ranked graduate programs.
—U.S. News & World Report
Top 50 nationwide for size of library collection.
—ALA
5th nationwide for service to veterans —"Best for Vets: Colleges," Military Times