progress testing

Since its inception in the late 1970s at both Maastricht University and the University of Missouri–Kansas CityArnold L, Willoughby TL. 1990. The quarterly profile examination. Academic Medicine 65(8):515–516. independently, the progress test of applied knowledge has been increasingly used in medical and health sciences programs across the globe. They are well established and increasingly used in medical education in both undergraduate and postgraduate medical education. They are used formatively and summatively.

The progress test is currently used by national progress test consortia in the United Kingdom,Swanson, D. B., Holtzman, K. Z., Butler, A., Langer, M. M., Nelson, M. V., Chow, J. W. M., et al. (2010). Collaboration across the pond: The multi-school progress testing project. Medical Teacher, 32, 480-485. Italy, The Netherlands,Schuwirth, L., Bosman, G., Henning, R. H., Rinkel, R., & Wenink, A. C. G. (2010). Collaboration on progress testing in medical schools in the Netherlands. Medical Teacher, 32, 476-479. in Germany (including Austria),Nouns, Z. M., & Georg, W. (2010). Progress testing in German speaking countries. Medical Teacher, 32, 467-470. and in individual schools in Africa,Aarts, R., Steidel, K., Manuel, B. A. F., & Driessen, E. W. (2010). Progress testing in resource-poor countries: A case from Mozambique. Medical Teacher, 32, 461-463. Saudi Arabia,Al Alwan, I., Al-Moamary, M., Al-Attas, N., Al Kushi, A., ALBanyan, E., Zamakhshary, M., et al. (2011). The progress test as a diagnostic tool for a new PBL curriculum. Education for Health (December, Article No. 493). South East Asia,Mardiastuti, H. W., & Werdhani, R. A. (2011). Grade point average, progress test, and try outs test as tools for curriculum evaluation and graduates' performance prediction at the national board examination. Journal of Medicine and Medical Sciences, 2(12), 1302-1305. the Caribbean, Australia, New Zealand, Sweden, Finland, UK, and the USA.Freeman, A., van der Vleuten, C., Nouns, Z., & Ricketts, C. (2010). Progress testing internationally. Medical Teacher, 32, 451-455. The National Board of Medical Examiners in the USA also provides progress testing in various countries De Champlain, A., Cuddy, M. M., Scoles, P. V., Brown, M., Swanson, D. B., Holtzman, K., et al. (2010). Progress testing in clinical science education: Results of a pilot project between the National Board of Medical Examiners and a US medical School. Medical Teacher, 32, 503-508.International Foundations of Medicine (2011). Retrieved 20 July 2011, from http://www.nbme.org/Schools/iFoM/index.html The feasibility of an international approach to progress testing has been recently acknowledged Finucane, P., Flannery, D., Keane, D., & Norman, G. (2010). Cross-institutional progress testing: Feasibility and value to a new medical school. Medical Education, 44, 184-186. and was first demonstrated by Albano et al.Albano, M. G., Cavallo, F., Hoogenboom, R., Magni, F., Majoor, G., Manenti, F., et al. (1996). An international comparison of knowledge levels of medical students: The Maastricht progress test. Medical Education, 30, 239-245. in 1996, who compared test scores across German, Dutch and Italian medical schools. An international consortium has been established in Canada International Partnership for Progress Testing (2011). Retrieved 18 July 2011, from http://ipptx.org/ involving faculties in Ireland, Australia, Canada, Portugal and the West Indies.

The progress test serves several important functions in academic programs. Considerable empirical evidence from medical schools in the Netherlands, Canada, United Kingdom and Ireland, as well postgraduate medical studies and schools in dentistry and psychology have shown that the longitudinal feature of the progress test provides a unique and demonstrable measurement of the growth and effectiveness of students’ knowledge acquisition throughout their course of study

Bennett, J., Freeman, A., Coombes, L., Kay, L., & Ricketts, C. (2010). Adaptation of medical progress testing to a dental setting. Medical Teacher, 32, 500-502.

Boshuizen, H. P. A., van der Vleuten, C. P. M., Schmidt, H., & Machiels-Bongaerts, M. (1997). Measuring knowledge and clinical reasoning skills in a problem-based curriculum. Medical Education, 31, 115-121.

Coombes, L., Ricketts, C., Freeman, A., & Stratford, J. (2010). Beyond assessment: Feedback for individuals and institutions based on the progress test. Medical Teacher, 32, 486-490.

Dijksterhuis, M. G. K., Scheele, F., Schuwirth, L. W. T., Essed, G. G. M., & Nijhuis, J. G. (2009). Progress testing in postgraduate medical education. Medical Teacher, 31, e464-e468.

Freeman, A., & Ricketts, C. (2010). Choosing and designing knowledge assessments: Experience at a new medical school. Medical Teacher, 32, 578-581.

Schaap, L., Schmidt, H., & Verkoeijen, P. J. L. (2011). Assessing knowledge growth in a psychology curriculum: which students improve most? Assessment & Evaluation in Higher Education, 1-13.

van der Vleuten, C. P. M., Verwijnen, G. M., & Wijnen, W. H. F. W. (1996). Fifteen years of experience with progress testing in a problem-based learning curriculum. Medical Teacher, 18(2), 103-109.

.van Diest, R., van Dalen, J., Bak, M., Schruers, K., van der Vleuten, C., Muijtjens, A. M. M., et al. (2004). Growth of knowledge in psychiatry and behavioural sciences in a problem-based learning curriculum. Medical Education, 38, 1295-1301.Verhoeven, B. H., Verwijnen, G. M., Scherpbier, A. J. J. A., & van der Vleuten, C. P. M. (2002). Growth of medical knowledge. Medical Education, 36, 711-717.

As a result, this information can be consistently used for diagnostic, remedial and prognostic teaching and learning interventions. In the Netherlands, these interventions have been aided by the provision of a web-based results feedback system known as ProF Muijtjens, A. M. M., Timmermans, I., Donkers, J., Peperkamp, R., Medema, H., Cohen-Schotanus, J., et al. (2010). Flexible electronic feedback using the virtues of progress testing. Medical Teacher, 32, 491-495. in which students can compare their results with their peers across different total and subtotal score perspectives, both across and within universities.

Additionally, the longitudinal data can serve as a transparent quality assurance measure for program reviews by providing an evaluation of the extent to which a school is meeting its curriculum objectives.Verhoeven, B. H., Snellen-Balendong, H. A. M., Hay, I. T., Boon, J. M., Van Der Linde, M. J., Blitz-Lindeque, J. J., et al. (2005). The versatility of progress testing assessed in an international context: a start for benchmarking global standardization? Medical Teacher, 27(6), 514-520. The test also provides more reliable data for high-stakes assessment decisions by using measures of continuous learning rather than a one-shot method (Schuwirth, 2007). Inter-university progress testing collaborations provide a means of improving the cost-effectiveness of assessments by sharing a larger pool of items, item writers, reviewers, and administrators. The collaborative approach adopted by the Dutch and other consortia has enabled the progress test to become a benchmarking instrument by which to measure the quality of educational outcomes in knowledge. The success of the progress test in these ways has led to consideration of developing an international progress test.Schauber, S., & Nouns, Z. B. (2010). Using the cumulative deviation method for cross-institutional benchmarking in the Berlin progress test. Medical Teacher, 32, 471-475.

The benefits for all main stakeholders in a medical or health sciences programme make the progress test an appealing tool to invest resources and time for inclusion in an assessment regime. This attractiveness is demonstrated by its increasingly widespread use in individual medical education institutions and inter-faculty consortia around the world, and by its use for national and international benchmarking practices.

Progress tests provide a rich source of information: the comprehensive nature in combination with the cross-sectional and longitudinal design offers a wealth of information both for individual learners as well as for curriculum evaluations.

Progress Testing fosters knowledge retention: the repeated testing of the same comprehensive domain of knowledge means that there is no point testing facts that could be remembered if studied the night before. Long term knowledge and knowledge retention is fostered because item content remains relevant long after the knowledge has been learned. Progress Testing removes the need for resit examinations: every new test occasion is a renewed opportunity to demonstrate growth of knowledge.

Progress Testing allows early detection of high achievers: some learners perform (far) beyond the expected level of their phase in training (e.g. they might have had relevant previous other training) and, depending on their performance, individual and more speeded pathways through the curriculum could be offered.

Progress Testing brings stability in assessment procedures: curriculum changes, changes in content, have no consequence for the progress test provided the end outcomes are unchanged.

Progress Testing provides excellent benchmarking opportunities: progress tests are not limited to a single school nor to PBL curricula and evaluations can easily be done to compare graduates and the effectiveness of different curriculum approaches.

Naturally, there are disadvantages. The required resources for test development and scoring and the need for a central organization are two very important ones.

Scoring,Muijtjens AM, Mameren HV, Hoogenboom RJ, Evers JL, van der Vleuten CP. 1999. The effect of a ‘don’t know’ option on test scores: Number-right and formula scoring compared. Medical Education 33(4):267–275. psychometric procedures Shen L. 2000. Progress testing for postgraduate medical education: A four year experiment of American College of Osteopathic Surgeons Resident Examinations. Advances in Health Sciences Education: Theory and Practice 5(2):117–129 for reducing test difficulty variation and standard setting procedures Verhoeven BH, Snellen-Balendong HA, Hay IT, Boon JM, van der Linde MJ, Blitz-Lindeque JJ, Hoogenboom RJI, Verwijnen GM, Wijnen WHFW, Scherpbier AJJA, et al. 2005. The versatility of progress testing assessed in an international context: A start for benchmarking global standardization? Medical Teacher 27(6):514–520 are more complex in progress testing.

Finally progress tests do not work in heterogeneous programs with early specialization (like in many health sciences programs). In more homogenous programs, such as most medical programs, they work really well and pay off in relation to driving learning and use of resources.

Information from 2010+ (this list may not be complete or up to date, please add any other known progress test administrations)

University of Minho, School of Medicine, Portugal

Netherlands Group - Five medical faculties in the Netherlands (Groningen, Leiden, Maastricht, Nijmegen and VU Amsterdam) and additionally, the Ghent University in Belgium use the test

International Partnership for Progress Testing (IPPT) - undergraduate medical programs at McMaster University Medical School, Canada; University of Limerick, Ireland; University of Algarve, Portugal; University of Western Sydney, Australia

Progress Test Medizin (PTM), Charité-Universitätsmedizin Berlin, Germany (Germany – Berlin, Aachen, Cologne, Münster, Witten, Duisburg-Essen, Oldenburg, Hamburg, Neuruppin, Düsseldorf, Augsburg, Bielefeld; Austria – Graz, Innsbruck, Vienna, Linz)

NBME 1 (Barts, St. George’s London, Leeds and Queens University, Belfast UK)

NBME 2 (University of South Florida and Case Western Reserve University)

Southern Illinois University, Vanderbilt, University of New Mexico, Penn State, Texas Tech, Medical College of Georgia, University of Minnesota

University of Manchester School of Medicine, UK

Peninsula College of Medicine and Dentistry, UK

Cardiff University School of Medicine, Wales, UK

Swansea University Medical School, Wales, UK

University of Tampere, Finland

University of Helsinki, Finland

Karaganda State Medical University, Kazakhstan

Otago Medical School, New Zealand

Auckland Medical School, New Zealand

São Paulo City Medical School (UNICID), Brazil

University of Indonesia, Medical School

Catholic University of Mozambique

Pretoria, South Africa

CMIRA Program, Syrian-Lebanese Hospital Institute for Education and Research, Brazil

Alfaisal University - College of Medicine, Saudi Arabia

The College of Medicine at King Saud bin Abdulaziz University for Health Sciences (KSAU-HS), Saudi Arabia

Batterjee Medical College for Senescences & Technology Jeddah- Saudi Arabia

Sulaiman Alrajhi Colleges - School of Medicine, Albukairiyah city, Qassim region - Saudi Arabia

Source:

King Fisal University KFU - College of Medicine, Al-Ahsaa, Saudi Arabia

The College of Dentistry at King Saud bin Abdulaziz University for Health Sciences (KSAU-HS), Saudi Arabia

Flinders University - School of Medicine (post-graduate), Adelaide, Australia

Royal College of Surgeons in Ireland - Dublin, Ireland

Royal College of Surgeons in Ireland - Medical University of Bahrain

• Medical education

[http://www.alfaisal.edu Alfaisal University]

{{reflist}}

• [https://progress-test-medizin.charite.de/en/ Progress Test Medizin, Charité-Universitätsmedizin Berlin]
• [https://web.archive.org/web/20140314163359/http://med.qu.edu.sa/eng/index.php/progress-test-results/progress-test.html Progress test Saudi Arabia at Qassim College of Medicine]
• [https://web.archive.org/web/20130921050422/http://ivtg.nl/en/node/69 interuniversity Progress Test Medicine, the Netherlands]

Category:Tests

Category:Medical education