Project LISTEN

Project LISTEN was led by (now Emeritus) Professor David 'Jack' Mostow,[http://www.cmu.edu/scs/robotutor/people/team-leader.html "Professor Jack Mostow"] who currently leads Carnegie Mellon's "RoboTutor" team in the Global Learning XPRIZE competition.[http://cmurobotutor.org "Carnegie Mellon University's RoboTutor XPRIZE Team"] Project LISTEN was supported by the Defense Advanced Research Projects Agency through DARPA Order 5167, the National Science Foundation under ITR/IERI Grant No. IEC-0326153,[http://drdc.uchicago.edu/community/project.php?projectID=102 “Integrating Speech and User Modeling in a Reading Tutor that Listens"] Grant No. REC-0326153 from the U.S. Department of Education’s Institute of Education Sciences[http://ies.ed.gov/search/?q=Project+Listen Institute of Education Sciences] and under Grants R305B070458, R305A080157 and R305A080628, and by the Heinz Endowments.[http://www.heinz.org/ Heinz Endowments] Project LISTEN's purpose was to develop, evaluate, and refine an intelligent tutor to listen to children read aloud, and help them learn to read.

As part of the research and testing, Project LISTEN's Reading Tutor has been used with positive results by hundreds of children in the United States, Canada, and other countries.{{cite web |url=https://www.cs.cmu.edu/~listen/index.html |title=Project LISTEN A summary |date=February 7, 2013 |work=Project Listen summary}} (See Prototype Testing below.) Results indicated that often the students whose initial proficiency was lowest benefited most from the Reading Tutor. Of particular interest was the strong performance of the Reading Tutor for English Language Learners.{{Cite journal |url=http://news.ubc.ca/ubcreports/2005/05aug04/box.html |title=A Magic Reading Box: New literacy software delivers amazing results among Vancouver grade schoolers who speak English as a second language |journal=UBC Reports |volume=51 |issue=8 |date=August 4, 2005}}

Project Listen fits well into Carnegie Mellon University's [https://www.cmu.edu/simon/ Simon Initiative], whose goal is to use learning science research to improve educational practice. As noted in the History of the Simon Initiative, "The National Science Foundation included Project LISTEN’s speech recognition system as one of its top 50 innovations from 1950-2000."[https://www.cmu.edu/simon/what-is-simon/history.html "The Simon Initiative"]

The goal of the Reading Tutor is to make the student experience of learning to read using it as effective or more effective than being tutored by a human coach - for example, as described at the Intervention Central website.[http://www.interventioncentral.org/academic-interventions/reading-fluency/assisted-reading-practice/ Assisted reading example] A child selects an item from a menu listing texts from a source such the Weekly Reader or authored stories. The Reading Tutor listens to the child read aloud using Carnegie Mellon’s Sphinx – II Speech Recognizer{{cite web |url=http://cmusphinx.sourceforge.net/ |title= CMU Sphinx Speech Recognition Toolkit|work=CMU Sphinx}}{{cite web |url=https://www.cs.cmu.edu/~archan/s_info/Sphinx2/sphinx2.html |title= CMU Sphinx-II User Guide|work=CMU Sphinx}} to process and interpret the student's oral reading. When the Reading Tutor notices a student misread a word, skip a word, get stuck, hesitate, or click for help, it responds with assistance modeled in part on expert reading teachers, adapted to the capabilities and limitations of technology.{{cite web |url=http://www.hcii.cmu.edu/research/project-listens-reading-tutor |title= Project Listen Description|work= HCI Project Listen}}

The Reading Tutor dynamically updates its estimate of a student’s reading level and picks stories a bit harder (or easier) according to the estimated level; this approach allows the Reading Tutor to aim for the zone of proximal development, that is, to expand the span of what a learner currently can do without help, toward what he or she can do with help.

The Tutor also scaffolds (provides support for) key processes in reading. It explains unfamiliar words and concepts by presenting short factoids (that is, comparisons to other words). It can provide both spoken and graphical assistance when the student has a problem. The Tutor represents Visual speech using talking mouth video clips of phonemes. It assists word identification by previewing new words, reading hard words aloud, and giving rhyming and other hints.

Detailed data on the interactions is saved in a database, and data-mining has been used to improve the Reading Tutor and investigate research questions.

Project LISTEN trials demonstrated usability, user acceptance, effective assistance, and pre-to-post-test gains. A number of controlled studies extended over several months, with student use of 20 minutes per day. Use of the Reading Tutor produced higher comprehension gains than current methods. To ensure there was no third variable involved, different treatments were compared within the same classrooms, with randomized assignment of children to treatments, stratified by pretest scores. Valid and reliable measures (Woodcock.1998){{cite web |url=http://www.education.com/definition/woodcock-reading-mastery-test-revised/ |title= WOODCOCK, R.W. 1998. Woodcock Reading Mastery Tests - Revised (WRMT-R/NU). American Guidance Service, Circle Pines, Minnesota.}} were used to measure gains between pre and post test.{{cite web |url=http://www.projectlisten.org/research/ |title= Project Listen Research Basis}} Data gathered during each trial was used to improve the efficacy of the tutor.

Various controlled studies were carried out as the Reading Tutor evolved, for example,

1. Pilot Study(1996–97){{cite web |url=https://www.cs.cmu.edu/~listen/pdfs/Mostow_Aist_pui97_final.pdf |title= When Speech Input is Not an Afterthought: A Reading Tutor that Listens, In Workshop on Perceptual User Interfaces, Banff, Canada, October, 1997}}
2. Within-classroom comparison(1998)[https://www.cs.cmu.edu/~listen/research.html#_ENREF_70/ Within-classroom comparison(1998)]{{cite journal |last1=Mostow |first1=Jack |last2=Aist |first2=Greg |last3=Burkhead |first3=Paul |last4=Corbett |first4=Albert |last5=Cuneo |first5=Andrew |last6=Eitelman |first6=Susan |last7=Huang |first7=Cathy |last8=Junker |first8=Brian |last9=Sklar |first9=Mary Beth |last10=Tobin |first10=Brian |title=Evaluation of an Automated Reading Tutor That Listens: Comparison to Human Tutoring and Classroom Instruction |journal=Journal of Educational Computing Research |date=22 July 2016 |volume=29 |issue=1 |pages=61–117 |doi=10.2190/06AX-QW99-EQ5G-RDCF|s2cid=46073812 }}
3. Comparison to human tutors(1999-2000){{citation |url=https://www.cs.cmu.edu/~listen/pdfs/2000-2001_Quaker_Valley_study_SSSR2002_paper.pdf|author1=Jack Mostow |author2=Greg Aist |author3=Paul Burkhead |author4=Albert Corbett |author5=Andrew Cuneo |author6=Susan Rossbach |author7=Brian Tobin |title=Independent practice versus computer-guided oral reading: Equal-time comparison of sustained silent reading to an automated reading tutor that listens}}{{cite web |url=https://www.ri.cmu.edu/pub_files/pub4/mostow_jack_2001_1/mostow_jack_2001_1.pdf |title=A controlled evaluation of computer- versus human-assisted oral reading}}A controlled evaluation of computer-versus human-assisted oral reading. Tenth Artificial Intelligence in Education (AI-ED) Conference, J.D. MOORE, C.L. REDFIELD and W.L. JOHNSON, Eds. Amsterdam: IOS Press, San Antonio, Texas, 586-588.
4. Equal-time comparison to Sustained Silent Reading (2000-2001){{Cite journal | doi=10.2190/EC.49.2.g|title = Computer-Guided Oral Reading versus Independent Practice: Comparison of Sustained Silent Reading to an Automated Reading Tutor That Listens| journal=Journal of Educational Computing Research| volume=49| issue=2| pages=249–276|year = 2013|last1 = Mostow|first1 = Jack| last2=Nelson-Taylor| first2=Jessica| last3=Beck| first3=Joseph E.|s2cid = 62382102}}

Since 2005, researchers both within and outside Project LISTEN have conducted and published controlled studies of the Reading Tutor. (See list in{{cite web |url=https://www.cs.cmu.edu/~listen/pubs.html |title= Published Studies of Project Listen}}).

Project Listen has received global recognition and many awards:

• 1994: Outstanding Paper Award at the Twelfth National Conference on Artificial Intelligence{{cite web |url=https://www.cs.cmu.edu/~listen/aaai94-online.ps|author1=Jack Mostow |author2=Steven F. Roth |author3=Alexander G. Hauptmann |author4=Matthew Kane |title=(Outstanding Paper) A Prototype Reading Coach that Listens}}
• 1998: Represented the Computing Research Association (CRA) at the Coalition for National Science Funding Exhibit (CNSF) for Congress
• 2000: The National Science Foundation included Project LISTEN in its "Nifty Fifty" list of top 50 innovations from 1950-2000.[https://www.cmu.edu/simon/what-is-simon/history.html The Simon Initiative]{{cite web |url=https://www.nsf.gov/about/history/nifty50/next25.jsp |title=Nifty 50}}
• 2002: Distinguished Finalist for the International Reading Association's Outstanding Dissertation of the Year Award{{cite web |url=https://www.cs.cmu.edu/~listen/pubs.html#IRA2002award |title= Helping Children Learn Vocabulary during Computer-Assisted Oral Reading}}
• 2002: Project LISTEN videos included in PBS series "Reading Rockets"{{cite web |url=https://www.cs.cmu.edu/~listen/mm.html#RR2002 |title= Project Listen Videos on Reading Rockets PBS Series}}
• 2008: Best Paper Award at the 9th International Conference on Intelligent Tutoring Systems.{{Cite book|author=Beck, J. E., Chang, K.-m., Mostow, J., & Corbett, A. |title=Introducing the Bayesian Evaluation and Assessment methodology |pages=383–394|url=https://www.springer.com/us/book/9783540691303|isbn=9783540691303 |publisher=Springer |year=2008 |series=Programming and Software Engineering }}{{cite web |url=https://www.cs.cmu.edu/~listen/pdfs/beck%20Does%20help%20help.pdf |title=(Best Paper Award) Does help help? Introducing the Bayesian Evaluation and Assessment methodology}}
• 2008: Best Paper Nominee at the 9th International Conference on Intelligent Tutoring Systems{{citation |doi=10.1007/978-3-540-69132-7_39|chapter = How Who Should Practice: Using Learning Decomposition to Evaluate the Efficacy of Different Types of Practice for Different Types of Students|title = Intelligent Tutoring Systems|volume = 5091|pages = 353–362|series = Lecture Notes in Computer Science|year = 2008|last1 = Beck|first1 = Joseph E.|last2 = Mostow|first2 = Jack|isbn = 978-3-540-69130-3}}{{cite web |url=https://www.cs.cmu.edu/~./listen/pdfs/beck%20its%20learning%20decomp.pdf |title= (Best Paper Nominee) How who should practice: Using learning decomposition to evaluate the efficacy of different types of practice for different types of students }}
• 2011: Best Paper Nominee at the 15th International Conference on Artificial Intelligence in Education.{{cite journal|author1=Jack Mostow |author2=Kai-min Chang |author3=Jessica Nelson |title=Toward Exploiting EEG Input in a Reading Tutor|journal=International Journal of Artificial Intelligence in Education|volume=22 |issue=1–2 |pages=230–237|url=http://content.iospress.com/articles/international-journal-of-artificial-intelligence-in-education/jai033|doi=10.3233/JAI-130033 |date=January 2013 }}{{cite web |url=https://www.cs.cmu.edu/~listen/pdfs/67380230.pdf |title=(Best Paper Award) Toward Exploiting EEG Input in a Reading Tutor, Mostow, J., Chang, K.-m., & Nelson, J., Proceedings of the 15th International Conference on Artificial Intelligence in Education, Auckland, NZ, 230-237, 2011}}
• 2012: Best Paper Award at the 25th Florida Artificial Intelligence Research Society Conference (FLAIRS-25).{{cite journal|author1=Sunayana Sitaram |author2=Jack Mostow|title=Mining Data from Project LISTEN's Reading Tutor to Analyze Development of Children's Oral Reading Prosody|journal=Proceedings of the Twenty-Fifth International Florida Artificial Intelligence Research Society Conference, AASI Digital Library|url=http://www.aaai.org/Library/FLAIRS/flairs12contents.php}}{{cite web |url=https://www.cs.cmu.edu/~listen/pdfs/MiningProsodyModels.pdf |title=(Best Paper Award) Mining Data from Project LISTEN's Reading Tutor to Analyze Development of Children's Oral Reading Prosody}}
• 2012: Best Student Paper Award at the 5th International Conference on Educational Data Mining{{cite journal |author=Xu, Y., Mostow, J.|title=Comparison of methods to trace multiple subskills: Is LR-DBN best? |journal=Proceedings of the Fifth International Conference on Educational Data Mining, Chania, Crete, Greece.|url=http://files.eric.ed.gov/fulltext/ED537074.pdf}}{{cite web |url=https://www.cs.cmu.edu/~listen/pdfs/LR-DBN%20vs%20CKT.pdf |title=(Best Student Paper Award) Comparison of methods to trace multiple subskills: Is LR-DBN best?}}
• 2013: Finalist for Best Paper Award at the 16th International Conference on Artificial Intelligence in Education, Memphis, TN{{cite book|title=Proceedings of the 16th International Conference on Artificial Intelligence in Education, Memphis, TN 2013|publisher=Springer US|chapter=Comparing Student Models in Different Formalisms by Predicting their Impact on Help Success|author=Lallé, S., Mostow, J., Luengo, V., & Guin, N.page=161-170|date=22 June 2013|isbn=9783642391125|chapter-url=https://books.google.com/books?id=4Fu5BQAAQBAJ&pg=PA161}}{{cite web |url=https://www.cs.cmu.edu/~listen/pdfs/AIED2013%20Lalle%2079260161.pdf |title=(Finalist for Best Paper Award) Comparing Student Models in Different Formalisms by Predicting their Impact on Help Success}}

• Intelligent tutoring system
• Speech recognition
• Artificial Intelligence
• Educational data mining
• Learning Sciences

{{Reflist}}

• Project LISTEN Home Page http://www.projectlisten.org/
• Project LISTEN Publications https://www.cs.cmu.edu/~listen/pubs.html

Category:Carnegie Mellon University