Too Little, Too Late: Improving Post-Primary Learning Outcomes in India

Context

Many of us in the education and research space are familiar with the Annual Status of Education Reports (ASER) surveys and reports which have highlighted the gravity of the learning crisis in India for the past 15 years. The reports have consistently shown that many children in primary school continue to lack basic skills in reading and arithmetic, which are foundational for learning and acquiring knowledge in the school system. In 2017, the report showed that more than half the students in the age group of 14-18 years were unable to perform simple mathematic operations like division and a fourth were unable to read basic texts fluently in their own language (ASER, 2017). It would be natural to presume that this must be the reality in backward rural areas. But this could not be possibly be the case at the heart of India’s rapidly developing metropolitan cities….right?

Wrong! We remember how struck we were by this fact when we surveyed students in public schools in Chennai. Even in the capital city of one of India’s most literate states, Grade 9 students on average could not read paragraphs or do basic mathematics like multiplication or division. They were being taught quadratic equations based on the prescribed school syllabus, and yet, over half the students could not do basic mathematics. A fraction could still not even recognise single digit numbers. Even though the schools were clean, the students and teachers very warm and welcoming, the classrooms had projectors and the students even had their own laptops (with the ever-familiar face of the Chief Minister smiling at you), there was a learning crisis that was unfolding in the heart of the city, and had gone largely unnoticed by the populace at large.

It was in this context that our implementation partner, Avanti Fellows (supported by the MacArthur Foundation), had an uphill task ahead of them as they launched their innovative, blended-learning after-school programme (described below) for teaching Science and Math to Grade 9 students. The programme was enthusiastically supported by the Chennai Municipal Corporation (CMC) since the crucial Grade 10 national exams were around the corner and these students had but a couple of years to prepare for them. The scores on these national exams would impact students’ ability to continue schooling, determine their ability to pursue subjects such as Science and be admitted to colleges, and even influence their early employment opportunities.

Programme

Building on a few years of pilots in Chennai schools, Avanti launched an intensive remedial education program for Grade 9 in CMC schools in 2015 which included trained facilitators, technology aids and parent counseling, along with an emphasis on peer learning and team work. Avanti’s pedagogy incorporates features from Eric Mazur’s peer instruction and collaborative learning pedagogy (Fagen et al., 2009; Schell et al., 2013; Zhang et al., 2017) and other blended learning programs. The program was adapted to low-resource settings for teaching Maths and Science, using basic technology available in government secondary schools in India, such as computers (without internet). Conducted after school for an hour for five days a week, Avanti classes run for 40 weeks in an academic year, led by trained facilitators rather than teachers. Facilitators are usually fresh graduates who are required to have a Bachelor’s degree, preferably (but not necessarily) in STEM, and do not having teaching certification or qualifications. They are primarily trained on facilitating classroom sessions which rely on peer learning, team work and student collaboration. The facilitators rely on pre-recorded video lectures, presentations, worksheets and other academic materials for Math and Science in the regional language and context. All instruction is in the local language, even if the school medium is in English, in order to make it more accessible for students.

Research Design

We were tasked with the responsibility of evaluating the effectiveness of the program across 12 schools (the budget and implementation constraint that Avanti faced). In partnership with the CMC, 24 eligible public schools across Chennai were selected for the study, and 12 schools were randomly allocated to receive Avanti’s program for a year. These 12 schools were selected by a paper lottery conducted at the CMC. Since the design of the Randomised Controlled Trial (RCT) ensured that a school getting Avanti’s program was by chance, any difference in student outcomes in these schools, as compared to those who did not get Avanti’s program, could credibly be attributed to the program itself.

To analyse the impact of the program, we collected data from over 1000 students in our sample on their basic learning competencies (measured by ASER), surveys on their time use and participation in the program, and most importantly, their scores on the national exam in Grade 10 one year later. As noted earlier, a stark revelation from our baseline survey was the fact that though these students were in Grade 9, their learning competencies were equivalent to those that would normally be taught in Grades 2 or 3.

Results

We now turn to discussing the results of our programme evaluation. First, we find that Avanti’s program was a resounding “success” when it came to improving basic learning outcomes of the students. The effect sizes were large (>0.5 standard deviations), and comparable to other remedial education programs that had been evaluated in the Indian context. Furthermore, it showed a promising potential for delivering programs that relied on basic technology, combined with in-person facilitation, collaboration and teamwork. However, a more sobering finding was that these impressive gains in basic learning failed to translate into any improvement on students’ scores on the national Grade 10 exam.

Furthermore, using our detailed data, we are able to show that in fact Avanti’s programme had no effect on the time students spent studying outside the school, or the take-up of private tuitions, or even the support they got from their parents. Instead, we conclude that the gap between where Grade 9 students are in terms of their academic learning (Grade 2 or 3 level) is far from what the exams at grade-level (Grades 9 and 10) actually require. So even though Avanti’s program is not “too little” in improving basic learning, it is definitely “too late” at this stage to move the needle sufficiently on impacting grade-level learning outcomes. This indicates that such programmes targeted at the post-primary level may not be sufficient to overcome the accumulated learning deficiencies at grade-level.

Concluding Thoughts

These results are in line with other recent studies such as Beg et al. (2020), who also find that learning outcomes do not translate into gains in official exams in Pakistan, a context similar to India. As Pritchett and Beatty (2012) note, the regular grade level curriculum may be excessively difficult for learners in these settings, who have continued to accumulate deficiencies over many years and still to need to build on basic skills.

Our findings indicate the need to rethink post-primary educational initiatives in the context of high learning gaps and the urgency to address learning levels earlier at the primary level itself. These models also reinforce the importance of models of ’Teaching at the Right Level’ (Banerjee et al., 2016), and ensuring that learning is driven based on the child’s needs rather than the dictates of the curriculum.

Unless we ensure that India’s youth, which is the largest population of youth in the world, can reach adulthood with foundational skills and knowledge, the long-awaited demographic dividend in India will remain elusive. We hope results like these can hold important lessons to inform the implementation of India’s recent National Education Policy 2020, which highlights the importance of foundational numeracy and literacy, and the role of education technology for achieving educational goals.

References

Banerjee, Abhijit V., Rukmini Banerji, James Berry, Esther Duflo, Harini Kannan, Shobhini Mukherji, Marc Shotland, and Michael Walton, “Mainstreaming an Effective Intervention: Evidence from Randomized Evaluations of Teaching at the Right Level in India,” Technical Report, National Bureau of Economic Research 2016.

Beg, Sabrin, Adrienne M. Lucas, Waqas Halim, and Umar Saif, “Engaging Teachers with Technology Increased Achievement, Bypassing Teachers Did Not,” 2020.

Fagen, Adam P., Catherine H. Crouch, and Eric Mazur, “Peer Instruction: Results from a Range of Classrooms,” The Review of Economics and Statistics, 2009, 91 (3), 437–456.

Pratham, “Annual Status of Education Report,” Technical Report, Pratham 2016.

Pritchett, Lant and Amanda Beatty, “The Negative Consequences of Overambitious Curricula in Developing Countries,” American Economic Journal: Applied Economics, 2012.

Schell, Julie, Brian Lukoff, and Eric Mazur, “Catalyzing Learner Engagement Using Cutting-Edge Response Systems in Higher Education,” 2013.

Zhang, Ping, Lin Ding, and Eric Mazur, “Peer Instruction in Introductory Physics: A Method to Bring about Positive Changes in Students Attitudes and Beliefs,” Physical Review Physics Education Research, 2017.

Author Information: Diva Dhar, University of Oxford and Gaurav Chiplunkar, University of Virginia

Suggested Citation: Diva Dhar and Gaurav Chiplunkar. 2021. 'Too Little, Too Late: Improving Post-Primary Learning Outcomes in India', Think Pieces Series No. 24. Education.SouthAsia (https://educationsouthasia.web.ox.ac.uk/).