An Impact Assessment of an Educational Pilot Project with Sisimpur and the Bangladesh Ministry of Primary and Mass Education (MOPME)

This is an evaluation of Sisimpur, an adaptation co-production of Sesame Street, the educational television series broadcast in the United States (US), localised by Bangladeshi producers to meet the needs of children in Bangladesh. (Please see Related Summaries below for further details, but, in brief:) Sisimpur is based on a Statement of Educational Objectives developed in consultation with Bangladeshi educational experts from a range of fields, including early child development, literacy, mathematics, science, and culture. Each segment of the series presents educational messages that are drawn from these objectives. Sisimpur aims to not only help prepare children for school by exposing them to basic skills but also to provide them with knowledge in the realms of socio-emotional development, life skills, and cultural understanding. In partnership with the Ministry of Primary and Mass Education (MOPME), the Sisimpur outreach team incorporated educational outreach materials into a 3-month curriculum that has been implemented at 16 registered non-government primary schools.

The prime objectives of the study were to assess the educational impact of the intervention (curriculum and materials) on children's school-readiness and the efficacy of the teacher training. The study was conducted among 3 participant groups: children (boys and girls aged 3 to 7 years), caregivers/parents, and MOPME school teachers who received training on Sisimpur. Children were assigned to 3 groups: a group that received Sisimpur print materials (Exp 1), a group that received print materials and DVDs (Exp 2), and a control group that did not receive Sisimpur materials. They were assessed on their skills prior to and after the intervention. Teachers participating in the intervention received training in implementing the curriculum and using the materials and activities in their classrooms; they participated in exit and in-depth interviews to gather feedback on the usefulness of the materials and training workshops. Field data were collected from May 2012 to December 2012.

In short, the project had a positive impact on some areas of children's knowledge, including school-readiness. This primarily occurred among those who were exposed to print materials only.

For example, compared to children in the control group, those who were in Exp 1 had higher scores in numeracy, knowledge of shapes, knowledge of the environment, and school readiness (as measured by the School Readiness Index). These findings were over and above the influence of key demographic factors such as family income, caregiver education, and the child's baseline performance. Evidence of impact was not found with literacy and health, hygiene, and nutrition outcomes. In most instances, children in Exp 2 (print + DVD) did not do significantly better than those in the control group. Findings with Exp 2 were rather inconsistent, and it is unclear why that was the case. It is possible that implementation in the Exp 2 schools was uneven.

Among the literacy materials, teachers used the workbook (Mojai Mojai Likhi O Pori) the most because children liked it, and it contained pictures, words, and letters. The Big Book (sobkhanei shikte pari), puzzle (Cobi-Shobdo-Borno), and matching cards (Chobi O Shobdo) were the least used because teachers did not have time to use them in class. The most used pre-math material was the Big Book (Borgo Raja O Trivuj Rani); the least used material was the domino game because children had some difficulty understanding the game. Among art, environment, and health materials, the most used was the art book (Ronge Rangao), flash cards (Shastho O Porissonota), and the Story Quilt (Golpo Khatha). The board game and the Big Book (Pani) were not used much because there was some difficulty in understanding how to use them or because children were less interested in those materials.

"It is important to note that the project materials were originally designed to be used among children and caregivers from disadvantaged backgrounds in informal settings. While the materials were easily integrated into a classroom setting, it is possible that the project would have had greater impact if materials were specifically designed for the MOPME classroom and addressed a range of basic as well as more advanced skills and knowledge."

According to the report, teachers felt that it was easy to integrate Sisimpur materials in their classroom curriculum. They reported using the materials daily. Classroom observations confirmed that teachers used the materials in class, but only a few materials were used and not all materials were available. Not all materials were used equally; their use depended on the availability of time and their complexity (children often did not understand the rules of some of the games, for example).

Teachers found the Sisimpur materials to be enjoyable for both teachers and children, effective for teaching, easy to understand and use; they also reported feeling confident in using the materials in the classroom. They thought that the project was helpful for early childhood education in general, and they introduced child-friendly materials into the classroom. Teachers felt that they learned a great deal through the project and that children learned many things through the materials.

The training workshops also received positive reviews - e.g., the teachers said that the trainers addressed participants' questions on different Sisimpur materials and teaching methods. "It was also observed that discussion was very useful in understanding [the] role of the community in creating awareness about early childhood development. Almost every teacher agreed that Sisimpur printed materials and DVDs were very attractive, high quality, and very easy to use. They also noted that discussions about the objectives of training, early childhood development, understanding the techniques to conduct lessons for children, and understanding the importance of pre-primary schooling were done properly. Most teachers liked group play during the training and mentioned the cooperativeness of instructors. They suggested increasing the duration of training, having more frequent trainings (at 2-3 month intervals), ensuring that participants have the opportunity to play games with a group, and have more time for demonstrating the materials".