Evaluations that make a difference is a collection of 8 evaluation stories from around the world which is one of the first pieces of systematic research looking at factors that contribute to high quality evaluations that are used by stakeholders to improve programs and improve people’s lives. This initiative collected stories about evaluations that made a difference, not only from the perspective of the evaluators but also from the commissioners and users. The stories in this collection tell powerful stories about the findings in the evaluations and the ways the evaluations contributed to the impact of the programs. You may access the report and all the stories here, in English, Spanish, and French.
In these weekly posts, we will be sharing each story… Comments are very welcome!!
Just before dawn Ashish nervously waits for Rama at the village bus stop. As soon as he spots his girlfriend, they cautiously board the bus toward the city planning their future as a couple. But their plans are thwarted when a police officer interrupts their journey and starts interrogating them.
‘We just want to get married,’ Ashish tells the officer. ‘Rama’s parents were forcing her to marry a stranger.’
‘We have done nothing wrong,’ Rama adds. The police officer then intervenes as the teenagers, both 16, request him to let them proceed.
‘It’s a crime,’ he says. ‘You need to be at least 20 years old to get married. Child marriage is a criminal offense – your parents can be prosecuted if they’re forcing you to marry young and you can be prosecuted too if you marry now.’
The legal age for marriage in Nepal is 20 years without parental consent and 18 years with consent.
While Rama and Ashish are fictional characters from a weekly radio drama aired on more than 40 stations across the Himalayan republic, their story paints an accurate picture of the crude reality of child marriage that prevails across the country’s remote villages. Nearly 3 of every 10 girls between 15 and 19 are presently married, and 4 of 10 are married before the age of 18, according to national statistics.
Radio dramas like this one reflect the ethos and pathos of Nepalese society. One programme in particular, a weekly called Saathi Sanga Man Ka Kura (SSMK), which means ‘Chatting with My Best Friend’, has been reaching out to young girls and boys for 15 years. SSMK educates young people about issues that are generally kept quiet in conservative Nepalese society. Started in 2001 with support from the United Nations Children’s Fund (UNICEF) and managed by Equal Access Nepal for the past decade, the 45-minute show has 7.2 million loyal listeners and receives more than 1,200 letters and 2,000 text messages each month.
Narrowing the focus
Before the introduction of the electronic media, street dramas and stage shows were popular throughout Nepal. Mostly mythological and sometimes satirical skits in local dialects emerged as a major form of entertainment in villages.
While this trend might have declined in cities, Ghanshyam Kumar Mishra, a radio producer from Nepal’s southern town of Janakpur in Dhanusa district, says dramas in local languages are still popular and preferred in Nepal’s rural pockets. Most people in these areas are low-income farmers with little educational backgrounds and no source of information other than radio, which is also one of the cheapest forms of entertainment. Therefore radio dramas, according to Mishra, are one of the best media through which to inform the masses.
In 2013, SSMK conducted an evaluation using the Ethnographic Action Research method to assess and evaluate the listenership trend and impact of its programme on its target audience. Community-based evaluators in Nepal’s rural districts conducted surveys and focus group discussions with community members and stakeholders.
One of the major findings from the evaluation was the need for a local language radio show.
The survey during the evaluation process highlighted the language conflict among SSMK’s listeners; the central version show was not able to fully capture the spirit of local communities. With the central programme produced in Nepali language, listeners in other parts of the country such as Dhanusa where Maithali is widely spoken, sometimes felt disconnected. Also, the issues varied from region to region and a national programme, which addressed topics on a macro level, lacked discourse on community-specific problems. The SSMK team therefore wanted to ensure meaningful participation of youth from across the country by starting local versions of the programme, which would give them a sense of ownership, according to Ayush Joshi, senior programme officer at Equal Access Nepal.
‘It was important to start a local versionto sustain SSMK in the long run,’ Joshi says. ‘The listeners were demanding a programme in a local language.’
To better equip rural youth with the knowledge and skills they need to participate in local and national policy development, SSMK launched local versions of the show in 15 marginalised districts. Equally important, its purpose was to provide necessary knowledge to the young people so that they could control their lives by applying what they learned.
The local programmes are customised according to their listenership with tailor- made content that helps the audience connect with prevailing issues in their communities. In a short span, Mishra says SSMK’s local version has been effective in highlighting issues like child marriage and child labour, both of which are considered socially acceptable in many rural communities. While the local-language radio dramas have played a pivotalrole in educating the public, segments incorporating local authorities and stakeholders have opened up a non-formal communication channel for discussion. It has also helped in holding authorities accountable for these issues.
Raising awareness of child labour
At Radio Mithila in Dhanusa, Sangi Sang Manat Baat became an offshoot of SSMK. It was targeted at the Maithali-speaking people living in the district’s 90 Village Development Committees. A show in Nepali, according to Mishra, simply would not resonate with the locals. A Maithali programme, on the other hand, pulls in listeners primarily interested in the problems that dominate their region.
‘When you’re communicating with people in their own language, the audience feels associated with the show,’ Mishra says. ‘Itis also easier to make them understand the topic. When the programme is strictly in Nepali, they might just think that it is not meant for them.’
In the 26 episodes of SSMK’s local offshoot, the show raised area-specific issues such as child marriage, dowry-related problems, and violence against women.
Mishra says the team also incorporated experts and law enforcement officials from the community to make episodes relevant locally. He says it adds extra credibility and makes the show more trustworthy.
Upon listening to the show on child labour broadcasted in Maithali, Mishra says people have become fairly hesitant in recruiting children. This episode also prompted authorities to at least warn businesses that employed children. ‘It takes time for these practices to end,’ Mishra says. ‘But I’m sure our radio show has started that much-needed conversation. It’s a small revolution of sorts.’
Changing attitudes about women’s issues
SSMK’s evaluators note that local programmes help represent community problems more accurately, and Manauda Ka Kura, SSMK’s local adaptation, is doing just that in the far western district of Accham.
Through radio drama, using an informational and emotional approach, the show has drawn attention toward child marriage, which is highly prevalent in the district.
Sarita, one of the characters, is forcefully married at an early age. She is deprived of her will to study and dies of pregnancy complications. On the other hand, her friend Renuka, who was against Sarita’s wedding, finishes her school and pursues her education to become a nurse.
‘My daughter will never forgive me,’ Sarita’s father later apologises to Renuka for dismissing her idea to at least wait for a few years.
Manauda Ka Kura receives considerable feedback and comments from its listeners, a few apologetic but most applauding
the programme’s efforts. The letters and text messages, according to programme producer Bidhutma Auji at Radio Ramaroshan, motivate them to continue the show and also help them ascertain the show’s impact.
One of the letters the show received after the episode on child marriage summarises the scale of Manauda Ka Kura’s effectiveness. A 16-year old girl writes:
My parents who are very conservative were forcing me to get married. But one day when I was listening to the radio programme, the hosts were talking about the hazards of child marriage. I requested my parents to listen to the show, and after listening they became aware for the first time about the realities of child marriage. They then decided not to force me into marriage. The local language radio show not only helped educate them but also helped them to empathise with me.
Auji says that their show in Acchameli language has increased the engagement and interaction level with the community members. It has also made it easier to disseminate information on topics that matter the most in their native tongue.
‘When the programme is in a local language, listeners feel that someone they know and trust is talking about their issues,’ Auji says. ‘It’s easier to communicate using local words and metaphors that might be difficult to convey in Nepali.’
Connecting youth, the elderly, and policymakers
For almost three years, listening to SSMK and now its local version has become a regular affair for Sagar Bhandari’s family in Accham’s Mangalsen village. Every week, the family sits by the radio set in the living room and listens to the show.
With higher mobile and internet penetration, young people have alternative means of accessing information. Bhandari, 18, says ‘Adolescents have many problems they cannot talk with their parents. These issues are always hidden from the parents. But when their parents to listen to a radio drama, it helps them understand what their children are going through.’
But statistics show that radio is still the popular choice of mass communication for people aged 15–19, and SSMK is the most important radio show in that age group. According to the National Demographic and Health Survey 2011, 58.5% of men and 49.7% women in the surveyed population listen to SSMK regularly.
While the local programmes have broken the silence between the two generations, SSMK’s evaluation also showed that they act as a tool to teach the adolescents about their rights – the legal age of marriage, the use of contraceptives, and charters regarding child labour. The programmes have also helped to initiate discussions between local policymakers and youth.
In village clubs and meetings led by local youth from the community, these episodes help to start conversations on important topics. Bhandari says:
In our monthly Child Club meetings, we discuss the issues raised in the radio show. The information is not limited to the radio show itself. The dramas serve as catalysts. Once the topics are introduced, they are then discussed and shared continuously at the workplace, in schools, and at home.
Prem Bahadur Buda, Accham’s District Child Rights Officer, also stressed the influence and impact of radio shows in the local language, particularly when the shows reaching areas inaccessible by the mainstream media.
‘When the programme is in a local language, people feel a sense of ownership – they think it’s a programme for and by them. And they are right!’
Buda added that local language programmes like Manauda Ka Kura in Acchameli also yield positive results for government-run projects.
‘Our [government] programmes are only targeted at certain groups. So we cannot incorporate everyone. But programmes like SSMK are for everyone. No one is left out when it comes to accessing information.’
Youngsters like Auji and Mishra believe that education and access to information are the keys to changing social attitudes. In districts like Accham and Dhanusa, young people are in the forefront of this slow, yet significant, change – making people aware of the issues and helping prevent some of the negative aspects of past practices through the information they receive through radio programmes.
Shows like Manauda Ka Kura and Sangi Sang Manak Baat have proved relevant in raising community-specific issues. They are programmes by the community and for the community.
‘This has helped grow SSMK’s listener base in many hard-to-reach areas,’ says Joshi from Equal Access Nepal. ‘SSMK is building a strong rapport in rural communities as a people-centric and inclusive media platform that listens to its listeners.’
Yvette Shirinian is the Business Development Officer for EqualAccess.
Ayush Joshi was the Program Officer for EqualAccess Nepal.
Story writer: Bibek Bhandari
Evaluations that Make a Difference 
Pingback: Evaluations that make a difference: evaluations stories around the world | Evaluations that Make a Difference