Evaluations that make a difference is a collection of 8 evaluation stories from around the world which is one of the first pieces of systematic research looking at factors that contribute to high quality evaluations that are used by stakeholders to improve programs and improve people’s lives. This initiative collected stories about evaluations that made a difference, not only from the perspective of the evaluators but also from the commissioners and users. The stories in this collection tell powerful stories about the findings in the evaluations and the ways the evaluations contributed to the impact of the programs. You may access the report and all the stories here, in English, Spanish, and French.
In these weekly posts, we will be sharing each story… Comments are very welcome!!
Since the end of Sri Lanka’s long civil war in 2009, the nation’s economy has recovered at a startling rate. GDP growth now hovers between 6% and 8%. Peace has certainly brought dividends – tourism has rebounded, China has made large investments in infrastructure, and trains once again run the length of the pearlshaped island in the Indian Ocean, just off the southern coast of India.
But it’s not all good news. As peace returned, small and medium enterprises (SMEs) found to their chagrin that China, India, and Bangladesh had all but cornered the world’s export markets in mass manufacturing and textiles – at their expense. To add to their woes, the national government had paid little attention to the sector over the years. For the hundreds of business owners renting land in the 18 government-owned industrial estates managed by the Industrial Development Board (IDB), the economic boom seemed to have all but passed them by.
The list of complaints was long and detailed. Rents were out of control, with some more than doubling year after year. And even when rents were paid, there seemed to be no correlation to the quality of infrastructure. Roads were more potholes than asphalt, electricity was unreliable, and coordinated marketing simply lacked focus. Although the government was investing heavily in national infrastructure, smaller businesses appeared to have been overlooked. R.S. Balanathan, the managing director of handloom silk exporter Ko Lanka, says all the industries in the estates were undeveloped. ‘The roadways were bad, and the rents were very high,’ he says.
In 2013, the situation began to change. The Ministry of Traditional Industries and Small Enterprise Development commissioned a comprehensive evaluation of the sector in 2013. A team of independent evaluators led by Ajith Ekanayaka began travelling to each of the 18 estates across the country to examine the issues plaguing the sector and to suggest direct solutions. Ekanayaka made it clear that this evaluation would not be relegated to a filing cabinet but would be put to immediate use.
For five months, the evaluators gathered their material, forming focus groups for industrial associations and interviewing managers and district officers. The three broad problems that came up again and again were decaying infrastructure, exorbitant rents, and expensive loans. Some had special concerns. Metal industrialists, for instance, could not lay their hands on raw materials because almost all scrap metal was being exported. The evaluation also found that some estates were being used to build housing by the owners of failed SMEs. The longterm weakness of the sector meant that employment for ethnic minorities and the rural poor – who provided much of the workforce in the estates – also suffered.
The evaluation report gave voice to longstanding grievances and recommended sweeping changes. What was needed, argued Ekanayaka, was direct intervention by the national government. The secretary of the small enterprise ministry, Velayuthan Sivagnanasothy, seized on the findings. He asserts:
The evaluation showed that the IDB was a mere rent collector and not improving the estates. With an evaluation like this, change would be far easier to generate. The evaluation process was a platform serving to carry the voices of the voiceless to the highest level of policy makers.
He took the evaluation to all relevant cabinet ministers to secure their support. With the cabinet on side, there was no way the issues raised by the evaluation could be ignored.
And so it came to pass. The ministry drafted a national policy on SME development, consulting widely and drawing on the expertise and advice of other ministries and agencies. But even before the policy could be adopted, change began to ripple outwards. The evaluation’s actionable recommendations had buy-in. Why?
Because policymakers had given frequent input through scoping sessions, and the evaluation was built on abroad, credible base of evidence and data as well as strong connections to stakeholders. According to Sivagnanasothy, a great deal has come from this single evaluation. He recalls:
When the rents went up, industrialists used to shout and cry: How can we afford it? We were able to convince the central government to cap rent increases at 7% annually. That’s a predictable environment, which is an enabling environment. Not only that, but the introduction of targeted loans with lower interest and longer grace periods meant that SMEs could now expand their operations more easily. In each of the estates, factory production is up, and with it, SME profits, leading to more jobs. It’s trickled down to the rural villages.
A tangible shift in attitude amongst IDB management towards the SMEs was confirmed by Ratnamalala, who praised the evaluation for what it revealed. ‘We expanded our service delivery and created a shift in mindsets of our officers to be more development oriented.’
The shift for the IDB was significant, with the agency undergoing a restructure in order to shift from a reactive to a proactive stance. The IDB now focuses on training, access to raw materials (such as previously scarce scrap metal), and access to credit. Not only that, but the rental income earmarked for infrastructure improvement around the estates was increased tenfold, from as little as 2–3% percent to 20–30% of the rent paid.
To address the problem of access to credit, the government rolled out a special scheme with concessionary rates to boost access. Joint marketing schemes were introduced to give potential buyers a streamlined way of examining the range and quality of products. Ailing SMEs can now gain access to a ‘nursing programme’ to bring them back to health, while the rules for gaining entry have been made more stringent to cut down on residential use of industrial estates. And the metal industries were given a lifeline at the expense of the scrap merchants, with a ban on the export of scrap. The amount injected into infrastructure upgrades immediately following the evaluation was 305 million Sri Lankan rupees (US$2.3 million), with more on the way. For IDB director general Ratnamalala, another benefit of the evaluation is simply being heard. ‘We have direct access to central government,’ he says. ‘Now, with these improvements, we will see growth in our industrial estates.’
On the ground, results of these rapid evaluation-led changes have been impressive. The voices of small and medium industrialists have been heard, and the lines of communication with government are open.
For silk exporter Balanathan, the result has been more cash in his pocket – and that means expansion into his key markets in Europe and the Middle East. From 100 employees in 2013, he now employs 130. ‘We have strong competition with China and Bangladesh,’ he says, ‘but these changes make us more competitive. Now, we have higher silk production than they do. We can beat them!’
Velayuthan Sivagnanasothy, Secretary, Ministry of Traditional Industries and Small Enterprise Development (sivagnanasothy@ hotmail.com) commissioned the evaluation of the SME Industrial Estate Programme and used the evaluation recommendations that led to policy changes and programmatic actions that made changes in people’s lives.
Ajith Ekanayake, Independent M&E Consultant, led the evaluation team.
Vaidehi Anushyanthan, Assistant Director, Ministry of Finance, cochaired the evaluation as focal point of the Government’s National Evaluation Department.
Story writers: Jessica Kenway and Doug Hendrie.