Evaluations that make a difference is a collection of 8 evaluation stories from around the world which is one of the first pieces of systematic research looking at factors that contribute to high quality evaluations that are used by stakeholders to improve programs and improve people’s lives. This initiative collected stories about evaluations that made a difference, not only from the perspective of the evaluators but also from the commissioners and users. The stories in this collection tell powerful stories about the findings in the evaluations and the ways the evaluations contributed to the impact of the programs. You may access the report and all the stories here, in English, Spanish, and French.
In these weekly posts, we will be sharing each story… Comments are very welcome!!
When you find the right direction in life it can be contagious. Inga didn’t realise it though. To her, she was doing what was right, but little did she know how many people would blossom from being in contact with her. Her clarity of purpose also helped build the support she needed.
Inga Mielitz is an atypical Christian minister in the Netherlands who works with people affected by HIV. ‘I’m a dyke,’ she says with unabashed candour. Her razor-short bleached hair and assertive demeanour surprise people who first meet her over the phone. ‘I started doing this work in 1996 when my gay friends, all white men, were dying from the disease. I felt a sense of responsibility.’
After working with various religious organisations for a few years, Inga felt that the restrictions and expectations placed on her diluted her sense of purpose. She felt a need to choose her work based on what was important to her.
They didn’t want to hear that discrimination of people with HIV has its roots in the negative way our churches talk about sexuality; they didn’t like my openness. But life is precious, and I wanted to do what I consider important in my life instead of doing what others expect from me. I believe that God wants every person to be who they are instead of pretending to be someone else. I still wanted to work with HIV-positive people, but I wanted to do it my way. Put together, sexuality and faith make a connection to something bigger than oneself.
Shortly thereafter, ShivA – an abbreviation for Spirituality, HIV, and AIDS – was born. The aim of this Netherlands organisation was to improve the quality of life of HIV- positive people and their loved ones through empowerment and by supporting people seeking meaning and spirituality. The idea was to provide a ready response for people asking Where do I find the strength to go on? Positive Sisters is a ShivA programme that provides support for HIV- positive people, especially migrant African and Caribbean women. Most of the women are referred to ShivA by hospitals where they’ve gone for treatment.
The gift of evaluation
After 4 years working with African and Caribbean women and 2 with the Positive Sisters project, having enroled more than 150 women and trained 18 Positive Sisters, Inga met two staff members of an evaluation firm called Results in Health at National AIDS day in Amsterdam. Aryanti Radyowijati and Maaike Esselink, impressed by Inga’s story and touched by ShivA’s apparent impact, offered a pro-bono evaluation using the Most Significant Change technique, as an opportunity for their team to gain experience in that methodology. Given the powerful stories and small sample size, Most Significant Change would be perfect.
Maaike’s team interviewed the women and then facilitated a collaborative workshop where different stakeholders reviewed the stories to identify the ones that felt most meaningful to them. The evaluation team then analysed the data and compiled a report. Finally they shared results and recommendations. Inga hoped the evaluation would help them secure funding going forward.
In an interview, Liako, a Positive Sister from Lesotho, shares her first conversation with Inga: ‘I explained to Inga how my life is and how open I am about HIV. Inga said, “Vive la Vie [one of ShivA’s early projects serving Afro-Caribbean women] is not for you, it’s for women who are closed and who don’t know other women.”’
Liako is a lively speaker and wears a broad smile. Her hands and body move rhythmically with her words as she speaks. Her dark skin looks translucent. She exudes positivity and joy. Inga offered Liako her newly started Positive Sisters volunteer training. Liako was enthusiastic, inspired to find a kindred spirit in Inga, and eager to help support others to live positive lives.
After Liako had been a Positive Sister for ShivA for 11 months, she met a young woman named Millie, aged 30 and recently diagnosed with HIV. Knowing that Liako was HIV-positive, Millie expected a thin, weak woman, her face creased in pain.
I was shocked that she looked so beautiful. It was reassuring for me to talk with her; she motivated me to push forward. She told me how she found out when she was my age that HIV/AIDS shouldn’t stop her from doing whatever she wanted to do with her life, and neither should I. I should go on and do it… [You] need a doctor or a nurse when you’re sick, [but you also] need someone who has been through that pain, who knows what it feels like. Without Liako I would not be who I am today.
Clearly, being at ShivA had positive effects on the women’s lives. So did the evaluation, but in different ways.1
Each of the five women interviewed for this article (two positive sisters, a project coordinator, a lead evaluator, and a referring nurse) spoke about different aspects of the evaluation process, but they all mentioned the positive impact it had on them and on the project as a whole. In the evaluation interviews, the women reflected on their experience with HIV/AIDS and how much they had changed since ShivA came into their lives. Millie had been quite shy about her HIV/AIDS status before ShivA, and now recognises that being interviewed was an extremely positive experience.
Whenever people talk about HIV they talk about how many people are infected in the world and how many people are taking medication, but it’s never the voice of those who are affected by it. This evaluation was the voice of the people infected…. Being interviewed made me want to be involved more. I didn’t want to only be a patient after that. Being interviewed helped me feel important and part of something good.
After the interview, Millie decided to become a Positive Sister, and she is now enroled in university.
For Liako, being interviewed meant becoming more aware of the impact of her work since ShivA: ‘The evaluation had a big impact on me. It was a Wow! moment for me. We are really doing a great job.’
Maaike’s team prepared the interviews into stories so they could be read at a half-day workshop to see which stories were more individual and which represented shared experiences.
The nurse, Lia, and the evaluator, Maaike, described the collaborative workshop day as a lively experience. The atmosphere that Positive Sisters brought was one where ‘the floor felt like it was vibrating.’ The environment was joyful: good food, laughter, music. The women talked, laughed, and sang spontaneously despite the intense sadness of their stories.
Lia remembers a shy, somber woman she supported. At the evaluation workshop, she was pleased to see this woman stand up and share her perspective:
When Positive Sisters are in the room, it is alive; I saw women become empowered. One woman transformed in front of me when she got up to speak. With Positive Sisters, women are finding meaning in life. Everyone searches for meaning but when you are diagnosed with HIV, you face it more.
With the collaborative workshop, the individual stories started to blend into a collective story. After the workshop, Inga told Maaike, ‘It is amazing that the women from the project, by hearing other stories, can now talk about the project as a whole.’
The stories grew from individual narratives, to project stories, and finally to a national dialogue. Inga says that the evaluation empowered African and Caribbean women to speak up on the national stage: ‘It was a snowballing process. Once it got started there was no stopping it. Policy makers are now contacting our women to contribute to their research, and there is a group of African women contributing who never spoke up before.’
Liako agrees. ‘We did short videos online. They are being shown on the internet and at conferences on HIV. Mine is about having a husband and a daughter who are negative. Healthy. And people get interested in who we are and what we do.’
Evaluation spurs action
Many Positive Sisters became more confident. There was a very small woman from Ethiopia. She was very quiet but there was a deep inner power in her…. She did an Ethiopian coffee ceremony, sitting on that little low chair just by herself. But she raised her head and contributed. Her voice was not loud, but her confidence shone through and everyone paid attention to her because they were hearing the voice of a free woman!
Although most of the stories collected by Results in Health were positive, they also offered insights into where improvements could be made. One suggestion was that Inga was doing too much. ‘The project was leaning too strongly on me,’ Inga recognises. ‘If I wasn’t there, nothing happened.’ But having more coordinators meant investing more money and Inga knew money was tight.
The evaluation suggested she assume a coordinator role to help mediate interactions with participants (now numbering several hundred) and Positive Sisters (now 31). Two Positive Sisters offered to take on a coordinating role without pay, including Liako, who with ShivA’s help is now pursuing a professional vocation as a counselor and a coach. At the time Liako was interviewed, this new coordinating role had just begun. She explains: ‘The evaluation made me want to do more. It made me feel very important, like I’m fulfilling a dream I didn’t even know I had. I feel like I’m more than I thought I could be. For me, success is not getting paid – it’s doing what I love.’
For ShivA, expanding roles are also important given that referrals are increasing too. Lia, the nurse, says that the evaluation increased her referrals from one in three to all women diagnosed. ‘Because of the evaluation I have a more professional view of Positive Sisters. After the evaluation I talked with my colleagues and we decided we would refer more people to Positive Sisters.’
The evaluation team felt that working with Positive Sisters was an opportunity to work with an exemplary client. Inga was hungry for insights and feedback and quick to apply them. Maaike says, ‘Inga was extremely interested in learning about the project. She really wanted to learn from it and get results. She always made time for us.’
One of Inga’s primary concerns for ShivA is, as with most community projects, to ensure continued funding. So when the evaluators proposed a workshop with multiple stakeholders to look at results and think about funding opportunities going forward, Inga was on board.
While steady funding for ShivA is not quite there yet, it is certainly on the way. The snowballing effect continues. At the workshop there were many ideas about how to get health providers to pay for the support participants get from ShivA. For example, a large foundation supported by the royal family of the Netherlands recently committed to help find the needed funding.
But the most important outcome from the workshop was something intangible and transformative.
In Inga’s own words:
It was very special to me because I do a lot of work alone and this time I was not alone. Everyone was trying to find a future for Positive Sisters. Through the evaluation, I was able to tap into the experience of professionals who also do good quality work. They could evaluate the quality of what I’m doing. It gave me a new surge of self-confidence, a stable ground to stand on. Now I can see how to make ShivA grow further and give it a brighter future.
We would like to credit the tellers of this story:
Inga Mielitz, ShivA Foundation
Liako Lekhooa-Oude Lansink and Millie, Positive Sisters
C. (Lia) Meerkerk, nurse practitioner
Maaike Esselink, Programme Oficer, Public Health, and all respondents of the evaluation who made this evaluation so valuable.
Story writer: Rita Fierro
ShivA website: http://www.shiva-positief.nl/a-letter-to-you
Results in Health: http://www.resultsinhealth.org/
Positive Sisters: http://www.shiva-positief.nl/a-letter-to-you