(1).jpg)
World YWCA's Podcast
World YWCA's Podcast
Episode 5: Evidence Generation
In this episode of RiseUp! we look at the ways that feminist techniques can inform one of the basic needs of community building and development work - Evidence Generation.
Host Chhavi Sachdev starts off the episode in conversation with Bijita Dev Sharma – the RiseUp! Monitoring Evaluation and Learning Specialist with the World YWCA, from India. Next, she talks about development work in the Global South with Michelle Reddy, the Co-Lead of the Pacific Feminist Fund. And finally, we hear from our third guest Veena Singh, also from Fiji, where she is the RiseUp! programme coordinator for the World YWCA in Asia Pacific. What values do these three women consider as feminist interventions in the process of evidence generation? Listen on to learn…
This podcast is brought to you by the Australian Government under the RiseUp! Young Women’s Leadership and Advocacy initiative in Asia-Pacific region.
Guests: Bijita Dev Sharma, Michelle Reddy and Veena Singh
Host: Chhavi Sachdev
Team: Dr. Suchi Gaur and Nirmala Gurung at World YWCA
Podcast production: Chhavi Sachdev and the team at Sonologue: Sharad Joshi & Deepa D.
Young women across the world are mobilizing for change and they're bringing their feminist values to every struggle they participate in. We believe the stories these leaders share will provide you resources for advocacy and training, because we believe that growing feminist leadership is one of the most powerful ways to effect change for this planet. I'm Chavi Sajdev, and this is the fifth episode of Rise Up Leadership, the latest podcast from the world YWCA. In each episode, I talk to young women leaders from many different nations about the concepts behind their activism. In this episode, we talk to three women about evidence generation how do we collect and analyze data in a way that upholds our feminist values? We'll kick off the discussion with Bajitha Dev Sharma, rise Up Monitoring, evaluation and Learning Specialist with the World YWCA, based out of Hyderabad in India. She breaks down the specific ways to look at evaluations in non-profit and development sectors.
Speaker 2:Evaluation in general, the way it is understood, really refers to a process where, by the end of a program phase, we are using social research methods to systematically investigate the effectiveness of an intervention. Traditionally, evaluation has been. You know it only concentrates on impact assessment. It is narrowed down to only looking at oh so, these are the objectives and oh so, this has been the impact.
Speaker 1:In the Rise Up program that Bajitha is helping to monitor and evaluate the World, YWCA has come up with ways to move from the limitations of traditional evaluation methods to an outcome mapping approach that they call a gender transformative evaluation.
Speaker 2:Outcome mapping approaches traditionally an approach and evaluation that is seen to be understood to be as a transformative approach. It looks at it as interested in capturing changes in attitudes and behaviors of the direct you know target beneficiaries. It is not just goals, objectives versus. You know what comes out at the end of, let's say, year one, year two or year three, or at the end of the project period. It really is meant to capture the significant changes in the whole process of transforming power structures.
Speaker 1:There are two kinds of data that their program collects quantitative and qualitative. In simpler terms, numbers and stories. Quantitative data sets provide the performance on broad basic indicators.
Speaker 2:Unfortunately, the not, you know the global, not the way classical evaluators from global not have passed down some of the approaches to us. They forget to look at the different identities, the intersecting identities like and there are so many, you know gender, ethnicity, caste class, sexual preference, abilities. These are not taken into account. But what a gender transformative evaluation is trying to do is give place a lot more focus on these intersecting identities and this is what makes you know the gender transformative evaluation approach unique Intersectional and gender these are the two lengths we bring to our data collection. So it has to be intersectional, which means we collect data based on different ethnicities, different backgrounds, based on religion, caste class, sex identity, sexual orientation, all that Also their marital status if they have children you know, do not have children their employment status, their education status.
Speaker 1:The other kind of data is qualitative, which Bidgeta describes as narrative change stories, anecdotes which help add key layers to the impact.
Speaker 2:Now the qualitative data is the more sort of detailed narrative data that helps us not just corroborate you know everything, you know the evidence we have with the quantitative data but it also provides in-depth information about this whole transformative change journey that we are interested in tracking. So our attempt is to also facilitate a process where these girls are constantly capturing stories from the field through in-depth interviews or focused group discussions. Also, with this, we are trying to measure the impact on the community, which is, you know, probably not the sphere of our direct engagement. If these girls are talking about these critical issues, is there a change that is taking place within the community? Do they have more allies in the community? Do they have more supporters within their families or communities or you know the larger ecosystem that surrounds them? So that kind of information is collected through qualitative tools.
Speaker 1:One of the ways that the World YWCA is upending the traditional power hierarchy involved in data collection is by empowering the young women leaders and participants of their rise-up program to do the data collection themselves, as well as provide them with the tools to analyze the data.
Speaker 2:So the girls are trained, they are given, you know, the skills to be able to collect that information and bring it onto the table. They analyze the whole process, where they sit together and analyze the data and send it back to us. If there are questions that they have or you're not at any point, they're also expected to provide their feedback. Similarly, if they are engaging with, let's say, their family members on an issue, or the community leaders, or gatekeepers, as we call them, on an issue, then they are able to capture that information themselves and bring it onto the table, do that analysis.
Speaker 1:And this ties in with the core feminist value that Bijita says informs their whole program implementation process the value of being inclusive. They co-create the program with the participants and alumna of the previous phases and they include their evaluation processes at every stage of the program implementation.
Speaker 2:So classically, evaluation, or monitoring and evaluation is seen to be a separate entity. You know it's outside the program. It appears periodically, maybe at the middle of the program or at the end of the program. But what makes Rise Up unique is that monitoring, evaluation and learning is embedded in the way Rise Up is designed. It is a part of its theory of change.
Speaker 1:As an expert in evaluation methodologies, bijita is familiar with the variety of approaches besides the outcome progress approach that the World YWCA uses for Rise Up, one that she regards highly is Naila Kabir's social institution relations approach, where she focuses on the four key institutions and the need to transform them.
Speaker 2:So when she's evaluating, she'll look at family, community, market, market and state. She'll do a layered kind of analysis and try to understand the intersection of gender relations through these four institutions. So what is happening at the family level, what is happening at the community level, market level, state level, so on and so forth? They focus on unraveling how intersecting identities affect the implementation of a program, and thus evaluation should typically focus on assessing the contribution of the program towards changing these power relations. So there are power relations in all these four institutions, clearly, but how can a program work in a way that these power relations are fundamentally transformed?
Speaker 1:Power relations in the nonprofit sector are also something that our next guest is concerned about.
Speaker 3:So the idea that evidence gathering is the only way is also problematic, because it is also around assuming that everyone has the ability to gather evidence. Who really is the one that's asking for evidence? And that's again about power, right? It's about dismantling, you know. Why is this evidence needed? Am I the best place person or the best place organizing To really tackle this? And, if this really is part of my organizational mandate, what is the approach that we are using in gathering data and in analyzing and in packaging and sharing that to the world? That's Michelle.
Speaker 1:Reddy from Fiji. She helped set up the first National Women's Fund in Fiji and is currently the co-lead of the Pacific Feminist Fund. She has a lot of experience parsing the dynamics between funders in the global north and recipients in the global south. One thing she points out is the inequity of researchers not making that data publicly available.
Speaker 3:If a lot of the mana or the, which is a Pacific word, if you're looking at all the wisdom that's being shared very freely by people, that does not have a prize value to it. But if someone is sharing their wisdom or their mana to you, it is absolutely, to me personally, absolutely important that you share that and make that accessible. The only time I think that you wouldn't practice that is if that person has really taken the time and told you that. I would rather not have the time to do that, I would rather not have this shared, but I think that when somebody does go down the journey of gathering and going into consultations and gathering evidence or research, I think the access to that is really paramount. And it is again about power. Who has access to this end product? How are they using all of this analysis? For what purpose? Where is the mandate coming from?
Speaker 1:Michelle sees a lot of value in gathering evidence, but she has several critiques about the processes involved in the ways that organisations go about it.
Speaker 3:In the Pacific in all the time that I've worked in the sector for more than 10, 15 years one of the things that I feel that's really, really important to debunk is this idea of what is this perceived rubric of skills and experience that makes someone tick all the right boxes so that their narrative and their evidence is on par to whatever the criteria is? And I think for me as a Pacific Islander now having to deal with climate injustice, it's absolutely paramount for me to know that everybody's experience is just as important and everybody's skills and knowledge is just as important, and I think that's why I'm really amazed and really proud of all these fierce Pacific feminists and women, human rights defenders, gender non-conforming. If you look at the disability sector in the Pacific, when you look at those working in the climate justice, they really embrace this notion of merging lived and a total experience with this also very dynamic need to find figures and data that really does complement each other.
Speaker 1:As Michelle points out, it is essential that organizations look at ways to engage people who live in that experience and who deal with the challenge of the time and of the context, who are active participants in that evidence generation.
Speaker 3:I think one of the things that's also really important for me is to ask you know who is involved in this, and then the most important question is to ask, like, who isn't involved and why not? You know, is there a rationale that I didn't hear of and that may actually be important to know, because that will make me understand this work more? And if there is not a really strong rationale or justification, then really the responsibility and co-responsibility lies with those who are undertaking the work, to make sure that you know those who are not involved are involved because, like I said earlier on, the process is just as important as the end product.
Speaker 1:Michelle also stresses that qualitative data, or, as she calls it, anecdotal evidence, is an important source of bringing attention to a problem.
Speaker 3:What was driving the work of the organization was hearing all this anecdotal evidence. You know you would hear from women's individual women or women's groups or individual human rights defenders and gender non-conforming people about the challenges that they were experienced. I have to say that anecdotal evidence is really important because they actually provide a basis for why you would want to move into a more evidence generation, which is really looking at a more systematized way of generating or looking into a particular area. So you can't just have you know a set of data and figures without having some narrative or some life story that makes it really real for people Like Bijita's concept of co-creating.
Speaker 1:Michelle has examples of her time at the Women's Fund Fiji, where they acted as facilitators to connect researchers with data skills and communities that were receiving grants from them.
Speaker 3:So really, the role of the Women's Fund, fiji in that instance was more like a co-hosting in an enabling space of bringing the partners together in that particular area, looking at finding a researcher that had really in-depth knowledge and had the mandate in a lot of ways by the grantee partners. So we had to run the consultant named by the grantee partners to see if they were comfortable working with this person, and a lot of it was also using research methodology that was more enabling rather than disen-enabling, and what I mean by that was because it was led by the grantee partners. It had to have the grantee partners in the data gathering, but also the data analysis and eventually the write-up of the research, and so all the partners were involved in all of the stages rather than just one of the stage.
Speaker 1:Michelle believes that feminist values really come into play when you scrutinize the process as much as the end product.
Speaker 3:So really challenging the dominant narratives around who is a storyteller and who is the person that's analyzing the data and who is the person that's publicizing the data. It's not my story as a person who is working at the Women's Fund. It really is the narrative of our grantee partners and they are the best people to first gather and analyze and they are the best people to really tell their stories and they are always going to be the best place people to find solutions to their problems and it cannot be from someone else that's outside of the country and I think that's really cool to a lot of the values of a feminist which is interrogating and challenging power dynamics.
Speaker 1:Critics of the process notwithstanding, Michelle agrees that evidence gathering is a really important approach and strategy.
Speaker 3:It's about making the case. It's to showcase what you're saying is a problem and what you think is the current approaches in the country that you live in. What are some of the gaps and recommendations? I think that's absolutely necessary and I think, being a feminist, it's really important to navigate and ground our work in both the collective care and protection. If we are not doing it in that way, then it's just not transformational at all. I think it's more transactional in nature and that, to me, is really detriment to the work that we are doing.
Speaker 1:Our third guest shares Michelle's apprehensions around data mining. We talked to Veena Singh, also from Fiji, where she is the Rise Up program coordinator for the World Wide WCA in Asia Pacific. She says that in her experience, mining of data is just an end to itself.
Speaker 4:So people come in academics or researchers. They come in and then they take this information but it does not come back to benefit the people, to benefit young women or women to improve policies and programs. So I think that's just a reflection of my involvement in the research space and how it can be really exhausting when there's a lot of people just coming in doing various types of research but you don't get to hear back on the work and how they utilize that piece of information.
Speaker 1:Veena also stresses that collecting data without applying an intersectional, feminist lens to it is harmful because then the data will not provide the support to the community it is taken from. Often a lot of time, money and resources are invested into the collation of evidence, but if quality time isn't spent in unpacking what the data is saying and analyzing it, the data is of no use.
Speaker 4:It is so critical because without a feminist lens, without a feminist approach, data will only be that just data and information. But when you apply a feminist, intersectional lens to this information that has been collated, you're really looking at things from a very different light and that way it really helps and speaks to people. You're able to develop policies, programs and design activities that is quite specific and takes into account the realities of people in all their diversities. So I think it is really really critical that we are analyzing data, looking at the information that is already available, highlighting the gaps that exist and building on the amazing work.
Speaker 1:She also stresses on the importance of making data easily accessible, especially to stakeholders. Even when it's collected or collated and shared, it is often incomprehensible to a lay person.
Speaker 4:So it's so so important to make research accessible, make it user-friendly, and information that you can easily understand and use.
Speaker 1:Veena feels that incorporating feminist values into an organization starts right from the beginning by asking questions of the process early.
Speaker 4:It's about looking within your organization and asking yourself who is missing, who needs to be included or who needs to be part of this organization. How the feminist values came into play was and something that I've already touched on is when you are planning even the early conversations is involving the young women and finding out from them like okay, we are planning on of developing this leadership program for young women, what are your thoughts? And in most cases, like the involvement of young women from the early stages, from the initial stages of just having that initial conversation, is really like you can see the ownership, the respect but valuing of each other's point of views and opinions and the way that the contributions from this early stage really affects and influences the design of the program and also the success of the program.
Speaker 1:Having seen projects from both sides, ones that have involved the participants from the beginning and ones that have not, veena has very clear examples of the different outcomes when their young women constituents are involved right from the initial conversations and participate in planning, designing, fundraising and actual programme implementation.
Speaker 4:And those who were in that decision-making power or leadership roles actually took a step back, because for them, this is for young women. It should be led by young women and we need to hear from them because they are the ones who would know what would work, what won't, what strategies would work and what will not be successful, what kind of platforms would be the best option and who would need to be invited. You know, like things like that, the logistics around it, and then you know, like the reflection process and the success rate for that was much higher because the young women themselves, they felt there was a level of respect, there was ownership and appreciation that you know, they were genuinely involved in these spaces because their voices mattered.
Speaker 1:On the flip side, veena has seen examples of development agencies who wanted to fund young women's leadership programmes but used a top-down approach where they didn't bother to consult and involve young women from the beginning.
Speaker 4:By the time that they did involve young women, it was at, you know, a later stage. It wasn't meaningful engagement, it was more tokenistic, just to show that, yes, we did involve young women. And then, you know, when it came to the implementation of the programme, we noticed that young women weren't really participating, they weren't really engaging and there were lots of challenges. Young women weren't feeling comfortable or confident to speak up because of the power imbalance in the room or even like within the actual implementation of the activities and not understanding and acknowledging the different realities and experiences of these young women, where they came from and their backgrounds. And you know, long story short, that particular programme was not successful. Young women slowly just left the programme and they just stopped engaging.
Speaker 1:There is no point in big funding organisations designing programmes for communities without centering their own self-defined needs and cultures at the heart of the process. As Veena says, to be a feminist is to bring that spirit of accountability and inclusiveness to all that you do, including organisational values.
Speaker 4:And depending on who you are, where you're from, the age group that you belong to, the way that you experience life, the way that you experience systems, it's unjust. So the programmes or activities that might be developed or will be developed or are developed, it needs to take into account the realities and experiences of everyone and the diversity and how they experience life. So that means, like for organisations when they say you know that they have feminist values or they need to incorporate feminist values. Like for me. It means too dismantled bias. You know, to check within ourselves, around us, but also within the institution or within the organization, the various, the power, the privilege and the advantages that one has.
Speaker 1:The World YWCA's Rise Up Monitoring, evaluation and Learning approach, or MEL, was co-created with young women over a period of two years, deep diving feminist values and building MEL systems that truly reflect realities on ground and yet are full of continuous learning potential around young women leadership. The World YWCA as an organization has developed several tools and strategies to ensure its evidence generation and data collection processes hold up to its standards of intersectional feminist inclusiveness. It tries to center young women's leadership by putting them in control of their own data with the skills to analyze it. The World YWCA is a global women's rights organization engaging millions of women, young women and girls around the world each year, across cultures and beliefs, to transform lives and the world for the better.
Speaker 1:With a presence in over a hundred countries, our work is grassroots driven, grounded in local communities and rooted in the transformational power of women. We provide support and opportunities for women, young women and girls to become leaders and change makers who not only protect their rights and impact their communities, but inspire their peers to do the same. We are focused on building a strong intergenerational network of women and young women leaders, with programs led by and for women and young women in response to the unique needs they see in their communities. Our goal is that by 2035, 100 million young women and girls transform power structures to create justice, gender equality, human dignity, freedom, a sustainable environment and a world free from violence and war. Leading a vital YWCA movement inclusive of all this podcast series has been funded by the Australian government under the Rise Up Young Women's Leadership and Advocacy Initiative in the Asia Pacific region. You can find out more about our work on our website. Our handle is worldwide YWCA on all social media.
Speaker 4:A sonologue production.