Share your ideas 
AES Blog

This is the AES Blog, where we regularly post articles by the Australasian evaluation community on the subjects that matter to us. If you have an idea, please contact us at . Blog guidelines can be found here.

 

Welcome to the AES Blog

Australasia has some excellent evaluators. More than that, we have an evaluation community full of ideas and a willingness to share. The AES has long provided a place for us to come together, at regional events and the annual conference, to develop our community together. Now we’re taking it online! The new AES blog will be a space for AES members – both new and experienced – to share their perspectives, reflecting on their theory... If you have an idea, please contact us on . Please also view our blog guidelines.

Practitioner Learnings in Creating a Culturally Safe Evaluation and Research Space

by Doyen Radcliffe, Donna-Maree Stephens and Sharon Babyack - Community First Development

Community First Development led an AES seminar on creating a culturally safe evaluation and research space in September 2021.Community First Development is a First Nations’ led organisation that works with First Nations’ communities by invitation only, every project we do is monitored throughout, using indicators designed and evaluated by communities themselves. Our vision is to see First Nations’ peoples and communities thriving.

At the seminar, we shared lessons learnt from a participatory action research project focused on gaining a better understanding of 11 communities with whom we had been working for three years on community development projects. We are pleased to be back to answer the questions from our AES seminar. Most importantly, we would like to thank attendees for the words of encouragement and support.

The focus of our presentation was the findings of our recently released action research final report, ‘Good governance practice leads to good relationships. Final Report: Findings and lessons learnt into the effectiveness of Community First Development’s community development approach in the Australian context’. You can watch the presentation recording here. AES members can also access the presentation slides via the WA regional network resources available via the AES resources page.

In the seminar, we talked about one of our big discoveries after phase 1 of the research. We found that while communities had common knowledge of the two forms of governance in operation, they were confused about the term ‘governance’. We called this our “fail forward” moment; we came to realise the methods we had used had stifled the conversations, and as a result, we had not learnt much about First Nations’ governance in the communities. After phase 1, we adapted to elicit organic and richer conversations. We replaced the formal semi structured interviews we had used in phase 1 with a Yarning Tool, consisting of visual prompts and questions, and trialled co-authoring of case studies. Through a research yarning approach, we began to unearth the challenges that occur in the interacting space between First Nations and Western forms of governance.

Bridges to Right Way Governance

Bridges to et al

One of the key findings of the report was about the intersecting space between First Nations Governance practices and Western Governance practices. Working in that space requires ‘Right Way Governance’. This is something relevant to evaluators engaging in and with First Nations’ people and communities.

Questions And Answers

Addressing Ongoing Colonial Impacts

What strategy/ies do you recommend for assisting community groups to move beyond a colonial model, especially when there is a 'default?

The key thing is to build trust and relationships with communities. Moving beyond colonial thinking requires coming in with no agenda and not to over-promise. You need to be genuine in your approach. An important part of that is sitting down, yarning, deep listening and understanding: to hear what is actually being said. It is a focus on community strengths and supporting from there.

Asking open ended questions, or taking an Appreciative Inquiry approach, may assist to see beyond the impacts colonial models are having.

What is the recommended strategy when there is a 'default' or 'comfort zone' still dominated by a white-dominant approach where Aboriginal community members are used to following white peoples’ priorities rather than take the lead from an Aboriginal standpoint?

The dominant society needs to rethink their approach and that is a challenge; further critical and reflective thinking needs to be undertaken by everyone working in communities. A good question to ask yourself may be, “is my approach giving First Nations’ people a voice?” An effective model to look at is the AES First Nations Cultural Safety Framework; this is a good reflective model to provide insights into Aboriginal and Torres Strait Islander ways of practice.

One practical way may be to share current community focused ‘data’ or research information to inform community focused decision making.

What are the benefits to Communities of your approach to Governance - how does it impact the work you do with Communities?

TIME! It cannot be emphasised enough. Allowing time to consult within community in a way that prompts thought rather than takes over actions. The trade-off for the time invested is positive, community focused outcomes.

There have always been Aboriginal and Torres Strait Islander ways of doing, and expressions of, community governance within First Nations’ communities. However, this is more often overlooked due to the influence of the dominant culture. The benefits of Community First Development’s approach to engaging with communities, and the governance they have in place, is that it provides community a voice. The way we engage with communities is based on communities’ terms in how they govern themselves – based on their Nation’s self-determination.

Research Methods/Methodology

 research methods

Community First Development’s Story of Change

Can you tell us a little about the process to develop the ‘story of change’?

The previous story of change we had was hierarchal and didn’t fit in with the work we do on the ground with communities as the landscape was always changing. The changes we could see were interconnected, evolving, and growing. The process to develop our current story of change came about by an organic light bulb moment!

The notion of a circle fitted better in our First Nations’ ways of thinking and doing. Interconnection is important to First Nations’ peoples as well as a holistic approach where First Nations’ voices are privileged, and more organic thinking happens. It’s important to note that this story of change reflects the theory of Community First Development’s approach — communities’ priorities and theories of change could be different.

More information on the Story of Change is here: https://www.communityfirstdevelopment.org.au/story-of-change

A bit more information on the process of how it was developed is available here: https://www.communityfirstdevelopment.org.au/stories/storyofchange?rq=story%20of%20change

How was your yarning tool different from a semi-structured interview technique?

The semi structured interview is more targeted and still involves yarning and deep listening. There is a greater richness of interviews that comes from Research Yarning that is based on relationships and trust. This takes time. One simple way to think of shifting to a Research Yarning approach is to understand that the ‘interviewee’ holds the power in the conversation to freely direct the conversation. The ‘interviewer’ is accepting that even if they feel the question has not been directly answered, there is a reason the ‘interviewee’ has taken the conversation in this direction. This requires deep listening, and to be flexible in the way questions and prompts are used.

I love the Research Yarning approach. Can you share a bit more about the visual prompts?

Visual prompts can provide a way to quickly convey a lot of information among the research team. In our participatory action research approach, this includes our community research partners. We also find that pictures and analogies promote reflection, conversation, connection and storytelling.

One of the prompts that we developed as part of the research project is the seed to tree scale and also illustrations of what ‘bridging governance’ might look like through a Venn diagram. The bridging governance illustration was very important in the end – we ended up adjusting our understanding all together based on how communities responded to it.

right way governance et al

Visual prompts in the Community First Development Yarning Tool developed as part of the Action Research Project. The one on the right is the version that was corrected based on what communities told us.

Visual prompts can be adapted to suit different situations. You can also use physical elements from your surroundings. An example of this is the Ten Seed Technique developed by Dr Ravi Jayakaran: https://www.communityfirstdevelopment.org.au/stories/the-ten-seed-technique.

You can use anything to prompt thinking and support where the community would like to go in terms of planning and goals. Some of our team have substituted in different objects readily available in the interview setting. For example, 10 plastic toys, through to 10 stones.

little bit lots

Community First Development’s adaptation of the Ten Seed Technique, using 10 plastic sheep.

The prompt questions that accompany the visuals, or the physical surroundings used, will always be dependent on the circumstances and type of relationship that you have with community. It’s imperative that the community is comfortable with the prompts, and when using this approach, that it is interactive and engaging.

With the yarning tool, I wonder if visual aids can be generic across a range of various projects to initiate yarning.

There would certainly be opportunity to have generic aids across a variety of projects. As an example, we have found the seed to tree scale has been adapted to multiple settings and used for multiple purposes beyond the research project. This has included strategic planning for communities. Most importantly, the visual aids should be something that communities are comfortable to use and will promote open ended conversation.

Metaphors are often used by community to share key messages and information, this links to our research yarning approach. While we use the Seed to Tree scale others might talk of boats/canoes, spears, baskets or other trees, plants or animals as a part of local cultural narratives. It would be an important part of the conversation about ‘what does success look like’ and ‘how do we know we progressing’?

For more information on the Seed to Tree scale, and how it can be adapted to different settings, visit our website: Seed to Tree scale – knowledge, practice and action — Community First Development.

seed to tree

Community First Development’s Seed to Tree scale.

Were there challenges developing these models as you are working with several different communities?

Our Community Development team are highly skilled in community engagement and had already built strong relationships with community research partners. This meant they had spent time understanding the unique aspects of each community. As a result, we were able to adapt and nuance our approach to Research Yarning and co-authoring case studies to each community setting.

In addition, we recognised there are inherent challenges with any research project working or partnering closely with communities. We took into account a range of aspects that can impact on how a project progress: Sorry Business, cultural obligations, shifting timelines. We maintained our flexible and adaptable approach, and this worked well.

Was it appropriate to record the yarning as audio or taking notes?

Permission is always sought before any recording takes place. Regardless of whether the community has pre-signed an ethics form at the start of a project, we will always check again beforehand, and then again before we share the information, even when it is de-identified. This is a reflection of the type of trusting relationship we build with communities, and the partnership approach we take in our Participatory Action Research. The community always has a choice on how and if they want to participate. There is no obligation, we operate in a culturally safe space, and this comes from years of experience.

Ethics

Doyen work

One of the communities who partnered with us in the research was the Aboriginal Males Healing Centre (AMHC). Pictured here is Uncle Colin Peterson, AMHC Council Elder; Devon Cuimara, AMHC CEO; and Djeran Cuimara, Devon’s son.

Are community members participating in the design phase paid for their expertise?

No, we are invited into the community by community, and as part of our work with communities monitoring and evaluation is tied into our agreement. As part of this approach, communities benefit from ongoing community development expertise from our team and specialised ongoing support from volunteers with cross-industry expertise.

We would encourage all evaluators to consider how any project involving Aboriginal and Torres Strait Islander people will benefit the community.

Did you need ethics approval for your research work? If so, how did you find the western ethics process - based on quite rigid pre-approved study designs - aligned with your more organic, ecosystem model?

Given the existing agreements we had in place with communities, and that the work was built into our community development activities, we were not formally required to receive ethics approval.

However, we chose to seek ethical approval through the Australian Institute of Aboriginal and Torres Strait Islander Studies (AIATSIS) Research Ethics Committee.

Like all of the work we do, we held fast to our values and principles as we navigated through a process that was new to us as an organisation. While the process required additional time, it was a worthwhile exercise that we would highly recommend to others. We realised that we had a lot of assumed knowledge about the ethical practices we have in place and the process pushed as to more clearly articulate these to a wider audience.

principles of practice

Community First Development principles of practice.

The downside was that we overcomplicated some aspects of the research design by trying to introduce technical language that wasn’t necessary in the end. This is something we would look out for in the future and was a good lesson learnt.

As a formal research project, it was beneficial to check we were covering off all ethical requirements, we found our ethics were to a high standard, but we often had different words and ways of describing the different aspects of ethics.

You might like to check out another resource we developed alongside some other First Nations’ evaluators, A protocol for ethical evaluation practice in Aboriginal and Torres Strait Islander settings. 

Great point about the potential to get tripped up by technical approaches when setting up M&E. On reflection do you think there were ways you could have started with your existing principles/values/approaches and built your M&E from that?

Our initial starting point was very much based on our principles and values. The feedback process through the ethics committee was where we were tripped up a little. On reflection, a helpful question when tackling feedback might be, “How does this feedback relate to ethics?” This frame would naturally guide as to answer the heart of the questions the ethics committee is wanting answered, and steer us away from getting caught up in technical aspects and methodologies within the research design.

Coding and Analysis

coding and analysis 

Online coding session with some of the Community First Development team.

Really keen to learn how you took a decolonised approach to coding and analysis. What was the approach used for developing evaluative criteria and synthesis methodology?

We did this by deep listening of the transcripts from the Research Yarning sessions and drawing out the key messages from what the communities were saying.

The approach we took was both deductive and inductive to break down key themes of what the community was saying and the key messages. We worked as a group and the criteria was discussed and agreed in a group setting, a very organic approach, with group members from across different regions in Australia. The relationship between our team and communities, as well as the relationships our team had with each other played a critical role in creating meaning, which was checked back with community to ensure we had the meaning and message right.

The key message was that cultural governance was significant to communities and the truth telling of the communities was also paramount. This is the start of any decolonisation process where the First Nations’ voices are privileged and central to the narrative. Once finished it was taken back to communities for clarity and permission to be used.

This process is something we plan to share about further on our website and will look for opportunities to link this in with the AES too.

Get involved!

Disappointing the funding fell through. Is funding essential to continuing your work? What is your workaround for this?

As a registered charity, funding is an ongoing challenge we tackle year on year. How far we can go in our research is greatly impacted on whether we can secure funding. This project ran a year longer than anticipated due to a range of reasons: one was stretching resourcing to take on this new and exciting approach for the organisation.

We currently have a fundraising strategy that supports our community development program that incorporates monitoring and evaluation. Where possible, we are growing in our Participatory Action Research (PAR) as part of this program. If you would like to be involved and support this work, see the donation and volunteer pages on our website.

We appreciate your feedback.

Have any of the report findings or answers above been beneficial to you? If so, how? Please comment via and share this post to your networks.

---------------- 

Doyen

 

Doyen Radcliffe is a Yamatji Naaguja man and is Community First Development’s Regional Manager – Western Region. He has been instrumental in the design of Community First Development theories of change, M,E&L frameworks and participatory approaches and tools. Doyen is also a Board member of the Australian Evaluation Society.

DonnaDonna-Maree Stephens is an Iwaidja woman of the Muran clan, northwest Arnhemland with 30 years’ experience in education, research and evaluation in the Northern Territory; focused on community engagement, educational and workforce outcomes for First Nations’ people. Donna is Community First Development’s inaugural First Nations’ Fellow for Research and Evaluation. She is currently Social and Emotional Wellbeing (SEWB) Workforce Development and Support Unit Coordinator at the Aboriginal Medical Service Alliance of the NT (AMSANT).

SharonSharon Babyack is Community First Development’s General Manager Impact and Strategy. Sharon brings 20 years of experience working for government, and not-for-profit organisations. She provides strategic direction and leadership on monitoring, evaluation, and research for Community First Development. She has played a leadership role in the development and roll-out of Community First Development’s M,E&L approach and database.

A return to the farm: Lessons from evaluation advo...
Ethical practice in evaluation is everyone’s busin...