Evaluate for integrity, inclusion & influence

Advice from the experts on community engagement evaluation

Evaluate for integrity, inclusion & influence
I asked engagement evaluation experts from across Australia to give me their top tips for community engagement. This post explores their advice and examines the benefits of engagement evaluation in more detail.

Evaluation is essential for best practice, innovative community engagement. But, it's one step in the engagement journey that public sector organisations still struggle to get right. But, never fear, our evaluation experts are coming to the rescue - both via this blog post and at upcoming evaluation training in Melbourne and Sydney.

Many thanks to our contributors for this post: Allison Hendricks (Darzin Software and Simply Stakeholders*), Max Hardy (Max Hardy Consulting), Don Sharples (AltometerBI) and Amanda Newbery (Articulous).

For many projects I’ve worked on, time is at a premium and community engagement is usually fitted in around other ‘more serious’ project delivery milestones. While this comes with the job to a certain extent, it’s not the best environment to foster a timely, authentic and meaningful engagement practice.

On project completion, a report on your project’s ‘community engagement activities’ will likely be required for compliance. However, many of us are rarely given time to reflect on the impact and effectiveness of our work. This is a shame, because evaluation helps us to understand what our engagement has achieved and how we can improve next time.

Through evaluation we explore whether our approach revealed insights that authentically influenced outcomes. The opportunity to be more ‘scientific’ about our approach and also grow and develop as practitioners. As you’ll read later in this article and see in the video – it’s evaluation that will help us to professionalise and legitimise our work as a sector.

Our definition of engagement evaluation

While many sector evaluate the effectiveness of their work. In the engagement sector, there are number of models we can use to measure the quality of our processes and outputs.

Some of the more common methods include:

  • The Brisbane Declaration measures: In 2005, a group of engagement practitioners signed the Brisbane Declaration at the International Conference on Engaging Communities. The declaration suggests that we evaluate our work for: integrity, inclusion, deliberation and influence. Allison Hendricks has created a guide for evaluation based on the standards - this will be presented during our upcoming evaluation training in Melbourne and Sydney.
  • IAP2 Evaluation considerations: evaluation is one of the courses people complete as a part of IAP2 certification. During that training, people are encouraged to think about ways they can measure: accessibility/reach, cost effectiveness, communication, feedback/input, and project management
What would you add to this definition? Do you agree or disagree? Join the conversation on Facebook or LinkedIn.

Benefits and advice for improving evaluation

What are the benefits of investing in engagement evaluation?

Allison: If you are interested in improving the process or the outcome of your community engagement, then you really need to be evaluating and reporting. It's like the old saying, what gets measured, gets done... and with community engagement, what gets measured often drives the process design and, sometimes, even the outcome.

Max: You start to measure things that count, and things that encourage much better behaviour with the whole project team. For example: What did you learn from the public? Do they feel like their concerns and interests are being understood? Are their contributions making a difference? Even though this might be challenging to measure at the start, it will revolutionise the way you work.

Don: Credibility is built within an organisation because of good evaluation. Recently, a client found a great deal of success engaging with their own internal stakeholders after they shared the evaluation they had been doing. It totally changed the way engagement was seen in the organisation, and opened up a range of doors for them to try new things.

Amanda: Evaluation might not be the sexiest or the most or the most creative part of our sector, but it is a critical element in allowing our profession to grow it's legitimacy and earn respect within organisations. It also allows us to demonstrate the true value that community engagement brings to projects, policies and organisational reform.

Megan: For digital engagement especially, evaluation helps us keep an eye on what's working and what we can do better next time. It's also crucial for getting blessing from your team or organisation to invest in similar (or different) tools and techniques next time.

What's your top tip for improving engagement evaluation?

Allison: What you ask and how you ask it is really important. Even if you ask just one question, make it meaningful, make it actionable. Remember, you're looking for insights you can use to improve the process or your outcomes.

Max: Start tuning into things that really matter, instead of just counting the number of 'bums on seats' or the number of 'meetings we held'.

Don: It's really hard to evaluate if you don't have a clear idea of what you set out to do in the first place. I help my clients to set clear goals at the start of a project, so we have measure of success to refer back to at the end. It's also good not to fear reaching those targets - use them as a guide. Not meeting your goals is better than not having any at all.

Amanda: As engagement continues to be incorporated into legislation there will be an increased requirement for our sector to evaluate its own practice. This includes 'are we delivering good process?',  'are we generally listening to what people want' and importantly ' what is the return on investment?'

Megan: If you're just starting out on upping your evaluation game, start with simple questions and build from there. You could explore: 'did we use the right tools for the job?' or 'was my project team actually able to use the insights we collected?'

How can readers up their evaluation game?

Megan: The first step is to recognise what stands in between you and effective evaluation for your community engagement activities. Only then can you start taking steps toward making a change.

While the reasons we don’t always evaluate effectively are many and varied, I’ve listed a few of the more common challenges and causes I’ve experienced below. There is no easy or one-size-fits-all solution because each organisation is different, but here are some of my thoughts/suggestions for moving in the right direction.

As with all innovation and improvement projects, my advice is to start with a series of smaller tests before locking yourself and your team into new evaluation frameworks that have not been road-tested sufficiently.

  1. Not enough time or budget allocated to evaluation might mean that decision-makers for your project (including you) have not allocated enough resources for evaluation. You'll need to discuss the benefits of evaluation and get buy-in for allocating more funds to these activities.
  2. No evaluation standards at your organisation might mean that evaluation is not embedded in your organisation’s community engagement culture. Have a conversation with other practitioners about how they’ve tackled evaluation for their organisation. They may have templates and guidelines you can draw on. Professional networking groups like Engage 2 Act also have online communities where you can seek advice online, and you can join the LinkedIn and Facebook groups for free.
  3. Or, perhaps you simply want to acquire or refresh skills/capacity for evaluation... This means you understand evaluation is important but you’re looking to learn the latest techniques and thinking. There are training courses out there to help you with this. Future.Boutique is able to deliver courses for your organisation. The International Association of Public Participation (IAP2) also has an engagement training offering.

*Simply Stakeholders is a Future.Boutique client. 

While you're here

Check out our other blog posts on community engagement. Thanks for taking the time to read our stuff! 🙂
What is digital deliberation?
Uncategorized add comment

What is digital deliberation?

Max Hardy and I caught up recently to discuss what makes digital deliberation possible and plan a groundbreaking new training program on the topic. Max is a world-leading deliberation guru who’s designed Citizens’ Juries and other varieties of deliberation since 1999. I’ve been working at the intersection of community engagement and technology since 2006. Together,…

Leave a Reply

Your email address will not be published. Required fields are marked *