No surprise, people don’t always like to “learn things about themselves about what’s gone wrong in someone else’s eyes,” says Sean Holowaychuk. “But if you want to get better, evaluation is part of the process.” With Chris Bennett, Holowaychuk teaches a dive-right-into-it Global Seminar to UBC students in Indonesia (virtual in 2022). The course guides students on valuing and giving accountability to international aid projects “because the No. 1 reason these types of projects fail is a lack of independent evaluation,” he notes.
Holowaychuk is a Food and Resource Economics instructor and session lecturer with the UBC Faculty of Land and Food Systems. Bennet is an adjunct professor in the same department, and the duo has been dropping students in the deep end, so to speak, for the past six years at partner IPB University outside Jakarta. Both do project evaluation in international development professionally. The course, Impact Evaluations Analytics, is a Global Seminar, one of many offered virtually, hybrid and in-person by the Office of the Vice-Provost International’s Go Global unit. Here are some of Holowaychuk’s thoughts.
Offered by Go Global, the UBC Global Seminar's focus is adding value to development project initiatives. A short video captures one student's experience in Indonesia (Credit: Philip Bertog)
The aim: To add value to development project initiatives – for example, assistance or aid programs implemented by governments, institutions or NGOs to correct market failures, such as low literacy levels in rural areas or watershed mismanagement upstream of local villages. Our course teaches how to apply accountability: that is, measuring the right things, in the right way, at the right time. Funding agencies, donors and taxpayers want a better understanding of where their money is going, and our role is to provide some evidence of what works. Chris (Bennett) and I have a criteria we’ve developed working together, the Impact Evaluation Analytics methodology.
What success looks like: We’ve seen real change in the way communities around IPB University approach project interventions over the six years we have offered this course. Our students have laid a foundation that each consecutive cohort builds upon. They ask tough questions and hold organizations accountable, which has led to improved outcomes for partner-beneficiaries.
The challenge: By default, evaluation is seen as a negative experience. Our job is to turn that around. It’s a tricky business. We hope students will leave with the tools to provide an accurate assessment of a project that adds value and promotes the success of future initiatives. We use international development as a setting, but the broad application of the instruments learned here are a critical element of successful project management – whether it is in the agriculture or the resource sector, food services industry, environmental sciences, forestry, nutrition and healthcare, education or banking and business-related fields.
UBC Global Seminar students work with Indonesia's IPB University on small-scale community projects in areas such as healthcare, waste management and agriculture (photo: Chris Bennett)
Your approach: We make it a very inclusive process of learning-by-doing. For students, we give them our methodology, but not a lot of answers. Learning becomes an iterative process of some time spent in the classroom; then taking that knowledge out in the field; then back to the classroom to debrief what they discovered. Then the cycle starts over again. Students are learning on the fly and a lot of failure goes on. To be successful, students have to work together to solve problems, present their findings to IPB University and then submit an evaluation report, as they would working in the development sector.
Biggest student takeaway: Ultimately, an appreciation of the challenges faced by both the communities who are receiving support in the form of services or knowledge or infrastructure, as well as the institutions, such as governments and NGOs, who attempt to help by improving local economic, environmental health and education conditions through projects.
Special about this Global Seminar: It’s an excellent way to get some experience in the development sector on a small scale. Our partner, IPB University in Indonesia, has diverse development project initiatives in the community, which are appealing across different faculties at UBC. They range from small-scale economic enterprises, such as goat or fish farming, to women’s healthcare programs, to waste management and composting initiatives in communities, community agroforestry and watershed management.
An Indonesian community leader who has collaborated with UBC's Impact the Evaluations Analytics Global Seminar group (photo: Katie Stannard)
What UBC needs more of: A multidisciplinary approach. All market failures have economic, environmental, social and political components. Our course attracts students from all over campus. It’s a real good mix when you have arts students working with forestry students, working with global resource students, working with geography students, working with econ students to solve these problems from different angles.
Less of? Silos. The world is made up of complex systems. And beginning with a multidisciplinary approach is a good starting point to solving problems.
What works best? We use simple examples. For instance, "help me plan my eight year old’s birthday party." Then we show how the best practices in project management – planning a party – translate and scale-up to rolling out a multi-million-dollar watershed management initiative. The project has to be logical, relevant, include a communication plan with stakeholder engagement, and the project has to have a net positive impact on people’s welfare. The same principles apply, and we show students the right way to evaluate this.
Partnering advice: It takes time to gain trust. But you have to begin with a trajectory through understanding, which leads to mutual respect, and you eventually get to “trust.” Like I’ve said, project evaluation, by default, can be seen as a negative experience, but our students over the years have worked hard to leave the communities where they’ve worked in a better place. And because of that, our course has earned the respect of the project partners. There’s been an evolution in how we’re viewed – we’ve been able to learn from them and they’ve been able to learn from us.
What’s key when working with partners? It’s about inclusivity. You need to include them in the process from the start. We try to inform partners and stakeholders about why we’re in their communities, why we’re asking questions, what students are doing and our reasons for being there – and that we can both profit from this experience.
What the world needs now: Ha! Better evaluation…
Small farming enterprises, such as this terraced agricultural operation in Indonesia, is one area to which students apply their impact evaluation analytics (photo: Chris Bennett)
Read more about the UBC Global Seminar, Impact Evaluations Analytics.
Find out more about UBC Global Seminars – “international experience while exploring a unique topic or location.”