Recent analysis of the diversity of UK university and research institute governing bodies has shown that governing boards do not reflect the diversity of their organisations, the wider society, and most importantly, the communities they serve (Leman, 2022; McMaster Codiroli, 2021). This discrepancy prompted the Wellcome Trust to commission Advance HE to design, deliver and evaluate the ‘Success on the Board’ programme, a bespoke development offer which aims to unlock the power of underrepresented voices in HE and research boards, governing bodies, and committees.
Success on the Board is a comprehensive development programme running over the course of one year, building participants’ knowledge of governing boards and equipping them with the skills needed to access board opportunities. The programme involves a combination of scheduled activities, as well as the appointment of an institutional champion. As part of the initial funding, Advance HE’s Insights team evaluated the programme’s content and impact across the first year of the programme’s implementation. This evaluation was embedded in the programme’s design, as the intention was to provide evidence of its success and ensure the programme’s continued funding.
Method
The focus of the pilot programme evaluation was the change brought to the individuals that made up the inaugural cohort: to assess the impact that engagement with the programme had on participants’ board-relevant knowledge, skills and confidence.
A pre- and post-test design was adopted to monitor the programme’s individual-level impact. In other words, evidence was collected from the participants through a bespoke online survey one week before their engagement with the programme (pre-test) and within one month after the programme’s conclusion (post-test). The differences in participants’ scores before and after participating in the programme were used to index changes in their board-relevant knowledge, skills and confidence.
The online surveys’ content was based on the programme’s intended learning outcomes and was almost identical, to allow for direct comparisons, although some questions related to overall programme satisfaction and suggestions for improvement were added at the post stage. Background personal characteristics were also collected to be able to have evidence on the cohort’s population make-up. Although the surveys included mostly close-ended questions (e.g. satisfaction/agreement statements to be rated using a Likert scale), some open-ended questions were also included to allow participants to express their experience and perceived gains from engaging with the programme in more detail.
Outcomes and impact
Advance HE produced a comprehensive evaluation report, which clearly outlined how the programme impacted participants. In its first year of running, the Success on the Board programme met its intended learning outcomes, with participants showing improved board knowledge, skills and confidence as well as increased board aspirations and achievements.
The evaluation showed that the biggest improvements were in participants’ perceived knowledge of:
- How boards operate.
- The selection processes for board positions.
This was particularly important as these areas presented some of the lowest ratings in the pre-engagement survey, thus providing further evidence that the programme benefited participants in the areas in which they perceived they were mostly in need.
Participants were very satisfied with their experience, being vocal about how it inspired them to pursue board opportunities.
Participants’ perceived benefits from engaging with the programme broadly matched their expected gains. The two most popular reported gains were:
- An extended understanding of how boards operate.
- Being able to identify one’s unique contributions to a board.
When prompted to describe how their practice has changed since engaging with the programme, participants mentioned either having already attained a board/committee role or being actively involved in attaining one. Specifically, the evaluation showed that:
63% of participants had submitted board applications since engaging with Success on the Board.
From the total of 17 board applications submitted by members of the first Success on the Board cohort, 30% were successful.
Based on these results, the programme’s funding has continued, with a second cohort recently embarking on their Success on the Board journey.
We’re really happy to support Success on the board for a second year. It’s great to see Wellcome’s continued enthusiasm for influencing who participates in higher education governing bodies and boards – an important piece of the larger puzzle to help get inclusion work as a larger priority for the research sector."Dr Diego Baptista, Head of Research and Funding Equity at Wellcome Trust
With evaluation still embedded in the programme’s delivery, we will continue to establish an evidence base for how it contributes to participants’ knowledge of governing board systems and processes, and their ability to successfully take up opportunities to join these.
Related links
McMaster Codiroli, N. (2020). Diversity of Governors in Higher Education. York: Advance HE.
Leman, J. (2022). Diversity of Governors in Higher Education. York: Advance HE.
Couch, L. (2021). How we’re putting diversity, equity and inclusion at the heart of our strategy. London: Wellcome Trust.
Wellcome Trust. Research culture: let's reimagine how we work together.
About the Researcher
Dr Panagiota (Peny) Sotiropoulou, Mixed Methods Researcher
Since joining Advance HE in September 2021, Peny has extensive experience in programme evaluation, leading our internal evaluations of the Aurora and Diversifying Leadership programmes, in addition to those embedded to our bespoke consultancy programmes like Success on the Board. Read some of her recent work here.
Peny has also further developed her EDI expertise in HE through getting involved with a wide array of projects, ranging from reviewing barriers to doctoral funding to institutional reporting and complaints processes. She is heavily involved in the production and dissemination of Advance HE’s annual Equality in Higher Education: Statistical Reports, as she loves to engage in outreach activities promoting EDI considerations to various audiences.
About our programme evaluations
We offer programme evaluations that can capture individual-level as well as institutional-level insights. We adopt a variety of methods, depending on the type and content of the programme being evaluated, offering a bespoke approach to measuring impact. Typically we include the perspective of multiple stakeholders to gain a holistic overview and provide insightful recommendations to institutions.
- Bespoke pre and post surveys: time-efficient and enables a broader perspective by including more participants and capturing data on multiple topics simultaneously (individual-level, institutional-level or both).
- Potential of follow-up surveys (e.g. at 6 months or a year later) to build a longitudinal perspective and measure changes that take longer to come to fruition.
- Potential to add control group to enable causal inferences.
- Interviews or reflective journals: a targeted approach to gaining deeper insights into participants’ and/or other key stakeholders’ experiences and perceptions of the programme and its impact.
- Focus groups or discussion forums: showcase the voices of different groups of individuals and allow unique experiences to be captured, as well as obtain valuable information on what works and what needs improving.
Each method has its own strengths and limitations, which complement each other, when employed in tandem. In this example, we applied this approach to evaluate the impact of development programme. However, the pre- and post-test design could just as easily be used to provide an institution with practical insights into the impact of training, teaching and learning interventions as well as newly implemented policies.
Contact insights@advance-he.ac.uk to learn how our bespoke research consultancy can be applied to your context.