Our Evidence Engine: using data insights and business intelligence to improve learner outcomes

Today, we sat down with Business Intelligence & Learning Analytics Manager Shane Wohlers. Having held a number of Educational Technology leadership roles in both New Zealand and the UK over the past 20 years, Shane brings a wealth of knowledge to the role.

Shane’s team delivers data-driven insights and actionable intelligence across the business to help others make informed decisions, understand problems, and identify solutions.

Can you tell me a little more how your team works to support eCampus NZ learners?

Absolutely. In a nutshell, our team uses data to understand the learner experience, allowing the whole eCampus NZ team to make timely, effective decisions and provide the best possible experience for all learners.

Each learner’s experience and success matters greatly to us and we work hard to develop solutions and approaches that help our learners achieve excellent outcomes. This has led us to explore exciting new ways to help our learners learn using artificial intelligence, advanced data modelling, networked data provisioning, automation, data visualisation and storytelling. While those are great, it all circles back to being able to use data well and to help others. We constantly ask ourselves: “How will this improve the learner experience and learner outcomes?”

The online learning experience is different to blended and face-to-face delivery, with different challenges and opportunities, and we want to understand how we can improve that experience for our learners. Our team has considerable experience and expertise in delivering learning in a range of settings—both nationally and internationally and across different kinds of delivery modes and educational settings. It’s fantastic to be able to draw on that depth of knowledge to help solve the challenges of today.

As well as improving what we do now, we are also looking ahead to emerging trends and developments. We are especially interested in applying our learning and expertise to other modes of learning, such as micro-credentials and work-integrated learning, as well as broadening what we define as ‘success’ for our learners.

How does your team work with other teams within eCampus NZ to turn data insights into good learner outcomes?

For us, it comes down to dialogue, relationships, and trust.

We work with teams across eCampus NZ to understand what matters to them and the work they do—their challenges, their goals and priorities, and what success means for them. This shared ‘sense-making’ helps us to understand what data and information will be valuable, where to start looking, and how to craft it into something actionable, relevant, and impactful.

This is an iterative, shared process where we work together to identify what is of value and what will lead to change or improvement.

We are a little different to other traditional data and analytics teams as our relationships and connections to others across eCampus NZ are something we consciously advocate for and champion.  Those relationships help us to be better at what we do and to develop a deep and nuanced understanding of our business, our stakeholders, and our learners. That in turn helps us to serve others more effectively. It’s a deliberate and proactive strategy.

When I interviewed Curriculum and Quality Manager Jackie Rees, she emphasised the culture of continuous improvement that underpins the work we do at eCampus NZ. What role does your team play in this?

What my team does essentially provides an ‘evidence engine’ for continuous improvement.

It’s an extremely powerful way to strengthen and evolve continuous improvement activities. This helps us to improve our performance, reduce friction across our systems and processes, and to move faster and with purpose. It also helps us to answer those hard questions:  Why are things the way they are? What does it mean? Where should we look to make an impact? What should we do?

In real terms, our use of analytics has contributed to a 23% improvement in overall learner success rates in 15 months, with gains ranging from 9% to 21 % across NZQA levels.

Simply put, our capabilities enable us to generate actionable intelligence and insights, which in turn helps us improve what we are doing and how we do it.

What tools do you use to support business intelligence and learner analytics?

Our infrastructure and data architecture is shaped by our need to service a national network of ITPs and stakeholders.

We look for tools and solutions that will enhance that network, integrate with what our stakeholders and partners have, and deliver what we need.

Scale, data interoperability, and robustness matter to us. We also understand that the needs and capabilities of our partners can vary greatly and our approaches need to accommodate that.

We use an integrated Microsoft technology stack for our data warehousing; machine learning/artificial intelligence; and data visualisation and presentation.

What changes have been implemented as a result of these capabilities and insights?

Our analysis of our learners’ experiences with us has led to a number of targeted interventions and changes in how we support our learners.

For example, we looked closely at the way learners interact with our Level 3 courses and used these insights to implement a range of new support strategies—including how we engage with learners and form relationships with them. These strategies contributed to lifting learner success rates by 15% in 5 months and lifting our Māori learner success rate by almost 30%.

Perhaps our most significant recent example has been our commitment to a recently completed business and data transformation project, a project we call Kowhai. This project has propelled us forward significantly.

eCampus NZ is learning by doing—learning what it means to be a data-driven educational organisation and taking the necessary maturity steps to achieve this.

This transformation is cultural and deliberate. It involves all tiers of our organisation, from board Level through to executive, management and operational teams.

We have reframed and refined our goals, objectives, and initiatives in terms of data-driven decision-making and have crafted the necessary metrics and measures required to describe and understand how we can be successful in this new world.

This commitment positions data and actionable intelligence at the heart of what we do: our decision-making; how and what we prioritise; and identifying how we can best serve our learners, stakeholders and communities.

eCampus NZ uses an online tool called Turnitin to check your assignment files against the content of other websites and databases. Turnitin has informed us that they have now added AI writing detection capabilities to their plagiarism review tools.Click here for more details.