Learning Analytics 2018 – An updated perspective

There are multiple definitions of learning analytics, but the industry preferred one comes from the International Conference on Learning Analytics (LAK 2011):

“Learning analytics is the measurement, collection, analysis, and reporting of data about learners and their contexts, for purposes of understanding and optimizing learning and the environments in which it occurs.”

 

There are three crucial elements involved in this definition

• Data: Data is the primary analytics asset. Data is the raw material that gets transformed into analytical insights.
• Analysis: Analysis is the process of adding intelligence to data using algorithms.
• Action: Action is the most important aspect of the definition. Taking action is the ultimate goal of any learning analytics process. The results of follow-up actions will determine the success or failure of our analytical efforts.

Let’s discuss these components one by one

Data

Learning analytics data is the set of information collected about the student, the learning environment, the learning interactions, and the learning outcomes. This information is usually gathered while the learning process is taking place.

Data comes from many sources:

• Student Information Systems (SIS): provide demographic and academic data.
• Learning Management Systems (LMS): provide students’ activity records as well as students’ performance information.
• Other systems: provide multiple types of information. Depending on how far we want to take our data collection, we could potentially collect library records, e-book consumption patterns, social network information, etc.

As you can see, data collection could potentially go far beyond the LMS.

It is also important to highlight that data is the primary raw material for learning analytics. Data is a very sensitive asset. There is a statement that summarizes what should be our approach towards data collection:

“Collect as much useful data as possible and as less sensitive data as required.”

But are we dealing with big data or small data?

Learning analytics is sometimes referred to as “big data in education”; however, when we run a learning analytics project, we won’t be dealing with “big data” as such.

Just to be precise, big data is defined as data generated at a high volume, high velocity, having a huge variety of sources, and having a limited level of veracity. Some of these characteristics (known as the 4 Vs of big data) may not apply to our collected data.

However, big data and mid/small data analysis techniques share a set of common machine learning and analytical algorithms.

Real big data processing requires a set of specific computing tools capable of dealing with the complexity of the 4 Vs.

Data Analysis

Data analysis is the process of obtaining actionable insights from the collected data. It is based on a set of mathematical and statistical algorithms known as machine learning techniques.

Generally speaking, more sophisticated data analysis algorithms lead to better insights. However, complex algorithms also impose higher demands on the collected data in terms of volume, type, timeframe, etc.

As a result of our data analysis, we can obtain two types of learning analytics:

• Descriptive learning analytics: These types of analytics are reactive. They allow understanding of the past and, based on this understanding, influence the future.
• Predictive learning analytics: These types of analytics are proactive. They influence the present and, therefore, improve ongoing learning processes.

Crossing the bridge that separates descriptive from predictive analytics requires the right data and the appropriate algorithms.

Selecting the right data and algorithms is the art behind properly implemented learning analytics processes.

Action

Taking action is the ultimate goal of any learning analytics process. The absence of action is a complete failure. It doesn’t matter how good or bad your predictions are if you are not willing to or are unable to act on them.

Taking action is a matter of leadership and culture:

• Leadership because analytics triggered actions need to be fully supported by the leadership team of your institution, school, company, etc.
• Culture because organizations need to create an internal data-driven culture. Intervention following a data analysis process should be the common behavior pattern for your organization.

At the end of the day, the right internal processes need to be in place for interventions to happen.

There is another aspect that needs to be considered.

What are we going to do when our analytical insights show students at risk, failed learning processes, or low-quality training content? Are we willing to let it go with no action? If this is the case, we will have to face other ethical considerations.

Benefits of learning analytics

There are many benefits of using learning analytics, as follows:

• Increase retention and performance: Learning analytics may be used to reduce dropout rates and increase students’ performance. Having the right insights allow for performing proactive tutoring and intervention.
• Improve content and course quality: Learning analytics may be used to discover content consumption patterns, understand content quality issues, and provide personalized learning experiences (adaptive learning).
• Proactively drive success: Learning analytics may be used to identify and promote success factors as well as to understand students’ pathways leading to graduation (curriculum design).
• Allocate costs efficiently: Learning analytics may help in discovering which resources work and which don’t. Selective investment strategies may well be designed based on our analytics.

At the end of the day, it is about improving students’ learning experiences and making sure funds are used in the areas that best contribute to student development.

Does it work? Is this a new learning era… or not?

Case studies show:

• the validity of predictive models applied to learning analytics,
• interventions with at-risk students are effective, and
• there are multiple benefits of following a data-driven approach to education.

The second question is a little bit more difficult to answer. If you look at how learning analytics is being described, it really looks promising:

“Learning Analytics: The coming third wave” – Malcolm Brown, Director EDUCAUSE

“The predictive analytics revolution” – EDUCAUSE ECAR Working Group Paper

“Adaptive Learning Holds Promise for the Future of Higher Education” – Barnes & Noble at EDUCATION DIVE

The reality is:

• the world is more and more data-driven – education is no exception,
• learning analytics is an important tool to improve education and to make educational institutions more competitive,
• learning analytics is useful only if there is “action” as a result of its implementation, and
• institutions must implement predictive analytics to really benefit from a new way of driving student success.

How do I implement my learning analytics project?

Implementing your first learning analytics project may be challenging. Nevertheless, there are a number of key elements that need to be considered:

• Start small: Implement an initial limited project. Limit your scope and duration. Don’t start with an institution-wide project.
• Don’t focus on technology: Focus on specific problems you want to address with your learning analytics.
• Go after quick wins: Show by example that learning analytics may improve your current learning processes.
• From day one, involve all the critical stakeholders: This will help you to get their support in the long run.

If you want to follow a more formal approach towards analytics implementation, there are a number of learning analytics readiness assessment tools. These tools provide you with a defined score on each of the elements that are supposed to be relevant when trying to establish an analytics-driven culture.

Among those elements we can highlight:

• strategy and vision,
• culture,
• governance and processes,
• ethics and legal issues,
• investment,
• technology, and
• data

The more you score on each of these readiness components, the better equipped you are for your learning analytics journey.

Our recommendation: “Start your seed project.”

Your seed project will provide you with initial experience.

Your seed project will make you experience all the elements you will find in the future once you implement analytics on a larger scale.

More details on learning analytics implementation on our post  “15 Strategies you can follow today to successfully deploy your learning analytics tool“.

Technology Challenges

Like any other technology, learning analytics also faces some challenges.

We describe some of them in the following sections:

Data capture: Not all student interactions happen in a digital environment. Analytics can only take into account events that provide a digital trace. In other words, learning analytics has a partial view of the educational environment.

Data variety: As we already described, learning analytics goes beyond the LMS. Combining data coming from multiple sources and systems may be difficult.

Comparable analytics: How do we compare learning analytics metrics? At the moment, no open standards are defining relevant metrics and, more importantly, metrics significance.

Predictions accuracy: All statistical processes draw conclusions subject to an error. Whether this error is acceptable or not depends on its estimated magnitude and its practical relevance. Learning analytics will never be perfect.

Partial view: Learning processes go beyond what data tells us. Don’t forget we are dealing with humans. There is no equation capable of defining us in full…yet.

Data literacy: Analytics consumers need the skills to interpret analytics and apply the correct follow-up actions.

If we were to select a challenge, I would point to ethics as the most relevant aspect to consider. Ethics must be taken into account at all stages of the learning analytics process: data collection, analysis, and action.

Don’t forget we are predicting human failure or success. Therefore, how we come up with those predictions and how we act on them is critical.

Being highly sensitive to ethics is definitely a must.

Are learning analytics something new?

For all of us who have been in the e-Learning world for a while, talking about learning analytics is like calling an old friend by a new name.

Since I can remember, Learning Management Systems have provided statistics showing student progress, average grades, time spent on the platform, etc. So what is new in learning analytics?

I would highlight two aspects:

• Learning analytics goes beyond the LMS: We are not just restricted to data provided by our LMS. Quite the opposite, we are trying to mix this information with data coming from other systems. Our goal is to enrich LMS data to obtain relevant insights.
• Learning analytics should be predictive: The game is no longer about average time, average progress, etc. The game is about using machine learning algorithms to predict and proactively act.

Don’t let the industry confuse you. Target for proactive learning analytics.

Key takeaways…

Let me summarize some key takeaways from our discussion:

• Learning analytics is an important asset for educational institutions, providing them with important benefits but also additional
• Institutions must target predictive analytics and start influencing the future by changing the present.
• Learning analytics is about action. No action = failure. Get ready to act and change!
• Launch your analytics seed project: start small, don’t focus on technology, go after quick wins, and involve all the stakeholders.

Call to Action

If you enjoyed this post, share it on your social networks and make it viral!!!

In the meantime, remember…

“Target for predictive analytics and be highly sensitive to ethics.”

NOTES:

• Get the “Handbook of Learning Analytics” if you wish to have more detailed info about some of the topics discussed on this post.
• The bridge shown on the post diagrams has been taken from “Enciclopedia Británica”
• Icons made by smashicons from www.flaticon.com
• Originally posted at https://www.iadlearning.com/learning-analytics-2018/