Building genuine Reporting & Impact Monitoring Process for Digital Learning Projects

This blog is written by our team member who has drawn inference from his real life experience to highlight a very important component which CSR, NGOs and State Governments should consider while designing a digital learning project.

That’s me training a group of girl students, most of whose parents are factory workers in a village in Maharashtra. And like them, all of the class 10 students in this school and several others in Maharashtra would now be learning from their personalised learning tablets using Marathi medium digital content.

While I was training, one of the girls in the batch was fiddling with her tablet when she stumbled upon the reports section. She told me that she could see her name and the video which she had just watched. She asked me what was the number in front of that. She was amazed when I told her that, that’s the amount of time she has spent in watching the video.

What she said next was both disheartening and encouraging. Here’s why.

She said that next time her baba would scold her for only wasting time not really studying, she can show him these numbers on her tablet. And then convince him to let her continue her studies. She said, unlike boys in her village, it is tough for girls to go for higher education.

But she was hopeful and with the tablet she was holding, she felt will further help her do what she wants to do in her life. While that was empowering for the young girl, for me it was a very powerful emotion of hope and positivity.

Since the last 5 years, I have worked with many CSR foundations and NGOs from around the country who have partnered with us to take tablet based digital learning to underserved government school students.

Because of the kind of investments they make on such projects both in terms of money and their time, it is natural to expect them to be personally interested in knowing how the project is unfolding on the ground. And most of them are actually very genuinely interested. They care.

But practically, you cannot expect them to be on the ground everyday and understand the kind of impact being created or what are the gaps and therefore how we can plug them or if there is an issue and how can it be fixed.

What you need is a feedback mechanism. Now, it is not as if it hasn’t existed in the social impact space. Traditionally, the approach has been where someone visits the beneficiaries periodically, speaks with them, conducts a physical assessment, records the findings and presents them to the stakeholders.

There are two challenges in this approach.

First, it is cost intensive. Someone needs to travel to such far off areas, stay there and be on the ground while assessing the impact. That is a cost.

Second is scale. If your project spans across multiple states, hundreds of schools or thousands of students, how would you conduct a genuine impact study. You may choose to rely on samples, but it has the risk of not giving you the true picture.

Time is also an important factor. You may start a project today and then wait six months, maybe more to get an impact study report in your hands. If you see gaps now, you have already lost a considerable time in which you could have acted and done the course correction.

So, how do we fix this.

At iDream Education, one of our core principles is to make absolutely authentic user data easily available to the stakeholders whenever they want and from wherever they want.

Today, we are running tablet and smart class based projects in 15 states across the country. I am proud to share that we have built a very transparent system of genuine reporting and monitoring of projects anytime, anywhere.

I am going to very quickly tell you how.

First thing we did was rather very simple. We built an offline reporting application, which could track every instance of usage of our learning platform on the tablet by the student or on the smart class by the teacher. Which means that the moment the student/teacher logs in and starts using the learning platform to access learning content, the application starts recording real time, where did the user clicked, which action was performed and for how long.

This data is then saved offline on the device.

We then enabled this offline reporting application to start syncing all the usage data saved on the device to a cloud based dashboard. So, every time the learning tablet of the students or the Smart TV / Interactive Flat Panel running the Smart Class connects to the internet, entire usage data syncs to this online reporting dashboard.

Now, any CSR head or members of the CSR teams, NGO coordinators, State education departments or any other stakeholder involved in the project can log onto this dashboard anytime, anywhere and see the data and get to know the impact instantly. .

What you see is absolutely genuine data as it syncs straight from the device of the students with no manual intervention in between. Whatever is saved offline on the device, the exact same information with no chances of error is present on the dashboard for you to see.

Over the years, we have seen very clear advantages of doing the above. Sharing them below:

1. Continuous impact monitoring: You do not have to now wait for 6 months or the annual reports. Every time you log into the dashboard, you can check the reports instantly and understand the kind of impact your interventions are making on the ground
2. Quick course correction: Since you are getting authentic data instantly, it also allows you to see gaps. And rather than waiting for the year end reports, you can decide and act upon the strategies to plug them month on month and improve the social ROI
3. Enrich the user experience: This is very particular in our case. On our learning platform, we offer multiple categories of content like videos, projects, books, assessments and much more. Based on usage data, we can remind and guide students to look at other categories of content and improve their experience of using the tablet
4. Encourage students to cover up Historical Learning Gaps: Many times when we look at data, we observe students switching classes and looking at content of junior grades. This is extremely relevant for our government school students. Wherever we find such instances, we encourage students to joyfully look at the content from junior grades and cover up on their previous year learning gaps. A message of reassurance from us just motivates students to continue on this learning journey of bridging historical gaps and mastering all topics

To summarise, I would just say one thing. If you are planning on funding a digital learning project for government schools or directly for the students, make sure you build a system of instant reporting which can enable you to continuously track the project and achieve best of the project impact.

If you have any questions or would like to tell me about your experiences of reporting and impact monitoring, please do share it in the comments below or write to us at share@idreameducation.org

Originally published at https://www.idreameducation.org on March 11, 2021.