Artikel

Laatst bewerkt door John Doove op 12-10-2012 14:13

10 Questions and Answers about CurriM

 

10 Questions and Answers about CurriM you may like to know (notes for the phone interview with Surf on October 9, 2012)

Q1. What is the reason for the TUE to develop the curriculum mining tool?

A1. Learning analytics and educational data mining in general and curriculum mining in particular have been recognized at TUE to be important and potentially useful as it can improve the quality of education. Curriculum mining is believed to feasible, i.e. possible to develop the corresponding tools and methodologies. At the Department of Computer Science we have a extensive experience and expertise in educational data mining (data mining and process mining in general), in adaptive educational hypermedia, in databases and data warehousing and in other related areas. TUE has accumulated students’ performance data over past 10-15 years. So, actually there is data to be mined. Last but not least CurriM comes at the right time – our study curriculum becomes more flexible, i.e. there are more opportunities to personalize an individual study plan, but this also implies that there are more “opportunities” for the students to fall into a trap taking a suboptimal curriculum path.

 

Q2. How does process mining work?

A2. Educational process/curriculum mining uses three main kinds of techniques:

(1) techniques for the educational process discovery including explorative data analysis, descriptive and predictive modeling (who will graduate when, whether someone is likely to pass a course, graduate or drop out, what the actual currimulum is and what its bottlenecks are);

(2) techniques for the educational process conformance checking (check whether an individual curriculum of a student conforms to the strict official regulations of the curriculum or there are some violations; whether recommendations are taken into account or not); and

(3) techniques for the extension of the formalized/explicit knowledge about the existing curriculum, e.g. showing success rates of students following particular paths.

If you eager to know more about the basics ideas behind these approaches, you may want to check:

Process Mining book by Wil van der Aalst, 

Introduction to Data Mining by Pang-Ning Tan et al. 

Handbook on Educational Data Mining by Romero et al. 

 

Q3. Does the tool work with existing data (quick wins)?

A3. Yes, and no. To be concise: there are no quick wins in the sense that a few month project won’t provide the required technology, infrastructure and support for adopting curriculum mining. There is a substantial amount of work to be done to bring curriculum mining to daily practice.

We use a generic approach allowing for use of a standardized input like e.g. MXML format of ProM tool. Basic data like student_id, course_id, semester_id, and grade can be easily imported from any database that stores such information.

So, if we take an ideal situation where the data is complete, consistent and clean, we can hope for good results to come out. But when it comes to practice we need to deal with lots of challenges. In case of TUE as with many other institutions, the data reflects the corresponding use practices relevant for a particular period in time. That data collection and management processes were good enough for the administrative needs, but not necessarily good for curriculum modeling.

To give a few examples – at TUE the curriculum changed from semesters to trimesters to semesters to quartiles in the past several years.

So, the course codes change over the years. These changes are not known to a computer. Therefore, it is difficult to keep all the data of one course together.  In general, if mappings of course ids from older to newer curriculum have not been put into a knowledge base, so it is not possible to make an automated reasoning about such changes;

Also, one can ask herself whether courses of 1992 can be compared to courses of 2012. It is likely that content of courses changes substantially over time and it is given in different ways by different teachers. A domain expert should be able to map the changes of the course codes each year, or remove courses for which can be said that they are not relevant for the later years.

Another typical difficulty is that for quite some students, there is an ambiguity whether they finished their study successfully or dropped or suspended their studies.

Until the very recent past in the database only the (first) attempt for an exam would be the first signal that the student actually attempted to follow this course.

These issues were not important for the administrative applications built on the top of the transactional database, but they become a serious barrier for the automated data analysis tools.

Besides, the data without meta-data or knowledge about the educational processes, present in completely separate sources like study guides (explicit), and in heads of the responsible educators (implicit) can easily mislead any intelligent data analysis tool.

There are study guides, but no formal curriculum model could be constructed by each faculty for each program. Of course, the faculties should not be blamed for this. It is essential to have right tools developed to make the curriculum modeling easy and transparent.

This is one of reasons why in this sense there are no quick wins as such.

Last but not least, after all the CurriM related challenged are overcome, it would be still a laborious process of developing and integrating this learning analytics tool for curriculum modeling into the existing infrastructure of the universities. Therefore, at the moment we consider it as a development of a stand-alone tool to be promoted for use in parallel with the existing infrastructure.


Q4. Which data-patterns you can recognize, for example?

A4. Anything predefined to be a pattern (template) can be recognized – as for discovery, as for the conformance or extension. The most useful patterns we currently utilize include:

Prerequisites – “the course A should be followed before taking course B”. And in particular, “Course A with grade advised grade not less than X should be passed before taking course B”.

Two or more courses that are (or are not) advised to be taken simultaneously.

A student must select M out of N courses and alike.


Q5. How can it help students (personal education route)?

A5. Awareness global – how good is their personal education route (does is conform with the official regulations?, is it realistic and would it allow to graduate in time?, would it allow to have a right job profile?), where they currently are in (their personal education route) curriculum, how they do in comparison with the averages and the top 10%, and awareness local – what is likely to happen if they take course X (in the next trimester/semester/quartile, or later).


Q6. Can it also help teachers?

A6. Yes, it can!

1) Indicate bottlenecks in the curriculum. This is not just about the courses that are often being failed, but course failures that are likely to cause a cascade of failures.

2) Suggest new prerequisites and other non-binding advice to the students.

3) Provide insights into the real (not assumed) curriculum

We did discuss and demonstrate CurriM to a few teachers and people involved in the education management. They can clear see the prospective benefit and appreciate the development of the tool.


Q7. What is/can be the effect on overall study success/output?

A7. As many other kinds of awareness tools e.g. in healthcare and wellbeing, we believe that the students will do better planning and monitoring of their own progress, get an evidence based motivation to raise their thresholds for the expected performance for the courses being crucial for their personal education route (and/or future career).  


Q8. And what is/can be the effect on the quality of education at the TUE?
A8. It goes without saying that we can consciously improve on certain aspects of the educational process only if we are aware of the current limitations.

The focus of this project was on helping individual students and teachers.  However, one of the big advantages of our tools is that it provides useful insights for the education management, that is, for the people who define the curriculum.

The curriculum modeling and awareness tool we have been developing can help to become aware of bottlenecks in the curriculum, so both teachers (and education management) and students can put special attention to the corresponding courses. For students it is important to know both explicit and implicit connections between the courses.

The more students are aware of where they are in the curriculum the more confident they can feel about their educational path.

Awareness often introduces a healthy competition wrt oneself and may stimulate to get into the top group.

 

Q9. Does the tool predict bad or good study results?

A9. Making prediction about the future is always hard. It is especially hard to make reliable predictions for the individuals. Especially when we know in advance that we do not even observe lots of factors affecting the performance of the students.

Thus instead of taking a too high risk of making a mistaken prediction, we think that it is more valuable to project essential patterns observed in the historical data.

 

Q10. Is the tool already a success? And will the TUE pass on with Learning Analytics/Curriculum Mining?

A10. The prototype of the software looks promising, we managed to demonstrate the concept and it has being liked by the students and lecturers. In this sense it is a success. Also, we did demonstrate that with slight adaptation of existing data mining and process mining techniques it is already possible to achieve useful results.

But it is too early to talk about a real tool and its success. Much more serious financial support and longer term commitments are needed to develop a fully functional and truly usable solution. We hope this pilot project continues and our efforts will help to change the educational process for better.

As a person with a strong belief in educational data mining and learning analytics, I hope that TUE will become more active in this area, not just in research but also in adopting it into daily practice. However, as with many other important developments, it will be a subject of funding availability.

Thanks to Surf we participated in this pilot project and made a small, but an important step in bringing learning analytics closer to the prospective end users, i.e. students and educators.

We hope that there will be a more serious follow up program that would make it possible to shift from the initial explorative to actual R&D and follow up user testing stages.

Once again – we consider CurriM as a pilot for a longer term project aiming for the technological innovation that would help to improve further the current experience of students and teachers with curriculum planning, and to increase the global i.e. overall awareness about the study curriculum for all the participants of the educational processes, from students to the education management.

Therefore, there are no quick wins with CurriM in the sense that a few month project won’t provide the required technology, infrastructure and support for adopting curriculum mining. There is a substantial amount of work to be done to bring curriculum mining to daily practice. This is not only about the existing data, this is about the effort required for curriculum knowledge engineering and making it accessible for both – computers and humans.


SIG's en subthema's: Learning Analytics

REACTIES


Geen reacties gevonden.

PLAATS REACTIE