NarenThanikesh

Introduction

Expertiza

Expertiza is an open source project for school assignment management for instructors and students based on the Ruby on Rails framework. Expertiza allows the instructor to create new assignments and customize new or existing assignments. It also allows the instructor to create a list of topics the students can sign up for. Students can form teams in Expertiza to work on various projects and assignments. Students can also peer review other students' submissions. Expertiza supports submission across various document types, including the URLs and word documents. Expertiza provides a dashboard for all the assignments corresponding to a course and provides absolute control to the Instructors and Teaching Assistants. In addition to assignments, it encompasses peer reviews wherein participants are allowed to provide feedback anonymously about each other's work thereby providing scope for the better outcome. The due_date.rb file is responsible for informing the users about the deadline for submission of the each assignment. Due dates in Expertiza have their association with many other components like assignments, reviews etc.

Purpose

The purpose of the project is to improve the functionality of score calculation. The scores in expertiza were calculated by taking the sum of weighted scores to calculate the total every time it is called. This takes considerable amount of time as there is a calculation involved each time. The methodology behind score calculation in expertiza can be significantly improved. The design behind calculating and storing scores can be tweaked to offer significant performance improvements. To achieve this, it is ideal to store overall holistic stores in addition to individual criterion scores.

Problem Statement

The current drawback of the expertiza system lies in how the scores are stored: Expertiza stores the scores based on each response to each criterion, but no holistic scores are stored. As a result of this mechanism, the process slows down. This means that if we need to know what score user A gives to user B based on 100, we have to rely on the code to calculate it (since in answers table we only have the score that A gives B on each criterion).

To overcome this bottleneck, two additional mechanisms can be included for handling the holistic scores.

Overall Description

Expertiza currently calculates scores dynamically when a page is called by taking the weighted scores of each question into consideration. This calculation is a time consuming process with a considerable overhead that significantly affects the performance. To solve the issue we propose a solution to store the holistic scores in a database, retrieving it every time it is required. This reduces the calculation that is required every time the score is called. When the user wants to check the scores of a completed assignment, a method localDB_calc is called and expertiza retrieves the scores of the student from the database rather than calculating the total from weighted scores. But, when the scores of an ongoing assignment is required, a method on_the_fly_calc is called that calculates the current score of the user. To determine which method will be invoked(on_the_fly_calc and localDB_calc), the current system time will be compared with last_review_deadline

• If system time is greater than last_review_deadline, indicating that the deadline has passed, the method localDbcal will retrieve the holistic score from the database
• If system time is lesser than last_review_deadline, indicating that the deadline has not passed, the method on_the_fly_cal will calculate the holistic score and display it.

Tasks Identified

The two suggested mechanisms for handling the storage and calculation of holistic scores for peer-reviews, as specified in the requirements document are :

• on_the_fly_calc
• localDB_calc

on_the_fly_calc

When the assignment is still ongoing , this method is called and it calculates holistic scores over a set of data (in this case, a set of record in answers table, namely the responses from user A to user B on each criterion) and store it in localdbscores. This stored score is retrieved and printed when the user A clicks on the "Your scores" link.

In the current implementation, when a user A wants to view the score given by user B, user A's score(i.e. out of 100) will be calculated everytime using the scores given by user B for each criterion (i.e.. score out of 5).

Handles the calculation of reputations for each reviewer in the case of an ongoing assignment.

localDB_calc

Once the deadline for an assignment is completed, the method localDB_calc calculates the holistic score over a single db query (in this case, a query would be: “get the score that user A gives user B on assignment 999 on round 2”). The query can also ask to print the average of all the scores in round 1 and round2, this average will be retrieved from localdbscores and printed.

This method handles the calculation of reputations for each reviewer in the case of a finished assignment. (all the reputations are calculated and stored since they are finalized).

Database

To store the holistic scores when an assignment is completed, a database localdbscores needs to be created. The value from this db is retrieved and displayed when the scores of a completed assignment needs to be viewed. The database Design is shown below.

Database Design

A table localdbscores will be created as:

class CreateLocalDB<ActiveRecord::Migration

def self.up

create_table :localdb_scores do |t|

t.column :id, :integer

t.column :score, :integer

t.column :round, :integer

t.column :type, :string

t.column :reference_id, :integer

end

with id, score, round, type, reference_id as its attributes as shown in the table below

"localdbscores" will have the following scores

	t.integer "id", limit: 24	id	Primary Key, Auto generated
	t.string "Type", limit: 255	Type	Type of score: either final or intermediate
	t.integer "Score", limit: 24	Score	Total score calculated based on each response to each criterion
	t.integer "Round", limit:24	Round	The completed round number
	t.integer "Reference_id" , limit:45	Reference_id	Acts as the foreign Key

Implementation

Assignment.rb

A new function get_last_review_date is added which takes assignid as a parameter and returns the last review deadline of the assignment. The deadline obtained from AssignmentDueDate for the corresponding assignid is stored in last_review_deadline and is returned.

Parent Class:ScoreCal

ScoreCal stores the score calculated by its sub classes: on_the_fly_cal and LocalDbcal to the database. In the database, a new table localdbscore is created with the schema as explained in section 2.2. Depending upon the deadline_type, object of either on_the_fly_cal or LocalDbcal is created and the appropriate functions within the classes are called.

Sub class 1: on_the_fly_cal

This is a subclass of class Scorecal contains a method calc_score whose functionality is to compute the holistic score. An object of on_the_fly_cal is called when the current time has not yet surpassed the time stored in last_review_deadline variable which contains the review duedate indicating that the assignment is still ongoing and has not been completed. The individual scores are retrieved from the database ScoreView and is stored in QuestionnaireData from which the holistic score of a student is calculated .

When the user accesses the scores for a review, the current implementation does not store a holistic score but rather calculates the holistic score from the score for every individual criterion. To calculate the total holistic score for a specific round, the individual scores are weighted and added.

This functionality is achieved by on_the_fly_calc.

A snapshot of the system before implementing the changes is shown below.

Sub class 2: LocalDbcal

This is a subclass of Scorecal and contains a function calc_score in class localDbcal. An object of LocalDbcal is called when the current time has surpassed the time stored in last_review_deadline variable which contains the review due date indicating that the assignment has been completed. This subclass retrieves the score from the database localDbscores for a corresponding student. If the score is not found in the database localDbscores, the function calc_score of on_the_fly_cal class is called to calculate the total from the weighted scores of each question. This calculated score is then inserted into the database.

This score is now returned to the view from where it was called.

_review_table.html.erb

The current implementation of this was to calculate the score and display it which as we know is a time consuming process. It has been changed to check if the current time has crossed the last_review_deadline obtained in assignments.rb and call the corresponding class as required.

If the current date has surpassed the last_review_deadline, it is an indication that the due date has passed and the class localDbcal is called to retrieve a value from the database as mentioned above. On the other hand, on_the_fly_cal is called when the current date has not surpassed the last_review_deadline, indicating that the assignment is ongoing and the holistic score is calculated and passed to the view.