You are now in the main content area

Use Crowdmark in your course

What is Crowdmark?

Crowdmark is an online collaborative grading and analytics platform that helps educators, working alone or in teams, to more effectively evaluate student work. The platform allows for easy distribution and collection of student work, offers tools for the assembly and management of grading teams, and provides an online workflow that streamlines grading and the delivery of rich feedback to students.

What are the benefits to using Crowdmark in your course?

  • Integrated with D2L Brightspace
  • Graders collaborate with ease and are able to see results and comments of other graders
  • Digital implimentation of marking reduces the need to carry physical papers 
  • Efficiency gains in exam administration, grading, data recording and analysis

 Support

For help in using Crowdmark features, please contact support@crowdmark.com. If you need any assistance with the integration with D2L Brightspace, please email courses@ryerson.ca, call 416-979-5000 ext. 556806 or visit our contact page.

Getting Started

Access the Content tab in your D2L Brightspace course shell. Add a module to the left-hand navigation bar by clicking Add a module and call it Crowdmark. 

 

Once you see the title at the top of the page, click the Published indicator and change it to Draft. This is to ensure students do not access the Crowdmark module you have just created.

 

Click on the link you have just created. The Crowdmark page will open, click Sign in with Brightspace and then enter in your Ryerson email. Click Next.

You must verify your email before continuing. To do this, go to your email and verify your account through the link that will be sent to you. 

Once your email is verified, click Go to my courses.

Crowdmark will ask if you are a Student or an Instructional team member. Click Instructional team member and then select Ryerson University. Click Continue. 

 

Create assessment

Click the Import a Course button, select your course, and click Import Course. Once selected, the course will appear in the My Courses section of your Crowdmark homepage.

In My Courses, click on the course you would like to create an assessment for. 

 

Create an Assessment

1. Click the Create assessment button on the top right corner.

2. A window will pop up where you can title your assesssment. Click the Create assessment button once you are finished.

Type 

In the type menu, you will be asked to select which type of assessment would you like to create.

Administered assignments are for monitored tests or exams written in person. The assignments are created with Crowdmark assessment templates, and then printed and administered by an instructor. Then, the assignment is scanned and uploaded to Crowdmark for grading.

Assigned assignments are completed and submitted by students independantly, such as regular assignments or take home exams. The assignment is emailed to the student and then the student must either complete the assignment digitally and submit it as a file or print and complete it on paper and then scan and upload a pdf.

 

Template

Checkmark the options/features that you would like to add to your assessment. 

  • Double-sided printing allows for the assessment sheets to be printed double-sided. If you will not be printing this assessment double-sided, please uncheck the box. Crowdmark needs this information to decide the placement of the QR codes in the printable file. 

The following features are in beta and may not work reliably, so we do not reccomend using them yet.

  • Enable automated matching will add a region to the cover page of your template for students to write first name, last name and student ID number. Crowdmark will then scan the page to automatically identify the student. 
  • Include multiple choice will add a 40-question bubble sheet to the end of each student’s booklet, then grade it automatically. Alternatively, we reccomend that you check out using Akindi for your courses bubble sheet exams.

An assessment template is the questions booklet that you will be printing and handing out to students. This menu allows you to upload a PDF of the assessment you have pre-created. Either drop your PDF into the box or click on the button to upload your PDF. This page can be formatted however you like, and can be any size — including 8.5" x 11", legal, A4, portrait or landscape format. Make sure to leave room under each question for the student to write out their answer.

After you upload your template, Crowdmark will add a unique QR code to each page and generate a printable PDF file containing a booklet for each student. 

It is important that you leave some space between the top of the page and the question, the QR codes are placed in this region. 

Once you upload your assessment template, you can review the pages with generated QR codes. If you are happy with the layout, click Next to continue with the setup. If not, click Try again to upload a different file.

During this step, be sure to verify that the QR codes aren't accidentally covering any of the questions.

Questions

In the questions menu, you will need to define the locations of each question (and subquestion) on your template. Crowdmark will automatically add a question label and a points label on each page of your assessment template where it identifies a question. The system will only detect 1 question per page, but you can manually add questions to include multiple.

If the automated system has missed a question, click Add question on the right and then drag the question on the page to change its location. You can also drag to reposition the existing question labels.

You may also edit the maximum score allocated for each question by editing the Max. points box on the side.

Once you are satisfied with the question label placements, click Save and Continue located at the bottom of the page.

Students

In the students menu, you are able to import the students in your courseshell from D2L Brightspace to Crowdmark. Click on import students from Brightspace and then review their details. Once finished, click Enroll students.

Generate

In the generate menu, click on Generate PDF to generate PDF files for the assessment. These files can be printed yourself or sent to Ricoh to be printed for you.

Once you have downloaded the PDF files, click Finished! Go to Dashboard. 

 

 

Assignment Details

  1. Set the due date and time of your assignment. Students will be able to submit and edit their submissions up until the time set.
  2. Automatically set the lateness penalty per day to be deducted from the final mark your teaching team allocates to the student.
  3. Allowing group submissions allows students to make their own groups and submit per group instead of individually. These groups can be set manually by an instructor, however students will still be able to modify them themselves.

 

Questions

You can write an optional assignment description to be displayed at the top of the page. This statement should include any general instructions about the assignment, but no specific questions.

You can create as many questions as you need by pressing the blue Add another question button at the bottom of the page.

  1. The label is the identifyer for each question. By default Crowdmark labels each question using the Q1, Q2, Q3 prefix, but this can be modified as every question has a unique identifier. It is not reccomended that these labels contain words and should be kept as short and consise as possible.
  2. The max points are the highest score the grader will be able to allocate to each question.
  3. The Question Content window is where you will write in the question for the student to answer. This window can use special formatting (click on Formatting tips for instructions) and can include attachments by clicking on Attach Files.

When you are finished, click Save and continue.

Adding Students

Press the blue button in the centre of the screen to automatically load your D2L Brightspace student roster into the assignment.

You can then look over the student roster to make sure everything is correct and then click Enroll students.

Manage Groups

Go to your assignment dashboard by clicking on Courses in the purple left-hand sidebar, then the name of your course, and then on the assignment name. Then, click Students from the sidebar on the left and then Edit Groups in the top right corner of the Students page.

You can edit groups by dragging and dropping them from the Unassigned students section on the left into groupings on the right.

Dashboard

  1. Clicking Distribute to students will send out an automatically generated email with a link to the assignment for students to read the questions and submit their responses.
  2. Clicking Overview grid will give you more information on who has submitted the assignment so far.

Printing, scanning & matching

This tutorial only applies to Administered Assignments.

Assigned Assignments do not require printing or scanning as they are handled online by the students.

You can print your Crowdmark assessment yourself using any regular printer. Please note that since Crowdmark supports different sizes of paper, the printer you use must support the size of paper you created your assessment on. Most printers default to regular 8.5x11" paper which is the same as the default page size in most writing applications such as Word and Google Docs.

Please Note: Getting your assessments printed and scanned by Ricoh will still require you to go through the Exam Matching process.

Ryerson has an agreement with Ricoh to take care of Crowdmark printing, binding, and scanning, as well as pick-up, delivery, and drop-offs. To do this:

  • Send your assessment/exam files to Ricoh by emailing them to jonathan.cebula@ricoh.ca, howard.lee@ricoh.ca and torontoworkflow@ricoh.ca. This is the file you generated at the end of the assessment creation process.
  • Arrange for Ricoh’s printing/scanning pick-ups and deliveries by filling out google formthis, external link form. When you submit the form, Ricoh staff will receive an email providing the details of the pick-up/delivery arrangement (be as specific as possible when specifying the pick-up/delivery location). Printing/Scanning and uploading usually takes 24 hrs.

This tutorial only applies to Administered Assignments as Assigned Assignments do not require scanning.

You can take an image of the completed booklets yourself by using a phone camera or regular scanner, and upload them as either individual files or packaged pdfs. The photos don't need to be labelled or in any specific order, as every question has its own QR code that identifies the question number and the package it is from.

Go to your assignment dashboard by clicking on Courses in the purple left-hand sidebar, then the name of your course, and then on the assignment name. Then, click on Upload booklets from the right-hand sidebar.

Drag and drop or click on the click here button to upload your files. You can upload multiple files at once. Wait until they have all uploaded and all of your pages have gone from the yellow pages pending indicator to the green pages completed indicator. Any issues will be indicated in red.

Make sure the green "Complete booklets" number eventually reaches the number of students that sat at the assessment.

If you choose to get your assessments printed by Ricoh in the Printing stage, they will pick up the exams at the time you scheduled on the Google Form you submitted and will scan them for you. You will get an email when your exams are uploaded, and can move on to Exam Matching

Crowdmark requires you to go through the process of Exam Matching, which is the process of digitally matching your students' exam booklets to their Ryerson email in the online system. 

After you have uploaded your assessment booklets:

Go to your assignment dashboard by clicking on Courses in the purple left-hand sidebar, then the name of your course, and then on the assignment name.

Click on Match booklets on the right-hand sidebar under Administration.

Search for the student's name, email, or student number. Regardless of which one you use to search, the results will show up as the student's Ryerson email.

  1. Click on the correct email displayed on the drop-down list.
  2. Once you have verified that the user is correct, click Next Unmatched.

Once you have matched all the booklets to a student, the page will tell you that all uploaded booklets have been matched. It will save your changes automatically, so you are free to start the grading process.

Note: This step is not recommended as it requires scanning booklets twice, but can be useful if you're already going around the room and taking attendance during an exam.

You can match the assessments while the students are writing their assessments by scanning each paper's QR code and searching in the app for the student's name that they have hand written on the cover sheet or have displayed in front of them on a student ID.

  1. To set up your account to use the Exam Matcher app, go to your My Account settings on the Crowdmark desktop site by clicking the abstract coloured icon in the top right corner of your screen. This icon may be a different colour for you, or may be a custom photo if you have already set it yourself.
  2. Enter your full name in the Full Name text field.
  3. Click Update my settings to save your changes.

Return to your assignment dashboard by clicking on Courses in the purple left-hand sidebar, then the name of your course, and then on the assignment name.

Click on Match with Exam Matcher App on the right-hand sidebar under Administration.

  1. Click Generate token.
  2. Take note of the number displayed on the screen under the most recent row of the Token column. Tokens can expire, so make sure to use the most recent one or generate a new one if neccesary.

Download the app by tapping on the Apple App Store or Google Play Store app on your phone, search for Exam Matcher, and then open the app once it has completed downloading.

This tutorial shows screenshots for the Apple iOS version of the app, but the Android version should work the same.

  1. Enter your full name, making sure it matches what you entered in the first step on the desktop version of the site.
  2. This can either be left blank or set to the room number your assessment is being written in. It will not affect your login.
  3. Enter the token as it is displayed on the desktop site.
  4. Click Sign in.

You will be prompted to enable Crowdmark's access to your device's camera. Tap OK.

Scan the first assessment's QR code. This can be the cover sheet, or the code at the top of any supplemental pages.

Search for the student's name, email, or student number in the app. Regardless of which one you use to search, the results will show up as the students Ryerson email.

  1. If you've made an error, tap Unmatch this student and try again. 
  2. If the results are accurate, tap Match another and repeat the process for all the assessment sheets.

The system will automatically save between each scan. When you are finished, close the app and continue to the desktop version of the site. When your exams are all scanned and uploaded, they will be ready to be marked there.

Grade student work

Before you start grading, you will need to import your team from Brightspace. To do this, click on your course in the My Courses area of the platform. 

 

Located on the left side bar, click Team.

 

Then click the Import from Brightspace button located on the top right corner of the page. 

After you import, your team members will receive emails from Crowdmark inviting them to the assessment.

 

Grading may be conducted by you or by your Teaching Assistants. Once all of the student booklets are uploaded to the system, select one of the Q boxes in the Grading area. 

Depending on which Q box you select, this determines what question you will be grading.

Continue to Step 2. 

The grading system has a tool sidebar which will aid you in marking up the student’s work. 

Continue to Step 3 when you are done. 

Comment

Allows you to comment on the student’s work and place the text box anywhere on the page. Useful for providing constructive feedback.

Freeform annotation

Allows you to draw lines on the page. Useful for marking up the page. 

 

Stamp menu

Allows you to choose from a checkmark, question mark and x stamp that can be placed anywhere on the page. Useful for marking correct and incorrect answers. 

 

Rectangle

Allows you to dragw a transparent rectangular shape anywhere on the page. Useful for highlighting items. 

Delete

When selected, you can click on any of the above tools placed on the page and they will be deleted.

Colour menu

Allows you to change the colour of the markings.

Comment library menu

Allows you to see all the comments made on the booklet.

Once comments and markups have been made, you must provide a grade for the question. 

1. On the right hand side, input a number using the number pad that describes the grade of the student’s work. Once done, click OK to save the grade. 

2. Click Next ungraded Q1 to continue to the next student’s booklet. 

Once all the questions of that specific Q box have been graded, exit the grading system and move onto the next Q box. 

 

Once all the booklets have been graded, you are able to export those grades to Brightspace. 

Go to your course in the My Courses area of Crowdmark and select the assessment that you would like to export from. In the assessment, click Results on the left-hand menu.

 

Click the Export Grades to Brightspace button. Click Export Grades (if prompted a second time).

 

Once this step is completed, a new section will be created in your D2L course Grades tool with the same name as the Crowdmark assessment.