Using Noteable for Assessment and Feedback
How to use Noteable to create code-based assignments.
Instructors can use Noteable to create notebook-based assessments using the nbgrader extension, and carry out marking and feedback processes for their students.
- Instructors can set up Noteable assignments, release them to students, collect the submissions, mark and give feedback, and release grades, in Formgrader.
- You can design assignments that are auto-graded for students' input. This can help with large classes where marking many code-based questions can become time-consuming.
- Submissions can also be manually reviewed and marked with feedback and comments by the instructor.
Multiple Marker has been enabled as default on 4th September 2023.
- All new Noteable LTI instances created in a new course in Learn will have Multiple Marker enabled as default.
- If you already opted in Multiple Marker during the piloting, your assignments will remain as they are.
- If you have never used Noteable for assessment in your course before, this does not matter to you.
- Note that if you use R studio from Noteable, NBGrader is not enabled, the release does not affect your course.
EDINA confirmed that Multiple Markers run in parallel with the Single Marker system but each Learn course (code) can only be associated with either one.
- If you have been using Noteable for assessment on the single marker system, you can continue to access the existing submissions and grade records from the Noteable LTI link in those Learn courses.
- If you want to change any previous Noteable assignments to the Multiple Marker system, please submit a call to is.helpline@ed.ac.uk with the course code and your UUN.
The new assessment file path in the Multiple Marker system is defaulted to "../instructor-courses/the learn course code/source/". In comparison, the previous Single Marker's assignment file path was "../source/". See screenshots in the attached PDF file:
Further information can be found on EDINA's Multiple Marker page.
Student Guide to Using Noteable for Assignments
YouTube Video: How to access and submit assignments in Noteable
If you have any questions regarding Noteable, please contact your instructor or course administrator.
For further detailed guidance you can also visit the nbgrader documentation, please be aware that there may be small configuration changes from source nbgrader - nbgrader documentation
In May 2019 The University of Edinburgh hosted a Hackathon to add new features and abilities to nbgrader. As part of this event the creator of nbgrader, Dr Jess Hamrick, gave a short introductionory talk that goes over the features of nbgrader - Nbgrader Presentation (Video)
Can I use nbgrader for purely manually graded assignments?
Yes, you can mark all the cells where students write their answers as "Manually Graded Answer" cells and then during grading run nbgrader autograde and the formgrader as normal.
Can I hide the test cells in a nbgrader assignment?
Nbgrader includes the ability to hide tests in “Autograder tests” cells through a special syntax such as ### BEGIN HIDDEN TESTS and ### END HIDDEN TESTS on either side of the code that needs to be hidden.
How does nbgrader ensure that students do not change the tests?
Nbgrader includes the ability to mark cells as "Read Only" in the cell menu in the creation phase. This means that the cell cannot be accidentally changed or removed by the student.
Does nbgrader protect against infinite loops?
Yes. nbgrader will stop executing a cell after a certain period of time, this amount of time is set at a system level and cannot be adjusted for each class. If you have issues with autograding timing out then please Contact Helpline
Do Students receive a submission receipt?
Students have a log showing all of their submissions with timestamps in the Assignments tab. We also keep detailed logs at a system level to check when students have submitted.
Can Students submit multiple times?
Students can submit as many times as they like until the submission deadline. When you, as an instructor, run the Collect Submissions function it will check if any new submissions have been made and show only the latest submission for you to mark.
I have made a mistake in an assignment that I released. Can I edit and re-release this?
Due to the way that nbgrader works we advise that you remove the original assignment and release a new assignment with a different name. This will ensure that any students who downloaded the original incorrect assignment will see the new version.
My Assignment tab says "No Course Found"
This is caused by not accessing Noteable via Learn. Please refer to the Requirements section of this page.