CS 291I: Bionic Vision

What would the world look like with a bionic eye?

This graduate course will introduce students to the multidisciplinary field of bionic vision, with an emphasis on both the computer science and neuroscience of the field.

The course will conclude with a programming project (teams of ≤ 3, any language/environment ok) in lieu of a final exam, giving students an opportunity to gain hands-on experience of working on open research problems using methods and tools best suited to their scientific background.

Instructor Michael Beyeler (first initial last name at ucsb dot edu)
Class WQ 2020, Tue/Thu 9:00 – 10:50 am, Phelps 3526
Office Hours Tue 4:00 – 5:00 pm or by appointment, Psych East 3822

This website and the Piazza Forum will be used as centers for communication. Homework submission will occur through GauchoSpace. Make sure you are enrolled! (Come to class to get an add code.)

Table of Contents

Course Objectives

The course will give an overview of current bionic eye technology designed to restore vision to people living with incurable blindness. By the end of the course, you should be able to:

  • Identify various types of bionic eye technologies, their differences and similarities
  • Have a basic understanding of the neuroscience of the human visual system
  • Be familiar with common preprocessing, encoding, and electrical stimulation methods
  • Understand the limitations of current bionic eye technologies
  • Have hands-on experience of working on open problems in the field

The course is targeted to a diverse audience spanning from computer science (human factors, neural networks, computer vision) to psychology (vision, psychophysics) and brain sciences (computational neuroscience, neuroengineering).

Prerequisites

  • There are no official prerequisites for this course. The instructor will do his best to make the course content self-contained.
  • However, prior programming experience (e.g., Python, Matlab, C++) will be highly beneficial as Homework 2 (HW2) and the final project require programming. Students will be introduced to pulse2percept, a Python-based simulation framework for bionic vision, which will form the basis for HW2 and (optionally) the final project.

Schedule

Import calendar:
Note: This schedule is subject to change over the course of the quarter.

Wk Date Reading Topics Action HW out HW due
1 Tue Jan 7
  • Introduction: Class requirements, policies
  • Bionic vision: then & now
Thu Jan 9 R1, R2
  • Blinding eye diseases
  • Sight restoration approaches & challenges
HW1
2 Tue Jan 14 R3
  • Fundamentals of neuroscience
  • The visual system
Thu Jan 16 R4
  • Computational neuroscience
  • Introduction to Python
Homework 1 (HW1) due by Sun, Jan 19, 11:59 pm.
A1 HW1
3 Tue Jan 21 R5, R6
  • Retina in health & disease
  • Retinal prostheses
Thu Jan 23 R7
  • Introduction to pulse2percept in Python
  • Project ideas: Discussion & brainstorming
A2
4 Tue Jan 28 R8
  • Visual psychophysics for retinal prostheses
  • Phosphene models: Scoreboard, axon map
Thu Jan 30 Teams present their project ideas
Team & project description (TPD) due by Sun, Feb 2, 11:59 pm.
TPD
5 Tue Feb 4 R9
  • Cortical prostheses: approaches, challenges
  • Phosphene models for cortical prostheses
HW2
Thu Feb 6
  • Mid-quarter review
A3
6 Tue Feb 11 R10, R11
  • Training & rehabilitation
  • Cortical plasticity & perceptual learning
Thu Feb 13 Guest Visit: Jason Esterhuizen, ORION implantee A4
7 Tue Feb 18 R12,
R13
  • Improving visual outcomes in bionic eye technologies
  • Advanced stimulation strategies
Thu Feb 20 R14
  • Optimizing electrical stimulation in an artificial retina
Homework 2 (HW2) due by Sun, Feb 23, 11:59 pm.
HW2
8 Tue Feb 25 R15,
R16
  • Scene representation for future bionic eye technologies
  • Advanced encoding methods
Thu Feb 27 Teams present project progress
9 Tue Mar 3 Teams work on projects -- Instructor out of the country
Thu Mar 5 Teams work on projects -- Instructor out of the country
10 Tue Mar 10 R17 Guest Lecture: Dr. Noelle Stiles, USC/Caltech
Thu Mar 12 R18
  • Outlook: Future of bionic vision
  • Alternatives to brain-computer interfaces
Project report (PR) & source code (SC) due Sun, Mar 15, 11:59 pm.
PR&SC
11 Tue Mar 17 Teams make their final project presentations

Course Requirements & Grading

Your final grade will be determined as follows:

  • 15% Class participation and attendance:
    • Students are expected to attend all class sessions and actively participate in class discussions and activities.
    • If a student must miss a session, they should email the instructor beforehand. Each student will be allowed 3 excused absences (no detailed explanation required) before their absence will start to negatively affect their participation grade.
    • However, late arrivals and unexcused absences will most definitely have a negative effect on a student’s participation grade.
  • 30% Homework assignments:
    • 10% Homework 1
    • 20% Homework 2
  • 55% Final project implementation, documentation, and presentation
    • 5% Project idea presentation (1 slide)
    • 10% Project progress presentation (2 slides: what have you done, what’s left to do)
    • 20% Project final presentation
    • 20% Project final report
    • (+5% extra credit if project shows promise of turning into a publication)

Lateness Policy

All assignments are due at 11:59:59 pm on the scheduled due date, typically a Sunday (timestamp of the online submission system).

  • Each student will be allowed 3 “late days” over the course of the quarter for which lateness will not be penalized. Late days cannot be applied to project deadlines. Late days may be applied to one or both homework assignments: Anything turned in between 12:00:00 am and 11:59:59 pm the next day is one day late; every day thereafter that an assignment is late, including weekends and holidays, counts as an additional late day.
  • Absolutely no late work will be accepted after the deadline if you have used up all your late days. If you’re not done on time, you must turn in what you have to receive partial credit. There will be no exceptions from this rule.
  • No exceptions will be made for the final project report.

Please make sure you understand this policy.

Final Project

In lieu of a final exam, students will conduct a programming project (team size ≤ 3). The goal of the project is to gain hands-on experience working on open research questions in bionic vision using tools and methods best suited to their scientific background.

All projects must address a research question and have a programming component. Students are free to use any programming language and development environment they choose. Building a project based on pulse2percept is encouraged (especially for students with relatively little programming experience) but is by no means required. Reproducing key research findings in the literature is allowed. No pure literature reviews, please.

Projects that show promise of turning into a publication will receive extra credit.

Students will present their project to the rest of the class during finals week. In addition, students will submit a write-up of their project and hand in their source code (see Milestones).

The project will be evaluated based on the:

  • originality/novelty of the idea
  • technical strength of the work (emphasis on the research, not the programming expertise)
  • organization, clarity, and style of the project report
  • effort and completeness of the work (normalized by the number of team members)

Project Milestones

Date Time Deliverable due
Thu, Jan 23 9:00 am Students start forming teams and discussing project ideas in class.
Thu, Jan 30 9:00 am Teams present their project ideas in class.
Sun, Feb 2 11:59 pm Teams submit a project title and 2-3 sentence project description.
Thu, Feb 27 9:00 am Teams present their project progress in class.
Sun, Mar 15 11:59 pm Teams hand in their final project report and all source code.
Tue, Mar 17 Teams make their final project presentations in class.

Students are encouraged to discuss ideas with the instructors, so that feedback can be incorporated early in the process.

Late days cannot be used on these project deadlines.

Project Presentation

Teams will present their project via Zoom on Tue, Mar 17.

Each team will have 20 mins to present (+5 mins for Questions & Answers). Sign up for a time slot here.

Before the meeting, decide who will host the slides/demo. This person will share their screen during the meeting. Other team members can choose to be physically present with the person sharing the screen or simply log in from their own computer.

There are at least two strategies to present your work:

  • Strategy A: Follow the outline of your report
    • Introduction, Methods, Results, Discussion
  • Strategy B: Top-down
    • Give an overview of the project’s end result
    • Follow with a detailed discussion of the various features/techniques

Make sure to address the challenges you faced and how you overcame them! What have you learned?

Every student in the team must say something.

Project Report

Each team will also submit a write-up of their project:

  • Use the CHI Extended Abstracts template
  • Structure your report like a short research paper (~4 pages):
    • Abstract: ~150 words
    • Introduction (1-2 paragraphs)
      • What did you study and why?
    • Related work (12 page)
      • Brief summary of the relevant literature.
      • Make sure to point out gaps in the literature that your project is trying to address.
    • Methods (1-2 pages)
      • First paragraph: Describe the big-picture idea behind your system/model/approach.
      • Subsections: Walk the reader through all the steps/features (with pictures/schematics).
    • Results (1-2 pages)
      • Structure based on research question(s) and/or experiments.
      • Have 2-3 figures to support your claims. Explain each figure and summarize the findings.
    • Discussion (12 page)
      • First sentence: Summarize your findings.
      • Discuss: What does it all mean? What have you learned? Future work?

When you’re done, zip up the PDF/DOC together with all your source code and upload the zip file to GauchoSpace.

Don’t forget to submit your source code.

Academic Integrity

The University of California has formal policies related to academic integrity.

Any act of academic dishonesty, such as cheating or plagiarism, will result in a University disciplinary action and an “F” in this course. In addition to academic integrity, I also expect everyone in this class to treat their fellow students and course staff with respect.

Basic Needs

If you are facing any challenges securing food or housing and believe this may affect your performance in the class, you are urged to meet with a Food Security and Calfresh Advocate who is aware of the broad variety of resources that UCSB has to offer (see their drop-in hours at food.ucsb.edu). You are also urged to contact the professor if you are comfortable doing so.

Please visit food.ucsb.edu for additional resources including Calfresh, the AS Food Bank, and more.