CS 491/591-005 Special Topics: Spatial Computing

Spring 2027

COURSE INFORMATION

COURSE DESCRIPTION

This course introduces the foundations and research frontiers of spatial computing, and gives students hands-on experience with physical and virtual 3D spaces through virtual, augmented, and mixed reality. From 3D math and human perception to hand and gaze interaction, tracking, and AI-driven interfaces, we explore how spatial computing renders virtual worlds, understands physical space, and supports interaction within it.

By the end of the semester, students will be equipped with fundamental knowledge and hands-on experience building XR applications on a standalone headset, along with the skills to read, critique, and produce XR research.

Note: “Spatial computing” in this course means XR and 3D interaction, not related to geographic information systems (GIS). Each student will be provided with an XR headset for the semester.

PREREQUISITES

This course requires students to have working knowledge of linear algebra (vectors, matrices, transformations) and prior programming experience. Assignments must be implemented in Unity (with C# or JavaScript). Students should also be familiar with basic usage of Unity (Unity Learn) and VR in Unity (Create with VR) by the 3rd week of the semester. If you are already familiar with Java, C++, Python, or a similar high-level language, you should be able to handle C#/JavaScript quickly and complete the course successfully. Prior experience in computer graphics, human-computer interaction, or computer vision is highly recommended.

WEEKLY TOPIC BREAKDOWN (TENTATIVE)

1. Introduction, history, and the XR pipeline
2. 3D math and the rendering pipeline
3. Visual perception and XR platforms
4. User tracking
5. Selection and manipulation
6. Multimodal input
7. Navigation and wayfinding
8. System control
9. Visualization
10. World tracking
11. Shared spaces and telepresence
12. Geometry, reconstruction and scene modeling
13. Content generation
14. Haptics, physical proxies and teleoperation
15. Final project presentation

PAPER READING AND PRESENTATION

Throughout the semester, we will read 12-20 research papers on the topics above. The reading list will be posted on Canvas.

• Reading: Every student must read the corresponding paper before the class, and understand its problem, approach, and results well enough to discuss them.
• Presentation: For each paper, one student is selected at random to present it. The presentation is 20 minutes and should cover the motivation, key ideas, method, evaluation, and limitations of the paper.
• Q&A and discussion: Each presentation is followed by 10 minutes of Q&A and discussion. All students are expected to take part with questions, critiques, and connections to other papers and to their own projects.

Since anyone may be selected to present, read every paper carefully. The Paper Reading and Presentation grade is based on your presentation and your participation in the discussions.

GRADING

Grade Breakdown

Incomplete and Late Submission

• Incomplete or no submission will receive zero point.
• Late submissions lose 1 pt per day.

Undergraduate (ST 491) vs. Graduate (ST 591)

Both sections cover the same content. However, students in the undergraduate section (ST 491) will either be graded under a less stringent grading scale; or have fewer or less advanced project requirements compared to the graduate section. Graduate students are expected to include a user or technical evaluation in their final project.

ACADEMIC INTEGRITY

• Academic dishonesty may result in failing the assignment or the course. All academic dishonesty activities will be reported to the department.

• It is allowed to use AI for learning purposes. But all students must fully understand their code! There will be a random check after each submission, where they may be asked questions about the part of the code they write (with comments removed). Submitting AI-generated code, copying from others without proper disclosure, or improper use of online resources will be considered as academic dishonesty. Reading responses and paper reviews must be written by yourself.