Enhancing the Sense of Presence in Virtual Reality.

The vision of virtual reality, since Ivan Sutherland's first head mounted device in 1968, has been the re-creation of reality, something indistinguishable from reality, akin to what was depicted in the 1999 film, The Matrix. However, researchers and developers have largely favored visual perception over other senses, leading to the design of virtual worlds that perhaps look real but do not feel real. This favoring of the visual sense and more recently visual and audio senses, overlooks theory in psychology and phenomenology that places embodied action at the center of perception. That is to say, it is the virtual environment's ability to support and enable user actions that impacts perception and perhaps by extension, the user's sense of presence, not just the visual fidelity. Starting with Gibson's approach to action-based perception, we proposed a 4-D framework for creating virtual reality experiences that seamlessly blend extrinsic elements such as the user's real world context with intrinsic elements such as the hardware specifications, application, and interactive content, with the goal to enable a higher sense of presence.

Designing Intelligent Tools for Creative People.

With artificial intelligence (AI) poised to transform the nature of creative media, it is especially important to design tools with the creative process in mind. While ample research demonstrates the importance of flow, playfulness, and exploration for creative tasks, these concepts are rarely considered when designing digital interfaces. My thesis develops principles for the design of intelligent and playful user interfaces through a series of concrete design tasks. I explore a number of approaches for establishing artist needs, develop digital representations that are amenable to both machine learning and user interaction, and design novel digital media that amplify, not stifle, creativity. I conclude with an informal design philosophy developed throughout this study and thoughts on how we can leverage AI to elevate human creativity.

Reliable Contact Simulation With IPC.

Simulating the contact of deformable and possibly thin solids in a robust, accurate, and efficient manner is challenging. Traditionally, the contact of solids is approximately modeled with linearized geometric information near the contacting regions. This approximation is prone to generating underconstrained or overconstrained subproblems that can produce interpenetrating or numerically unstable results, especially when large deformation of solids is also present. To avoid these issues, we propose incremental potential contact (IPC) by formulating a mathematically consistent and general noninterpenetration constraint based on precisely calculated unsigned distances between boundary elements. IPC applies a customized barrier potential to directly relate the distances to the contact forces, which can grow infinitely large as the distance approaches zero to guarantee noninterpenetration. Results show reliable contact simulation even with versatile materials, large timestep sizes, fast impact velocities, severe deformation, and varying boundary conditions-bringing the intricate and important dynamical details to computer graphics in a reliable way for the first time.

Differentiable Visual Computing: Challenges and Opportunities.

Classical algorithms typically contain domain-specific insights. This makes them often more robust, interpretable, and efficient. On the other hand, deep-learning models must learn domain-specific insight from scratch from a large amount of data using gradient-based optimization techniques. To have the best of both worlds, we should make classical visual computing algorithms differentiable to enable gradient-based optimization. Computing derivatives of classical visual computing algorithms is challenging: there can be discontinuities, and the computation pattern is often irregular compared to high-arithmetic intensity neural networks. In this article, we discuss the benefits and challenges of combining classical visual computing algorithms and modern data-driven methods, with particular emphasis to my thesis, which took one of the first steps toward addressing these challenges.

Rendering grass in real time with dynamic lighting.

The authors present a real-time grass rendering technique that works for large, arbitrary terrains with dynamic lighting, shadows, and a good parallax effect. A novel combination of geometry and lit volume slices provides accurate, per-pixel lighting. A fast grass-density management scheme allows the rendering of arbitrarily shaped patches of grass.