Hi there !

I'm Jack Yang a XR developer an Entrepreneur

Award-winning XR developer with 7+ years of experience in
developing immersive and interactive virtual experiences.

about me

Working in HCI Lab
  • Hi! I'm Jack Yang, an AR/VR developer specializing in XR Interactions and Virtual Simulations. Currently I'm working at Microsoft Mixed Reality.

    Since diving into the XR industry in 2015, I've been fervently dedicated to harnessing immersive technologies to empower everyone.

    Feel free to reach out if you want to collaborate or just want to chat!


experience

June 2022 - Present Software Engineer, Mixed Reality Microsoft

Working on Microsoft Mesh - the next generation communication and collaboration platform

February 2019 - May 2022 AR/VR Interaction Engineer Holos

I developed an AR/VR training system with advanced features like gesture recognition and hand tracking. I Introduced multiplayer networking, 3D modeling processing, and other features. I helped Holos to secure a $750K U.S. Air Force contract and get into the TechStars Program.

October 2017 - May 2022 AR/VR Researcher Kats Lab of Applied Physics

I formulated algorithms for simulating color blindness on hyperspectral images, designed VR simulations with a color-calibrated Oculus Rift, and developed an AI mimicking human color vision behavior, achieving 90% chromatic adjustment accuracy.

September 2020 - May 2022 Graphics Researcher UW Graphics Group

I developed a low-latency VR system for remote robot control via gestures and created a UI for replicating robot arm movements, analyzing data from 15+ lab experiments in Unity.

Education

2016 - 2020 Bachelor Degree University of Wisconsin - Madison

I studied Computer Science and Communication Arts. I also researched under Professor Mikhail Kats and Professor Michael Gleicher . For two semesters, I was the Teaching Assistant for CS 559 (Computer Graphics).

Summer 2015 National High School Game Academy Carnegie Mellon University

I studied game design at CMU Entertainment Technology Center, where I also led and published a VR game development with Oculus Rift DK2 called Kitchen Kraving under the instruction of Professor Chris Klug

Publications

Z. Yang, B. Rubio-Perez, M. A. Kats, “Breaking Binocular Redundancy Through Virtual Reality” , (2021)

J. Salman, M. Gangishetty, B. Rubio-Perez, D. Feng, Z. Yu, Z. Yang, C. Wang, A. Shahsafi, D. Congreve, M. A. Kats, “Passive frequency conversion of ultraviolet images into the visible using perovskite nanocrystals” , Journal of Optics (2021)

skills

AR/VR Development
95%
AR/VR Interaction Design
95%
Unity / C#
95%
Accessibility Design
90%
Unreal Engine / C++
85%
OpenGL / Vulkan
80%

Robotic Mimicry Control in VR

  • Position : Researcher
  • Technologies : Unity, ROS, IK Solver
  • Overview : This project explores remote manipulation of robots through virtual reality and inverse kinematics . There are two parts: mimcry control of robot arms, and re-creation of robot motion through data.
  • Mimicry Control : In the first half of the research, I implemented a virtual reality system that allows users to remote control robots with their hand and arm gestures by passing ROS (Robot Operating System) data between Unity and the robot through network socket with minimal latency. The video below demonstrates this capability in action.

  • Recreate Robot Movement : During the second half of the research, due to COVID-19 restriction, I could not be in the lab and research with the robot, so I switched gear into replicating movement by interpolating data from actual robotic experiments in Unity. By providing a spreadsheet of joint angles with timestamps, the interpreter I designed can create animation clips that replicate the exact robot arm movement. Here is a demo of what it looks like.
  • Application : This system allows us to perform robotic manipulation remotely, which can benefit many fields, such as the operation of dangerous equipment and more delicate robot motions, while reducing the cost of robot maintenance.
Close

Vision Disorder Through VR

  • Position : Researcher
  • Technologies : Unity, Matlab, Oculus, Python
  • Overview : This project explores and simulates the vision of several visual impairments such as colorblindness and binocular disparities.
  • Color vision Deficiency : In the first phase, I devised chromatic adjustment algorithms using computer vision to simulate this condition. Next, I confirmed my hypothesis by combining optics with AI to develop a system to mimic human interactions during a colorblindness test with 90% accuracy. In the second phase, I programmed simulation models into VR environments to visualize my research findings.
  • Binocular rivalry : The human visual system perceives the world by combining information received from the convergence of two eyes and infer three-dimensional cues such as depth. In the second phase, we demonstrate a virtual reality system that breaks the inherent binocular redundancy by projecting two different dynamic and static contents to each eye.
  • Project Image
    Project Image
  • Application : This technique harnesses mathematical optical simulation for a realistic VR representation of Color Vision Deficiency. Its potential stretches beyond mere simulation; it can be reverse-engineered for enhanced color perception in humans. Additionally, by altering binocular redundancy in VR, we pioneer custom binocular rivalry simulations without physical tools like mirror stereoscopes. This advances binocular disparity experiments, VR accessibility, and paves the way for vision enhancements.
Binocular Disparities Technical Report

Project Source Code

Passive Frequency Conversion Paper

Vision Deficiency Simulation Paper

Close

Kitchen Kraving

  • Position : Project Lead & Developer
  • Technologies : Unity, Maya, Oculus
  • Overview : Kitchen Kraving is a fast-paced kitchen-themed VR game.
  • Description : Kitchen Kraving is a Virtual Reality game in a kitchen setting where the users have to make food while stealing food to eat without being caught by the boss, who randomly check in on the user. The game was first developed at Carnegie Mellon ETC with a group of artists, programmers, and sound designers. It was later published on the Oculus store
Github

Close
Project Image

XR Design Handbook

  • Position : Founder, Developer
  • Technologies : ChatGPT, Ruby, Jekyll
  • Overview : XR Design Handbook is a compilation of XR design guidelines from numerous leading companies in the field.
  • Description : The repository contains a compilation of XR design guidelines from numerous leading companies in the field. This is a goldmine for both seasoned designers and newcomers to XR, ensuring we can create user-friendly and effective experiences.
  • Inspiration : I've been noticing that while there are myriad AR/VR design guidelines spread across various companies and platforms, there hasn't been a consolidated place where we can see them all together. Recognizing this gap, I took the initiative to start this project.
XR Design Handbook

Github

Close

MRTK for Unity

  • Position : Open Source Developer
  • Technologies : Unity, Ultraleap, MRTK
  • Overview : MRTK is a Unity toolkit used for developing XR content.
  • Description : In this open Source project, I work with Microsoft developers and other open-source contributors on MRTK for Unity. Some of the features I worked on are integrating Leap Motion Hand Tracking with MRTK (see demo video above) and adding new features for the upcoming MRTK 2.7.
2.7 Release Note

Github

Close
Project Image

Poly Space VR

  • Position : Founder, Developer
  • Technologies : Unity, Blender, Photon Engine
  • Overview : PolySpaceVR is a virtual reality social platform for customizability and small group gatherings.
  • Inspiration : When I played VRChat, I noticed that the giant platform is great for meeting random people but not so much for intimate or customized experiences. After some research, I realized there is no VR social media platform targeted for small groups, so I created one myself. Due to the size of the project, I choose to make it open source and lightweight so that other developers who share the vision can contribute.
  • Description (from Oculus Store): Poly Space VR is a low poly, lightweight, open-source social platform where you can meet and chat with your friends in VR. You can create and share your own Poly Spaces by following the guidelines. Each month, an incredible version of Poly Space VR will be selected and uploaded to Oculus Store so everyone can join in on the fun!
Oculus Store

Github

Close

Project Virtualso

  • Position : Founder, Developer
  • Technologies : Eye-tracking, IBM Watson, Unity
  • Overview : Project Virtualso combines artifical intelligence (AI), natural language processing (NLP), and virtual reality to create customizable conversational humanoid agent capable of making emotion-driven facial expressions and body gestures.
  • Inspiration : This project attempts to combine my knowledge in communication arts and VR to create realistic human interaction wthin VR. Due to COVID, many do not have access to in-person interactions and cannot develop the soft skills necessary, so I created several simulations such as job interviews and public speaking.
  • Virtual Interview : The user interviewed by a conversational humanoid agent. The virtual interviewer can hold basic conversations and engage in realistic emotion-driven interactions such as facial expression, body gestures and much more thanks to Natural Language Processing (NLP) techniques.

    • Virtual Presentation : The user presents their slides in front of a room of AI audiences capable of reacting to the speech in VR. This project aims to help those with fears in public speaking and has been tested by formal employees from several companies and received very positive feedbacks.
    Close

Video Format

  • Project : Website
  • Client : Envato
  • Technologies : html, javascript
  • Budget : 1550 USD
preview

Close

Collect

  • Position : Co-Founder, CTO
  • Technologies : Flutter, Adobe XD, Firebase
  • Overview : Collect is a social media startup aiming to enable people collect experiences.
  • Description : Collect is a platform that allows people to collect and store experiences in pictures. The saved images will be stored and sorted by collection type, making it easy for users to build unique collections, look back at their collections and share their photographs with users that have similar interests.
Blog Post

Close
Project Image

UpNote

  • Position : Founder
  • Technologies : Flutter, AdobeXD, Firebase
  • Overview : UpNote is a startup aiming to democratize music for bar goers.
  • Description : Upvote’s B2B and B2C platform allows bars, restaurants, event organizers and, individuals democratize music playlists and capture data on music preferences. Upvote enables individuals to nominate songs through integration with their music streaming service of choice (ex. Spotify or Apple Music). Once a song is nominated, other users in the same location can up- or down-vote each song selection to determine which will play next.

    Watch our pitch below:
  • Note : Unfortunately, the project was cut short due to COVID-19 and bar shutdowns
Business Brief

Business Canvas

Close

get in touch

  • locationSeattle, washington
  • emailjackyangzzh [at] gmail [dot] com

Want to connect? Please fill out the form below and I will reply you shortly.

My Blog

Blog Overview

I enjoy writing blogs on Medium, whether a book review or thoughts on the current state of technology. Check out My Medium Page



Jack

Yang