Hi there !
I'm Jack Yang a XR developer an Entrepreneur
Award-winning XR developer with 7+ years of experience in
developing highly immersive and
interactive virtual
experiences.
about me
I design and develop immersive virtual experiences, and I love what I do.
-
Hi There! My name is Jack Yang. I'm a mission-driven AR/VR developer specializing in XR Interactions and Virtual Simulations.
I have been in the XR industry since 2015, and I am passionate about building a world where anyone could reach their full potential with immersive technologies.
Moreover, with a background in communications, I am very interested in Social Computing and cultural implications of XR.
experience
• Working on Microsoft Mesh - the next generation communication and collaboration platform
• Championed the building of an interactive, networked AR/VR content management and training simulation system, featuring hand tracking with integrated gesture recognition and simulated physics.
• Design prototypes; test and deploy key features including multiplayer networking, 3D-model processing, and virtual object manipulation interface.
• Formulate and implement new design decisions and product directions based on user testing observations.
• Leverage user testing observations to create and implement innovative product directions and design decisions.
• Employ critical thinking and complex problem-solving skills to identify potential areas for improvement and create actionable solutions.
• Collaborate with cross-functional teams toward the timely achievement of project and organizational objectives.
• Awarded a $750K research contract with the U.S. Air Force and acceptance into the TechStars Program.
• Wrote 4,000+ lines of code to incorporate, optimize, and maintain Ultraleap (Leap Motion) hand tracking support and demonstration projects for Microsoft's Unity Mixed Reality Toolkit (MRTK).
• Partnered with developers and support personnel to handle bug fixes and repository documentation management.
• Devise chromatic adjustment algorithms with computer vision techniques on hyperspectral images to simulate color blindness.
• Create and program virtual reality simulations to visualize research findings through color-calibrated Oculus Head Mounted Display (HMD), leading to practical design implications for potential human vision enhancement optical lenses.
• Accelerate development of an artificial intelligence algorithm to replicate human behavior during color vision deficiency tests such as Farnsworth-Munsell 100 Hue Test and D-15 Test; subsequently examine the accuracy of chromatic adjustment with 90% confidence.
• Constructed a virtual reality teleoperation system where users could remotely control robots via hand and arm gestures by passing ROS data between Unity and robot through network socket with little latency.
• Crafted a motion playback system with an intuitive user interface to dynamically replicate virtual robot arm movement; this was accomplished by interpolating robotic data from experiments in Unity that were used to analyze more than 15 lab experiments.
Education
• Computer Science (with a focus on computer graphics) and Communication Arts (with a focus on rhetoric)
• Research under Professor Mikhail Kats and Professor Michael Gleicher
• Teaching Assistant for CS 559 Computer Graphics (Fall 2019 and Spring 2020)
• Studied game design and development at CMU Entertainment Technology Center
• Developed a VR game with the Oculus Rift DK2 called Kitchen Kraving (published on the Oculus Store) under the instruction of Professor Chris Klug
Certificates
• 3-courses program includes: 1) Computer Graphics, 2) How VR Works, 3) Creating VR Apps
• 5-courses program includes: 1) Introduction to VR, 2) 3D Models for VR, 3) 3D Interaction Design in VR, 4) Building Interactive 3D Characters and Social VR, 5) Making Your First VR Game
• 4-courses program includes: 1) Fundemental & Design Principles, 2) Cognition, Context & Culture, 3) Ethics, Needfinding & Prototyping, 4) Evaluation, Agile Methods & Beyond
• 8-courses program includes: 1) Human-Centered Design, 2) Design Principles, 3) Social Computing, 4) Input and Interaction, 5) User Experience, 6) Information Design, 7) Designing, Running, Analyzing Experiments, 8) Capstone Project
• 6-courses program includes: 1) Foundations of Project Management, 2) Project Initiation, 3) Project Planning, 4) Project Execution 5) Agile Project Management, 6) Capstone Project
• 3-courses program includes: 1) Introduction to CV, 2) Advanced CV & Deep Learning, 3) Object Tracking and Localization
Publications
Z. Yang, B. Rubio-Perez, M. A. Kats, “Breaking Binocular Redundancy Through Virtual Reality” , (2021)
J. Salman, M. Gangishetty, B. Rubio-Perez, D. Feng, Z. Yu, Z. Yang, C. Wang, A. Shahsafi, D. Congreve, M. A. Kats, “Passive frequency conversion of ultraviolet images into the visible using perovskite nanocrystals” , Journal of Optics (2021)
Featured in：Cameron, Mike, “Effective Leaders: Four Attributes That Underpin The Core Characteristics of Effective Leadership” , SpiritCast Network (2021)
skills
my portfolio
A brief collection of my work. Contact me if you would like to see more.-
Project Virtualso
Conversational AI Assistant -
MRTK for Unity
MR development kit -
Aladdin Unity
AI Unity Assistant -
Poly Space VR
VR Social Media Platform -
Neon Painter
VR painting experience -
Kitchen Kraving
interactive VR game -
Color Vision deficiency Simulation
Simulate color bindliness -
Binocular Rivalry Through Virtual Reality
simulate vision disorder -
Robotic Mimicry Control in Virtual Reality
control robot through vr -
Collect
Minimalistic social media -
UpNote
music voting app -
enroll.wisc.edu
course enrollment website
AR/VR
Research
Entrepreneurship & UX
Color deficiency Simulation
- Organization : Kats Laboratory of Applied Physics
- Professor : Dr. Mikhail Kats
- Duration : 5 Years
- Technologies : Matlab, Hyperspectral Imaging, Unity, Oculus
- Overview : Our project explored color vision deficiency (CVD) and the potential to correct it through optical lenses.
- Description : In the first phase, I devised chromatic adjustment algorithms using computer vision to simulate this condition. Next, I confirmed my hypothesis by combining optics with AI to develop a system to mimic human interactions during a colorblindness test with 90% accuracy. In the second phase, I programmed simulation models into VR environments to visualize my research findings.
- Application : This technique enables mathematical optical simulation Color Vision Deficiency and provides a realistic simulation through VR. The process can potentially be reverse-engineered as a vision enhancement algorithm to allow humans more see more colors.
Passive Frequency Conversion Paper
Vision Deficiency Simulation Paper (In Progress)
Close
Robotic Mimicry Control in VR
- Organization : UW Graphics Group
- Professor : Dr. Michael Gleicher
- Duration : 1 year and 3 months
- Technologies : Unity, ROS, Linux, StreamVR, IK Solver
- Overview : This project explores remote manipulation of robots through virtual reality and inverse kinematics . There are two parts: mimcry control of robot arms, and re-creation of robot motion through data.
- Mimicry Control : In the first half of the research, I implemented a virtual reality system that allows users to remote control robots with their hand and arm gestures by passing ROS (Robot Operating System) data between Unity and the robot through network socket with minimal latency. The video below demonstrates this capability in action.
- Recreate Robot Movement : During the second half of the research, due to COVID-19 restriction, I could not be in the lab and research with the robot, so I switched gear into replicating movement by interpolating data from actual robotic experiments in Unity. By providing a spreadsheet of joint angles with timestamps, the interpreter I designed can create animation clips that replicate the exact robot arm movement. Here is a demo of what it looks like.
- Application : This system allows us to perform robotic manipulation remotely, which can benefit many fields, such as the operation of dangerous equipment and more delicate robot motions, while reducing the cost of robot maintenance.
Binocular Rivalry Through Virtual Reality
- Organization : KATS LABORATORY OF APPLIED PHYSICS
- Professor : Dr. Mikhail Kats
- Duration : 6 months
- Technologies : Unity, SteamVR
- Overview : This project explores and simulates the vision of several visual impairments such as binocular disparities.
- Description : The human visual system perceives the world by combining information received from the convergence of two eyes and infer three-dimensional cues such as depth. Here, we demonstrate a virtual reality system that breaks the inherent binocular redundancy by projecting two different dynamic and static contents to each eye.
- Application : By breaking binocular redundancy in virtual reality, our method aims to provide new ways to customize and simulate binocular rivalry without the need for physical equipment such as a mirror stereoscope. With the foundation of our technique, it should be possible to create more complex and personalized binocular disparities. This technique yields implications to binocular disparity experiments, virtual reality accessibility design, and potential human vision enhancement.
Source Code
Close
Kitchen Kraving
- Organization : Carnegie Mellon ETC
- Position : Project Manager & Developer
- Duration : 4 months
- Technologies : Unity, Maya, Oculus, C#
- Overview : Kitchen Kraving is a fast-paced kitchen-themed VR game.
- Description : Kitchen Kraving is a Virtual Reality game in a kitchen setting where the users have to make food while stealing food to eat without being caught by the boss, who randomly check in on the user. The game was first developed at Carnegie Mellon ETC with a group of artists, programmers, and sound designers. It was later published on the Oculus store
Close
Neon Painter
- Position : Developer
- Duration : 8 months
- Technologies : Unreal Engine, Blender
- Overview : Neon Painter is a space-themed VR game for self-expression and contemplation.
- Description : Neon Painter is a 3D Painting VR application where users stand on top of a lone planet in randomly generated space scene. Combined with gorgeous graphics and an intuitive painting system, users will have an immersive and expressive VR experience.
- Inspiration : The theme is inspired by the concept of the "overview effect," euphoria astronauts often experience during flight. The visual of this game is inspired by Cyberpunk 2077. I intend to use this game to capture that shift in awareness and use painting as a way for users to express themselves, almost as a meditative experience.
Close
MRTK for Unity
- Organization : Microsoft Mixed Reality Toolkit (MRTK)
- Position : Open Source Developer
- Duration : 1 Year and 9 Months
- Technologies : Unity, Ultraleap, MRTK
- Overview : MRTK is a Unity toolkit used for developing XR content.
- Description : In this open Source project, I work with Microsoft developers and other open-source contributors on MRTK for Unity. Some of the features I worked on are integrating Leap Motion Hand Tracking with MRTK (see demo video above) and adding new features for the upcoming MRTK 2.7.
Github
Close
Poly Space VR
- Position : Creator & Developer
- Duration : 1 Year
- Technologies : Unity, Blender, Photon Engine
- Overview : PolySpaceVR is a virtual reality social platform for customizability and small group gatherings.
- Inspiration : When I played VRChat, I noticed that the giant platform is great for meeting random people but not so much for intimate or customized experiences. After some research, I realized there is no VR social media platform targeted for small groups, so I created one myself. Due to the size of the project, I choose to make it open source and lightweight so that other developers who share the vision can contribute.
- Description (from Oculus Store): Poly Space VR is a low poly, lightweight, open-source social platform where you can meet and chat with your friends in VR. You can create and share your own Poly Spaces by following the guidelines. Each month, an incredible version of Poly Space VR will be selected and uploaded to Oculus Store so everyone can join in on the fun!
Github
Close
Aladdin Unity
- Position : Founder & Developer
- Duration : Ongoing
- Technologies : Open AI, Unity
- Overview : Aladdin is a Unity plugin uses AI to assist developers in writing code and generating textures. It aims to make the development process faster and more efficient.
- Outlook : I'm planning to make Aladdin Unity a general purpose plugin (not just Unity development tooling) that developers can use in their Unity application to quickly get generative AI up and running. In fact, I'm currently using Aladdin Unity to make my own Unity app and will continuously improve it based on my experience.
Close
Project Virtualso
- Position : Creator & Developer
- Duration : 2 years and 6 months
- Technologies : NLP, Unity, VR, AI
- Overview : Project Virtualso combines artifical intelligence (AI), natural language processing (NLP), and virtual reality to create customizable conversational humanoid agent capable of making emotion-driven facial expressions and body gestures.
- Inspiration : This project attempts to combine my knowledge in communication arts and VR to create realistic human interaction wthin VR. Due to COVID, many do not have access to in-person interactions and cannot develop the soft skills necessary, so I created several simulations such as job interviews and public speaking.
- Virtual Interview : The user interviewed by a conversational humanoid agent. The virtual interviewer can hold basic conversations and engage in realistic emotion-driven interactions such as facial expression, body gestures and much more thanks to Natural Language Processing (NLP) techniques.
- Virtual Presentation : The user presents their slides in front of a room of AI audiences capable of reacting to the speech in VR. This project aims to help those with fears in public speaking and has been tested by formal employees from several companies and received very positive feedbacks.
enroll.wisc.edu
- Position : UX Researcher
- Duration : 6 months
- Technologies : Tobii Eye Tracking, Morae Software, InVision
- Overview : I conducted UX research for new UW course enrollment website.
- Description : To better understand user behavior and usability patterns, I conducted user research with 20+ participants. I generated an evaulation report (available below) that summarizes my findings and offers my practical suggestions. I then worked with project managers and web developers to implement those findings, which eventually resulted in the successful launch of the course enrollment website.
Close
Collect
- Position : Co-Founder, CTO
- Duration : 1 Year
- Technologies : Flutter, Adobe XD, Firebase
- Overview : Collect is a social media startup aiming to enable people collect experiences.
- Description : Collect is a platform that allows people to collect and store experiences in pictures. The saved images will be stored and sorted by collection type, making it easy for users to build unique collections, look back at their collections and share their photographs with users that have similar interests.
Close
UpNote
- Position : Founder
- Duration : 3 Years
- Technologies : Flutter, AdobeXD
- Overview : UpNote is a startup aiming to democratize music for bar goers.
-
Description : Upvote’s B2B and B2C platform allows bars,
restaurants, event organizers and, individuals democratize music playlists and
capture data on music preferences. Upvote enables individuals to nominate songs
through integration with their music streaming service of choice (ex. Spotify or Apple Music). Once a song is
nominated,
other users in the same location can up- or down-vote each song selection to determine which will play next.
Watch our pitch below: - Note : Unfortunately, the project was cut short due to COVID-19 and bar shutdowns
Business Canvas
Close
get in touch
I’m always open to discussing ideas and collaboration!-
locationSeattle, washington
-
emailjackyangzzh [at] gmail [dot] com
Want to connect? Please fill out the form below and I will reply you shortly.
My Blog
A Collection of My Latest BlogsBlog Overview
I enjoy writing blogs on Medium, whether a book review or thoughts on the current state of technology. Check out My Medium Page