How to Build Kinect-Powered Web Motion Games with HTML5 and Node-Kinect2

Zero – Preface

For front‑end engineers, developing Kinect motion games may seem out of reach because most existing solutions use C# or C++, but with HTML5’s new features we can create browser‑based Kinect games.

1 – Introduction

At the Chengdu TGC2016 expo we developed a motion game based on the Naruto mobile game chapter “Nine‑Tailed Fox Attack”. The game runs entirely in Chrome, demonstrating that front‑end technologies are sufficient to build a Kinect‑based web motion game.

2 – How to Implement

Using HTML5 to develop a Kinect motion game involves two main steps: collecting data from the Kinect (color, depth, skeleton) and making that data accessible to the browser.

1. Data Collection

Color data – color image

Depth data – infrared information

Skeleton data – calculated from depth and color data

2. Exposing Kinect Data to the Browser

We evaluated several frameworks that use sockets to transmit data from a server process to the browser:

Kinect‑HTML5 (C# server)

ZigFu (supports H5, Unity3D, Flash; commercial)

DepthJS (browser plugin)

Node‑Kinect2 (Node.js server, many examples, fast feedback)

We chose Node‑Kinect2 despite the lack of official documentation because of its extensive examples and familiarity for front‑end developers.

Kinect – captures player data (depth, color, etc.)

Node‑Kinect2 – reads and processes Kinect data

Browser – listens to the Node server and receives player data for game logic

3 – Preparation

System Requirements

USB 3.0

DX11‑compatible GPU

Windows 8 or later

Browser with WebSocket support

Kinect v2 sensor

Environment Setup

Connect Kinect v2

Install Kinect SDK v2.0

Install Node.js

Install Node‑Kinect2

4 – Demo

Below we demonstrate how to obtain skeleton data and highlight the spine mid‑section and hand gestures.

Server Side

Create a web server that sends skeleton data to the browser.

Browser Side

Receive skeleton data in the browser and render it on a canvas.

5 – Development Documentation

Node‑Kinect2 lacks official docs, so we compiled the following reference.

Server‑Side Data Types

bodyFrame – skeleton data

infraredFrame – infrared data

longExposureInfraredFrame – high‑precision infrared

rawDepthFrame – raw depth data

depthFrame – processed depth data

colorFrame – color image

multiSourceFrame – all data streams

audio – audio data (untested)

Skeleton Joint Types

0 spineBase – 脊椎基部

1 spineMid – 脊椎中部

2 neck – 颈部

3 head – 头部

4 shoulderLeft – 左肩

5 elbowLeft – 左肘

6 wristLeft – 左腕

7 handLeft – 左手掌

8 shoulderRight – 右肩

9 elbowRight – 右肘

10 wristRight – 右腕

11 handRight – 右手掌

12 hipLeft – 左臀

13 kneeLeft – 左膝

14 ankleLeft – 左踝

15 footLeft – 左脚

16 hipRight – 右臀

17 kneeRight – 右膝

18 ankleRight – 右踝

19 footRight – 右脚

20 spineShoulder – 颈下脊椎

21 handTipLeft – 左手指

22 thumbLeft – 左拇指

23 handTipRight – 右手指

24 thumbRight – 右拇指

Gesture Types

0 unknown – cannot recognize

1 notTracked – not detected

2 open – palm

3 closed – fist

4 lasso – scissors hand (merged index finger)

Skeleton Data Structure

body {<br/>  bodyIndex [number]: index (up to 6 users),<br/>  joints [array]: joint objects with coordinates and colors,<br/>  leftHandState [number]: left hand gesture,<br/>  rightHandState [number]: right hand gesture,<br/>  tracked [boolean]: whether the body is captured,<br/>  trackingId ...<br/>}

Kinect Object Methods

on – listen to data events

open – open Kinect device

close – close device

openBodyReader – read skeleton data

open**Reader – read other data types

6 – Practical Summary

1. Using HTML5 to develop Kinect motion games lowers the technical barrier, allowing front‑end engineers to create such games easily.

2. Numerous frameworks can be leveraged, e.g., jQuery, CreateJS, Three.js for different rendering approaches.

3. The possibilities are vast: combining motion games with WebAR, WebAudio, or mobile devices opens exciting new opportunities.