One-Stroke Gesture Recognition Using Canvas: From Drawing to Comparison

Introduction

Recently I explored graphics and built a one‑stroke gesture recognition demo using canvas; the result looks like the animated GIF below. The article walks through the entire pipeline—from capturing raw points to comparing gestures—without involving deep learning or AI libraries.

Step 1: Gesture Drawing

Users draw on a canvas; mouse events are captured and consecutive points are linked with lines while each point is also drawn as a small circle. The core code is:

handleMousemove(e: MouseEvent) {
    if (!this.isMove) return;
    const curPoint = this.getCanvasPos(e);
    const lastPoint = this.inputPoints[this.inputPoints.length - 1];
    // draw line
    CanvasUtils.drawLine(this.ctx2d, lastPoint[0], lastPoint[1], curPoint[0], curPoint[1], 'blue', 3);
    // draw point
    CanvasUtils.drawCircle(this.ctx2d, curPoint[0], curPoint[1], 5);
    this.inputPoints.push(curPoint);
}

After drawing, the raw points are unevenly spaced because of varying drawing speed.

Step 2: Resampling

To obtain uniform data, the gesture is resampled into n equally spaced points. First the total path length is computed, then points are interpolated along the path until the desired count is reached. The implementation is:

export type Point = [number, number];
static resample(inputPoints: Point[], sampleCount: number): Point[] {
    const len = GeoUtils.getLength(inputPoints);
    const unit = len / (sampleCount - 1);
    const outputPoints: Point[] = [[...inputPoints[0]]];
    let curLen = 0;
    let prevPoint = inputPoints[0];
    for (let i = 1; i < inputPoints.length; i++) {
        const curPoint = inputPoints[i];
        let dx = curPoint[0] - prevPoint[0];
        let dy = curPoint[1] - prevPoint[1];
        let tempLen = GeoUtils.getLength([prevPoint, curPoint]);
        while (curLen + tempLen >= unit) {
            const ds = unit - curLen;
            const ratio = ds / tempLen;
            const newPoint: Point = [prevPoint[0] + dx * ratio, prevPoint[1] + dy * ratio];
            outputPoints.push(newPoint);
            curLen = 0;
            prevPoint = newPoint;
            dx = curPoint[0] - prevPoint[0];
            dy = curPoint[1] - prevPoint[1];
            tempLen = GeoUtils.getLength([prevPoint, curPoint]);
        }
        prevPoint = curPoint;
        curLen += tempLen;
    }
    while (outputPoints.length < sampleCount) {
        outputPoints.push([...prevPoint]);
    }
    return outputPoints;
}

Resampling yields a uniform set of points ready for further processing.

Step 3: Translation

The gesture is shifted so that its centroid aligns with the canvas center. The translation distance is computed from the centroid, and each point is moved accordingly:

// translate each point
static translate(points: Point[], dx: number, dy: number) {
    points.forEach(p => {
        p[0] += dx;
        p[1] += dy;
    });
}

After translation the origin is moved to the canvas centre, simplifying later rotation and scaling.

Step 4: Rotation

The direction of the gesture is normalized by rotating it to the nearest of eight reference lines. The required radian is computed from the start point and the centroid, then all points are rotated:

// compute radian to nearest sub‑line
static computeRadianToSubline(center: Point, startPoint: Point, sublineCount: number): number {
    const dy = startPoint[1] - center[1];
    const dx = startPoint[0] - center[0];
    let radian = Math.atan2(dy, dx);
    if (radian < 0) radian += TWO_PI;
    const unitRadian = TWO_PI / sublineCount;
    const targetRadian = Math.round(radian / unitRadian) * unitRadian;
    radian -= targetRadian;
    return radian;
}
// rotate each point
static rotate(points: Point[], radian: number) {
    const sin = Math.sin(radian);
    const cos = Math.cos(radian);
    points.forEach(p => {
        const [x, y] = p;
        p[0] = cos * x - sin * y;
        p[1] = sin * x + cos * y;
    });
}

Step 5: Scaling

Gestures are normalized to a common size by computing the axis‑aligned bounding box (AABB) of the resampled points and scaling them so that the longest side matches a target dimension (e.g., 100 × 100). The scaling code is:

// scale each point
static scale(points: Point[], scale: number) {
    points.forEach(p => {
        let [x, y] = p;
        p[0] = x * scale;
        p[1] = y * scale;
    });
}

Step 6: Gesture Recording

The normalized points are saved as a feature vector; a thumbnail can be generated by drawing the points on a small off‑screen canvas.

Step 7: Comparison (Core)

Two normalized gestures are compared by measuring the distance between their point sequences. A simple Euclidean‑squared distance is provided:

static squaredEuclideanDistance(points1, points2) {
    let squaredDistance = 0;
    const count = points1.length;
    for (let i = 0; i < count; i++) {
        const p1 = points1[i];
        const p2 = points2[i];
        const dx = p1[0] - p2[0];
        const dy = p1[1] - p2[1];
        squaredDistance += dx * dx + dy * dy;
    }
    return squaredDistance;
}

For direction‑focused similarity, cosine similarity is used:

static calcCosDistance(vector1: number[], vector2: number[]): number {
    let similarity = 0;
    vector1.forEach((v1, i) => {
        const v2 = vector2[i];
        similarity += v1 * v2;
    });
    return similarity; // range -1 to 1
}

Both metrics illustrate how to quantify gesture similarity.

Important Considerations

Gestures have direction; vertical and horizontal strokes must be distinguished.

Aspect ratio influences similarity—squares differ from long rectangles.

Number of sampled points affects both efficiency and robustness.

Complexity of the shape does not directly correlate with recognition accuracy.

Potential applications include real‑time correction of hand‑drawn shapes in remote teaching.

General Comparison Pipeline (Optional)

Feature extraction (standardizing data into equal‑length vectors) followed by algorithmic comparison (distance or cosine similarity) is a common pattern for many similarity problems.

Multi‑Stroke Extension (Optional)

For characters composed of multiple strokes, each stroke can be processed individually, then their similarity scores summed; additional heuristics handle stroke order, missing strokes, and varying point counts.

Conclusion

The article presented a complete, code‑driven workflow for one‑stroke gesture recognition, from raw canvas input to normalized feature vectors and similarity measurement, demonstrating that even simple geometric processing can achieve useful pattern matching.