Implementing a Twisted Gallery Effect with Three.js and WebGL

Introduction The article discusses why typical CSS and SVG cannot achieve pixel‑level distortion effects and introduces WebGL fragment shaders as the solution for creating impressive twisted visual effects.

Preparation A three.js template is provided (forkable from the linked CodePen) as the starting point for the project.

Implementation Idea The core concept is to keep the HTML world and the WebGL world synchronized so that any visual effect can be applied to native HTML elements without breaking interaction.

World Synchronization

Building HTML

<main class="overflow-hidden">
  <div data-scroll>
    <div class="relative w-screen h-screen flex-center">
      <img class="w-240 h-120" src="https://i.loli.net/2019/11/16/cqyJiYlRwnTeHmj.jpg" crossorigin="anonymous" />
    </div>
    ... (additional image divs) ...
  </div>
</main>
<div class="twisted-gallery fixed -z-1 inset-0 w-screen h-screen"></div>

The images are initially hidden with CSS:

img {
  opacity: 0;
}

Synchronizing HTML and WebGL Units To render HTML images in WebGL, the pixel dimensions of each img element must be transferred to the WebGL scene. The field of view (FOV) is calculated so that 1 WebGL unit equals 1 CSS pixel.

class TwistedGallery extends Base {
  constructor(sel, debug) {
    ...
    this.cameraPosition = new THREE.Vector3(0, 0, 600);
    const fov = this.getScreenFov();
    this.perspectiveCameraParams = { fov, near: 100, far: 2000 };
  }
  getScreenFov() {
    return ky.rad2deg(2 * Math.atan(window.innerHeight / 2 / this.cameraPosition.z));
  }
}

For testing, the plane geometry is set to 100 × 100 units, matching the HTML image size.

Ensuring Images Are Loaded The imagesLoaded library is used to wait for all images before proceeding.

import imagesLoaded from "https://cdn.skypack.dev/[email protected]";
const preloadImages = (sel = "img") => {
  return new Promise(resolve => {
    imagesLoaded(sel, { background: true }, resolve);
  });
};

Displaying Images in WebGL A DOMMeshObject class bridges DOM elements to WebGL meshes by reading the element’s bounding rectangle and creating a matching PlaneBufferGeometry .

class DOMMeshObject {
  constructor(el, scene, material = new THREE.MeshBasicMaterial({ color: 0xff0000 })) {
    this.el = el;
    const rect = el.getBoundingClientRect();
    this.rect = rect;
    const { width, height } = rect;
    const geometry = new THREE.PlaneBufferGeometry(width, height, 10, 10);
    const mesh = new THREE.Mesh(geometry, material);
    scene.add(mesh);
    this.mesh = mesh;
  }
  setPosition() {
    const { top, left, width, height } = this.rect;
    const x = left + width / 2 - window.innerWidth / 2;
    const y = -(top + height / 2 - window.innerHeight / 2) + window.scrollY;
    this.mesh.position.set(x, y, 0);
  }
}

Images are turned into textures and attached to a custom shader material:

class TwistedGallery extends Base {
  ...
  createDistortImageMaterial() {
    const distortImageMaterial = new THREE.ShaderMaterial({
      vertexShader: twistedGalleryMainVertexShader,
      fragmentShader: twistedGalleryMainFragmentShader,
      side: THREE.DoubleSide,
      uniforms: { uTexture: { value: 0 } }
    });
    this.distortImageMaterial = distortImageMaterial;
  }
  createImageDOMMeshObjs() {
    const { images, scene, distortImageMaterial } = this;
    const imageDOMMeshObjs = images.map(image => {
      const texture = new THREE.Texture(image);
      texture.needsUpdate = true;
      const material = distortImageMaterial.clone();
      material.uniforms.uTexture.value = texture;
      return new DOMMeshObject(image, scene, material);
    });
    this.imageDOMMeshObjs = imageDOMMeshObjs;
  }
  setImagesPosition() {
    this.imageDOMMeshObjs.forEach(obj => obj.setPosition());
  }
}

Vertex Shader (twistedGalleryMainVertexShader)

varying vec2 vUv;
void main(){
  vec4 modelPosition = modelMatrix * vec4(position, 1.);
  vec4 viewPosition = viewMatrix * modelPosition;
  vec4 projectedPosition = projectionMatrix * viewPosition;
  gl_Position = projectedPosition;
  vUv = uv;
}

Fragment Shader (twistedGalleryMainFragmentShader)

uniform sampler2D uTexture;
varying vec2 vUv;
void main(){
  vec2 newUv = vUv;
  vec4 texture = texture2D(uTexture, newUv);
  gl_FragColor = vec4(texture.rgb, 1.);
}

Scrolling Native scroll events lack velocity information, so locomotive-scroll is used to capture both position and speed.

import LocomotiveScroll from "https://cdn.skypack.dev/[email protected]";
class TwistedGallery extends Base {
  listenScroll() {
    const scroll = new LocomotiveScroll({ getSpeed: true });
    scroll.on("scroll", () => {
      this.setImagesPosition();
    });
    this.scroll = scroll;
  }
}

Twisted Effect (Post‑processing) A post‑processing pass manipulates the rendered scene based on scroll speed.

import { RenderPass } from "https://cdn.skypack.dev/[email protected]/examples/jsm/postprocessing/RenderPass.js";
import { ShaderPass } from "https://cdn.skypack.dev/[email protected]/examples/jsm/postprocessing/ShaderPass.js";
import gsap from "https://cdn.skypack.dev/[email protected]";
class TwistedGallery extends Base {
  constructor(sel, debug) {
    ...
    this.scrollSpeed = 0;
  }
  createPostprocessingEffect() {
    const composer = new EffectComposer(this.renderer);
    const renderPass = new RenderPass(this.scene, this.camera);
    composer.addPass(renderPass);
    const customPass = new ShaderPass({
      vertexShader: twistedGalleryPostprocessingVertexShader,
      fragmentShader: twistedGalleryPostprocessingFragmentShader,
      uniforms: {
        tDiffuse: { value: null },
        uRadius: { value: 0.75 },
        uPower: { value: 0 }
      }
    });
    customPass.renderToScreen = true;
    composer.addPass(customPass);
    this.composer = composer;
    this.customPass = customPass;
  }
  setScrollSpeed() {
    const scrollSpeed = this.scroll.scroll.instance.speed || 0;
    gsap.to(this, {
      scrollSpeed: Math.min(Math.abs(scrollSpeed) * 1.25, 2),
      duration: 1
    });
  }
  update() {
    if (this.customPass) {
      this.setScrollSpeed();
      this.customPass.uniforms.uPower.value = this.scrollSpeed;
    }
  }
}

Post‑processing vertex shader is identical to the main vertex shader (omitted). The fragment shader applies a radial distortion based on the uniform uPower :

uniform sampler2D tDiffuse;
uniform float uRadius;
uniform float uPower;
varying vec2 vUv;
void main(){
  vec2 pivot = vec2(.5);
  vec2 d = vUv - pivot;
  float rDist = length(d);
  float gr = pow(rDist / uRadius, uPower);
  float mag = 2. - cos(gr - 1.);
  vec2 uvR = pivot + d * mag;
  gl_FragColor = texture2D(tDiffuse, uvR);
}

Conclusion The tutorial demonstrates one specific twisted effect but emphasizes that mastering the synchronization between HTML and WebGL opens the door to many more sophisticated visual experiences.

References

Three.js template: https://codepen.io/alphardex/pen/yLaQdOq

imagesLoaded: https://github.com/desandro/imagesloaded

locomotive-scroll: https://github.com/locomotivemtl/locomotive-scroll

GSAP: https://github.com/greensock/GSAP