My first time animating a 3D scene on the scroll position was with the Wind City project, where the camera position gets updated based on the scroll value. I determined which camera value I needed by first modifying the values with a Graphic User Interface, and then writing them down one by one. JavaScript then detects the current scrolling position of the user and updates the camera position based on those values.

// controls the 3D state
  useEffect(() => {
    if (cameraSettings[scrollProgress]) {
      setCameraInteractivity(cameraSettings[scrollProgress].interactivity)
      setCameraX(round(cameraSettings[scrollProgress].position.x(width)))
      setCameraY(round(cameraSettings[scrollProgress].position.y(width)))
      setCameraZ(round(cameraSettings[scrollProgress].position.z(width)))
      setCameraRotationX(
        round(cameraSettings[scrollProgress].rotation.x(width))
      )
      setCameraRotationY(
        round(cameraSettings[scrollProgress].rotation.y(width))
      )
      setCameraRotationZ(
        round(cameraSettings[scrollProgress].rotation.z(width))
      )
    }
  }, [scrollProgress, width])

This project has 2 values for every camera position: 1 for the smartphone and 1 for the desktop. The final value will be within this range and decided based on the current screen width.

{
    position: {
      x: scaleLinear([breakpoints.smartphone, breakpoints.desktopXL], [23, 20]),
      y: scaleLinear([breakpoints.smartphone, breakpoints.desktopXL], [2, -3]),
      z: scaleLinear([breakpoints.smartphone, breakpoints.desktopXL], [23, 18]),
    },
    rotation: {
      x: scaleLinear([breakpoints.smartphone, breakpoints.desktopXL], [0, 0]),
      y: scaleLinear(
        [breakpoints.smartphone, breakpoints.desktopXL],
        [0.84, 0.84]
      ),
      z: scaleLinear([breakpoints.smartphone, breakpoints.desktopXL], [0, 0]),
    },
    interactivity: { x: 0.4, y: 2 },
  }

We wanted to reuse this mechanic for another project, which shows the 3D model of a building from various angles. As an effort to improve our workflow, we decided to use Theatre.js. It aims to facilitate animating 3D scenes, by allowing the developer to update values with an editor with an interface similar to a simplified version of After Effects. Once the animation sequence is made, it’s possible to export it as a JSON file. This also has the advantage that a non-developer member of the team can update the animation and give us the JSON file at the end.

Using Theatre.js for animations is an improvement, since it is faster to set up (instead of adding our own GUI ourselves the copy-pasting the values manually). It’s also great for animating multiple elements at the same time and previewing this in the browser.

This time, since the animation needs to be more precise, using an interpolation between the smartphone values and desktop ones could have caused issues. Indeed, the camera can be stuck behind a wall if the values are a bit off. Instead, I made 1 animation per breakpoint. Setting up the additional animations at the beginning was easy (I copied and pasted the main one and modified it slightly on every breakpoint) but the feedback round required a lot more animation updates than I anticipated, which ended up being time-consuming. What makes it even worse is that animations need to be tested on every breakpoint every time to make sure that none of them have issues.

This mechanic was needed again for the 2nd story of RSB. Since having 1 animation per breakpoint was a major pain point in the previous project, I decided to search for an alternative way. Thankfully, Three.js offers the setViewOption property on its camera, which can move the final camera position in a similar way that you would move 2D content. Adding this offset without this method is quite complex since we’re in a 3D scene, so moving the camera on the x,y, and z axis would also change from which angle the building is seen and so on… This is perfect to make the animation responsive since it means I can have the 3D scene centered on smartphone and on the right on desktop.

  useEffect(() => {
    if (camera && width > 0) {
       // adds the X and Y offset to make the 3D scene's view responsive
      const offsetY =
        height * yOffsetResponsive[currentBreakpoint] + viewOffset.y;
      const offsetX =
        width * xOffsetResponsive[currentBreakpoint] + viewOffset.x;
      // used for the transition between the Berlin SVG map
      // and the satellite image of the Masurenallee
      const worldOffsetY = svgSize.height * worldOffset.y;
      const worldOffsetX = svgSize.width * 0.21 * worldOffset.x;

      // Adds the offset to the camera
      camera.setViewOffset(
        width,
        height,
        worldOffsetX + offsetX,
        worldOffsetY + offsetY,
        width,
        height
      );
    }
  }, [currentBreakpoint, width, height, viewOffset.x, worldOffset.x]);

This ended up working really well. Later, when I needed to update the first scene’s animation again I updated it so that it also uses the same mechanic.

It caused a small issue, since, for one of the sequences, I needed the building to be centered on both the smartphone and the desktop, but it was easy to solve by animating the offset values.

Another update was to use the camera target to facilitate creating the camera viewpoints. Indeed, currently, I have to manually move the camera on the x, y, and z axis and also find out which rotation angle I need so that the building stays visible. Having a camera target on the building would mean that the camera is always focused on it, and I can just focus on finding the camera position values I need. Unfortunately, this caused the camera movement to jump during the animation and did not always show the building under the right angle, so I had to go fall back on the previous animation method.