React Three Fiber Integration

Editframe provides first-class integration with React Three Fiber (R3F) for rendering 3D scenes in video compositions. Import from @editframe/react/r3f to access components and utilities for synchronizing Three.js animations with your timeline.

Import

import {
  CompositionCanvas,
  OffscreenCompositionCanvas,
  useCompositionTime
} from "@editframe/react/r3f";

CompositionCanvas

Renders a React Three Fiber scene on the main thread with automatic timeline synchronization.

Basic Usage

import { Timegroup } from "@editframe/react";
import { CompositionCanvas, useCompositionTime } from "@editframe/react/r3f";
import { Box } from "@react-three/drei";

function RotatingBox() {
  const { timeMs } = useCompositionTime();
  const rotation = (timeMs / 1000) * Math.PI * 2; // Full rotation per second

  return (
    <Box rotation={[0, rotation, 0]}>
      <meshStandardMaterial color="orange" />
    </Box>
  );
}

export const Video = () => {
  return (
    <Timegroup mode="fixed" duration="5s" className="w-[1920px] h-[1080px]">
      <CompositionCanvas shadows>
        <ambientLight intensity={0.5} />
        <pointLight position={[10, 10, 10]} />
        <RotatingBox />
      </CompositionCanvas>
    </Timegroup>
  );
};

Features

• Automatic Time Sync: Integrates with ef-timegroup's addFrameTask to synchronize 3D animations with your timeline
• Deterministic Rendering: Uses frameloop="demand" for frame-by-frame rendering during video export
• WebGL Sync: Calls gl.finish() after each frame to ensure all GPU commands complete before capture
• Preserves Drawing Buffer: Automatically sets preserveDrawingBuffer: true for video export

useCompositionTime Hook

Access the current composition time inside your R3F scene:

function AnimatedSphere() {
  const { timeMs, durationMs } = useCompositionTime();
  const progress = timeMs / durationMs; // 0 to 1
  const scale = 1 + Math.sin(progress * Math.PI * 2) * 0.5;

  return (
    <mesh scale={[scale, scale, scale]}>
      <sphereGeometry args={[1, 32, 32]} />
      <meshStandardMaterial color="hotpink" />
    </mesh>
  );
}

Custom WebGL Options

<CompositionCanvas
  shadows
  gl={{
    antialias: true,
    alpha: true,
    // preserveDrawingBuffer is automatically added
  }}
  camera={{ position: [0, 0, 5], fov: 75 }}

>
  {/* Scene content */}
</CompositionCanvas>

OffscreenCompositionCanvas

Renders a React Three Fiber scene in a web worker using OffscreenCanvas. This keeps the main thread free and enables rendering to continue even when the browser tab is hidden.

Worker Setup

Create a worker file:

// scene-worker.ts
import { renderOffscreen } from "@editframe/react/r3f";
import { useCompositionTime } from "@editframe/react/r3f";
import { Box } from "@react-three/drei";

function Scene() {
  const { timeMs } = useCompositionTime();
  const rotation = (timeMs / 1000) * Math.PI;

  return (
    <>
      <ambientLight intensity={0.5} />
      <pointLight position={[10, 10, 10]} />
      <Box rotation={[0, rotation, 0]}>
        <meshStandardMaterial color="cyan" />
      </Box>
    </>
  );
}

renderOffscreen(<Scene />);

Component Usage

import { Timegroup } from "@editframe/react";
import { OffscreenCompositionCanvas } from "@editframe/react/r3f";

const worker = new Worker(
  new URL('./scene-worker.ts', import.meta.url),
  { type: 'module' }
);

export const Video = () => {
  return (
    <Timegroup mode="fixed" duration="10s" className="w-[1920px] h-[1080px]">
      <OffscreenCompositionCanvas
        worker={worker}
        canvasProps={{ shadows: true, dpr: [1, 2] }}
        fallback={
          <div className="flex items-center justify-center w-full h-full">
            <p>OffscreenCanvas not supported</p>
          </div>
        }
      />
    </Timegroup>
  );
};

Features

• Web Worker Rendering: All R3F rendering happens in a separate thread
• Background Tab Resilience: Continues rendering when the browser tab is hidden
• Zero-Copy Transfer: Uses ImageBitmap transfer for efficient pixel data
• Automatic Sync: Integrates with addFrameTask for frame-by-frame rendering
• Safari Fallback: Shows fallback content when OffscreenCanvas is unavailable

Worker Protocol

The worker communicates via structured messages:

Main → Worker:

{
  type: 'renderFrame',
  timeMs: number,
  durationMs: number,
  requestId: number
}

Worker → Main:

{
  type: 'frameRendered',
  requestId: number,
  bitmap: ImageBitmap  // Transferred, not copied
}

When to Use OffscreenCanvas vs CompositionCanvas

Use OffscreenCompositionCanvas when:

• Complex 3D scenes with heavy computation
• Long render times where main thread responsiveness matters
• Need guaranteed rendering in background tabs
• Rendering multiple compositions simultaneously

Use CompositionCanvas when:

• Simple 3D scenes
• Browser compatibility is critical (Safari support)
• Debugging (easier to inspect in main thread)
• Using Three.js features incompatible with workers

Advanced: Custom Animations

Combine useCompositionTime with Three.js for complex animations:

Particle System

function ParticleSystem() {
  const { timeMs, durationMs } = useCompositionTime();
  const meshRef = useRef<THREE.InstancedMesh>(null);

  useEffect(() => {
    if (!meshRef.current) return;

    const count = 1000;
    const dummy = new THREE.Object3D();
    const progress = timeMs / durationMs;

    for (let i = 0; i < count; i++) {
      const t = progress + (i / count) * 0.1;
      const angle = t * Math.PI * 2;
      const radius = 5 + Math.sin(t * 10) * 2;

      dummy.position.set(
        Math.cos(angle) * radius,
        Math.sin(t * 5) * 3,
        Math.sin(angle) * radius
      );
      dummy.rotation.set(t * Math.PI, t * Math.PI * 2, 0);
      dummy.updateMatrix();

      meshRef.current.setMatrixAt(i, dummy.matrix);
    }

    meshRef.current.instanceMatrix.needsUpdate = true;
  }, [timeMs, durationMs]);

  return (
    <instancedMesh ref={meshRef} args={[undefined, undefined, 1000]}>
      <sphereGeometry args={[0.1, 16, 16]} />
      <meshStandardMaterial color="white" />
    </instancedMesh>
  );
}

Rendering to Video

3D scenes render to video just like any other Editframe element:

import { Timegroup } from "@editframe/react";
import { CompositionCanvas } from "@editframe/react/r3f";

export const Video = () => {
  return (
    <Timegroup
      mode="fixed"
      duration="10s"
      className="w-[1920px] h-[1080px]"
    >
      <CompositionCanvas>
        {/* Your 3D scene */}
      </CompositionCanvas>
    </Timegroup>
  );
};

// Render with CLI
// npx editframe render ./src/Video.tsx

The R3F integration ensures:

1. Each frame renders synchronously at the correct timeline position
2. WebGL finishes all commands before frame capture
3. Deterministic output regardless of playback state
4. Proper integration with Editframe's rendering pipeline

Browser Support

• CompositionCanvas: All modern browsers (Chrome, Firefox, Safari, Edge)
• OffscreenCompositionCanvas: Chrome, Firefox, Edge (Safari does not support OffscreenCanvas)

Related

• timegroup.md - Timeline container element- render-to-video.md - Rendering compositions to video
• scripting.md - Programmatic element control