Yeah I think I'm making a THREE.js extension

[Brackets-Coder]

Insanity? Yes. Probably not going to get merged? Who cares. Waste of time? Potentially, but I might learn a lot :D

With the rise of #2285, it's become clear to me that users are looking for more advanced 3D rendering capabilities than what Simple3D provides out of the box without having the complexity of writing everything from Scratch in Pen+.
Yes, I'm aware that this has already been attempted, but it seems like no one's succeeded. The closest true THREE.js scratch port I could find was this: https://github.com/DarksideAlex/ScratchTHREE
Even then, it looks like an unfinished and deprecated archive.

Though it could take years and potentially be a wasted effort, I think that with the recent innovations Turbowarp has been making in custom extension functionality this could be very much possible; in fact, I've already gotten the barebones extension demo in the docs to load three.js and log the library to the console.

[fath11]

There has been better implementation of Three.js such as:

• https://github.com/Civ3ro/extensions

And unreleased/not on GitHub attempts on Discord (for the sake of documentation):

• https://discord.com/channels/837024174865776680/1353776614898401422
• https://discord.com/channels/837024174865776680/1299782673186816121 (Released on their Discord server)
• https://discord.com/channels/837024174865776680/1387119825758982144 (Released on their Discord server)
• https://discord.com/channels/837024174865776680/1366110906777337949 (uses Babylon.js)

The attempts I shown has gotten basic functionality working like rendering without trying to break TurboWarp's normal rendering. One thing I want to say about ScratchTHREE though is their use of UUID to reference objects which is really unique because it incentives the use of variables and I think might be a better approach than using names. Maybe it should be used along side names? I am not sure.

[Brackets-Coder]

Sure there's a lot of Three.js extensions, but they feel unfinished (to quote you, "attempts") and/or unprofessional to me. A lot of them have good concepts in one area, but don't apply them to another, or similar

The lack of a complete, simple Three.js extension is what I'm trying to solve. I'm not trying to simply "Make a three.js extension"

I just wanna make one that's consistent, readable code, and makes sense for beginning users.
Yes, I doubt it will be merged
Yes, I'm aware of the numerous other Three.js extensions
Yes, I'm insane

I just want to make the best one because I like making cool things things that are easy for other people to use

[fath11]

I wouldn't call it something insane but it will be interesting to see your attempt since you are more skilled than others that I listed.

[Brackets-Coder]

I wouldn't call it something insane but it will be interesting to see your attempt since you are more skilled than others that I listed.

It'll be a while before you see anything from me. Also, I wouldn't consider myself "skilled"; the best candidate for making this extension would be @Xeltalliv but I'm quite sure he doesn't want to make another 3D extension.

Hooking the renderer into the scratch stage drawable is what I've been trying to figure out because it's more complicated than stage layers, Three likes to have its own canvas (especially for interaction like OrbitalControls) and slapping a new one on top of the stage won't work.

[Brackets-Coder]

I got the barebones working

video.mp4

[Brackets-Coder]

@Xeltalliv since you seem to know all about this kind of thing would you mind taking a look at the renderer? I've attempted to get it to work and it kinda works, but when doing higher framerates or high quality pen or opening the developer inspector can adjust the speed at which the box moves. Also it requires really tiny increments for changing things, I don't really know why

this is kinda the first time I've worked with skin renderers so I'm glad I got this far at all

if you need the whole extension please let me know

I have a few other concerns about the logistics of this but they're more related to THREE.js so I'll attempt to resolve them myself...

let deltaTime = 0;
  let previousTime = 0;

  vm.runtime.on('BEFORE_EXECUTE', () => {
    const now = performance.now();

    if (previousTime === 0) {
      // First frame. We used to always return 0 here, but that can break projects that
      // expect delta time to always be non-zero. Instead we'll make our best guess.
      deltaTime = 1 / vm.runtime.frameLoop.framerate;
    } else {
      deltaTime = (now - previousTime) / 1000;
    }

    previousTime = now;
  });

  // Now that Three.js is loaded, create a stage layer to draw the rendered output to
  // Some of this skin class is based on the fantastic Simple3D extension
  class ThreeSkin extends Scratch.vm.renderer.exports.Skin {
    constructor(id, renderer) {
      super(id, renderer);
  
      this._renderer = renderer;
  
      const canvas = document.createElement('canvas');
      this._canvas = canvas;
  
      this.threeRenderer = new THREE.WebGLRenderer({
        alpha: true,
        antialias: true,
        canvas: canvas,
      });

      this.threeRenderer.setClearColor(0x000000, 0);
  
      const gl = renderer.gl;
      const texture = gl.createTexture();
      gl.bindTexture(gl.TEXTURE_2D, texture);
      gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_S, gl.CLAMP_TO_EDGE);
      gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_T, gl.CLAMP_TO_EDGE);
      gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MIN_FILTER, gl.NEAREST);
      gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MAG_FILTER, gl.NEAREST);
      this._texture = texture;
      this._nativeSize = renderer.getNativeSize();
      this._boundOnNativeSizeChanged = this.onNativeSizeChanged.bind(this);
      this._rotationCenter = [this._nativeSize[0] / 2, this._nativeSize[1] / 2];
      renderer.on('NativeSizeChanged', this._boundOnNativeSizeChanged);
      this.resizeCanvas();
    }
  
    dispose() {
      this._renderer.removeListener('NativeSizeChanged', this._boundOnNativeSizeChanged);
      if (this._texture) this._renderer.gl.deleteTexture(this._texture);
      super.dispose();
    }
  
    get size() {
      return this._nativeSize;
    }
  
    getTexture() {
      return this._texture;
    }
  
    updateContent() {
      this.threeRenderer.render(scene, camera);
  
      const gl = this._renderer.gl;
      gl.pixelStorei(gl.UNPACK_PREMULTIPLY_ALPHA_WEBGL, true);
      gl.bindTexture(gl.TEXTURE_2D, this._texture);
      gl.texImage2D(
        gl.TEXTURE_2D,
        0,
        gl.RGBA,
        gl.RGBA,
        gl.UNSIGNED_BYTE,
        this._canvas
      );
      gl.pixelStorei(gl.UNPACK_PREMULTIPLY_ALPHA_WEBGL, false);
  
      this.emitWasAltered();
    }
  
    resizeCanvas() {
      let w, h;
      if (this._renderer.useHighQualityRender) {
        w = this._renderer.canvas.width;
        h = this._renderer.canvas.height;
      } else {
        [w, h] = this._nativeSize;
      }
    
      this._canvas.width = w;
      this._canvas.height = h;
    
      this.threeRenderer.setSize(w, h);
      camera.aspect = w / h;
      camera.updateProjectionMatrix();
    
      this.updateContent();
    }
  
    onNativeSizeChanged(event) {
      this._nativeSize = event.newSize;
      this._rotationCenter = [this._nativeSize[0] / 2, this._nativeSize[1] / 2];
      this.resizeCanvas();
    }
  } 

  // use the drawable order to create a new three drawable between the pen and video layers
  // It's wrapped in an IIFE for collapsibilitiy 
  (() => {
    let videoIndex = renderer._groupOrdering.indexOf('video');
    const groupName = 'threeJSLayer';
    renderer._groupOrdering.splice(videoIndex + 1, 0, groupName);
    renderer._layerGroups[groupName] = {
      groupIndex: videoIndex + 1,
      drawListOffset: renderer._layerGroups['video'].drawListOffset
    };
  
    renderer._groupOrdering.forEach((name, i) => {
      renderer._layerGroups[name].groupIndex = i;
    });
  
    const skinId = renderer._nextSkinId++;
    const skin = new ThreeSkin(skinId, renderer); // your Three.js skin
    renderer._allSkins[skinId] = skin;
  
    const drawableId = renderer.createDrawable(groupName);
    const drawable = renderer._allDrawables[drawableId];
    renderer.updateDrawableSkinId(drawableId, skinId);
  
    const originalDraw = renderer.draw;
  
    renderer.draw = function() {
      skin.updateContent();
      originalDraw.call(this);
    };
  
    drawable.setHighQuality = function (...args) {
      Object.getPrototypeOf(this).setHighQuality(...args);
      this.skin.resizeCanvas();
    }
  })();