feat(layers): use opacity transfer function except for PT

[PaulHax]

Instead of a layer's opacity being uniform, despite voxel values, use the opacity points in the preset. Basicly reverts #449

But, to maintain existing behavior for PT/PET modality, add a new preset, "2hot-opaque", with just one opacity point at 1.

Good CT-PT dataset for testing https://demo.orthanc-server.com/app/explorer.html#study?uuid=6b9e19d9-62094390-5f9ddb01-4a191ae7-9766b715

A CT (with missing slices) and 2 DICOM SEG instances
chest-seg-sr-lite.zip

[netlify]

✅ Deploy Preview for volview-dev ready!

Built without sensitive environment variables

Name	Link
🔨 Latest commit	f9896c7
🔍 Latest deploy log	https://app.netlify.com/sites/volview-dev/deploys/6548f523ccf39a00080c52ef
😎 Deploy Preview	https://deploy-preview-482--volview-dev.netlify.app/
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

---

To edit notification comments on pull requests, go to your Netlify site configuration.

[aylward]

Using the CT and 2SEG data in the zip file, dragged-and-dropped, and the SEG objects don't align with the CT data.

Also, I could not find the "2hot-opaque" transfer function.

[PaulHax]

I deleted instances from the CT series to fit zip under 25 mb, so alignment is wrong.

2hot-opaque is just for PT DICOMS's added as 2D layers.

[floryst]

Overall LGTM. I'll wait for functionality confirmation from @aylward.

[sedghi]

I'm trying to understand this change so we can apply it to our example and make our fusion opacity linear as well.

Currently, we are modifying the opacity like this:

// CT not touched

// PT
const ofun = volumeActor.getProperty().getScalarOpacity(0);
ofun.addPoint(range[0], colormap.opacity);
ofun.addPoint(range[1], colormap.opacity);

However, as discussed earlier, this approach results in undesired non-linear behavior.

CleanShot.2024-11-18.at.17.11.59.mp4

I attempted to understand the opacity function transfer function proxy but couldn't figure out how to adjust the layer-wise opacity of the fusion without altering the opacity transfer function of my PT volume actor.

@PaulHax, could you advise on what changes are needed in this codebase? The solution in this PR seems specific to VolView.

• What is the purpose of the proxy, and how does it differ from a standard piecewise opacity function? is the proxy only applied to the forground (in this case pt?)
• Do we encounter the case vtkPiecewiseFunctionProxy.Mode.Points that ensures linearity?

Thanks in advance for your help

[PaulHax]

Hi @sedghi

VolView's slice opacity sliders change the opacity of the whole mesh:
https://github.com/Kitware/VolView/blob/main/src/components/vtk/VtkLayerSliceRepresentation.vue#L97
The PiecewiseFunction points are set once, when the "color preset" is changed.

Proxies are a red herring. (floryst ripped out VolView's use of proxies a while back anyways. Proxies were used to replicate a single PiecewiseFunction/etc across views, but JS frameworks do a more predictable job.)

I'm not sure why your example does not work. The approach you outlined, setting the opacity peicewise function directly, seems sound. Beside the usual bug suspect, ImageMapper, perhaps the ImageSlice mesh is getting tossed in the opaqe render pass.

[sedghi]

@PaulHax
I'm sorry for my basic question, but what is a mesh in a volume actor/mapper?

The demo above uses a volume mapper, not an image mapper. We perform thin slice rendering of volume mappers for MPR, which differs from how VTK or VolView handle it (I guess?). The original reason for not using imageSliceMapper was texture sharing and memory optimization. However, since you've added these optimizations in vtk.js, perhaps we should consider using an image mapper for it as well.

Can you refer me to the place that the mesh is defined/handled in the ImageSliceMapper?

[PaulHax]

Right, VolView is using the ImageSlice/ImageMapper CPU slicing approach for the 2D views. Where the ImageMapper defines the "mesh" it draws on: https://github.com/Kitware/vtk-js/blob/master/Sources/Rendering/OpenGL/ImageMapper/index.js#L1357

It would be nice if we did Cornerstone's approach and just used the VolumeMapper, for a unified pipeline/shader for the 2D and 3D views. We would be limited to 4 "layers" as OpenGL can only do 4 components in a texture. vtk.js does not blend multiple VolumeMappers, yet: Kitware/vtk-js#3133

So that might explain the behavior in your demo. If you have 2 ImageDatas, one each for CT/PT, and 2 VolumeMappers, then the PT volume goes completely opaque and there is no blending with the background CT.

If the Cornerstone example is using one ImageData/texture with 2 components, a single VolumeMapper should have robust support for blending them, with the 4 component limitation. That said, folks more informed than me expect a certen type of blending with CT+PET: #355

[sedghi]

Are you suggesting a single volume mapper with independent components one for CT and one for PT?

I used that trick for implementing MIP + Labelmap here Kitware/vtk-js#3103 I remember

Is there any benefit to using a multi-volume mapper? If the data layers are fewer than four, is it generally better to use a single-volume mapper with multiple independent components?

[PaulHax]

Are you suggesting a single volume mapper with independent components one for CT and one for PT?

Yes.

What I've heard is that vtk.js does not do multi-volume at the moment, but the real experts will know: finetjul, floryst, sankhesh

[sedghi]

Thanks a lot for your time