3d model generation parameters

[AugmentedRealityCat]

What are the parameters we can possibly modify for the 3d model generation process ?

I discovered that the resolution of the image you are using as a source as a direct impact on the resolution of the 3d mesh produced by the 3d-photo-inpainting process. The larger the image, the more polygons you get.

I also discovered that the alpha channel of the vertex color map where the RGB values are stored seems to identify with different values the different inpainting surfaces it has produced. Like the main surface area is value 064, then the extension on the left has value 128 and the top one 196 (those are just random examples to explain the concept, not actual values). Can anyone else confirm that ?

What are your favourite algorithms for depth extraction when used to generate a 3d model ? My best successes so fare were with the newest Midas (v3.1, both the 512 and the 384) and the DPT_Hybrid, almost all the time with the BOOST option turned OFF. But I also had scenes that worked better with other algos, like Leres+Boost. What about you ?

Speaking of boost, it looks like that for most situations it ruins the 3d model generation. It seems to be good to extract local relief, but it distorts the overall positioning of objects in the scene. At least, so it seems to me so far with the parameters I use (boost res = 768).

Also, looking for more parameters to control, can we use change the parameters listed over here : https://github.com/vt-vl-lab/3d-photo-inpainting/blob/master/DOCUMENTATION.md
as per here: https://github.com/thygate/stable-diffusion-webui-depthmap-script/blob/main/scripts/inpaint/argument.yml
and be hopeful that this will work ?

[thygate]

Yes, the mesh size is directly related to the image resolution. Interesting observation on the alpha channel, I have only observed very small values. My best results have been with res101 and boost. (boost res ?)

The parameters for the inpainting (the yaml file you list) are hardcoded atm, there are a lot of them, too many to include in the ui.

I had already discovered an optional parameter I missed, that should be enabled (repeat_inpaint_edge = true), which will be enabled in the next version.

We could discuss which parameters to include, perhaps in settings instead of the ui ..

edit: The bilateral filter params should not be changed, as they are critical to remove features that would otherwise crash the algorithm. (i found out the hard way)

[thygate]

These are the remaining parameters we could change, but I assume it's already been optimized to get the best results ..

config['extrapolation_thickness'] = 60
config['extrapolate_border'] = True
config['depth_threshold'] = 0.04
config['redundant_number'] = 12
config['ext_edge_threshold'] = 0.002
config['background_thickness'] = 70
config['context_thickness'] = 140
config['background_thickness_2'] = 70
config['context_thickness_2'] = 70
config['log_depth'] = True
config['depth_edge_dilate'] = 10
config['depth_edge_dilate_2'] = 5
config['repeat_inpaint_edge'] = True

The settings like context_thickness, background_thickness, and depth_edge_dilate that have 2 settings, the first value is used in the first pass, the second value in the second pass.