How to use own depth map?

[RyuMaster]

Hi! What if I already have my depth map and color map, how can I use this just to generate inpainting mesh that I need? Or this is not possible?

[WorldofDepth]

If uploading one's depth map were possible, that would be great! AI depth maps often need some editing. And would that also lower VRAM load?

[thygate]

@RyuMaster @WorldofDepth
Custom depth maps now possible in single image and batch.

[mikocevar]

How can one use the custom depth map? What are the requirements? I get errors.

[thygate]

use a 16 bit png for best results, in single image, and directory batch mode simply check the new checkbox.
in single file mode depthmap generation will be skipped if checkbox is checked and a depthmap is given.
in batch directory mode, if the depthmap output filename exists and the checkbox is checked, it will skip generation and load the file ..

I get errors.

The actual error messages would be helpful.

[mikocevar]

I'm so sorry about not providing the error message. I'm using an 8-bit JPG snatched from the internet. The error is:

Computing depthmap(s) ..
  0%|                                                                                                                                                                                                     | 0/1 [00:00<?, ?it/s] 

  0%|                                                                                                                                                                                                     | 0/1 [00:00<?, ?it/s]
Error completing request
Arguments: (0, <PIL.Image.Image image mode=RGB size=1024x743 at 0x4B8E5720>, None, '', '', 0, 6, 512, 512, False, True, False, True, 0.333, 1, False, 1, True, True, False, False, True, True, True, True, True, 2.5, 'polylines_sharp', 0, False, False, False, False, False, 'u2net', False, 0, 2, True, <tempfile._TemporaryFileWrapper object at 0x000000004B8E71F0>, True) {}
Traceback (most recent call last):
  File "E:\stable_difusion\modules\call_queue.py", line 56, in f
    res = list(func(*args, **kwargs))
  File "E:\stable_difusion\modules\call_queue.py", line 37, in f
    res = func(*args, **kwargs)
  File "E:\stable_difusion\extensions\stable-diffusion-webui-depthmap-script\scripts\depthmap.py", line 1069, in run_generate
    outputs, mesh_fi = run_depthmap(
  File "E:\stable_difusion\extensions\stable-diffusion-webui-depthmap-script\scripts\depthmap.py", line 464, in run_depthmap
    dimg = Image.open(custom_depthmap_img)
  File "E:\stable_difusion\venv\lib\site-packages\PIL\Image.py", line 3283, in open
    raise UnidentifiedImageError(msg)
PIL.UnidentifiedImageError: cannot identify image file <tempfile._TemporaryFileWrapper object at 0x000000004B8E71F0>

Traceback (most recent call last):
  File "E:\stable_difusion\venv\lib\site-packages\gradio\blocks.py", line 987, in postprocess_data
    if predictions[i] is components._Keywords.FINISHED_ITERATING:
IndexError: tuple index out of range

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "E:\stable_difusion\venv\lib\site-packages\gradio\routes.py", line 394, in run_predict
    output = await app.get_blocks().process_api(
  File "E:\stable_difusion\venv\lib\site-packages\gradio\blocks.py", line 1078, in process_api
    data = self.postprocess_data(fn_index, result["prediction"], state)
  File "E:\stable_difusion\venv\lib\site-packages\gradio\blocks.py", line 991, in postprocess_data
    raise ValueError(
ValueError: Number of output components does not match number of values returned from from function f

While I understand it cannot identify the image I have no clue on why exactly. The image I'm using: https://i.pinimg.com/originals/c1/14/75/c11475c3aa88aa2eeb5d4e5e6c19242e.jpg

Do you have any "working" custom depth maps?

I also tried to provide the depth map the script saves in output and I got the error:

Computing depthmap(s) ..
  0%|                                                                                                                                                                                                     | 0/1 [00:00<?, ?it/s] 

  0%|                                                                                                                                                                                                     | 0/1 [00:00<?, ?it/s]
Error completing request
Arguments: (0, <PIL.Image.Image image mode=RGB size=1024x743 at 0x4B5295D0>, None, '', '', 0, 6, 512, 512, False, True, False, True, 0.333, 1, False, 1, True, True, False, False, True, True, True, True, True, 2.5, 'polylines_sharp', 0, False, False, False, False, False, 'u2net', False, 0, 2, True, <tempfile._TemporaryFileWrapper object at 0x000000004B52A500>, True) {}
Traceback (most recent call last):
  File "E:\stable_difusion\modules\call_queue.py", line 56, in f
    res = list(func(*args, **kwargs))
  File "E:\stable_difusion\modules\call_queue.py", line 37, in f
    res = func(*args, **kwargs)
  File "E:\stable_difusion\extensions\stable-diffusion-webui-depthmap-script\scripts\depthmap.py", line 1069, in run_generate
    outputs, mesh_fi = run_depthmap(
  File "E:\stable_difusion\extensions\stable-diffusion-webui-depthmap-script\scripts\depthmap.py", line 464, in run_depthmap
    dimg = Image.open(custom_depthmap_img)
  File "E:\stable_difusion\venv\lib\site-packages\PIL\Image.py", line 3283, in open
    raise UnidentifiedImageError(msg)
PIL.UnidentifiedImageError: cannot identify image file <tempfile._TemporaryFileWrapper object at 0x000000004B52A500>

Traceback (most recent call last):
  File "E:\stable_difusion\venv\lib\site-packages\gradio\blocks.py", line 987, in postprocess_data
    if predictions[i] is components._Keywords.FINISHED_ITERATING:
IndexError: tuple index out of range

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "E:\stable_difusion\venv\lib\site-packages\gradio\routes.py", line 394, in run_predict
    output = await app.get_blocks().process_api(
  File "E:\stable_difusion\venv\lib\site-packages\gradio\blocks.py", line 1078, in process_api
    data = self.postprocess_data(fn_index, result["prediction"], state)
  File "E:\stable_difusion\venv\lib\site-packages\gradio\blocks.py", line 991, in postprocess_data
    raise ValueError(
ValueError: Number of output components does not match number of values returned from from function f

[thygate]

While it should work with jpg images, or 8 bit per channel RGB images, it is not the best idea to store depthmaps in this form.
It is a lossy format leading to artifacts and is always 8 bit with 2/3 of the values being redundant.
For best quality, use 16 bit single channel PNG files.
(edit: apparently jpeg does have a single channel mode (L), but it is still 8 bit)

The custom depth maps allow one to generate depth maps first, edit them, and then use the edited depthmap.
While it should also work, it is not intended to use with random online images that might not have the same size as the input image.

The error above is thrown when it tries to open the image in the custom depth map box, all it says is "PIL.UnidentifiedImageError: cannot identify image file" ... I'm not sure what's going on here.

[thygate]

The file you linked is a mode L jpeg, I have updated the code to support it too.
It opens the image just fine for me, but failed a bit further because of the format, this is fixed in the latest update.

I'm not sure what's going on for you that it can't open the file ..
What are all the options you are running with ?

[mikocevar]

In the image below you can probably see everything that is set/changed from the default.

https://i.imgur.com/PvSjzl0.png

I still get an error. I tried with a depth map that is saved in the output, here is the error:

DepthMap v0.3.11
device: cuda
Computing depthmap(s) ..
  0%|                                                                                                                                                                                                     | 0/1 [00:00<?, ?it/s]

  0%|                                                                                                                                                                                                     | 0/1 [00:00<?, ?it/s]
Error completing request
Arguments: (0, <PIL.Image.Image image mode=RGB size=1024x743 at 0x52CFF670>, None, '', '', 0, 6, 512, 512, False, True, False, True, 0.2, 1, False, 1, True, True, False, False, ['left-right', 'red-cyan-anaglyph'], 2.5, 'polylines_sharp', 0, False, False, False, False, False, 'u2net', False, 0, 2, True, <tempfile._TemporaryFileWrapper object at 0x0000000052CFF640>, True, False, True, False) {}
Traceback (most recent call last):
  File "E:\stable_difusion\modules\call_queue.py", line 56, in f
    res = list(func(*args, **kwargs))
  File "E:\stable_difusion\modules\call_queue.py", line 37, in f
    res = func(*args, **kwargs)
  File "E:\stable_difusion\extensions\stable-diffusion-webui-depthmap-script\scripts\depthmap.py", line 1110, in run_generate
    outputs, mesh_fi, meshsimple_fi = run_depthmap(
  File "E:\stable_difusion\extensions\stable-diffusion-webui-depthmap-script\scripts\depthmap.py", line 506, in run_depthmap
    dimg = Image.open(custom_depthmap_img)
  File "E:\stable_difusion\venv\lib\site-packages\PIL\Image.py", line 3283, in open
    raise UnidentifiedImageError(msg)
PIL.UnidentifiedImageError: cannot identify image file <tempfile._TemporaryFileWrapper object at 0x0000000052CFF640>

Traceback (most recent call last):
  File "E:\stable_difusion\venv\lib\site-packages\gradio\routes.py", line 394, in run_predict
    output = await app.get_blocks().process_api(
  File "E:\stable_difusion\venv\lib\site-packages\gradio\blocks.py", line 1078, in process_api
    data = self.postprocess_data(fn_index, result["prediction"], state)
  File "E:\stable_difusion\venv\lib\site-packages\gradio\blocks.py", line 1012, in postprocess_data
    prediction_value = block.postprocess(prediction_value)
  File "E:\stable_difusion\venv\lib\site-packages\gradio\components.py", line 4307, in postprocess
    "name": self.make_temp_copy_if_needed(y),
  File "E:\stable_difusion\venv\lib\site-packages\gradio\processing_utils.py", line 362, in make_temp_copy_if_needed
    temp_dir = self.hash_file(file_path)
  File "E:\stable_difusion\venv\lib\site-packages\gradio\processing_utils.py", line 334, in hash_file
    with open(file_path, "rb") as f:
OSError: [Errno 22] Invalid argument: "<div class='error'>UnidentifiedImageError: cannot identify image file &lt;tempfile._TemporaryFileWrapper object at 0x0000000052CFF640&gt;</div><div class='performance'><p class='time'>Time taken: <wbr>0.94s</p><p class='vram'>Torch active/reserved: 2079/2102 MiB, <wbr>Sys VRAM: 3687/4096 MiB (90.01%)</p></div>"

I am getting near-perfect results though, so this is epic. The right hand is perfect, but the left hand has barely any detail.., that's why I thought if I provide a detailed custom depth map it would yield better results.

The error above is from a depth map I provided and is actually the last one it saved to output when I generated the depth map without checking the custom depth map. Link below:

https://i.imgur.com/wi0f53i.png

P.s.: Thank you very much for taking your time and checking this. <3

[thygate]

In the image below you can probably see everything that is set/changed from the default.
https://i.imgur.com/PvSjzl0.png

You're not using a custom depth map, the input field is empty in that screenshot ?!

[mikocevar]

https://i.imgur.com/stXDv2Q.png

Here's the specific screenshot. Sorry, I confused you.

[thygate]

I have no idea why it can't open the file for you ..

Some googling suggests a file lock maybe, is the file open in another program ? have you tried rebooting ?

What command-line options are you running with ?

[mikocevar]

I will try. If possible, share a depth map you are sure it works! Thanks.

…

On Wed, Apr 12, 2023, 20:43 Bob Thiry ***@***.***> wrote: I have not idea why it can't open the file for you .. Some googling suggests a file lock maybe, is the file open in another program ? have you tried rebooting ? — Reply to this email directly, view it on GitHub <#146 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ADVLKK4B3A7WU2WNKQ4WBMTXA3ZUTANCNFSM6AAAAAAWHEN6CQ> . You are receiving this because you commented.Message ID: <thygate/stable-diffusion-webui-depthmap-script/repo-discussions/146/comments/5596190 @github.com>

[thygate]

It should work with any generated depthmap by the extension, saved as png or jpeg.

What command-line options are you running with ?

[mikocevar]

I'm working on Windows with https://github.com/AUTOMATIC1111/stable-diffusion-webui

I just ran the command line with Administrator privilege and I get the same error. I'm honestly confused, if this is not a code bug I really don't know what the issue is, huh?

[thygate]

What command line arguments did you configure ?

As in ..
COMMANDLINE_ARGS= --allow-code --xformers --api

[mikocevar]

I did not know I could call with specific args. I checked the code and printed them out and there are none passed by default. So my answer is No command-line arguments.