Dataset generation fails with FileNotFoundError #30
Description
TL;DR:
Running the env_geneneration/generate_dataset.py file results in error concering missing files in a generated data directory.
Error Message:
Processing trench nr 2000
Processing trench nr 2000
Processing trench nr 2000
Traceback (most recent call last):
File "/home/mario/Projects/environment_traversability/terra/terra/env_generation/generate_dataset.py", line 96, in <module>
generate_complete_dataset(args.config)
File "/home/mario/Projects/environment_traversability/terra/terra/env_generation/generate_dataset.py", line 70, in generate_complete_dataset
create_train_foundations(config)
File "/home/mario/Projects/environment_traversability/terra/terra/env_generation/create_train_data.py", line 152, in create_foundations
filename_start = sorted(os.listdir(img_folder))[0].split("_")[0]
^^^^^^^^^^^^^^^^^^^^^^
FileNotFoundError: [Errno 2] No such file or directory: '/home/mario/Projects/environment_traversability/terra/data/openstreet/foundations/images'
Steps to reproduce
- Checkout commit
f15bc17(most recentmainat time of writing) - Follow conda section of
README.mdto set up the environment. - Run
generate_data.pyas described inenv_generation/README.md
Addional Info:
- Output of
conda info
active environment : terra
active env location : /home/mario/anaconda3/envs/terra
shell level : 2
user config file : /home/mario/.condarc
populated config files :
conda version : 23.7.2
conda-build version : 3.26.0
python version : 3.11.4.final.0
virtual packages : __archspec=1=x86_64
__cuda=12.8=0
__glibc=2.35=0
__linux=6.8.0=0
__unix=0=0
base environment : /home/mario/anaconda3 (writable)
conda av data dir : /home/mario/anaconda3/etc/conda
conda av metadata url : None
channel URLs : https://repo.anaconda.com/pkgs/main/linux-64
https://repo.anaconda.com/pkgs/main/noarch
https://repo.anaconda.com/pkgs/r/linux-64
https://repo.anaconda.com/pkgs/r/noarch
package cache : /home/mario/anaconda3/pkgs
/home/mario/.conda/pkgs
envs directories : /home/mario/anaconda3/envs
/home/mario/.conda/envs
platform : linux-64
user-agent : conda/23.7.2 requests/2.31.0 CPython/3.11.4 Linux/6.8.0-60-generic ubuntu/22.04.5 glibc/2.35
UID:GID : 1000:1000
netrc file : /home/mario/.netrc
offline mode : False
- Output of
conda list
# packages in environment at /home/mario/anaconda3/envs/terra:
#
# Name Version Build Channel
_libgcc_mutex 0.1 main
_openmp_mutex 5.1 1_gnu
absl-py 2.1.0 pypi_0 pypi
appdirs 1.4.4 pypi_0 pypi
astunparse 1.6.3 pypi_0 pypi
attrs 23.2.0 pypi_0 pypi
black 24.3.0 pypi_0 pypi
bzip2 1.0.8 h5eee18b_5
ca-certificates 2023.12.12 h06a4308_0
certifi 2024.2.2 pypi_0 pypi
cfgv 3.4.0 pypi_0 pypi
charset-normalizer 3.3.2 pypi_0 pypi
chex 0.1.85 pypi_0 pypi
click 8.1.7 pypi_0 pypi
click-plugins 1.1.1 pypi_0 pypi
cligj 0.7.2 pypi_0 pypi
cloudpickle 3.0.0 pypi_0 pypi
contourpy 1.2.0 pypi_0 pypi
cycler 0.12.1 pypi_0 pypi
decorator 5.1.1 pypi_0 pypi
distlib 0.3.8 pypi_0 pypi
dm-tree 0.1.8 pypi_0 pypi
docker-pycreds 0.4.0 pypi_0 pypi
etils 1.7.0 pypi_0 pypi
expat 2.5.0 h6a678d5_0
filelock 3.13.1 pypi_0 pypi
fiona 1.9.6 pypi_0 pypi
flake8 7.0.0 pypi_0 pypi
flatbuffers 24.3.7 pypi_0 pypi
flax 0.8.2 pypi_0 pypi
fonttools 4.50.0 pypi_0 pypi
fsspec 2024.3.0 pypi_0 pypi
gast 0.5.4 pypi_0 pypi
geopandas 0.14.3 pypi_0 pypi
gitdb 4.0.11 pypi_0 pypi
gitpython 3.1.42 pypi_0 pypi
google-pasta 0.2.0 pypi_0 pypi
grpcio 1.62.1 pypi_0 pypi
gviz-api 1.10.0 pypi_0 pypi
h5py 3.10.0 pypi_0 pypi
identify 2.5.35 pypi_0 pypi
idna 3.6 pypi_0 pypi
imageio 2.34.0 pypi_0 pypi
importlib-resources 6.3.1 pypi_0 pypi
jax 0.6.1 pypi_0 pypi
jax-cuda12-pjrt 0.6.1 pypi_0 pypi
jax-cuda12-plugin 0.6.1 pypi_0 pypi
jaxlib 0.6.1 pypi_0 pypi
keras 3.0.5 pypi_0 pypi
kiwisolver 1.4.5 pypi_0 pypi
lazy-loader 0.3 pypi_0 pypi
ld_impl_linux-64 2.38 h1181459_1
libclang 18.1.1 pypi_0 pypi
libffi 3.4.4 h6a678d5_0
libgcc-ng 11.2.0 h1234567_1
libgomp 11.2.0 h1234567_1
libstdcxx-ng 11.2.0 h1234567_1
libuuid 1.41.5 h5eee18b_0
markdown 3.6 pypi_0 pypi
markdown-it-py 3.0.0 pypi_0 pypi
markupsafe 2.1.5 pypi_0 pypi
matplotlib 3.8.3 pypi_0 pypi
mccabe 0.7.0 pypi_0 pypi
mdurl 0.1.2 pypi_0 pypi
ml-dtypes 0.5.1 pypi_0 pypi
msgpack 1.0.8 pypi_0 pypi
mypy-extensions 1.0.0 pypi_0 pypi
namex 0.0.7 pypi_0 pypi
ncurses 6.4 h6a678d5_0
nest-asyncio 1.6.0 pypi_0 pypi
networkx 3.2.1 pypi_0 pypi
nodeenv 1.8.0 pypi_0 pypi
numpy 1.26.4 pypi_0 pypi
nvidia-cublas-cu12 12.4.2.65 pypi_0 pypi
nvidia-cuda-cupti-cu12 12.4.99 pypi_0 pypi
nvidia-cuda-nvcc-cu12 12.9.86 pypi_0 pypi
nvidia-cuda-nvrtc-cu12 12.4.99 pypi_0 pypi
nvidia-cuda-runtime-cu12 12.4.99 pypi_0 pypi
nvidia-cudnn-cu12 9.10.2.21 pypi_0 pypi
nvidia-cufft-cu12 11.2.0.44 pypi_0 pypi
nvidia-cusolver-cu12 11.6.0.99 pypi_0 pypi
nvidia-cusparse-cu12 12.3.0.142 pypi_0 pypi
nvidia-nccl-cu12 2.20.5 pypi_0 pypi
nvidia-nvjitlink-cu12 12.4.99 pypi_0 pypi
nvidia-nvshmem-cu12 3.2.5 pypi_0 pypi
opencv-python 4.9.0.80 pypi_0 pypi
openssl 3.0.13 h7f8727e_0
opt-einsum 3.3.0 pypi_0 pypi
optax 0.2.1 pypi_0 pypi
orbax-checkpoint 0.5.6 pypi_0 pypi
osmnx 1.9.1 pypi_0 pypi
packaging 24.0 pypi_0 pypi
pandas 2.2.1 pypi_0 pypi
pathlib 1.0.1 pypi_0 pypi
pathspec 0.12.1 pypi_0 pypi
pillow 10.2.0 pypi_0 pypi
pip 24.0 pypi_0 pypi
platformdirs 4.2.0 pypi_0 pypi
pre-commit 3.6.2 pypi_0 pypi
protobuf 3.20.3 pypi_0 pypi
psutil 5.9.8 pypi_0 pypi
pycodestyle 2.11.1 pypi_0 pypi
pyflakes 3.2.0 pypi_0 pypi
pygame 2.5.2 pypi_0 pypi
pygments 2.17.2 pypi_0 pypi
pyparsing 3.1.2 pypi_0 pypi
pyproj 3.6.1 pypi_0 pypi
python 3.12.2 h996f2a0_0
python-dateutil 2.9.0.post0 pypi_0 pypi
pytz 2024.1 pypi_0 pypi
pyyaml 6.0.1 pypi_0 pypi
readline 8.2 h5eee18b_0
requests 2.31.0 pypi_0 pypi
rich 13.7.1 pypi_0 pypi
scikit-image 0.22.0 pypi_0 pypi
scipy 1.12.0 pypi_0 pypi
sentry-sdk 1.42.0 pypi_0 pypi
setproctitle 1.3.3 pypi_0 pypi
setuptools 68.2.2 py312h06a4308_0
shapely 2.0.3 pypi_0 pypi
six 1.16.0 pypi_0 pypi
smmap 5.0.1 pypi_0 pypi
sqlite 3.41.2 h5eee18b_0
tdqm 0.0.1 pypi_0 pypi
tensorboard 2.16.2 pypi_0 pypi
tensorboard-data-server 0.7.2 pypi_0 pypi
tensorboard-plugin-profile 2.15.1 pypi_0 pypi
tensorflow 2.16.1 pypi_0 pypi
tensorflow-probability 0.24.0 pypi_0 pypi
tensorstore 0.1.56 pypi_0 pypi
termcolor 2.4.0 pypi_0 pypi
terra 0.0.1 pypi_0 pypi
tifffile 2024.2.12 pypi_0 pypi
tk 8.6.12 h1ccaba5_0
toolz 0.12.1 pypi_0 pypi
tqdm 4.66.2 pypi_0 pypi
typing-extensions 4.10.0 pypi_0 pypi
tzdata 2024.1 pypi_0 pypi
urllib3 2.2.1 pypi_0 pypi
virtualenv 20.25.1 pypi_0 pypi
wandb 0.16.4 pypi_0 pypi
werkzeug 3.0.1 pypi_0 pypi
wheel 0.41.2 py312h06a4308_0
wrapt 1.16.0 pypi_0 pypi
xz 5.4.6 h5eee18b_0
zipp 3.18.1 pypi_0 pypi
zlib 1.2.13 h5eee18b_0
- Output of
python --version
Python 3.12.2