grch38_to_grch37_ref_chain path not found

[li1ba]

Hi!
I'm trying to install and run seqr locally, but it fails with an error:

Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/luigi/worker.py", line 199, in run
    new_deps = self._run_get_new_deps()
  File "/usr/local/lib/python3.7/site-packages/luigi/worker.py", line 141, in _run_get_new_deps
    task_gen = self.task.run()
  File "/seqr-loading-pipelines/luigi_pipeline/seqr_loading.py", line 89, in run
    if self.grch38_to_grch37_ref_chain: check_if_path_exists(self.grch38_to_grch37_ref_chain, "grch38_to_grch37_ref_chain")
  File "/seqr-loading-pipelines/luigi_pipeline/seqr_loading.py", line 23, in check_if_path_exists
    raise ValueError(f"{label} path not found: {path}")
ValueError: grch38_to_grch37_ref_chain path not found: gs://hail-common/references/grch38_to_grch37.over.chain.gz

Command:
docker-compose exec pipeline-runner load_data.sh $BUILD_VERSION $SAMPLE_TYPE $INDEX_NAME $INPUT_FILE_PATH

Full output:

+ BUILD_VERSION=38
+ SAMPLE_TYPE=WES
+ INDEX_NAME=test
+ INPUT_FILE_PATH=test/NA128782.F_rmdup.DV_AC.vcf.gz
+ case ${BUILD_VERSION} in
+ FULL_BUILD_VERSION=GRCh38
+ SOURCE_FILE=/input_vcfs/test/NA128782.F_rmdup.DV_AC.vcf.gz
+ DEST_FILE=/input_vcfs/test/NA128782.mt
+ python3 -m seqr_loading SeqrMTToESTask --local-scheduler --reference-ht-path /seqr-reference-data/GRCh38/combined_reference_data_grch38.ht --clinvar-ht-path /seqr-reference-data/GRCh38/clinvar.GRCh38.ht --vep-config-json-path /vep_configs/vep-GRCh38-loftee.json --es-host elasticsearch --es-index-min-num-shards 1 --sample-type WES --es-index test --genome-version 38 --source-paths /input_vcfs/test/NA128782.F_rmdup.DV_AC.vcf.gz --dest-path /input_vcfs/test/NA128782.mt
/usr/local/lib/python3.7/site-packages/luigi/parameter.py:347: UserWarning: OptionalParameter "source_path" with value "<luigi.parameter.Parameter object at 0x7f5e15e3a8d0>" is not of type string or None.
  param_name, param_value))
/usr/local/lib/python3.7/site-packages/luigi/parameter.py:283: UserWarning: Parameter "es_password" with value "None" is not of type string.
  warnings.warn('Parameter "{}" with value "{}" is not of type string.'.format(param_name, param_value))
/usr/local/lib/python3.7/site-packages/luigi/parameter.py:283: UserWarning: Parameter "interval_ref_ht_path" with value "None" is not of type string.
  warnings.warn('Parameter "{}" with value "{}" is not of type string.'.format(param_name, param_value))
/usr/local/lib/python3.7/site-packages/luigi/parameter.py:283: UserWarning: Parameter "hgmd_ht_path" with value "None" is not of type string.
  warnings.warn('Parameter "{}" with value "{}" is not of type string.'.format(param_name, param_value))
/usr/local/lib/python3.7/site-packages/elasticsearch/connection/base.py:190: ElasticsearchDeprecationWarning: Elasticsearch built-in security features are not enabled. Without authentication, your cluster could be accessible to anyone. See https://www.elastic.co/guide/en/elasticsearch/reference/7.16/security-minimal-setup.html to enable security.
  warnings.warn(message, category=ElasticsearchDeprecationWarning)
DEBUG: Checking if SeqrMTToESTask(source_path=<luigi.parameter.Parameter object at 0x7f5e15e3a8d0>, use_temp_loading_nodes=True, es_host=elasticsearch, es_port=9200, es_index=test, es_username=pipeline, es_password=None, es_index_min_num_shards=1, source_paths=/input_vcfs/test/NA128782.F_rmdup.DV_AC.vcf.gz, dest_path=/input_vcfs/test/NA128782.mt, genome_version=38, vep_runner=VEP, reference_ht_path=/seqr-reference-data/GRCh38/combined_reference_data_grch38.ht, interval_ref_ht_path=None, clinvar_ht_path=/seqr-reference-data/GRCh38/clinvar.GRCh38.ht, hgmd_ht_path=None, sample_type=WES, dont_validate=False, dataset_type=VARIANTS, remap_path=, subset_path=, vep_config_json_path=/vep_configs/vep-GRCh38-loftee.json, grch38_to_grch37_ref_chain=gs://hail-common/references/grch38_to_grch37.over.chain.gz) is complete
DEBUG: Checking if SeqrVCFToMTTask(source_paths=/input_vcfs/test/NA128782.F_rmdup.DV_AC.vcf.gz, dest_path=/input_vcfs/test/NA128782.mt, genome_version=38, vep_runner=VEP, reference_ht_path=/seqr-reference-data/GRCh38/combined_reference_data_grch38.ht, interval_ref_ht_path=, clinvar_ht_path=/seqr-reference-data/GRCh38/clinvar.GRCh38.ht, hgmd_ht_path=None, sample_type=WES, dont_validate=False, dataset_type=VARIANTS, remap_path=, subset_path=, vep_config_json_path=/vep_configs/vep-GRCh38-loftee.json, grch38_to_grch37_ref_chain=gs://hail-common/references/grch38_to_grch37.over.chain.gz, hail_temp_dir=) is complete
INFO: Informed scheduler that task   SeqrMTToESTask__seqr_reference__VARIANTS__input_vcfs_test_db43ea0f8d   has status   PENDING
DEBUG: Checking if VcfFile(filename=/input_vcfs/test/NA128782.F_rmdup.DV_AC.vcf.gz) is complete
INFO: Informed scheduler that task   SeqrVCFToMTTask__seqr_reference__VARIANTS__input_vcfs_test_34bb3e7b6a   has status   PENDING
INFO: Informed scheduler that task   VcfFile__input_vcfs_test_c974e08b6c   has status   DONE
INFO: Done scheduling tasks
INFO: Running Worker with 1 processes
DEBUG: Asking scheduler for work...
DEBUG: Pending tasks: 2
INFO: [pid 15] Worker Worker(salt=554367766, workers=1, host=b4769f946b28, username=root, pid=15) running   SeqrVCFToMTTask(source_paths=/input_vcfs/test/NA128782.F_rmdup.DV_AC.vcf.gz, dest_path=/input_vcfs/test/NA128782.mt, genome_version=38, vep_runner=VEP, reference_ht_path=/seqr-reference-data/GRCh38/combined_reference_data_grch38.ht, interval_ref_ht_path=, clinvar_ht_path=/seqr-reference-data/GRCh38/clinvar.GRCh38.ht, hgmd_ht_path=None, sample_type=WES, dont_validate=False, dataset_type=VARIANTS, remap_path=, subset_path=, vep_config_json_path=/vep_configs/vep-GRCh38-loftee.json, grch38_to_grch37_ref_chain=gs://hail-common/references/grch38_to_grch37.over.chain.gz, hail_temp_dir=)
Initializing Hail with default parameters...
2022-12-21 09:16:56.873 WARN  NativeCodeLoader:60 - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Setting default log level to "WARN".
To adjust logging level use sc.setLogLevel(newLevel). For SparkR, use setLogLevel(newLevel).
Running on Apache Spark version 3.1.3
SparkUI available at http://b4769f946b28:4040
Welcome to
     __  __     <>__
    / /_/ /__  __/ /
   / __  / _ `/ / /
  /_/ /_/\_,_/_/_/   version 0.2.105-acd89e80c345
LOGGING: writing to /hail-20221221-0916-0.2.105-acd89e80c345.log
ERROR: [pid 15] Worker Worker(salt=554367766, workers=1, host=b4769f946b28, username=root, pid=15) failed    SeqrVCFToMTTask(source_paths=/input_vcfs/test/NA128782.F_rmdup.DV_AC.vcf.gz, dest_path=/input_vcfs/test/NA128782.mt, genome_version=38, vep_runner=VEP, reference_ht_path=/seqr-reference-data/GRCh38/combined_reference_data_grch38.ht, interval_ref_ht_path=, clinvar_ht_path=/seqr-reference-data/GRCh38/clinvar.GRCh38.ht, hgmd_ht_path=None, sample_type=WES, dont_validate=False, dataset_type=VARIANTS, remap_path=, subset_path=, vep_config_json_path=/vep_configs/vep-GRCh38-loftee.json, grch38_to_grch37_ref_chain=gs://hail-common/references/grch38_to_grch37.over.chain.gz, hail_temp_dir=)
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/luigi/worker.py", line 199, in run
    new_deps = self._run_get_new_deps()
  File "/usr/local/lib/python3.7/site-packages/luigi/worker.py", line 141, in _run_get_new_deps
    task_gen = self.task.run()
  File "/seqr-loading-pipelines/luigi_pipeline/seqr_loading.py", line 89, in run
    if self.grch38_to_grch37_ref_chain: check_if_path_exists(self.grch38_to_grch37_ref_chain, "grch38_to_grch37_ref_chain")
  File "/seqr-loading-pipelines/luigi_pipeline/seqr_loading.py", line 23, in check_if_path_exists
    raise ValueError(f"{label} path not found: {path}")
ValueError: grch38_to_grch37_ref_chain path not found: gs://hail-common/references/grch38_to_grch37.over.chain.gz
DEBUG: 1 running tasks, waiting for next task to finish
INFO: Informed scheduler that task   SeqrVCFToMTTask__seqr_reference__VARIANTS__input_vcfs_test_34bb3e7b6a   has status   FAILED
DEBUG: Asking scheduler for work...
DEBUG: Done
DEBUG: There are no more tasks to run at this time
DEBUG: There are 2 pending tasks possibly being run by other workers
DEBUG: There are 2 pending tasks unique to this worker
DEBUG: There are 2 pending tasks last scheduled by this worker
INFO: Worker Worker(salt=554367766, workers=1, host=b4769f946b28, username=root, pid=15) was stopped. Shutting down Keep-Alive thread
INFO:
===== Luigi Execution Summary =====

Scheduled 3 tasks of which:
* 1 complete ones were encountered:
    - 1 VcfFile(filename=/input_vcfs/test/NA128782.F_rmdup.DV_AC.vcf.gz)
* 1 failed:
    - 1 SeqrVCFToMTTask(...)
* 1 were left pending, among these:
    * 1 had failed dependencies:
        - 1 SeqrMTToESTask(...)

This progress looks :( because there were failed tasks

===== Luigi Execution Summary =====

Here are the docker images I use:

REPOSITORY                                      TAG           IMAGE ID       CREATED         SIZE
gcr.io/seqr-project/seqr                        gcloud-prod   2e49e22c912a   5 days ago      2.31GB
gcr.io/seqr-project/pipeline-runner             gcloud-prod   6dfe8699a825   2 weeks ago     3.43GB
redis                                           6.2.7         4b1123a829a1   2 weeks ago     113MB
docker.elastic.co/elasticsearch/elasticsearch   7.16.3        3a5e93284781   11 months ago   611MB
docker.elastic.co/kibana/kibana                 7.16.3        df64a680d590   11 months ago   1.3GB
postgres                                        12.6          145bb83bd157   19 months ago   314MB

I'm able to download the grch38_to_grch37.over.chain.gz file with gsutil, so it shouldn't be a problem with gcloud installation:
gsutil cp gs://hail-common/references/grch38_to_grch37.over.chain.gz .

[hanars]

Can you try running docker-compose exec pipeline-runner gcloud auth application-default login to confirm your pipeline runner container is authenticated to google cloud?

[li1ba]

I was trying to follow Option 2 of the local install: https://github.com/broadinstitute/seqr/blob/master/deploy/LOCAL_INSTALL.md#option-2-annotate-and-load-on-prem
Is the authentication to google cloud necessary?

[hanars]

Thats what I'm trying to figure out

[li1ba]

When I run the command you recommended, it asks to follow a link and log in with a google account.

[hanars]

And if you do that does that fix the problem?

[li1ba]

Hi! Yes, the authorization with a google account fixed the issue. Thank you very much!

[GeorginaAmbriz]

Hello,

I had the same problem when I tried to upload a WGS trio dataset with GRCH38 reference genome.
I tried to login to gcloud with:
docker-compose exec pipeline-runner gcloud auth application-default login
but I had this message:

Access blocked: Google Auth Library’s request is invalid

[email protected]
You can’t sign in because Google Auth Library sent an invalid request. You can try again later, or contact the developer about this issue. Learn more about this error
If you are a developer of Google Auth Library, see error details.
Error 400: invalid_request

Then, I stoped the pipeline-runner container, and I logged in to gcloud with:
./google-cloud-sdk/bin/gcloud init
I restarted pipeline-runner with;
docker-compose up -d pipeline-runner
But it still doesn't work. I still have the message: Access blocked: Google Auth Library request is invalid when I run docker-compose exec pipeline-runner gcloud auth application-default login.

I tried to upload the dataset again with:
docker-compose exec pipeline-runner load_data.sh $BUILD_VERSION $SAMPLE_TYPE $INDEX_NAME $INPUT_FILE_PATH
But this time It didn't open hail. I had this output:

+ BUILD_VERSION=38
+ SAMPLE_TYPE=WGS
+ INDEX_NAME=wgs_trio3
+ INPUT_FILE_PATH=GRCh38/trio1_wgs_hg38_final.recalibrated_Clean2.vcf.gz
+ case ${BUILD_VERSION} in
+ FULL_BUILD_VERSION=GRCh38
+ SOURCE_FILE=/input_vcfs/GRCh38/trio1_wgs_hg38_final.recalibrated_Clean2.vcf.gz
+ DEST_FILE=/input_vcfs/GRCh38/trio1_wgs_hg38_final.mt
+ python3 -m seqr_loading SeqrMTToESTask --local-scheduler --reference-ht-path /seqr-reference-data/GRCh38/combined_reference_data_grch38.ht --clinvar-ht-path /seqr-reference-data/GRCh38/clinvar.GRCh38.ht --vep-config-json-path /vep_configs/vep-GRCh38-loftee.json --es-host elasticsearch --es-index-min-num-shards 1 --sample-type WGS --es-index wgs_trio3 --genome-version 38 --source-paths /input_vcfs/GRCh38/trio1_wgs_hg38_final.recalibrated_Clean2.vcf.gz --dest-path /input_vcfs/GRCh38/trio1_wgs_hg38_final.mt
/usr/local/lib/python3.7/site-packages/luigi/parameter.py:347: UserWarning: OptionalParameter "source_path" with value "<luigi.parameter.Parameter object at 0x7f54980ae940>" is not of type string or None.
  param_name, param_value))
/usr/local/lib/python3.7/site-packages/luigi/parameter.py:283: UserWarning: Parameter "es_password" with value "None" is not of type string.
  warnings.warn('Parameter "{}" with value "{}" is not of type string.'.format(param_name, param_value))
/usr/local/lib/python3.7/site-packages/luigi/parameter.py:283: UserWarning: Parameter "interval_ref_ht_path" with value "None" is not of type string.
  warnings.warn('Parameter "{}" with value "{}" is not of type string.'.format(param_name, param_value))
/usr/local/lib/python3.7/site-packages/luigi/parameter.py:283: UserWarning: Parameter "hgmd_ht_path" with value "None" is not of type string.
  warnings.warn('Parameter "{}" with value "{}" is not of type string.'.format(param_name, param_value))
/usr/local/lib/python3.7/site-packages/elasticsearch/connection/base.py:190: ElasticsearchDeprecationWarning: Elasticsearch built-in security features are not enabled. Without authentication, your cluster could be accessible to anyone. See https://www.elastic.co/guide/en/elasticsearch/reference/7.16/security-minimal-setup.html to enable security.
  warnings.warn(message, category=ElasticsearchDeprecationWarning)
DEBUG: Checking if SeqrMTToESTask(source_path=<luigi.parameter.Parameter object at 0x7f54980ae940>, use_temp_loading_nodes=True, es_host=elasticsearch, es_port=9200, es_index=wgs_trio3, es_username=pipeline, es_password=None, es_index_min_num_shards=1, source_paths=/input_vcfs/GRCh38/trio1_wgs_hg38_final.recalibrated_Clean2.vcf.gz, dest_path=/input_vcfs/GRCh38/trio1_wgs_hg38_final.mt, genome_version=38, vep_runner=VEP, reference_ht_path=/seqr-reference-data/GRCh38/combined_reference_data_grch38.ht, interval_ref_ht_path=None, clinvar_ht_path=/seqr-reference-data/GRCh38/clinvar.GRCh38.ht, hgmd_ht_path=None, sample_type=WGS, dont_validate=False, dataset_type=VARIANTS, remap_path=, subset_path=, vep_config_json_path=/vep_configs/vep-GRCh38-loftee.json, grch38_to_grch37_ref_chain=gs://hail-common/references/grch38_to_grch37.over.chain.gz) is complete
DEBUG: Checking if SeqrVCFToMTTask(source_paths=/input_vcfs/GRCh38/trio1_wgs_hg38_final.recalibrated_Clean2.vcf.gz, dest_path=/input_vcfs/GRCh38/trio1_wgs_hg38_final.mt, genome_version=38, vep_runner=VEP, reference_ht_path=/seqr-reference-data/GRCh38/combined_reference_data_grch38.ht, interval_ref_ht_path=, clinvar_ht_path=/seqr-reference-data/GRCh38/clinvar.GRCh38.ht, hgmd_ht_path=None, sample_type=WGS, dont_validate=False, dataset_type=VARIANTS, remap_path=, subset_path=, vep_config_json_path=/vep_configs/vep-GRCh38-loftee.json, grch38_to_grch37_ref_chain=gs://hail-common/references/grch38_to_grch37.over.chain.gz, hail_temp_dir=) is complete
INFO: Informed scheduler that task   SeqrMTToESTask__seqr_reference__VARIANTS__input_vcfs_GRCh_fe735991be   has status   PENDING
DEBUG: Checking if VcfFile(filename=/input_vcfs/GRCh38/trio1_wgs_hg38_final.recalibrated_Clean2.vcf.gz) is complete
INFO: Informed scheduler that task   SeqrVCFToMTTask__seqr_reference__VARIANTS__input_vcfs_GRCh_54a7f63522   has status   PENDING
INFO: Informed scheduler that task   VcfFile__input_vcfs_GRCh_e834fce497   has status   PENDING
INFO: Done scheduling tasks
INFO: Running Worker with 1 processes
DEBUG: Asking scheduler for work...
DEBUG: Pending tasks: 3
INFO: [pid 94] Worker Worker(salt=838232608, workers=1, host=3a11103263e8, username=root, pid=94) running   VcfFile(filename=/input_vcfs/GRCh38/trio1_wgs_hg38_final.recalibrated_Clean2.vcf.gz)
INFO: [pid 94] Worker Worker(salt=838232608, workers=1, host=3a11103263e8, username=root, pid=94) done      VcfFile(filename=/input_vcfs/GRCh38/trio1_wgs_hg38_final.recalibrated_Clean2.vcf.gz)
DEBUG: 1 running tasks, waiting for next task to finish
INFO: Informed scheduler that task   VcfFile__input_vcfs_GRCh_e834fce497   has status   DONE
DEBUG: Asking scheduler for work...
DEBUG: Pending tasks: 2
INFO: [pid 94] Worker Worker(salt=838232608, workers=1, host=3a11103263e8, username=root, pid=94) running   SeqrVCFToMTTask(source_paths=/input_vcfs/GRCh38/trio1_wgs_hg38_final.recalibrated_Clean2.vcf.gz, dest_path=/input_vcfs/GRCh38/trio1_wgs_hg38_final.mt, genome_version=38, vep_runner=VEP, reference_ht_path=/seqr-reference-data/GRCh38/combined_reference_data_grch38.ht, interval_ref_ht_path=, clinvar_ht_path=/seqr-reference-data/GRCh38/clinvar.GRCh38.ht, hgmd_ht_path=None, sample_type=WGS, dont_validate=False, dataset_type=VARIANTS, remap_path=, subset_path=, vep_config_json_path=/vep_configs/vep-GRCh38-loftee.json, grch38_to_grch37_ref_chain=gs://hail-common/references/grch38_to_grch37.over.chain.gz, hail_temp_dir=)
ERROR: [pid 94] Worker Worker(salt=838232608, workers=1, host=3a11103263e8, username=root, pid=94) failed    SeqrVCFToMTTask(source_paths=/input_vcfs/GRCh38/trio1_wgs_hg38_final.recalibrated_Clean2.vcf.gz, dest_path=/input_vcfs/GRCh38/trio1_wgs_hg38_final.mt, genome_version=38, vep_runner=VEP, reference_ht_path=/seqr-reference-data/GRCh38/combined_reference_data_grch38.ht, interval_ref_ht_path=, clinvar_ht_path=/seqr-reference-data/GRCh38/clinvar.GRCh38.ht, hgmd_ht_path=None, sample_type=WGS, dont_validate=False, dataset_type=VARIANTS, remap_path=, subset_path=, vep_config_json_path=/vep_configs/vep-GRCh38-loftee.json, grch38_to_grch37_ref_chain=gs://hail-common/references/grch38_to_grch37.over.chain.gz, hail_temp_dir=)
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/luigi/worker.py", line 184, in run
    raise RuntimeError('Unfulfilled %s at run time: %s' % (deps, ', '.join(missing)))
RuntimeError: Unfulfilled dependency at run time: VcfFile__input_vcfs_GRCh_e834fce497
DEBUG: 1 running tasks, waiting for next task to finish
INFO: Informed scheduler that task   SeqrVCFToMTTask__seqr_reference__VARIANTS__input_vcfs_GRCh_54a7f63522   has status   FAILED
DEBUG: Checking if SeqrVCFToMTTask(source_paths=/input_vcfs/GRCh38/trio1_wgs_hg38_final.recalibrated_Clean2.vcf.gz, dest_path=/input_vcfs/GRCh38/trio1_wgs_hg38_final.mt, genome_version=38, vep_runner=VEP, reference_ht_path=/seqr-reference-data/GRCh38/combined_reference_data_grch38.ht, interval_ref_ht_path=, clinvar_ht_path=/seqr-reference-data/GRCh38/clinvar.GRCh38.ht, hgmd_ht_path=None, sample_type=WGS, dont_validate=False, dataset_type=VARIANTS, remap_path=, subset_path=, vep_config_json_path=/vep_configs/vep-GRCh38-loftee.json, grch38_to_grch37_ref_chain=gs://hail-common/references/grch38_to_grch37.over.chain.gz, hail_temp_dir=) is complete
DEBUG: Checking if VcfFile(filename=/input_vcfs/GRCh38/trio1_wgs_hg38_final.recalibrated_Clean2.vcf.gz) is complete
INFO: Informed scheduler that task   SeqrVCFToMTTask__seqr_reference__VARIANTS__input_vcfs_GRCh_54a7f63522   has status   PENDING
INFO: Informed scheduler that task   VcfFile__input_vcfs_GRCh_e834fce497   has status   PENDING
DEBUG: Asking scheduler for work...
DEBUG: Pending tasks: 2
INFO: [pid 94] Worker Worker(salt=838232608, workers=1, host=3a11103263e8, username=root, pid=94) running   SeqrVCFToMTTask(source_paths=/input_vcfs/GRCh38/trio1_wgs_hg38_final.recalibrated_Clean2.vcf.gz, dest_path=/input_vcfs/GRCh38/trio1_wgs_hg38_final.mt, genome_version=38, vep_runner=VEP, reference_ht_path=/seqr-reference-data/GRCh38/combined_reference_data_grch38.ht, interval_ref_ht_path=, clinvar_ht_path=/seqr-reference-data/GRCh38/clinvar.GRCh38.ht, hgmd_ht_path=None, sample_type=WGS, dont_validate=False, dataset_type=VARIANTS, remap_path=, subset_path=, vep_config_json_path=/vep_configs/vep-GRCh38-loftee.json, grch38_to_grch37_ref_chain=gs://hail-common/references/grch38_to_grch37.over.chain.gz, hail_temp_dir=)
ERROR: [pid 94] Worker Worker(salt=838232608, workers=1, host=3a11103263e8, username=root, pid=94) failed    SeqrVCFToMTTask(source_paths=/input_vcfs/GRCh38/trio1_wgs_hg38_final.recalibrated_Clean2.vcf.gz, dest_path=/input_vcfs/GRCh38/trio1_wgs_hg38_final.mt, genome_version=38, vep_runner=VEP, reference_ht_path=/seqr-reference-data/GRCh38/combined_reference_data_grch38.ht, interval_ref_ht_path=, clinvar_ht_path=/seqr-reference-data/GRCh38/clinvar.GRCh38.ht, hgmd_ht_path=None, sample_type=WGS, dont_validate=False, dataset_type=VARIANTS, remap_path=, subset_path=, vep_config_json_path=/vep_configs/vep-GRCh38-loftee.json, grch38_to_grch37_ref_chain=gs://hail-common/references/grch38_to_grch37.over.chain.gz, hail_temp_dir=)
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/luigi/worker.py", line 184, in run
    raise RuntimeError('Unfulfilled %s at run time: %s' % (deps, ', '.join(missing)))
RuntimeError: Unfulfilled dependency at run time: VcfFile__input_vcfs_GRCh_e834fce497
DEBUG: 1 running tasks, waiting for next task to finish
INFO: Informed scheduler that task   SeqrVCFToMTTask__seqr_reference__VARIANTS__input_vcfs_GRCh_54a7f63522   has status   FAILED
DEBUG: Asking scheduler for work...
DEBUG: Done
DEBUG: There are no more tasks to run at this time
DEBUG: There are 2 pending tasks possibly being run by other workers
DEBUG: There are 2 pending tasks unique to this worker
DEBUG: There are 2 pending tasks last scheduled by this worker
INFO: Worker Worker(salt=838232608, workers=1, host=3a11103263e8, username=root, pid=94) was stopped. Shutting down Keep-Alive thread
INFO: 
===== Luigi Execution Summary =====

Scheduled 3 tasks of which:
* 1 ran successfully:
    - 1 VcfFile(filename=/input_vcfs/GRCh38/trio1_wgs_hg38_final.recalibrated_Clean2.vcf.gz)
* 1 failed:
    - 1 SeqrVCFToMTTask(...)
* 1 were left pending, among these:
    * 1 had failed dependencies:
        - 1 SeqrMTToESTask(...)

This progress looks :( because there were failed tasks

===== Luigi Execution Summary =====

When I ran this, I had already deleted the hail matrix that was created before.

I'll really appreciate any suggestion to fix this.
Thank you
Georgina

[GeorginaAmbriz]

I've just reviewed my command, in fact the name of the input file was wrong. That's why Hail didn't initialized. However, I still have the error when trying to authenticated the pipeline runner container to google cloud.

[hanars]

you appear to be having a different issue - the issue reported in this ticket was ValueError: grch38_to_grch37_ref_chain path not found:, which I do not see anywhere in your output. The output you provided is consistent with a typo in the input name so that does make sense. In order to debug any remaining issues you have I need the full output of the command that you are running with the correct input

[WMAPernice]

@GeorginaAmbriz what helped me was to run gcloud auth application-default login (not using docker-compose) to set up a default login config. That avoided the authentication error ("Google Auth Library’s request is invalid" which may have something to do with a recent policy change by Google) you get when running docker-compose exec pipeline-runner gcloud auth application-default login. This seems to fix the authentication error that yields the ValueError: grch38_to_grch37_ref_chain path not message you receive.