etl_simple_femr expects code field, but OMOP tables don’t include it directly

[nico9902]

I'm working on multimodal modeling with the INSPECT dataset and wanted to use etl_simple_femr to generate the required extract from OMOP data tables by using the etl_simple_femr pipeline on OMOP-compatible tables (e.g., condition_occurrence.csv, drug_exposure.csv, etc.), but I’m running into this error: KeyError: 'code'.
The traceback points to the function get_concept_ids_from_file() in simple.py: resulting_concepts.add(row["code"]).

The problem is that OMOP tables such as condition_occurrence do not contain a code column. Instead, they use *_concept_id fields (e.g., condition_concept_id) that refer to numeric concept IDs, which need to be mapped to FEMR-style codes. However, this mapping step does not seem to be included in etl_simple_femr.

Can you clarify:
• Whether etl_simple_femr is meant to be used after an external preprocessing step that adds the code column?
• If so, could you provide an example or script that performs this transformation from OMOP concept IDs to FEMR-style code strings?
• Alternatively: would you consider sharing the 'extract' dataset used to train MOTOR, or a preprocessed version of OMOP → FEMR, ready to be used with etl_simple_femr?

[ndonyapour]

The EHR data are in OMOP format and you need to use the FEMR OMOP ETL.

import os
import argparse
import subprocess
from loguru import logger
import femr.datasets

if __name__ == "__main__":

    INPUT_DIR = ""
    TARGET_DIR = ""

    # The number of threads depends on the number of cores available on the machine.
    # We use 3 threads to avoid overloading the machine.
    num_threads = 3
    logger.info(f"Number of threads: {num_threads}")
    
    os.makedirs(TARGET_DIR, exist_ok=True)

    TEMP_DIR = os.path.join(TARGET_DIR, "temp")
    EXTRACT_DIR = os.path.join(TARGET_DIR, "extract")

    etl_command = [
        "etl_generic_omop",
        INPUT_DIR,
        EXTRACT_DIR,
        TEMP_DIR,
        "--num_threads",
        str(num_threads),
    ]
    subprocess.run(etl_command, check=True)

    logger.info(f"Femr database saved in path: {TARGET_DIR}")
    logger.info("Testing the database")

    database = femr.datasets.PatientDatabase(EXTRACT_DIR)
    try:
        logger.info(f"Number of patients in the database: {len(database)}")
        omop_id = next(iter(database))
        patient = database[omop_id]
        events = patient.events
        logger.info(f"Number of events in patients with omop_id {omop_id}: {events}")
        logger.info(f"First event of the patient: {events[0]}")
    finally:
        database.close()

I used FEMR V0.1.16 and it worked.