Nuscene dataset processing

[tongnie]

Hi, I found that in the prepare_data_internvl.py, the image path is stored with the idx. However, the raw nuscene data doesn't have this id.
This will lead to an error:

FileNotFoundError: [Errno 2] No such file or directory: 'F:\research\DriveLMM-o1-main\data\nuscene\e7ef871f77f44331aefdebc24ec034b7_b10f0cd792b64d16a1a5e8349b20504c.png'

Could you please provide the file processing script for the nuscene data? Thanks

[ayesha-ishaq]

Hi,

We use a single image collage of the multiviews in our case which are saved with the keys. Here's a snippet of how we create them:

   cameras = ["CAM_FRONT_LEFT", "CAM_FRONT", "CAM_FRONT_RIGHT","CAM_BACK_RIGHT", "CAM_BACK", "CAM_BACK_LEFT"]
   _, axes = plt.subplots(2, 3, figsize=(60, 24), gridspec_kw={'wspace': 0, 'hspace': 0})     
        for i, cam in enumerate(cameras):
            camera = nusc.get('sample_data', current_sample['data'][cam])
            camera_transform = nusc.get('calibrated_sensor', camera['calibrated_sensor_token'])
            img_path = os.path.join(nusc.dataroot, camera['filename'])
            # Load the image
            image = cv2.imread(img_path)
            image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)
            if i < 3:
                axes[0, i].imshow(image)
                axes[0, i].axis('off')
            else:
                axes[1, i-3].imshow(image)
                axes[1, i-3].axis('off')

        # cv2.imwrite(savepath_img / f'{scene_token}_{current_token}.png', image)
    plt.savefig(savepath_img / f'{scene_token}_{current_token}.png', bbox_inches='tight')
    plt.close()

You can create a similar function to create stitched images of the multiviews using the image paths provided in the dataset.

[tongnie]

Thank you for your prompt reply! Could you please provide the complete script to process multiveiw images? This will be very helpful to my research and reproduce the evaluation results!

Thanks again for your kind help!