Skip to content

GalaxyCore GC2145 image corruption on Nicla Vision over serial #1077

Open
@torydebra

Description

@torydebra

Hi,

We have implemented a library to trasmit the sensors data of the Nicla: https://github.com/ADVRHumanoids/nicla_vision_drivers through WiFi (UDP or TCP)

Recently I am trying to transmit such data over Serial, using the USB cable, which if I am not wrong utilizes UART connection.
By doing so, some images are received (partially) corrupted. This is noticeable since it happens frequently, but most of the time images are fine:

RGB565, 320x240

Image Image

With WiFi connection, the problem happens but very infrequently and hence tolerable.

Hardware

Hardware-wise, I am trying with different USB cables and different nicla boards and the behaviour is the same. If the problem is not already there, just shaking the nicla (to try to see if cable/port connection is bad) does not make the corruption to occur.

Software

Software-wise, I am sending data like this:

Serial.write((uint8_t*)header_buffer, headerLength);
Serial.write((uint8_t*)img_ptr, camPtr->frameSize());
Serial.write(END_BYTE);

In receiving the package (on PC, python) where I check for the START_BYTE, END_BYTE. Furthermore I check the packet size, and checksum/CRC-32 (tried both), from bytes inside the header. They are always correct, even when the corruption happen.

Digging out more, I found out that the corruption is indeed just a byte's shift. On the receiver side, when shifting the data of one byte with data_shifted = data[1:] + data[:1], the corruption is "dual":

Image

I am not super sure, but this corruption seems to happen as soon strong light/white areas are captured by the camera:
https://github.com/user-attachments/assets/4fbb36ea-12e4-4101-9246-63add3e82088

Bayer

For now, I am practically solving trasmitting CAMERA_BAYER pixel format instead of CAMERA_RGB565 (tried CAMERA_R320x240 as well CAMERA_R320x320 this time) . But I am not sure if the problem is solved or only less noticeable because of the Bayer representation. Indeed it seems that images turns from more "blue-ish" to "red-ish" and viceversa:

BAYER, 320x320

Image Image <\div>
Screencast.2025-07-09.14.23.37.mp4

I can also see some RGB spurious pixel in the contours but this may related to the Bayer demosacing.
More, arduino code put a warning for BAYER mode but I am unsure about its meaning and if I should handle the data differently:

        case CAMERA_BAYER:
            // There's no BAYER support so it will just look off.
            // Make sure odd/even row are switched to work with our bayer conversion.
            ret |= regWrite(GC2145_I2C_ADDR,
                    REG_SYNC_MODE, REG_SYNC_MODE_DEF | REG_SYNC_MODE_ROW_SWITCH);
            ret |= regWrite(GC2145_I2C_ADDR,
                    REG_OUTPUT_FMT, REG_OUTPUT_SET_FMT(reg, REG_OUTPUT_FMT_BAYER));
            break;

Bonus

As a bonus issue, here is what happens with bayer and 640 resolution: (CAMERA_R640x480)
BAYER, 640x480

Image

Colors are always wrong, no matter all the OpenCV available conversions for Bayer (with cv2.COLOR_BayerGBRG2BGR good for 320 resolution and that should be the right one according to GC2145 documentation).
I can also see the mosaic RGB squared pattern on the image.


Anyone has any suggestions?
Thanks

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions