fix: correct calc to uT based on res and gain.

[Kukuffs]

• the resolution and the gain settings of the sensor is properly accounted with fix I propose.
• the info specified in the datasheet not completely clear. And it takes a while from my side contacting the Melexis support to find out, how the gain should be accounted.

It should be done as follows:
The recalculation factor/formula for the magnetic data measurements is dependent on the two sensitivity settings: GainSel and [ResX, ResY, ResZ].
To perform the calculation from ADC Counts to uT follow the steps:

1. Resolution Correction.

    Mag_X_ADC_ResCorr = Mag_X_ADC << ResX;
    Mag_Y_ADC_ResCorr = Mag_Y_ADC << ResY;
    Mag_Z_ADC_ResCorr = Mag_Z_ADC << ResZ;
   

   << - state for the left arithmetic bit shift (1-bit left shift -> multiplication by 2).
   Note. Make sure no variable overflow is possible during the left bit shift operation.

2. ADC Counts -> uT calculation (uT is µT).
   Here, the correct setting for GainSel should be accounted.
   In the DS 16.1. Medium-Field Variant, page 17 mentioned: Sensitivity X-, Y-, Z-axis is 2.5 uT/LSB16 or 400 LSB16/mT.
   This is true for the default GainSel = 9 setting, which is: gain_gainSel_9_default = 0.125.
   However, in case the altered gain is used, then the relative change in the sensitivity should be taken into account.
   E.g. GainSel = 7 -> gain_gainSel_7 = 1.0 -> The sensitivity is increased in 8 times comparing to default GainSel:
   1.0 / 0.125 = 8.

Generally, the uTinLSB factor could be expressed as:
uTinLSB = uTinLSB_default * gain_gainSel_9_default / gain_current

    For the specific case, GainSel = 7, the formula will be:
    
    uTinLSB = 2.5 * 0.125 / 1.0;
    uTinLSB = 2.5 / 8;
    uTinLSB = 0.3125;
    Mag_X_uT = uTinLSB * Mag_X_ADC_ResCorr;
    Mag_X_uT = uTinLSB * Mag_X_ADC_ResCorr;
    Mag_Z_uT = uTinLSB * Mag_X_ADC_ResCorr;

[djw33]

Please pull these changes in! It greatly improves this library.

[caternuson]

Is this essentially a datasheet error and the only solution was obtained via email's with Melexis?

Is there a simple example sketch that can be used to demonstrate the issue?

[djw33]

Here's an example sketch where I modified the example supplied with the library:

#include "Adafruit_MLX90395.h"

Adafruit_MLX90395 sensor = Adafruit_MLX90395();

void setup(void)
{
  Serial.begin(115200);

  /* Wait for serial on USB platforms. */
  while (!Serial) {
      delay(10);
  }

  Serial.println("Starting Adafruit MLX90395 Demo");
  
  if (! sensor.begin_I2C()) {          // hardware I2C mode, can pass in address & alt Wire
    Serial.println("No sensor found ... check your wiring?");
    while (1) { delay(10); }
  }
  Serial.print("Found a MLX90395 sensor with unique id 0x");
  Serial.print(sensor.uniqueID[0], HEX);
  Serial.print(sensor.uniqueID[1], HEX);
  Serial.println(sensor.uniqueID[2], HEX);

  sensor.setOSR(MLX90395_OSR_8);
  Serial.print("OSR set to: ");
  switch (sensor.getOSR()) {
    case MLX90395_OSR_1: Serial.println("1 x"); break;
    case MLX90395_OSR_2: Serial.println("2 x"); break;
    case MLX90395_OSR_4: Serial.println("4 x"); break;
    case MLX90395_OSR_8: Serial.println("8 x"); break;
  }
  
  sensor.setResolution(MLX90395_RES_19);
  Serial.print("Resolution: ");
  switch (sensor.getResolution()) {
    case MLX90395_RES_16: Serial.println("16b"); break;
    case MLX90395_RES_17: Serial.println("17b"); break;
    case MLX90395_RES_18: Serial.println("18b"); break;
    case MLX90395_RES_19: Serial.println("19b"); break;
  }
  Serial.print("Gain: "); Serial.println(sensor.getGain());
  
}

void loop(void) {

  /* Get a new sensor event, normalized to uTesla */
  sensors_event_t event; 
  sensor.setGain(8);
  
  sensor.getEvent(&event);

  Serial.print("Gain: "); Serial.print(sensor.getGain());
  /* Display the results (magnetic field is measured in uTesla) */
  Serial.print(" X:"); 
  Serial.print(event.magnetic.x);
  Serial.print(", ");
  Serial.print("Y:"); 
  Serial.print(event.magnetic.y);
  Serial.print(", "); 
  Serial.print("Z:"); 
  Serial.print(event.magnetic.z); 
  Serial.println(" ");

  
  sensor.setGain(7);
  sensor.getEvent(&event);
  Serial.print("Gain: "); Serial.print(sensor.getGain());
  /* Display the results (magnetic field is measured in uTesla) */
  Serial.print(" X:"); 
  Serial.print(event.magnetic.x);
  Serial.print(", ");
  Serial.print("Y:"); 
  Serial.print(event.magnetic.y);
  Serial.print(", "); 
  Serial.print("Z:"); 
  Serial.print(event.magnetic.z); 
  Serial.println(" ");

  //Serial.println(" uTesla ");

  delay(100);
}

With the current library implementation the resulting output if you hold a magnet up to the sensor will jump between 1x and 100x the value. This is an example of a run I did:

Gain: 8 X:2.50, Y:-38.50, Z:119.75 
Gain: 7 X:280.00, Y:-3495.00, Z:11872.50 
Gain: 8 X:4.50, Y:-43.75, Z:118.25 
Gain: 7 X:327.50, Y:-3565.00, Z:11802.50

The current library is multiplying by the gain value, when I believe it should be dividing. So in this example the actual measurement of the magnetic field is probably in the x:20, y35, z:1190 range. As for multiplying by the gaindefault, I dont know where the original poster got that, but I can atleast scale my result to account for that. This is the output by the same script if I use the Original poster's library instead, I cant hold the magnet in the exact same place but you can atleast see the results are on the same order of magnitude:

Gain: 8 X:5650.00, Y:-1550.00, Z:5925.00 
Gain: 7 X:5680.00, Y:-1152.50, Z:6050.00 
Gain: 8 X:5800.00, Y:-1425.00, Z:6025.00 
Gain: 7 X:5625.00, Y:-1117.50, Z:5632.50 

The same thing can be done for modifying the resolution and seeing that the way it is currently implemented will shift the returned value up and down accordingly when the returned values should align until one of the signals is saturated.

Does this help?