Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

magnetometer x y z calculations can overflow at less than half the sensor range #304

Open
martinwork opened this issue Apr 7, 2023 · 0 comments
Assignees
Labels
bug Something isn't working dal-change-required p3
Milestone

Comments

@martinwork
Copy link
Collaborator

The CALIBRATED_SAMPLE calculation can overflow 32bit signed integer arithmetic.

#define CALIBRATED_SAMPLE(sample, axis) (((sample.axis - calibration.centre.axis) * calibration.scale.axis) >> 10)
#define LSM303_M_NORMALIZE_SAMPLE(x) (150 * (int)(x))

Raw sensor value range is up to 32767, so sampleENU is up to 4915050. calibration.scale.axis is around 1024, so the CALIBRATED_SAMPLE calculation can involve numbers greater than 5 billion, and overflow will occur just above 2 billion.

To see it happening, add trace as follows, and move a magnet close to the sensor.

https://github.com/lancaster-university/codal-core/blob/master/source/drivers/LSM303Magnetometer.cpp#L181
DMESGN( "raw %d enu %d ", (int) (*z), (int) sampleENU.z);
https://github.com/lancaster-university/codal-core/blob/master/source/driver-models/Compass.cpp#L320
DMESG( "calibrated %d ", (int) sampleENU.z);

raw 13584 enu 2037600 calibrated 2037600
raw 14542 enu 2181300 calibrated -2013004
raw 13843 enu 2076450 calibrated 2076450

These figures are without calibration, but calibrated calibration.centre.xyz are small compared to these sampleENU.xyz values.

Related: lancaster-university/microbit-dal#472

@microbit-carlos microbit-carlos added this to the v0.2.52 milestone Apr 20, 2023
@microbit-carlos microbit-carlos modified the milestones: v0.2.52, v0.2.53 May 5, 2023
@microbit-carlos microbit-carlos modified the milestones: v0.2.56, v0.2.59 Jun 15, 2023
@microbit-carlos microbit-carlos modified the milestones: v0.2.59, v0.2.60 Jul 25, 2023
@microbit-carlos microbit-carlos modified the milestones: v0.2.61, v0.2.62 Sep 20, 2023
@microbit-carlos microbit-carlos modified the milestones: v0.2.64, v0.2.65 Nov 14, 2023
@microbit-carlos microbit-carlos modified the milestones: v0.2.67, v0.2.68 Feb 27, 2024
@microbit-carlos microbit-carlos added the bug Something isn't working label Mar 28, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working dal-change-required p3
Projects
None yet
Development

No branches or pull requests

4 participants