Scalable tsdf integration: parallel_for failed: cudaErrorMemoryAllocation: out of memory

[Apeiria]

Hello,

I tried the ScalableTSDF integration and got this error:
terminate called after throwing an instance of 'thrust::system::system_error' what(): parallel_for failed: cudaErrorMemoryAllocation: out of memory Aborted (core dumped)

I thought it was a little bit strange because it took a lot more memory than UniformTSDF, which worked with about 4GB, and my graphics card has 8GB in total.

I've tried to play with the parameters like making the voxel size larger, but I always got the same error.

Below is the code I'm trying to run.

Thanks for your great work and any help you can offer.

#include "cupoch/cupoch.h"
#include "cupoch/integration/scalable_tsdfvolume.h"
#include "cupoch/integration/uniform_tsdfvolume.h"
#include "sys/time.h"

int main(int argc, char *argv[]) {
    using namespace cupoch;
    using namespace std;

    string datasetDir = "/home/apeiron/data/tum_clouds/";
    auto intrinsic = camera::PinholeCameraIntrinsic(
            640, 480, 517.306408, 516.469215, 318.643040, 255.313989);

    //-- read imgs
    geometry::Image depth, color;
    io::ReadImage("/home/apeiron/data/tum_clouds/depth/0.png", depth);
    io::ReadImage("/home/apeiron/data/tum_clouds/color/0.jpg", color);
    auto rgbd = geometry::RGBDImage::CreateFromColorAndDepth(
            color, depth, 5000.0, 2.0, false);

    //-- tsdf
    float length = 2;
    float resolution = 512;
    float sdf_trunc_percentage = 0.01;

    // uniform works
    // integration::UniformTSDFVolume volume(
    //         length, resolution, length * sdf_trunc_percentage,
    //         integration::TSDFVolumeColorType::RGB8);

    integration::ScalableTSDFVolume volume(
            length / resolution, length * sdf_trunc_percentage,
            integration::TSDFVolumeColorType::RGB8);

    Eigen::Matrix4f extrinsic = Eigen::Matrix4f::Identity();
    extrinsic(0, 3) = -0.5;
    extrinsic(1, 3) = -0.5;
    extrinsic(2, 3) = 0.5;
    volume.Integrate(*rgbd, intrinsic, extrinsic);

    auto mesh = volume.ExtractTriangleMesh();

    io::WriteTriangleMesh("test.ply", *mesh);
    visualization::DrawGeometries({mesh});
    return 0;
}

[neka-nat]

Hi @Apeiria ,

Thank you for reporting.
ScalableTSDF uses the unordered_map implemented in STDGPU, but it is not very memory efficient.
I am currently exploring ways to use memory more efficiently.

[Apeiria]

Hi @Apeiria ,

Thank you for reporting.
ScalableTSDF uses the unordered_map implemented in STDGPU, but it is not very memory efficient.
I am currently exploring ways to use memory more efficiently.

Thank for your reply! Looking forward to your next version.

[Apeiria]

Hi,
I notice that you use float for tsdf value and weight, Vector3i for index, and Vector3f for color. So it takes 32 bytes per TSDFVoxel, and a 512^3 volume takes 4GB.
I'm not sure but wouldn't it be better to use, for example, unsigned char *3 for color and short for index, and maybe short is enough for quantized tsdf and weight. Therefore it only takes 13 bytes per TSDFVoxel

[Apeiria]

Hi,
I notice that you use float for tsdf value and weight, Vector3i for index, and Vector3f for color. So it takes 32 bytes per TSDFVoxel, and a 512^3 volume takes 4GB.
I'm not sure but wouldn't it be better to use, for example, unsigned char *3 for color and short for index, and maybe short is enough for quantized tsdf and weight. Therefore it only takes 13 bytes per TSDFVoxel

Is it necessary to store indices? I think the (x,y,z) index of a voxel can be derived from its index in the vector.

[neka-nat]

Hi,

Your suggestion is correct.
The grid_index can be eliminated and the size of the TSDFVoxel can be further reduced.
However, I find the problem is that scalable tsdf uses more memory than uniform tsdf.
Basically, scalable tsdf is a more memory-saving algorithm than uniform tsdf.
I'd like to leave the issue of TSDFVoxel's size for another issue.