Does cstruct provides marshaling to/from a char bigarray (bigstring)?

[UnixJunkie]

It would be nice to have such a functionality.
Since bigarrays are the only thing that can be memory mapped on unices.

[reynir]

Does Cstruct.of_bigarray or Cstruct.buffer do what you're looking for? (I'm not sure what the difference is)

https://github.com/mirage/ocaml-cstruct/blob/main/lib/cstruct.mli#L178
https://github.com/mirage/ocaml-cstruct/blob/main/lib/cstruct.mli#L629

[UnixJunkie]

I don't think so.
What I am looking for is like:

val Marshal.to_buffer                                                        
    bytes -> int -> int -> 'a -> Stdlib__Marshal.extern_flags list -> int

but instead of bytes we would have a bigstring.
Also:

val Marshal.from_bytes bytes -> int -> 'a

with also bytes replaced by bigstring.
Also, the number of bytes that were consumed should be returned.

[reynir]

Sorry, I think I needed another cup of coffee :-)

I found that there are C symbols caml_output_value_to_malloc and caml_input_value_from_malloc. The former could be used to implement marshal_to_cstruct I think, but the latter implicitly calls free() on the buffer :/ I don't find it in cstruct.

[UnixJunkie]

I know some related code in parmap, if you are interested:
https://github.com/rdicosmo/parmap/blob/master/src/bytearray_stubs.c
ml_marshal_to_bigarray_buffer and ml_unmarshal_from_bigarray are interesting.
I am not sure they have the exact required interface though.
Esp., if you don't return the number of bytes read, it is annoying
for the software layer which is managing the buffer.

[avsm]

It's worth noting that bytes are non-moving in OCaml 5.0 (and will likely remain so for 1KB+ allocations even if a compactor is added in the future), and so we should probably just resolve the question of whether Cstruct can be backed by bytes in OCaml 5.0+ and eliminate this bigarray split.

[UnixJunkie]

The problem with bytes is that they cannot be memory mapped, as far as I know.