-
Notifications
You must be signed in to change notification settings - Fork 24
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Callbacks please! Usable for async? #307
Comments
Would be nicer if the generated class would be named There could/should be an option to the decorator how the generated class is named. At least that might be usefull for those who do not generate the C# code automatically but call a tool which does so. |
Besides the missing Maybe a layered approach gives the most options for all... |
is wrong as To be able to pass the final wrapped python func to the interpreter in a way its compatible with the developers model, an instance of That way Beste design case would be the need to override just the " |
Generators are the best option for calbacks, async functions (coroutines) are fancy wrappers around the generator system in Python. Utilizing C#.NET's Task system you would execute a callback from a generator whenever it yielded a value in a thread: from typing import Generator
def example_generator(length: int) -> Generator[str, int, bool]:
for i in range(length):
x = yield f"Item {i}"
if x:
yield f"Received {x}"
return True var mod = Env.TestGenerators();
var generator = mod.ExampleGenerator(3);
var callback = new Action<string>(
// This is the callback that will be called from the generator
s => Assert.Equal(generator.Current, s)
);
// Wait for the generator to finish
var task = Task.Run(() =>
{
while (generator.MoveNext())
{
// Simulate a callback
callback(generator.Current);
// Optionally send a value back to the generator
generator.Send(10);
}
// Optionally return a value from the generator
Assert.True(generator.Return);
}); I'll submit this test to main to demonstrate it and make sure it's tracked for regressions. The Pending calls API is really unreliable and only gets called in the interpreter loop (so not when embedded like CSnakes) and in certain scenarios. |
pythonnet enables C# functions being called from Python. Very usefull, like redirecting stdout and stderr to whatever the developer likes...
Here a modified example from pythonnet/pythonnet#1794
Callbacks
Is something like that possible in CSnakes? Too bad I have no clue how to call into dotnet from Python and havn't analyzed yet how pythonnet does it. But as pythonnet does "crazy" stuff, like accessing CPython object internals it might not be the preferred way anyway.
With CSnakes magic something like this should be possible:
CSnakes SourceGenerator could generate the sources for a class like:
which could be use like:
This might be not thread safe, having "side effects" (here: changing a class definition while other python code might use it at the same time). But this should be no argument against it as developer how will use it that way should know what they are doing (documentation...) .
Those developers who want to have no thread safety issues etc. could use
PythonEngineerCallable<T>.CreateFor()
to create a callback which is then used as shown in the pythonnet code...As it is possible to give Python decorators parameters, the generated C# source could also:
InjectWrite(Action<PyObject, string, PyFunc> f);
where the last PyFunc argument is the original func . Would enable to create "hybrid" Python/C# classes where C# can inherit the method and call the "super" function... for those who want that. Maybe automatically generated this way if the decorator is given an "override" (or something else C# like) argument?To be able to prevent the dotnet runtime from catching outside variables etc. :
or any possible combination as requested by the decorators arguments. The provided CSnakes python decorators are dummys, just return the decorated function "as is"...
Of course it should be possible to replace every python function defined anywhere. And static function calls like
PythonEngineerCallable<T>.CreateFor()
should enable created a Python function object from C# usable as a callback. When used in this way its up to the developer how to pass additional needed parameters on callback.As I'm not a C# language expert I guess that the shown API could be better. But the most important problem to solve is how CPython can call back into dotnet.
I guess it might be possible using CPython cffi or ctypes . Best would be a "native" implementation provided as a module by the one who compiled the used CPython DLL . Maybe its possible to come up with something the maintainer of the python NuGet package will include or provide as an additional NuGet package...
This way CSnakes would be better then pythonnet regarding compability. Looks like pythonnet does some "interersting" stuff, detecting the layout of the Python object struct. Fact is: pythonnet isn't compatible with 3.14 yet (which might be of other reasons).
Generic async wrapper possible?
Having a callback from CPython to C# it should be possible to generate a async C# function as
that used from C# like
or with syntax sugar if possible:
or internally by the SourceGenerator etc. ...
Some work to finish the design the API and implement it, but as if done as shown its only the "easy" part. Just the C# conventions for async are fullfilled.
( CSnakes could restrict the usage in any way suitable for the implementation. Like telling the developer not to recurse async calls etc. or anything which will fail. )
More of a problem is that how Python "async" calls from C# into Python should work depends on the Python code: the
// give the python interpreter the syncPyFunc to execute in a "fire and forget" manner
part will be different and should be customizable somehow.It could be as "simple" as generating a wrapper to be used with
int Py_AddPendingCall(int (*func)(void*), void *arg)
which is part of the Stable API and looks promising. OK - https://docs.python.org/3/c-api/init.html#sub-interpreter-support says that there is no guarantee that func is being called, but maybe it is called with a usable success rate? If it is not - the developer might provide a CancellationToken to set a timeout ...
Other might want to run the python function in a new python thread. Or queue it to a worker ...
How that C# API could be made compatible with possible sub-interpreters is beyond my knowledge like how to call/address specific subinterpreter from the CAPI ?
Much work to do, as a start anything how to call C# from Python is helpfull for me... C Code is welcome too, just have to get CPython compiled by myself to write a module.
PS: Tony - if such C# DLL forth and back calls are in your Python Internals book - let me know. will buy it... otherwise money is an "issue" for me at the moment.
The text was updated successfully, but these errors were encountered: