Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Automatically wrapping with Clang.jl #149

Draft
wants to merge 54 commits into
base: main
Choose a base branch
from
Draft

Automatically wrapping with Clang.jl #149

wants to merge 54 commits into from
+31,122 −2,325

Conversation

jkozdon
Copy link
Member

@jkozdon jkozdon commented Jul 15, 2021 •
edited
Loading

Experimenting to play with #147

Todo

@jkozdon
Copy link
Member Author

jkozdon commented Jul 15, 2021

Doesn't work with everything yet but you can call initialize and finalize.

@jkozdon
Copy link
Member Author

jkozdon commented Jul 15, 2021 •
edited
Loading

Input:

# For some reason you need to initialize MPI before using multiple libraries.
# Need to debug this
using MPI
MPI.Initialized() || MPI.Init()
using PETSc
for petsclib in PETSc.LibPETSc.petsclibs
    println(splitpath(petsclib.petsc_library)[end])
    @show PETSc.initialized(petsclib)
    @show PETSc.initialize(petsclib)
    @show PETSc.initialized(petsclib)
    @show PETSc.finalized(petsclib)
    @show PETSc.finalize(petsclib)
    @show PETSc.initialized(petsclib)
    @show PETSc.finalized(petsclib)
    println()
end

Output:

libpetsc_double_real_Int64.dylib
PETSc.initialized(petsclib) = false
PETSc.initialize(petsclib) = nothing
PETSc.initialized(petsclib) = true
PETSc.finalized(petsclib) = false
PETSc.finalize(petsclib) = nothing
PETSc.initialized(petsclib) = false
PETSc.finalized(petsclib) = true

libpetsc_single_real_Int64.dylib
PETSc.initialized(petsclib) = false
PETSc.initialize(petsclib) = nothing
PETSc.initialized(petsclib) = true
PETSc.finalized(petsclib) = false
PETSc.finalize(petsclib) = nothing
PETSc.initialized(petsclib) = false
PETSc.finalized(petsclib) = true

libpetsc_double_complex_Int64.dylib
PETSc.initialized(petsclib) = false
PETSc.initialize(petsclib) = nothing
PETSc.initialized(petsclib) = true
PETSc.finalized(petsclib) = false
PETSc.finalize(petsclib) = nothing
PETSc.initialized(petsclib) = false
PETSc.finalized(petsclib) = true

libpetsc_single_complex_Int64.dylib
PETSc.initialized(petsclib) = false
PETSc.initialize(petsclib) = nothing
PETSc.initialized(petsclib) = true
PETSc.finalized(petsclib) = false
PETSc.finalize(petsclib) = nothing
PETSc.initialized(petsclib) = false
PETSc.finalized(petsclib) = true

@jkozdon
Copy link
Member Author

jkozdon commented Jul 16, 2021

I guess I still need to sort out the types (But I think it should be possible with more walking of the syntax tree).

@jkozdon jkozdon mentioned this pull request Jul 16, 2021
Open
@jkozdon jkozdon force-pushed the jek/gen branch 3 times, most recently from 1e39b30 to 97e39a4 Compare July 19, 2021 23:32
This was referenced Jul 20, 2021
Merged
Open
@jkozdon jkozdon force-pushed the jek/gen branch 3 times, most recently from 16b5f8e to fe80c8f Compare July 20, 2021 04:03
@psanan
Copy link
Contributor

psanan commented Jul 20, 2021

Lazy question, but what's the high-level procedure here? What is now automatic and what still has to be maintained "by hand"? Are the specific things here related to various PETSc components all of the "by hand" flavor, or are those things that have been generated automatically and are then checked in?

@vchuravy
Copy link
Member

I think everything in lib is autogenerated.

@jkozdon
Copy link
Member Author

jkozdon commented Jul 20, 2021

As @vchuravy said, the file in lib is generated from the PETSc header files with the code in gen. These produce pretty raw wrappers, that I am then expanding on to make it all a little more friendly to work with.

There is a bunch of stuff we do not currently wrap (look at generator.toml), some of it will need to be sorted out and some of it won't (I think I know / have a plan for how to support most of what will be needed, just waiting to fix it until I am ready to use it / write the test).

There probably some gotchas left in there with respect to system / build dependent types that will need to be sorted out, but right now it does support multiple PETSc builds living side-by-side in Julia.

@psanan shoot me an email if you'd like to VTC to chat about what I am doing, I'd be more than happy to have feedback.

@jkozdon jkozdon force-pushed the jek/gen branch 2 times, most recently from 53fc7ab to aa8e660 Compare July 21, 2021 23:09
@jkozdon jkozdon force-pushed the jek/gen branch 4 times, most recently from afbe26f to 0f22d51 Compare August 3, 2021 19:55
@boriskaus boriskaus marked this pull request as ready for review August 3, 2021 21:49
@jkozdon jkozdon force-pushed the jek/gen branch 4 times, most recently from 3e3e0f9 to 1850ac4 Compare August 4, 2021 17:53
@psanan
Copy link
Contributor

psanan commented Aug 5, 2021

Docs suggestion, re our chat our yesterday. Perhaps somewhere like here one could add language like "PETSc.jl's high-level, easy-to-use interface exposes only some of the functionality of PETSc. However, PETSc.jl includes an automatically-generated wrapper layer which directly exposes more of the PETSc API, which may be of use to existing expert users. See (link to minimal example/test of using the wrappers, directly) for an example".

@jkozdon
Copy link
Member Author

jkozdon commented Aug 5, 2021

Thanks for the recommendation. I've put it as part of the PR task list.

@jkozdon jkozdon linked an issue Aug 31, 2021 that may be closed by this pull request
Revisit MPI after / with #149 #150
Open
@jkozdon jkozdon force-pushed the jek/gen branch 2 times, most recently from 4db9766 to 581f379 Compare September 3, 2021 18:48
jkozdon
jkozdon commented Sep 8, 2021
View reviewed changes
src/viewer.jl
# ideally we would capture the output directly, but this looks difficult
# easiest option is to redirect stdout
# based on suggestion from https://github.com/JuliaLang/julia/issues/32567
function _show(io::IO, obj)
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This still has problems with large matrices.

Might be better to create a temp file with mktemp(), redirect the PETSC output to an ascii file with PetscViewerASCIIOpenWithFILE, then dump the contents to io.

Or something is wrong with the flush?

@jkozdon jkozdon mentioned this pull request Dec 3, 2021
Closed
3 tasks
@jkozdon jkozdon closed this Feb 10, 2023
@ViralBShah
Copy link
Member

It would be really cool to get this working and merged. I have no idea what the challenges were (and plan to dive in), but is it possible that improvements in Clang.jl will help us get this done?

@boriskaus
Copy link
Collaborator

I think there were no real challenges, but it was probably mainly caused by the main developer wanting things to be perfect before merging it back into main.

Clang already used to do a good job in wrapping the functions. Making a nice julia-like interface around this would be good to have, but that had to be done manually which is what took a long time (and was never finished).
I guess Clang can now even take the help into account in the documentation

@ViralBShah
Copy link
Member

I feel that it would be nice to get it refreshed and merged and it may help others to contribute if all the auto-wrapping is in place. Kind of imagining moving things one step forward.

Since the package is not even loading right now, once that is fixed, this sort of thing may help us find other maintainers.

@boriskaus
Copy link
Collaborator

Another thing was the interaction with the GC. There was a presentation about how this was solved in python (for firedrake) at this year’s PETSc conference, so I suppose the same method can be used in julia

@boriskaus
Copy link
Collaborator

I fully agree with you. If the basic functions are wrapped and we have some basic documentation it would motivate others to help.
It probably won’t take professionals that long. I, however, don’t belong to that group. Happy to help with usage examples etc.

@ViralBShah
Copy link
Member

I'll see how I can help with getting the package back up and running and get this PR merged.

@boriskaus
Copy link
Collaborator

That would be fantastic!

@ViralBShah
Copy link
Member

Just leaving this one open to be able to drive work on it.

@ViralBShah ViralBShah reopened this Oct 14, 2024
@ViralBShah
Copy link
Member

@boriskaus This has a pretty large set of conflicts. Any suggestions on how to get it rebased to main quickly? Are some of these just auto-generated files?

@ViralBShah
Copy link
Member

ViralBShah commented Oct 14, 2024 •
edited
Loading

Tried to update this branch to Julia 1.11. The package loads fine in Julia 1.11 with some minor tweaks (I checked in my commits).

The clang autowrap stuff prints a few errors (naturally it's been a long time), but it does write a petsc_library.jl. I see this:

┌ Error: missing implementation for ExprNode{Clang.Generators.FunctionDefault, Clang.CLFunctionDecl}(:SNESObjectiveComputeFunctionDefaultFD, Clang.Generators.FunctionDefault(), CLCursor (Clang.CLFunctionDecl) SNESObjectiveComputeFunctionDefaultFD(SNES, Vec, Vec, void *), Expr[], Expr[], Int64[]), please file an issue to Clang.jl.
└ @ Clang.Generators ~/.julia/packages/Clang/uJQ2L/src/generator/codegen.jl:716
┌ Error: missing implementation for ExprNode{Clang.Generators.FunctionDefault, Clang.CLFunctionDecl}(:SNESComputeJacobianDefault, Clang.Generators.FunctionDefault(), CLCursor (Clang.CLFunctionDecl) SNESComputeJacobianDefault(SNES, Vec, Mat, Mat, void *), Expr[], Expr[], Int64[]), please file an issue to Clang.jl.
└ @ Clang.Generators ~/.julia/packages/Clang/uJQ2L/src/generator/codegen.jl:716
┌ Error: missing implementation for ExprNode{Clang.Generators.FunctionDefault, Clang.CLFunctionDecl}(:SNESComputeJacobianDefaultColor, Clang.Generators.FunctionDefault(), CLCursor (Clang.CLFunctionDecl) SNESComputeJacobianDefaultColor(SNES, Vec, Mat, Mat, void *), Expr[], Expr[], Int64[]), please file an issue to Clang.jl.
└ @ Clang.Generators ~/.julia/packages/Clang/uJQ2L/src/generator/codegen.jl:716
┌ Error: missing implementation for ExprNode{Clang.Generators.FunctionDefault, Clang.CLFunctionDecl}(:SNESPicardComputeFunction, Clang.Generators.FunctionDefault(), CLCursor (Clang.CLFunctionDecl) SNESPicardComputeFunction(SNES, Vec, Vec, void *), Expr[], Expr[], Int64[]), please file an issue to Clang.jl.
└ @ Clang.Generators ~/.julia/packages/Clang/uJQ2L/src/generator/codegen.jl:716
┌ Error: missing implementation for ExprNode{Clang.Generators.FunctionDefault, Clang.CLFunctionDecl}(:SNESPicardComputeMFFunction, Clang.Generators.FunctionDefault(), CLCursor (Clang.CLFunctionDecl) SNESPicardComputeMFFunction(SNES, Vec, Vec, void *), Expr[], Expr[], Int64[]), please file an issue to Clang.jl.
└ @ Clang.Generators ~/.julia/packages/Clang/uJQ2L/src/generator/codegen.jl:716
┌ Error: missing implementation for ExprNode{Clang.Generators.FunctionDefault, Clang.CLFunctionDecl}(:SNESPicardComputeJacobian, Clang.Generators.FunctionDefault(), CLCursor (Clang.CLFunctionDecl) SNESPicardComputeJacobian(SNES, Vec, Mat, Mat, void *), Expr[], Expr[], Int64[]), please file an issue to Clang.jl.
└ @ Clang.Generators ~/.julia/packages/Clang/uJQ2L/src/generator/codegen.jl:716
┌ Error: missing implementation for ExprNode{Clang.Generators.FunctionDefault, Clang.CLFunctionDecl}(:TSComputeRHSFunctionLinear, Clang.Generators.FunctionDefault(), CLCursor (Clang.CLFunctionDecl) TSComputeRHSFunctionLinear(TS, PetscReal, Vec, Vec, void *), Expr[], Expr[], Int64[]), please file an issue to Clang.jl.
└ @ Clang.Generators ~/.julia/packages/Clang/uJQ2L/src/generator/codegen.jl:716
┌ Error: missing implementation for ExprNode{Clang.Generators.FunctionDefault, Clang.CLFunctionDecl}(:TSComputeRHSJacobianConstant, Clang.Generators.FunctionDefault(), CLCursor (Clang.CLFunctionDecl) TSComputeRHSJacobianConstant(TS, PetscReal, Vec, Mat, Mat, void *), Expr[], Expr[], Int64[]), please file an issue to Clang.jl.

@ViralBShah
Copy link
Member

Part of the issue is that the generator seems to be for the default Float64/Int32 combination, and I wonder if it needs to be extended to generate all the type combinations.

@boriskaus
Copy link
Collaborator

I think we can first focus on getting this working before doing the rest. Part of the problem is likely that some of the PETSc functions have been renamed.

@ViralBShah
Copy link
Member

I'm going to try to just get this branch working without worrying about rebasing. I am already able to generate the full petsc_library.jl but it doesn't load yet. You can try it out on this branch by running generator.jl.

I don't think major changes have happened since this branch was created and now.

@boriskaus
Copy link
Collaborator

fine with me; ideally we would change the existing tests that are currently in main to only use the auto-generated wrappers, which would simplify maintaining it.

@boriskaus
Copy link
Collaborator

We don't have to strictly follow what was done before, since Clang.jl has presumably improved a lot over the last few years.

If I simplify generator.toml to:

[general]
library_name = "$petsc_library"

use_julia_native_enum_type = true
use_deterministic_symbol = true

prologue_file_path = "./prologue.jl"
# epilogue_file_path = "./epilogue.jl"

auto_mutability = true              # Create mutable structures:
auto_mutability_with_new = true     # Allow creating empty structs
auto_mutability_ignorelist = []
auto_mutability_includelist = []

extract_c_comment_style = "doxygen"

export_symbol_prefixes = ["CX", "clang_"]

It essentially compiles the whole library with a few exceptions:

julia> include("generator.jl")
[ Info: Found dependent header: /Users/kausb/.julia/artifacts/a69b8f699a39f274f95752394759a3e9973e02b0/lib/petsc/double_real_Int64/include/petscbag.h
[ Info: Found dependent header: /Users/kausb/.julia/artifacts/a69b8f699a39f274f95752394759a3e9973e02b0/lib/petsc/double_real_Int64/include/petscsys.h
[ Info: Found dependent header: /Users/kausb/.julia/artifacts/a69b8f699a39f274f95752394759a3e9973e02b0/lib/petsc/double_real_Int64/include/petscconf.h
[ Info: Found dependent header: /Users/kausb/.julia/artifacts/a69b8f699a39f274f95752394759a3e9973e02b0/lib/petsc/double_real_Int64/include/petscconf_poison.h
[ Info: Found dependent header: /Users/kausb/.julia/artifacts/a69b8f699a39f274f95752394759a3e9973e02b0/lib/petsc/double_real_Int64/include/petscfix.h
[ Info: Found dependent header: /Users/kausb/.julia/artifacts/a69b8f699a39f274f95752394759a3e9973e02b0/lib/petsc/double_real_Int64/include/petscmacros.h
[ Info: Found dependent header: /Users/kausb/.julia/artifacts/a69b8f699a39f274f95752394759a3e9973e02b0/lib/petsc/double_real_Int64/include/petscversion.h
[ Info: Found dependent header: /Users/kausb/.julia/artifacts/a69b8f699a39f274f95752394759a3e9973e02b0/lib/petsc/double_real_Int64/include/petsc/private/petscadvancedmacros.h
[ Info: Found dependent header: /Users/kausb/.julia/artifacts/a69b8f699a39f274f95752394759a3e9973e02b0/lib/petsc/double_real_Int64/include/petscsystypes.h
[ Info: Found dependent header: /Users/kausb/.julia/artifacts/a69b8f699a39f274f95752394759a3e9973e02b0/lib/petsc/double_real_Int64/include/petscmath.h
[ Info: Found dependent header: /Users/kausb/.julia/artifacts/a69b8f699a39f274f95752394759a3e9973e02b0/lib/petsc/double_real_Int64/include/petscerror.h
[ Info: Found dependent header: /Users/kausb/.julia/artifacts/a69b8f699a39f274f95752394759a3e9973e02b0/lib/petsc/double_real_Int64/include/petscviewertypes.h
[ Info: Found dependent header: /Users/kausb/.julia/artifacts/a69b8f699a39f274f95752394759a3e9973e02b0/lib/petsc/double_real_Int64/include/petscoptions.h
[ Info: Found dependent header: /Users/kausb/.julia/artifacts/a69b8f699a39f274f95752394759a3e9973e02b0/lib/petsc/double_real_Int64/include/petsclog.h
[ Info: Found dependent header: /Users/kausb/.julia/artifacts/a69b8f699a39f274f95752394759a3e9973e02b0/lib/petsc/double_real_Int64/include/petsctime.h
[ Info: Found dependent header: /Users/kausb/.julia/artifacts/a69b8f699a39f274f95752394759a3e9973e02b0/lib/petsc/double_real_Int64/include/petscbt.h
[ Info: Found dependent header: /Users/kausb/.julia/artifacts/a69b8f699a39f274f95752394759a3e9973e02b0/lib/petsc/double_real_Int64/include/petscstring.h
[ Info: Found dependent header: /Users/kausb/.julia/artifacts/a69b8f699a39f274f95752394759a3e9973e02b0/lib/petsc/double_real_Int64/include/petsclogtypes.h
[ Info: Found dependent header: /Users/kausb/.julia/artifacts/a69b8f699a39f274f95752394759a3e9973e02b0/lib/petsc/double_real_Int64/include/petsclogdeprecated.h
[ Info: Found dependent header: /Users/kausb/.julia/artifacts/a69b8f699a39f274f95752394759a3e9973e02b0/lib/petsc/double_real_Int64/include/petscmatlab.h
[ Info: Found dependent header: /Users/kausb/.julia/artifacts/a69b8f699a39f274f95752394759a3e9973e02b0/lib/petsc/double_real_Int64/include/petscdraw.h
[ Info: Found dependent header: /Users/kausb/.julia/artifacts/a69b8f699a39f274f95752394759a3e9973e02b0/lib/petsc/double_real_Int64/include/petscdrawtypes.h
[ Info: Found dependent header: /Users/kausb/.julia/artifacts/a69b8f699a39f274f95752394759a3e9973e02b0/lib/petsc/double_real_Int64/include/petscdevice.h
[ Info: Found dependent header: /Users/kausb/.julia/artifacts/a69b8f699a39f274f95752394759a3e9973e02b0/lib/petsc/double_real_Int64/include/petscdevicetypes.h
[ Info: Found dependent header: /Users/kausb/.julia/artifacts/a69b8f699a39f274f95752394759a3e9973e02b0/lib/petsc/double_real_Int64/include/petscsf.h
[ Info: Found dependent header: /Users/kausb/.julia/artifacts/a69b8f699a39f274f95752394759a3e9973e02b0/lib/petsc/double_real_Int64/include/petscsftypes.h
[ Info: Found dependent header: /Users/kausb/.julia/artifacts/a69b8f699a39f274f95752394759a3e9973e02b0/lib/petsc/double_real_Int64/include/petscvec.h
[ Info: Found dependent header: /Users/kausb/.julia/artifacts/a69b8f699a39f274f95752394759a3e9973e02b0/lib/petsc/double_real_Int64/include/petscis.h
[ Info: Found dependent header: /Users/kausb/.julia/artifacts/a69b8f699a39f274f95752394759a3e9973e02b0/lib/petsc/double_real_Int64/include/petscsectiontypes.h
[ Info: Found dependent header: /Users/kausb/.julia/artifacts/a69b8f699a39f274f95752394759a3e9973e02b0/lib/petsc/double_real_Int64/include/petscistypes.h
[ Info: Found dependent header: /Users/kausb/.julia/artifacts/a69b8f699a39f274f95752394759a3e9973e02b0/lib/petsc/double_real_Int64/include/petscviewer.h
[ Info: Found dependent header: /Users/kausb/.julia/artifacts/a69b8f699a39f274f95752394759a3e9973e02b0/lib/petsc/double_real_Int64/include/petscsection.h
[ Info: Found dependent header: /Users/kausb/.julia/artifacts/a69b8f699a39f274f95752394759a3e9973e02b0/lib/petsc/double_real_Int64/include/petscdmda.h
[ Info: Found dependent header: /Users/kausb/.julia/artifacts/a69b8f699a39f274f95752394759a3e9973e02b0/lib/petsc/double_real_Int64/include/petscdm.h
[ Info: Found dependent header: /Users/kausb/.julia/artifacts/a69b8f699a39f274f95752394759a3e9973e02b0/lib/petsc/double_real_Int64/include/petscmat.h
[ Info: Found dependent header: /Users/kausb/.julia/artifacts/a69b8f699a39f274f95752394759a3e9973e02b0/lib/petsc/double_real_Int64/include/petscmatcoarsen.h
[ Info: Found dependent header: /Users/kausb/.julia/artifacts/a69b8f699a39f274f95752394759a3e9973e02b0/lib/petsc/double_real_Int64/include/petscdmtypes.h
[ Info: Found dependent header: /Users/kausb/.julia/artifacts/a69b8f699a39f274f95752394759a3e9973e02b0/lib/petsc/double_real_Int64/include/petscfetypes.h
[ Info: Found dependent header: /Users/kausb/.julia/artifacts/a69b8f699a39f274f95752394759a3e9973e02b0/lib/petsc/double_real_Int64/include/petscdstypes.h
[ Info: Found dependent header: /Users/kausb/.julia/artifacts/a69b8f699a39f274f95752394759a3e9973e02b0/lib/petsc/double_real_Int64/include/petscdmlabel.h
[ Info: Found dependent header: /Users/kausb/.julia/artifacts/a69b8f699a39f274f95752394759a3e9973e02b0/lib/petsc/double_real_Int64/include/petscdt.h
[ Info: Found dependent header: /Users/kausb/.julia/artifacts/a69b8f699a39f274f95752394759a3e9973e02b0/lib/petsc/double_real_Int64/include/petscdmdatypes.h
[ Info: Found dependent header: /Users/kausb/.julia/artifacts/a69b8f699a39f274f95752394759a3e9973e02b0/lib/petsc/double_real_Int64/include/petscpf.h
[ Info: Found dependent header: /Users/kausb/.julia/artifacts/a69b8f699a39f274f95752394759a3e9973e02b0/lib/petsc/double_real_Int64/include/petscao.h
[ Info: Found dependent header: /Users/kausb/.julia/artifacts/a69b8f699a39f274f95752394759a3e9973e02b0/lib/petsc/double_real_Int64/include/petscfe.h
[ Info: Found dependent header: /Users/kausb/.julia/artifacts/a69b8f699a39f274f95752394759a3e9973e02b0/lib/petsc/double_real_Int64/include/petscspace.h
[ Info: Found dependent header: /Users/kausb/.julia/artifacts/a69b8f699a39f274f95752394759a3e9973e02b0/lib/petsc/double_real_Int64/include/petscdualspace.h
[ Info: Found dependent header: /Users/kausb/.julia/artifacts/a69b8f699a39f274f95752394759a3e9973e02b0/lib/petsc/double_real_Int64/include/petscdmcomposite.h
[ Info: Found dependent header: /Users/kausb/.julia/artifacts/a69b8f699a39f274f95752394759a3e9973e02b0/lib/petsc/double_real_Int64/include/petscdmpatch.h
[ Info: Found dependent header: /Users/kausb/.julia/artifacts/a69b8f699a39f274f95752394759a3e9973e02b0/lib/petsc/double_real_Int64/include/petscdmplex.h
[ Info: Found dependent header: /Users/kausb/.julia/artifacts/a69b8f699a39f274f95752394759a3e9973e02b0/lib/petsc/double_real_Int64/include/petscpartitioner.h
[ Info: Found dependent header: /Users/kausb/.julia/artifacts/a69b8f699a39f274f95752394759a3e9973e02b0/lib/petsc/double_real_Int64/include/petscdmplextypes.h
[ Info: Found dependent header: /Users/kausb/.julia/artifacts/a69b8f699a39f274f95752394759a3e9973e02b0/lib/petsc/double_real_Int64/include/petscfv.h
[ Info: Found dependent header: /Users/kausb/.julia/artifacts/a69b8f699a39f274f95752394759a3e9973e02b0/lib/petsc/double_real_Int64/include/petscfvtypes.h
[ Info: Found dependent header: /Users/kausb/.julia/artifacts/a69b8f699a39f274f95752394759a3e9973e02b0/lib/petsc/double_real_Int64/include/petscdmfield.h
[ Info: Found dependent header: /Users/kausb/.julia/artifacts/a69b8f699a39f274f95752394759a3e9973e02b0/lib/petsc/double_real_Int64/include/petscdmplextransform.h
[ Info: Found dependent header: /Users/kausb/.julia/artifacts/a69b8f699a39f274f95752394759a3e9973e02b0/lib/petsc/double_real_Int64/include/petscdmplextransformtypes.h
[ Info: Found dependent header: /Users/kausb/.julia/artifacts/a69b8f699a39f274f95752394759a3e9973e02b0/lib/petsc/double_real_Int64/include/petscdmredundant.h
[ Info: Found dependent header: /Users/kausb/.julia/artifacts/a69b8f699a39f274f95752394759a3e9973e02b0/lib/petsc/double_real_Int64/include/petscdmshell.h
[ Info: Found dependent header: /Users/kausb/.julia/artifacts/a69b8f699a39f274f95752394759a3e9973e02b0/lib/petsc/double_real_Int64/include/petscdmsliced.h
[ Info: Found dependent header: /Users/kausb/.julia/artifacts/a69b8f699a39f274f95752394759a3e9973e02b0/lib/petsc/double_real_Int64/include/petscdmswarm.h
[ Info: Found dependent header: /Users/kausb/.julia/artifacts/a69b8f699a39f274f95752394759a3e9973e02b0/lib/petsc/double_real_Int64/include/petscdmstag.h
[ Info: Found dependent header: /Users/kausb/.julia/artifacts/a69b8f699a39f274f95752394759a3e9973e02b0/lib/petsc/double_real_Int64/include/petscdmproduct.h
[ Info: Found dependent header: /Users/kausb/.julia/artifacts/a69b8f699a39f274f95752394759a3e9973e02b0/lib/petsc/double_real_Int64/include/petscds.h
[ Info: Found dependent header: /Users/kausb/.julia/artifacts/a69b8f699a39f274f95752394759a3e9973e02b0/lib/petsc/double_real_Int64/include/petsccharacteristic.h
[ Info: Found dependent header: /Users/kausb/.julia/artifacts/a69b8f699a39f274f95752394759a3e9973e02b0/lib/petsc/double_real_Int64/include/petscts.h
[ Info: Found dependent header: /Users/kausb/.julia/artifacts/a69b8f699a39f274f95752394759a3e9973e02b0/lib/petsc/double_real_Int64/include/petscsnes.h
[ Info: Found dependent header: /Users/kausb/.julia/artifacts/a69b8f699a39f274f95752394759a3e9973e02b0/lib/petsc/double_real_Int64/include/petscksp.h
[ Info: Found dependent header: /Users/kausb/.julia/artifacts/a69b8f699a39f274f95752394759a3e9973e02b0/lib/petsc/double_real_Int64/include/petscpc.h
[ Info: Found dependent header: /Users/kausb/.julia/artifacts/a69b8f699a39f274f95752394759a3e9973e02b0/lib/petsc/double_real_Int64/include/petscpctypes.h
[ Info: Found dependent header: /Users/kausb/.julia/artifacts/a69b8f699a39f274f95752394759a3e9973e02b0/lib/petsc/double_real_Int64/include/petscconvest.h
[ Info: Found dependent header: /Users/kausb/.julia/artifacts/a69b8f699a39f274f95752394759a3e9973e02b0/lib/petsc/double_real_Int64/include/petsctao.h
[ Info: Found dependent header: /Users/kausb/.julia/artifacts/a69b8f699a39f274f95752394759a3e9973e02b0/lib/petsc/double_real_Int64/include/petsctaolinesearch.h
[ Info: Found dependent header: /Users/kausb/.julia/artifacts/a69b8f699a39f274f95752394759a3e9973e02b0/lib/petsc/double_real_Int64/include/petsctao_deprecations.h
[ Info: Parsing headers...
/Users/kausb/.julia/artifacts/a69b8f699a39f274f95752394759a3e9973e02b0/lib/petsc/double_real_Int64/include/petscsys.h:98:6: error: "PETSc was configured with Open MPI but now appears to be compiling using a non-Open MPI mpi.h"
[ Info: Processing header: /Users/kausb/.julia/artifacts/a69b8f699a39f274f95752394759a3e9973e02b0/lib/petsc/double_real_Int64/include/petsc.h
[ Info: Building the DAG...
┌ Warning: [CollectDependentSystemNode]: found symbols in the system headers: [:MPI_Info, :MPI_Op, :MPI_Status, :MPI_Status, :MPI_Datatype, :off_t, :__darwin_off_t, :MPI_Comm, :MPI_Aint, :MPI_File, :ADIOI_FileD, :MPI_Request, :MPI_Group, :MPI_Offset, :MPI_Win]
└ @ Clang.Generators ~/.julia/packages/Clang/uJQ2L/src/generator/passes.jl:95
[ Info: Emit Julia expressions...
┌ Error: missing implementation for ExprNode{Clang.Generators.FunctionDefault, Clang.CLFunctionDecl}(:SNESObjectiveComputeFunctionDefaultFD, Clang.Generators.FunctionDefault(), CLCursor (Clang.CLFunctionDecl) SNESObjectiveComputeFunctionDefaultFD(SNES, Vec, Vec, void *), Expr[], Expr[], Int64[]), please file an issue to Clang.jl.
└ @ Clang.Generators ~/.julia/packages/Clang/uJQ2L/src/generator/codegen.jl:716
┌ Error: missing implementation for ExprNode{Clang.Generators.FunctionDefault, Clang.CLFunctionDecl}(:SNESComputeJacobianDefault, Clang.Generators.FunctionDefault(), CLCursor (Clang.CLFunctionDecl) SNESComputeJacobianDefault(SNES, Vec, Mat, Mat, void *), Expr[], Expr[], Int64[]), please file an issue to Clang.jl.
└ @ Clang.Generators ~/.julia/packages/Clang/uJQ2L/src/generator/codegen.jl:716
┌ Error: missing implementation for ExprNode{Clang.Generators.FunctionDefault, Clang.CLFunctionDecl}(:SNESComputeJacobianDefaultColor, Clang.Generators.FunctionDefault(), CLCursor (Clang.CLFunctionDecl) SNESComputeJacobianDefaultColor(SNES, Vec, Mat, Mat, void *), Expr[], Expr[], Int64[]), please file an issue to Clang.jl.
└ @ Clang.Generators ~/.julia/packages/Clang/uJQ2L/src/generator/codegen.jl:716
┌ Error: missing implementation for ExprNode{Clang.Generators.FunctionDefault, Clang.CLFunctionDecl}(:SNESPicardComputeFunction, Clang.Generators.FunctionDefault(), CLCursor (Clang.CLFunctionDecl) SNESPicardComputeFunction(SNES, Vec, Vec, void *), Expr[], Expr[], Int64[]), please file an issue to Clang.jl.
└ @ Clang.Generators ~/.julia/packages/Clang/uJQ2L/src/generator/codegen.jl:716
┌ Error: missing implementation for ExprNode{Clang.Generators.FunctionDefault, Clang.CLFunctionDecl}(:SNESPicardComputeMFFunction, Clang.Generators.FunctionDefault(), CLCursor (Clang.CLFunctionDecl) SNESPicardComputeMFFunction(SNES, Vec, Vec, void *), Expr[], Expr[], Int64[]), please file an issue to Clang.jl.
└ @ Clang.Generators ~/.julia/packages/Clang/uJQ2L/src/generator/codegen.jl:716
┌ Error: missing implementation for ExprNode{Clang.Generators.FunctionDefault, Clang.CLFunctionDecl}(:SNESPicardComputeJacobian, Clang.Generators.FunctionDefault(), CLCursor (Clang.CLFunctionDecl) SNESPicardComputeJacobian(SNES, Vec, Mat, Mat, void *), Expr[], Expr[], Int64[]), please file an issue to Clang.jl.
└ @ Clang.Generators ~/.julia/packages/Clang/uJQ2L/src/generator/codegen.jl:716
┌ Error: missing implementation for ExprNode{Clang.Generators.FunctionDefault, Clang.CLFunctionDecl}(:TSComputeRHSFunctionLinear, Clang.Generators.FunctionDefault(), CLCursor (Clang.CLFunctionDecl) TSComputeRHSFunctionLinear(TS, PetscReal, Vec, Vec, void *), Expr[], Expr[], Int64[]), please file an issue to Clang.jl.
└ @ Clang.Generators ~/.julia/packages/Clang/uJQ2L/src/generator/codegen.jl:716
┌ Error: missing implementation for ExprNode{Clang.Generators.FunctionDefault, Clang.CLFunctionDecl}(:TSComputeRHSJacobianConstant, Clang.Generators.FunctionDefault(), CLCursor (Clang.CLFunctionDecl) TSComputeRHSJacobianConstant(TS, PetscReal, Vec, Mat, Mat, void *), Expr[], Expr[], Int64[]), please file an issue to Clang.jl.
└ @ Clang.Generators ~/.julia/packages/Clang/uJQ2L/src/generator/codegen.jl:716
[ Info: [ProloguePrinter]: print to /Users/kausb/.julia/dev/PETSc.jl/gen/../lib/petsc_library.jl
[ Info: [GeneralPrinter]: print to /Users/kausb/.julia/dev/PETSc.jl/gen/../lib/petsc_library.jl
[ Info: [EpiloguePrinter]: print to /Users/kausb/.julia/dev/PETSc.jl/gen/../lib/petsc_library.jl
[ Info: Done!
"/Users/kausb/.julia/dev/PETSc.jl/gen/../lib/petsc_library.jl"

@boriskaus
Copy link
Collaborator

boriskaus commented Oct 16, 2024 •
edited
Loading

@ViralBShah
I've looked at the tests that are marked with FIXME, and most of the changes are due to PETSc changing the function calls (slightly) during more recent versions. We can port that manually but it probably makes more sense to try and autowrap the full library and use that instead (surely easier to maintain).

Also, this PR is very large. Perhaps it is simpler to start a new one from main?

@ViralBShah
Copy link
Member

Yes, the simpler thing would be to start the generator on a new PR in main, and just get that running first. And then piecemeal port things over to the new generated API. But not being a user of PETSc, that is where I will start running into my limitations here. Let's push one step forward at least.

I do feel that on main, it would be nice to fix all the FIXME things first so that everything that was working in the past is working again now, and tagged - giving a solid baseline for the new stuff.

@boriskaus
Copy link
Collaborator

I’m a PETSc user (well at least of part of the library). Yet I’m only a beginner when it comes to using clang to port it all.
I did notice in a recent project that it is possible to automatically make julia docs out of doxygen strings (which every routine in petsc has).
It would be fantastic if we can automatically provide help for all of the routines (even when the help is strictly speaking only valid for the c routines)

If you leave out the functions that have ccalls in the current main, there are only a limited number of additional functions remaining that are mainly there to make using petsc more julia-like. Those ones could be gradually expanded to include more functionally, but as long as we call autowrapped routines throughout it will be much easier to maintain this.

@ViralBShah
Copy link
Member

ViralBShah commented Oct 16, 2024 •
edited
Loading

So with your generated library, did you try doing include petsc_library.jl? Getting it to load cleanly is the first step. I assume you used the generator.toml that is in this branch (which has a lot of stuff to exclude and such).

I see what you are saying about the auto-generated comments for the wrappers. Wouldn't we still need the docs that are in the Julian interface as it stands right now - or are you thinking those could be completely replaced with this auto-generated stuff?

@boriskaus
Copy link
Collaborator

So with your generated library, did you try doing include petsc_library.jl? Getting it to load cleanly is the first step. I assume you used the generator.toml that is in this branch (which has a lot of stuff to exclude and such).

I didn’t manage to load it just now; I was using a simpler generator.toml shown in one of the comments

I see what you are saying about the auto-generated comments for the wrappers. Wouldn't we still need the docs that are in the Julian interface as it stands right now - or are you thinking those could be completely replaced with this auto-generated stuff?

I think it can be replaced with autogenerated stuff. Multiple dispatch will take care of the rest.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
5 participants
@jkozdon @psanan @vchuravy @ViralBShah @boriskaus