Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update + Add Syntaxes #3

Open
wants to merge 19 commits into
base: master
Choose a base branch
from
Open

Update + Add Syntaxes #3

wants to merge 19 commits into from

Conversation

CrossR
Copy link
Member

@CrossR CrossR commented Dec 30, 2019

This bumps us up to the latest master of tree-sitter.

I've also added CPP, JS, TS, TSX and Python parsers. I can add more later, but wanted to check these were actually added correctly.

Steps taken:

  • Copy over the whole repo to update it.
  • Copy over the changes made to the build-lib script to bring back our changes.
  • Clone each parser repo, npm install and copy over the parser.c.
  • If needed, add to build-lib build step and ar step.

Still WIP as I've not tested with the upstream reason binds yet.

@CrossR
Copy link
Member Author

CrossR commented Dec 30, 2019

Hm, possibly something not quite right in this repo.

Builds fine with a local link against the reason binds, but trying to build esy @test gives me

Undefined symbols for architecture x86_64:
  "_tree_sitter_cpp_external_scanner_create", referenced from:
      _tree_sitter_cpp.language in libtree-sitter.a(cpp_parser.o)
  "_tree_sitter_cpp_external_scanner_deserialize", referenced from:
      _tree_sitter_cpp.language in libtree-sitter.a(cpp_parser.o)
  "_tree_sitter_cpp_external_scanner_destroy", referenced from:
      _tree_sitter_cpp.language in libtree-sitter.a(cpp_parser.o)
  "_tree_sitter_cpp_external_scanner_scan", referenced from:
      _tree_sitter_cpp.language in libtree-sitter.a(cpp_parser.o)
  "_tree_sitter_cpp_external_scanner_serialize", referenced from:
      _tree_sitter_cpp.language in libtree-sitter.a(cpp_parser.o)
  "_tree_sitter_js", referenced from:
      _rets_parser_new_js in libtreesitter_stubs.a(bindings.o)
     (maybe you meant: _tree_sitter_json)
  "_tree_sitter_python_external_scanner_create", referenced from:
      _tree_sitter_python.language in libtree-sitter.a(python_parser.o)
  "_tree_sitter_python_external_scanner_deserialize", referenced from:
      _tree_sitter_python.language in libtree-sitter.a(python_parser.o)
  "_tree_sitter_python_external_scanner_destroy", referenced from:
      _tree_sitter_python.language in libtree-sitter.a(python_parser.o)
  "_tree_sitter_python_external_scanner_scan", referenced from:
      _tree_sitter_python.language in libtree-sitter.a(python_parser.o)
  "_tree_sitter_python_external_scanner_serialize", referenced from:
      _tree_sitter_python.language in libtree-sitter.a(python_parser.o)
  "_tree_sitter_ts", referenced from:
      _rets_parser_new_ts in libtreesitter_stubs.a(bindings.o)
     (maybe you meant: _tree_sitter_tsx)
  "_tree_sitter_tsx_external_scanner_create", referenced from:
      _tree_sitter_tsx.language in libtree-sitter.a(tsx_parser.o)
  "_tree_sitter_tsx_external_scanner_deserialize", referenced from:
      _tree_sitter_tsx.language in libtree-sitter.a(tsx_parser.o)
  "_tree_sitter_tsx_external_scanner_destroy", referenced from:
      _tree_sitter_tsx.language in libtree-sitter.a(tsx_parser.o)
  "_tree_sitter_tsx_external_scanner_scan", referenced from:
      _tree_sitter_tsx.language in libtree-sitter.a(tsx_parser.o)
  "_tree_sitter_tsx_external_scanner_serialize", referenced from:
      _tree_sitter_tsx.language in libtree-sitter.a(tsx_parser.o)
ld: symbol(s) not found for architecture x86_64

Need to actually understand that and work out if its an API change (and if so, where to change that) or if its a build issue.

EDIT: Ah! Everyone of the new grammars includes a scanner.c(c) file alongside it, which I wasn't including. Notes on what they are here. I assume that I also need to be including them, as they are referenced in the .external_scanner part of the languages. I also had to include a shared scanner header used in the TypeScript parser scanner.

These are needed for these more complex parsers.
@CrossR
Copy link
Member Author

CrossR commented Dec 30, 2019

Looks like this might need to be a bit more involved actually, since we currently build with gcc, but some of these scanner files are written in C++: tree-sitter/tree-sitter-python#60

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant