Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
63 commits
Select commit Hold shift + click to select a range
e6c13b4
Init customizable FAST importer + add helpers in TSFASTImpoter
jecisc Feb 18, 2026
f6462fb
Add some helper methods to customize a FASTTree
jecisc Feb 19, 2026
a216481
Reify context entry
jecisc Feb 19, 2026
029335e
Speed up FAST import by improving the positions management
jecisc Feb 19, 2026
b001feb
Fix typo
jecisc Feb 20, 2026
8e9a16b
Add an error management to not stop the visit of a node in case an er…
jecisc Feb 20, 2026
2217b1c
Add a way to alias fields names to match a FM property with a differe…
jecisc Feb 22, 2026
e01ec00
Add some hooks to better set the parent entity
jecisc Feb 22, 2026
a2eaf57
Add utility method
jecisc Feb 24, 2026
85b683e
Speed up TSSymbolsBuilderVisitor
jecisc Feb 24, 2026
0754b8c
Add next context customizations
jecisc Feb 25, 2026
2595b8a
Some customizations to make the customizable importer more powerful +…
jecisc Feb 26, 2026
d3bbc4a
Speed up import by caching the add selectors
jecisc Feb 26, 2026
fbb097b
Add error report management
jecisc Feb 27, 2026
757bdb4
Add missing setter
jecisc Feb 27, 2026
9034bf7
Add possibility to change the branch to use
jecisc Feb 27, 2026
cf81d65
Add utility method
jecisc Mar 2, 2026
d5a0bd6
Fix API used.
jecisc Mar 4, 2026
4d9c0c1
Rename API after discussion with Clotilde
jecisc Mar 9, 2026
1ff5fd4
Improve TSSymbolExplorerPresenter to display number of children by fi…
jecisc Mar 9, 2026
33cf091
TSSymbolsBuilderVisitor should not crash on non UTF8 files
jecisc Mar 10, 2026
fb733d0
Add TSNodeFinderVisitor
jecisc Mar 13, 2026
d980166
Merge branch 'main' into customizableimporter
jecisc Mar 13, 2026
7197c8b
Fix bug when the node found is not in the last file
jecisc Mar 13, 2026
6bde300
Merge d9801668f2a6212dab899bd4186aaf711c9463ea
jecisc Mar 13, 2026
ebba2c4
Add utility method
jecisc Mar 16, 2026
1d005d9
Remove halt I deft
jecisc Mar 17, 2026
5b6fe33
Reduce the number of encoder instantiations
jecisc Mar 20, 2026
bb30f53
Speed up import by disabling some checks that should not fail
jecisc Mar 20, 2026
4d03503
Speed up a lot #sourceCodeOf:
jecisc Mar 20, 2026
25e638c
Fix bad start byte
jecisc Mar 20, 2026
7a4d91d
Rename importers to visitors
jecisc Mar 25, 2026
53aa91d
Introduce abstract importer.
jecisc Mar 25, 2026
a5a120b
Start to work on doc
jecisc Mar 25, 2026
fb19973
Add documentation on utilities
jecisc Mar 25, 2026
a416de7
Start doc on FAST utils
jecisc Mar 25, 2026
6157450
Add doc on FAST builder
jecisc Mar 25, 2026
3b6a766
Merge a416de7f2e889df0c6a7897df8e7c4be46f9b409
jecisc Mar 25, 2026
1806740
Add doc on MM generation
jecisc Mar 25, 2026
30aa0aa
Add some utilities
jecisc Mar 25, 2026
ef057c9
Add doc on import
jecisc Mar 25, 2026
e467268
Improve english
jecisc Mar 25, 2026
c439437
Merge remote-tracking branch 'mine/customizableimporter' into customi…
jecisc Mar 25, 2026
9535a78
Begin mm customization section
jecisc Mar 25, 2026
210c4a7
Progress on model customization
jecisc Mar 25, 2026
6013670
Polishing of doc
jecisc Mar 25, 2026
3a8e3eb
Add doc on field aliasing
jecisc Mar 25, 2026
f1d714c
Add empty sections to remember what to document
jecisc Mar 25, 2026
85624f4
Finish (I hope?) documentation on the customizable visitor
jecisc Mar 26, 2026
ee88ed9
Add tests utilities
jecisc Mar 26, 2026
c5142b1
Merge 85624f4e27a8577b8cec4658ada06c4913632083
jecisc Mar 26, 2026
ee56761
Add doc on tests
jecisc Mar 26, 2026
6895947
Doc on error handling
jecisc Mar 26, 2026
9ab6873
Finish first pass on the doc
jecisc Mar 26, 2026
3d78f53
Merge branch 'main' into customizableimporter
jecisc Mar 26, 2026
bc5d042
Comment
jecisc Mar 26, 2026
c0022b1
Merge 3d78f531607882af000024b2107f3861d1891977
jecisc Mar 26, 2026
1ad2862
Use #newTenured
jecisc Mar 26, 2026
c489616
Add Pharo 14 to the CI
jecisc Mar 26, 2026
ee1d7de
TSFASTBuilder should use #newClassNamed: instead of #ensureClassNamed:
jecisc Mar 27, 2026
ca4b477
Merge c4896160456f1260c81ece27986d232d1301d908
jecisc Mar 27, 2026
1623015
Generate the importer in TSFASTBuilder
jecisc Mar 27, 2026
c06b2fe
Update doc
jecisc Mar 27, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ jobs:
runs-on: ubuntu-latest
strategy:
matrix:
smalltalk: [ Pharo64-13 ]
smalltalk: [ Pharo64-13, Pharo64-14 ]
name: ${{ matrix.smalltalk }}
steps:
- uses: actions/checkout@v4
Expand Down
72 changes: 54 additions & 18 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,30 @@
This project provides **Pharo bindings to [Tree-sitter](https://tree-sitter.github.io/tree-sitter/)**.
It allows you to analyze and highlight code using existing Tree-sitter parsers developed by the community.


<!-- TOC -->

- [Pharo Tree-Sitter](#pharo-tree-sitter)
- [Installation](#installation)
- [Native Libraries Required](#native-libraries-required)
- [Automated Library Generation](#automated-library-generation)
- [How It Works](#how-it-works)
- [Required Tools for MacOS and Linux](#required-tools-for-macos-and-linux)
- [Required Tools for Windows](#required-tools-for-windows)
- [Manual Installation](#manual-installation)
- [macOS](#macos)
- [Building Tree-sitter Shared Library](#building-tree-sitter-shared-library)
- [Archlinux](#archlinux)
- [Quick Example](#quick-example)
- [Moose: TreeSitter to build FAST and Famix models](#moose-treesitter-to-build-fast-and-famix-models)
- [Utilities and Debugging](#utilities-and-debugging)
- [Pre-built Libraries (Temporary Solution)](#pre-built-libraries-temporary-solution)
- [Create a new FASTLanguageMetamodel using PharoTreeSitter](#create-a-new-fastlanguagemetamodel-using-pharotreesitter)

<!-- /TOC -->

## Installation

To install it in **Pharo**, run the following script in a Playground:

```smalltalk
Expand All @@ -16,6 +40,14 @@ Metacello new
load.
```

To add this project to your baseline you can copy paste:

```Smalltalk
spec
baseline: 'TreeSitter'
with: [ spec repository: 'github://Evref-BL/Pharo-Tree-Sitter:main/src' ]
```

## Native Libraries Required

To function correctly, the following native libraries are required depending on your OS:
Expand Down Expand Up @@ -131,24 +163,6 @@ So you only have to install tree-sitter and the grammar you wanna use with your
yay tree-sitter tree-sitter-grammars
```


## Debug

You might want to debug a tree sitter parser from Pharo.
Whereas we did not find _yet_ a way to use the pharo debugger.
You can create a logger attached to the parser.

```st
callback := (TSLogCallback on: [ :payload :log_type :buffer | Transcript crShow: buffer ] ).
logger := TSLogger new log: callback .
parser logger: logger.
```

## Pre-built Libraries (Temporary Solution)

If you face issues building the libraries, you can download pre-built libraries (for Windows and macOS) here:
https://doi.org/10.5281/zenodo.15423234

## Quick Example

Here is a basic usage example in Pharo:
Expand All @@ -161,6 +175,28 @@ tree := parser parseString: string.
tree rootNode
```

## Moose: TreeSitter to build FAST and Famix models

[`Moose`](https://github.com/moosetechnology/Moose)is a software analysis platform. It can manage two kinds of models:
- Famix been a dependency model
- FAST (Famix AST) been an AST representation of the language

It exist some utilities to help building a Famix model for a language supported by TreeSitter. You can find this project here: [https://github.com/moosetechnology/TreeSitterFamixIntegration](https://github.com/moosetechnology/TreeSitterFamixIntegration)

And this project provides a way to easily create a FAST importer. You can find the documentation here: [Documentation](resources/doc/fast_importer.md)

## Utilities and Debugging

This project provides a few utilities to help you debug a tree sitter parser or to help you with its manipulation.

Those utilises are documented here: [Documentation](resources/doc/ts_utilities.md)

## Pre-built Libraries (Temporary Solution)

If you face issues building the libraries, you can download pre-built libraries (for Windows and macOS) here:
https://doi.org/10.5281/zenodo.15423234


# Create a new FASTLanguageMetamodel using PharoTreeSitter

This library allows you to create a first version of any FASTLanguageMetamodel, in condition that this language is supported by treesitter and by PharoTreeSitter. To do that you can follow a detailed documentation [here](https://modularmoose.org/blog/2025-09-16-generation-of-fast-metamodel-using-treesitter/).
Expand Down
Binary file added resources/doc/TSNodeFinderVisitor.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added resources/doc/TSSymbolsBuilderVisitor.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added resources/doc/basicMM.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Loading