reason

Core Reason

Contributor Setup

With esy

# Make sure you have the latest esy
npm install -g esy@next
git clone https://github.com/facebook/reason.git
cd reason
esy

Testing:

Test Suite:

esy test # Run tests

One Off Tests:

Start up the rtop top level with your changes:

esy x rtop

Pipe some text to refmt with your changes:

echo "let a = 1" | esy x refmt

esy tips:

• esy x your command will run one command your command in an environment where the projects are built/installed. esy x which refmt will build the packages and install them for the duration of one command - which refmt. This will print the location of the built refmt binary.
• For more, see the esy documentation.

All the built binaries are in esy echo '#{self.target_dir}/install/default/bin'.

With opam

# On OSX, install opam via Homebrew:
brew update
brew install opam
# On Linux, see here (you will need opam >= 1.2.2): http://opam.ocaml.org/doc/Install.html

opam init
# Add this to your ~/.bashrc (or ~/.zshrc), then do `source ~/.bashrc`
#   eval $(opam config env)

opam update
opam switch 4.04.2
eval $(opam config env)
git clone https://github.com/facebook/reason.git
cd reason
opam pin add -y reason .
opam pin add -y rtop .

Opam Troubleshooting:

• Is the previous pinning unsuccessful? We might have updated a dependency; try opam update then opam upgrade.
• During the last opam pin step, make sure your local repo is clean. In particular, remove artifacts and node_modules. Otherwise the pinning might go stale or stall due to the big node_modules.

Repo Walkthrough

 (_Click to see a larger version)

Reason is the orange part. The core of the codebase is a parser + a printer, plus other miscellaneous utilities we expose.

Throughout the codebase, you might see mentions of “migrate-parsetree”, Ast_404, etc. These refer to https://github.com/let-def/ocaml-migrate-parsetree. It’s a library that allows you to convert between different versions of the OCaml AST. This way, the Reason repo can be written in OCaml 4.04’s AST data structures, while being usable on OCaml 4.02’s libraries (BuckleScript’s on 4.02 too).

The Reason lexer & parser use Menhir, a library that generates parsers. You can read more about Menhir here.

Core Files

• src/reason-parser/reason_lexer.mll: the lexer that chunks a raw string into tokens. See the file for more comments.

• src/reason-parser/reason_parser.mly: the parser that takes the lexer’s result and turns it into a proper AST (abstract syntax tree). See the file for more comments.

• src/reason-parser/reason_pprint_ast.ml: the pretty-printer! This is the reverse of parsing: it takes in the AST (abstract syntax tree) and prints out the nicely formatted code text.

• src/reason-parser/reason_parser.messages.checked-in: this is the huge table of mostly generated, sometimes hand-written, syntax error messages. When the parser ends up at an invalid parsing state (aka ends up with a syntax error), it’d refer to that file’s content and see if that case has a specific error message assigned to it. For an example fix, see this PR and the follow-up. To add a syntax error message see the “Add a Menhir Error Message” section below.
  • When running esy, and a new reason_parser.messages file is generated, do a mv reason_parser.messages reason_parser.messages.checked-in to persist the updated messages.

• src/reason-parser/reason_oprint.ml: the “outcome printer” used by Merlin, rtop and terminal, that prints the errors in Reason syntax. More info in the file itself.

• src/reason-parser/menhir_error_processor.ml, reason_parser_explain.ml: two files that allows us to batch assign a better syntax error message for a category of errors, like accidentally using a reserved token. More info in the comments of these files.

Miscellaneous Files

• ocamlmerlin_reason.ml: produces the ocamlmerlin-reason binary, used in conjunction with Merlin-extend. This is an extension to Merlin, which picks up this binary from your environment to analyze Reason files when your editor calls Merlin.

• *.mllib: related: see the OCaml extensions list. These are generated file from pkg/build.ml, which describes the package we distribute. No need to worry about them.

• src/reason-parser/reason_config.ml: global configuration that says whether the parser should run in “recoverable” mode. Merlin has a neat feature which lets it continue diagnosing e.g. type errors even when the file is syntactically invalid (at the expense of the accuracy of those type error reports’ quality). Searching reason_config in the codebase will show you how this is used.

• src/refmttype/reason_format_type.ml, reason_type_of_ocaml_type.ml: again, see pkg/build.ml. These produce the refmttype binary, used by BetterErrors to output compiler errors in Reason syntax rather than the OCaml one.

• src/reason-parser/reason_parser.messages: auto-generated from parser changes. Menhir generates parsing code that assigns each syntax error to a code, and lets us customize these errors. Syntax errors can be very precisely pinpointed and explained this way.

• src/reason-parser/reason_toolchain.ml, src/reason-parser/refmt_impl.ml: the entry point that calls the parsing logic.

• src/rtop/reason_utop.ml, src/rtop/reason_toploop.ml, src/rtop/rtop_init.ml: Reason’s Utop integration. Utop’s the terminal-based REPL you see when executing utop (in Reason’s case, the wrapper rtop).

• *.sh: some of the binaries’ entries.

• src/rtop/reason_util.ml, reason_syntax_util.ml: utils.

• src/reason-parser/reactjs_jsx_ppx_v2.ml: the ReactJS interop that translates Reason JSX into something that ReactJS understands. See the comments in the file and the description in ReasonReact.

• src/reason-parser-tests/testOprint.ml: unit tests for the outcome printer mentioned above. See the file for more info on how outcome printing is tested.

Working With Parser

Here’s a recommended workflow:

• First put your code in the current master syntax in a file test.re
• esy x refmt --print ast test.re
• look closely at the ast, spot the thing you need
• Search for your item in reason_parser.mly
• Change the logic
• esy test

Lexer helpers doc: http://caml.inria.fr/pub/docs/manual-ocaml/libref/Lexing.html Parser helper docs: http://caml.inria.fr/pub/docs/manual-ocaml/libref/Parsetree.html Menhir manual: http://gallium.inria.fr/~fpottier/menhir/manual.pdf Small Menhir example: https://github.com/derdon/menhir-example Random Stack Overflow answer: https://stackoverflow.com/questions/9897358/ocaml-menhir-compiling-writing (Ok seriously, we need some more Menhir examples. But hey, nobody said it was easy… for now!)

Want some example pull requests? Here are a few:

• Fix outcome printer object printing
• Add more spacing when printing Ptyp_package
• Implement spread for jsx3
• Make deref be a prefix operator
• Print MyConstructor(()) as MyConstructor()
• Ensure valid parsing of constraint expressions after printing
• Record punning for value & pattern for fields with module prefix
• Rage implement everything
• Print functions as javascript
• Transform ocaml pervasives interfaces to reason correctly
• Special case printing of foo(bar)##value
• Use ~ for named args
• Bring back parentheses-less switch foo
• Remove extra parens printed in type a = Foo((unit => unit))
• Don’t apply sugar to Js.t({.}) and Js.t({..})
• Pun record destructuring with renaming
• Add support for simple pattern direct argument with array, list & record
• Fix outcome printer record value printing
• Printfoo(()) as foo() + update parser
• Allow parsing of constraint expressions without parens inside constructor expr
• Don’t print fun in record expressions with Pexp_fun values
• Force breaks for nested records
• Always break object def with two or more rows
• Make exponentiation operator print with right associativity

Debugging Grammar Conflicts

Run the main parser through Menhir with the --explain flag to have it print out details about the conflict. esy menhir --explain src/reason-parser/reason_parser.mly. The debug information can be found at src/reason-parser/reason_parser.conflicts.

Use the --dump flag to have Menhir print the state transitions for debugging the parser rules that are applied esy menhir --dump src/reason-parser/reason_parser.mly It will generate src/reason-parser/reason_parser.automaton which you can inspect. It will also drop a reason_parser.ml and reason_parser.mli in that directory which you need to remove before building again.

Debugging the Lexer/Parser State at Runtime

Add the --trace flag to the end of the menhir (flags..) section in the dune file for the reason-parser, and it will print out tokens that are lexed along with parser state transitions.

Add a Menhir Error Message

To add a Menhir error message, you first need to know the error code. To find the error code, you can run the following commands from the Reason project root:

esy x refmt --parse re foo.re

Where foo.re contains a syntax error. This will result in an error message like:

File "test2.re", line 4, characters 2-6:
Error: 2665: <syntax error>

Here, the error code is 2665. We then search for this code in src/reason-parser/reason_parser.messages.

• If you find it, you can add a better error message instead of the not so descriptive <syntax error>.

To test the new error message you can run the following commands again:

esy x refmt --parse re foo.re

Then submit a PR!

Improve Error Message Locations

In some situations, Reason might report errors in a different place than where it occurs. This is caused by the AST not having a correct location for a node. When the error reports a location that’s simply at the top of the file, that means we likely didn’t attach the location in the parser correctly, altogether.

Before digging into Reason parser, make sure this isn’t actually caused by some PPX. Otherwise, run:

esy x refmt --parse re --print ast test.re

Where test.re has the code that produces the error message at the wrong location.

In the printed AST, you can see nodes such as Ptyp_constr "int" (test.re[1,0+15]..[1,0+18]) where the part between parentheses is the location of the node.

The error message’s own location will look like ([0,0+-1]..[0,0+-1]) too.

To fix this, we need to find the AST node in src/reason-parser/reason_parser.mly. It’s a big file, but if you search for the AST node, you should be able to find the location (if not, bug us on Discord). It will probably involve a mkexp or mkpat without a proper ~loc attached to it.

As you can see from other parts in the parser, many do have a ~loc assigned to it. For example

| LIDENT jsx_arguments
      {
        (* a (punning) *)
        let loc_lident = mklocation $startpos($1) $endpos($1) in
        [($1, mkexp (Pexp_ident {txt = Lident $1; loc = loc_lident}) ~loc:loc_lident)] @ $2
      }

Testing Two Different Syntax Versions

If you’d like to convert from an old Reason syntax version to the one in master (whether to debug things, or if you’re interested in trying out some syntax changes from master and then explaining to us why your perspective on the Reason syntax is the best, lol):

• Revert the repo to the old commit you want
• Build, through esy
• Move the built refmt binary esy x which refmt somewhere else
• Revert back to master
• esy x which refmt again to get the master binary.

Then do:

esy x refmt --parse my_old_syntax_file.re --print binary_reason | ./refmt_impl --parse binary_reason --print re

Basically, turn your old syntax into an AST (which is resilient to syntax changes), then turn it back into the new, textual code. If you’re reverting to an old enough version, the old binary’s flags and/or the old build instructions might be different. In that case, see esy x refmt --help and/or the old README.

This site is open source. Improve this page.