660 Commits

Author SHA1 Message Date
Niko Matsakis
248c752f62 extend the tutorial to explain lexing vs parsing 2017-04-04 06:24:38 -04:00
Niko Matsakis
5cb728e5bb Merge pull request #212 from nikomatsakis/fix-lane-table-unit-test
Fix lane table unit test
2017-04-03 11:22:01 -04:00
Niko Matsakis
5d30989369 ignore match_section.rs 2017-04-03 05:55:57 -04:00
Niko Matsakis
fa31cf265c stop using test.sh and instead use cargo test --all 2017-04-03 05:54:43 -04:00
Niko Matsakis
3b3c1719f7 break apart the unit tests into two 2017-04-03 05:54:43 -04:00
Niko Matsakis
5c0d15b40f lane table: add EOF as lookahead when reducing start terminal 2017-04-03 05:54:43 -04:00
Niko Matsakis
c489c805d0 print the set of things we are interpreting
makes debugging easier
2017-04-03 05:54:43 -04:00
Niko Matsakis
f924cb857d in Lane Table mode, we use fewer states 2017-04-03 05:54:43 -04:00
Niko Matsakis
dc08834f3a only add ! to the set of terminals of we use it 2017-04-03 05:54:43 -04:00
Niko Matsakis
007d6a810e Merge pull request #211 from nixpulvis/ignore-doc-generated-files
Remove and ignore generated files.
2017-04-01 21:10:58 -04:00
Nathan Lilienthal
5fb5c638bc Remove and ignore generated files. 2017-03-30 13:11:27 -07:00
Niko Matsakis
d17ea101c7 Merge pull request #206 from tildeio/match
`match` declaration for controlling precedence in the generated tokenizer
2017-03-30 05:24:33 -06:00
Niko Matsakis
6ee7355ec4 add various match tests 2017-03-30 05:18:08 -06:00
Niko Matsakis
4ebd265f86 silence some warnings that were driving me crazy 2017-03-30 05:18:08 -06:00
Niko Matsakis
243e9983cf add negative parsing to existing test 2017-03-30 05:18:08 -06:00
Niko Matsakis
e0725251d0 cleanup the fields in token-check further 2017-03-30 05:18:07 -06:00
Niko Matsakis
6fe7377c22 revamp how we tokenck and store the InternToken
In `InternToken`, we now coallesce everything into one
`Vec<MatchEntry>`, rather than using a vector and a map.  In the
token-check code, the various fields associated with a match are moved
into a struct.
2017-03-30 05:18:07 -06:00
Niko Matsakis
b75669c8d6 make the map/catch-all non-optional in validator 2017-03-30 05:18:07 -06:00
Niko Matsakis
61c2ce1522 use TerminalLiteral instead of TerminalString where appropriate
This avoids a lot of unwraps and panics. I did copy the precedence code
but I plan to refactor that shortly.
2017-03-30 05:18:07 -06:00
Peter Wagenet
9c1317508e implement customizable match section 2017-03-30 05:17:43 -06:00
Niko Matsakis
6f12cfe41b Merge pull request #208 from wieczyk/issue203_enable_constructing_lr1_states_via_envvar
Enabling lane-table-algorithm via environment variable
2017-03-29 07:50:57 -06:00
Pawel Wieczorek
8a0fae86b3 Enabling lane-table-algorithm via environment variable 2017-03-28 19:40:48 +02:00
Niko Matsakis
e9a06edd85 Merge pull request #207 from tildeio/upgrade-regex-syntax
Upgrade regex-syntax
2017-03-28 05:19:13 -04:00
Niko Matsakis
9e529089d3 convert panics into Err results for byte-based regex
We don't currently support byte-based regular expressions,
though it does seem like something that would be potentially
useful in the future.
2017-03-28 04:44:03 -04:00
Peter Wagenet
b12643d413 Upgrade regex-syntax
This fixes a bug where some regexes are not correctly printed as string literals.
2017-03-27 18:54:57 -07:00
Niko Matsakis
b78b2aaaa1 Merge pull request #198 from nikomatsakis/lane-table-algorithm
Further progress towards lane table algorithm
2017-03-23 06:09:21 -04:00
Niko Matsakis
301a2a23e2 remove unstable features 2017-03-23 05:59:05 -04:00
Niko Matsakis
dfb4040da1 Merge pull request #199 from federicomenaquintero/master
Fix for the README.md, and a small documentation fix
2017-03-23 05:16:58 -04:00
Federico Mena Quintero
c2118466de tutorial.md: Fix grammar and make a picture more legible 2017-03-21 21:02:38 -06:00
Federico Mena Quintero
c6bfeb80c3 tutorial: Use the same suggested version for the regex crate as in the toplevel README.md 2017-03-21 20:35:17 -06:00
Federico Mena Quintero
b64b1480b5 README.md: Mention a version of the regex crate that actually exists 2017-03-21 20:34:52 -06:00
Niko Matsakis
4cc5e50166 fix typo in the README 2017-03-21 05:15:03 -04:00
Niko Matsakis
a26cd51862 Merge pull request #197 from wieczyk/issue179_inconsistency_when_switching_parser_modes
parse table generator is now packing terminal parameters into tuple
2017-03-21 05:14:07 -04:00
Pawel Wieczorek
d6ebe3b0f0 Added comments explaining usage of artifical token Tok::Fraction 2017-03-19 15:49:38 +01:00
Pawel Wieczorek
baf722a403 parse table generator is now packing terminal parameters into tuple 2017-03-19 04:59:30 +01:00
Niko Matsakis
4ca00adf07 Merge pull request #196 from wieczyk/issue157_bracket_chars_are_perceived_as_part_of_syntax
Issue157 bracket chars are perceived as part of syntax
2017-03-18 07:57:05 -04:00
Pawel Wieczorek
040c76776e Added tests for error-cases for tokenizer 2017-03-17 22:32:23 +01:00
Niko Matsakis
e4f73c3757 try the large example, which revealed a few bugs 2017-03-17 06:19:46 -04:00
Niko Matsakis
b774a53fa7 add a comment 2017-03-17 05:15:12 -04:00
Niko Matsakis
25d89effd3 implement the rest of the lane table algorithm
Or, at least, enough to pass the "torturous" `paper_example_g1_build`
test.
2017-03-17 05:14:38 -04:00
Pawel Wieczorek
6fb8f7e8de Better coverage for new method of handling character literals in
tokenizer.
2017-03-16 19:49:50 +01:00
Niko Matsakis
7515fccef6 inefficient version of the core lane table algorithm
The missing piece is updating the lookaheads of the new states that wind
up getting created.
2017-03-16 08:54:47 -04:00
Pawel Wieczorek
ebfdb238f6 Initial fix for nikomatsakis/lalrpop#157 2017-03-16 00:25:37 +01:00
Niko Matsakis
5144980de5 some limited comments referring back to the README 2017-03-15 06:16:13 -04:00
Niko Matsakis
16c0cbad26 create a state set where each state maps to a ContextSet 2017-03-15 06:13:45 -04:00
Niko Matsakis
e6a2c29c1d add method rows() that creates a ContextSet for each state 2017-03-15 05:40:40 -04:00
Niko Matsakis
27e154f072 update README to describe The Plan 2017-03-10 06:14:02 -05:00
Niko Matsakis
416d3801d6 factor out the first-sets and state-graph
Rebuilding them is just silly
2017-03-08 05:44:32 -05:00
Niko Matsakis
e02bca0a92 add an initial version of the LR(1) construction algorithm
It only handles LALR(1) grammars right now.
2017-03-08 05:27:03 -05:00
Niko Matsakis
8203eac67c rewrite the lane table tracing to start with a conflicting action
This removes all the redundant columns due to shifts.
2017-03-07 05:59:03 -05:00