657 Commits

Author SHA1 Message Date
Niko Matsakis
fa31cf265c stop using test.sh and instead use cargo test --all 2017-04-03 05:54:43 -04:00
Niko Matsakis
3b3c1719f7 break apart the unit tests into two 2017-04-03 05:54:43 -04:00
Niko Matsakis
5c0d15b40f lane table: add EOF as lookahead when reducing start terminal 2017-04-03 05:54:43 -04:00
Niko Matsakis
c489c805d0 print the set of things we are interpreting
makes debugging easier
2017-04-03 05:54:43 -04:00
Niko Matsakis
f924cb857d in Lane Table mode, we use fewer states 2017-04-03 05:54:43 -04:00
Niko Matsakis
dc08834f3a only add ! to the set of terminals of we use it 2017-04-03 05:54:43 -04:00
Niko Matsakis
007d6a810e Merge pull request #211 from nixpulvis/ignore-doc-generated-files
Remove and ignore generated files.
2017-04-01 21:10:58 -04:00
Nathan Lilienthal
5fb5c638bc Remove and ignore generated files. 2017-03-30 13:11:27 -07:00
Niko Matsakis
d17ea101c7 Merge pull request #206 from tildeio/match
`match` declaration for controlling precedence in the generated tokenizer
2017-03-30 05:24:33 -06:00
Niko Matsakis
6ee7355ec4 add various match tests 2017-03-30 05:18:08 -06:00
Niko Matsakis
4ebd265f86 silence some warnings that were driving me crazy 2017-03-30 05:18:08 -06:00
Niko Matsakis
243e9983cf add negative parsing to existing test 2017-03-30 05:18:08 -06:00
Niko Matsakis
e0725251d0 cleanup the fields in token-check further 2017-03-30 05:18:07 -06:00
Niko Matsakis
6fe7377c22 revamp how we tokenck and store the InternToken
In `InternToken`, we now coallesce everything into one
`Vec<MatchEntry>`, rather than using a vector and a map.  In the
token-check code, the various fields associated with a match are moved
into a struct.
2017-03-30 05:18:07 -06:00
Niko Matsakis
b75669c8d6 make the map/catch-all non-optional in validator 2017-03-30 05:18:07 -06:00
Niko Matsakis
61c2ce1522 use TerminalLiteral instead of TerminalString where appropriate
This avoids a lot of unwraps and panics. I did copy the precedence code
but I plan to refactor that shortly.
2017-03-30 05:18:07 -06:00
Peter Wagenet
9c1317508e implement customizable match section 2017-03-30 05:17:43 -06:00
Niko Matsakis
6f12cfe41b Merge pull request #208 from wieczyk/issue203_enable_constructing_lr1_states_via_envvar
Enabling lane-table-algorithm via environment variable
2017-03-29 07:50:57 -06:00
Pawel Wieczorek
8a0fae86b3 Enabling lane-table-algorithm via environment variable 2017-03-28 19:40:48 +02:00
Niko Matsakis
e9a06edd85 Merge pull request #207 from tildeio/upgrade-regex-syntax
Upgrade regex-syntax
2017-03-28 05:19:13 -04:00
Niko Matsakis
9e529089d3 convert panics into Err results for byte-based regex
We don't currently support byte-based regular expressions,
though it does seem like something that would be potentially
useful in the future.
2017-03-28 04:44:03 -04:00
Peter Wagenet
b12643d413 Upgrade regex-syntax
This fixes a bug where some regexes are not correctly printed as string literals.
2017-03-27 18:54:57 -07:00
Niko Matsakis
b78b2aaaa1 Merge pull request #198 from nikomatsakis/lane-table-algorithm
Further progress towards lane table algorithm
2017-03-23 06:09:21 -04:00
Niko Matsakis
301a2a23e2 remove unstable features 2017-03-23 05:59:05 -04:00
Niko Matsakis
dfb4040da1 Merge pull request #199 from federicomenaquintero/master
Fix for the README.md, and a small documentation fix
2017-03-23 05:16:58 -04:00
Federico Mena Quintero
c2118466de tutorial.md: Fix grammar and make a picture more legible 2017-03-21 21:02:38 -06:00
Federico Mena Quintero
c6bfeb80c3 tutorial: Use the same suggested version for the regex crate as in the toplevel README.md 2017-03-21 20:35:17 -06:00
Federico Mena Quintero
b64b1480b5 README.md: Mention a version of the regex crate that actually exists 2017-03-21 20:34:52 -06:00
Niko Matsakis
4cc5e50166 fix typo in the README 2017-03-21 05:15:03 -04:00
Niko Matsakis
a26cd51862 Merge pull request #197 from wieczyk/issue179_inconsistency_when_switching_parser_modes
parse table generator is now packing terminal parameters into tuple
2017-03-21 05:14:07 -04:00
Pawel Wieczorek
d6ebe3b0f0 Added comments explaining usage of artifical token Tok::Fraction 2017-03-19 15:49:38 +01:00
Pawel Wieczorek
baf722a403 parse table generator is now packing terminal parameters into tuple 2017-03-19 04:59:30 +01:00
Niko Matsakis
4ca00adf07 Merge pull request #196 from wieczyk/issue157_bracket_chars_are_perceived_as_part_of_syntax
Issue157 bracket chars are perceived as part of syntax
2017-03-18 07:57:05 -04:00
Pawel Wieczorek
040c76776e Added tests for error-cases for tokenizer 2017-03-17 22:32:23 +01:00
Niko Matsakis
e4f73c3757 try the large example, which revealed a few bugs 2017-03-17 06:19:46 -04:00
Niko Matsakis
b774a53fa7 add a comment 2017-03-17 05:15:12 -04:00
Niko Matsakis
25d89effd3 implement the rest of the lane table algorithm
Or, at least, enough to pass the "torturous" `paper_example_g1_build`
test.
2017-03-17 05:14:38 -04:00
Pawel Wieczorek
6fb8f7e8de Better coverage for new method of handling character literals in
tokenizer.
2017-03-16 19:49:50 +01:00
Niko Matsakis
7515fccef6 inefficient version of the core lane table algorithm
The missing piece is updating the lookaheads of the new states that wind
up getting created.
2017-03-16 08:54:47 -04:00
Pawel Wieczorek
ebfdb238f6 Initial fix for nikomatsakis/lalrpop#157 2017-03-16 00:25:37 +01:00
Niko Matsakis
5144980de5 some limited comments referring back to the README 2017-03-15 06:16:13 -04:00
Niko Matsakis
16c0cbad26 create a state set where each state maps to a ContextSet 2017-03-15 06:13:45 -04:00
Niko Matsakis
e6a2c29c1d add method rows() that creates a ContextSet for each state 2017-03-15 05:40:40 -04:00
Niko Matsakis
27e154f072 update README to describe The Plan 2017-03-10 06:14:02 -05:00
Niko Matsakis
416d3801d6 factor out the first-sets and state-graph
Rebuilding them is just silly
2017-03-08 05:44:32 -05:00
Niko Matsakis
e02bca0a92 add an initial version of the LR(1) construction algorithm
It only handles LALR(1) grammars right now.
2017-03-08 05:27:03 -05:00
Niko Matsakis
8203eac67c rewrite the lane table tracing to start with a conflicting action
This removes all the redundant columns due to shifts.
2017-03-07 05:59:03 -05:00
Niko Matsakis
c7ba5efd79 convert the lane-table construction to always use LR0 internally
It's more efficient.
2017-03-07 05:48:59 -05:00
Niko Matsakis
18b775112d start writing up how the lane table stuff works
This is my best effort to supplement the existing paper and also just
document my own personal understanding.
2017-03-07 05:11:41 -05:00
Niko Matsakis
68532b5439 make lane table construction generic in lookahead L 2017-03-06 06:48:40 -05:00