Branch: Tag:

2013-04-21

2013-04-21 12:36:58 by Henrik Grubbström (Grubba) <grubba@grubba.org>

Merge pike-modules/Tokenizer into grubba/tokenizer

This merges the old experimental Tokenizer module into current Pike 7.9.

* /home/grubba/src/cvsmigration/Tokenizer: (72 commits)

2006-10-03

2006-10-03 16:47:34 by Henrik Grubbström (Grubba) <grubba@grubba.org>

Implemented BreakNoLabel in C.

Rev: src/modules/Tokenizer/Tokenizer.cmod:1.37
Rev: src/modules/Tokenizer/module.pmod.in:1.37

2006-09-30

2006-09-30 14:33:47 by Henrik Grubbström (Grubba) <grubba@grubba.org>

Implemented more stuff in C. Removed some pike code that had already been implemented in C.

Rev: src/modules/Tokenizer/Tokenizer.cmod:1.35
Rev: src/modules/Tokenizer/module.pmod.in:1.36

2004-12-16

2004-12-16 16:22:02 by Henrik Grubbström (Grubba) <grubba@grubba.org>

Removed or replaced with "->is_group" all use of "->group_start" as appropriate.
Replaced all use of "->contents" with "->get_contents()".
Note: There are now lots of redundant ->is_group checks.
Note: May want to cache the return value from ->get_contents() is several places.

Rev: src/modules/Tokenizer/module.pmod.in:1.35

2004-12-03

2004-12-03 15:32:12 by Henrik Grubbström (Grubba) <grubba@grubba.org>

Changed all ->`+=(1) to ->next().
Moved Scope class.
Separated parse_meta_identifier() to parse_meta_function().
Updated for the new *Group classes.
Added For and Sscanf meta programs.

Rev: src/modules/Tokenizer/module.pmod.in:1.34

2004-11-28

2004-11-28 19:39:24 by Henrik Grubbström (Grubba) <grubba@grubba.org>

Updated to the new Group/EndGroup API.

Rev: src/modules/Tokenizer/module.pmod.in:1.33

2004-09-08

2004-09-08 19:20:58 by Henrik Grubbström (Grubba) <grubba@grubba.org>

Added Case.

Rev: src/modules/Tokenizer/module.pmod.in:1.32

2004-09-07

2004-09-07 17:01:30 by Henrik Grubbström (Grubba) <grubba@grubba.org>

Added meta programs for Dot and ColonColon.
Added support for adjacent string concat.

Rev: src/modules/Tokenizer/module.pmod.in:1.31

2004-09-07 16:15:12 by Henrik Grubbström (Grubba) <grubba@grubba.org>

Added a lot of symbols to Pike_0_1_Tokens.

Rev: src/modules/Tokenizer/module.pmod.in:1.30

2004-09-04

2004-09-04 19:10:35 by Henrik Grubbström (Grubba) <grubba@grubba.org>

Added TokenList()->_sizeof().
Iterator(Tokenlist)->insert() no longer advances the current position.
Various parser fixes.
Added some debug code.
Splited parse_meta_program() into two, with the second being parse_meta_identifier().
Added support for parsing of soft casts, splice and return.
Improved several error messages.

Rev: src/modules/Tokenizer/module.pmod.in:1.29

2004-09-03

2004-09-03 12:04:03 by Henrik Grubbström (Grubba) <grubba@grubba.org>

Various parser fixes.

Rev: src/modules/Tokenizer/module.pmod.in:1.28

2003-12-28

2003-12-28 15:50:03 by Henrik Grubbström (Grubba) <grubba@grubba.org>

Updated TokenHandler type to confirm to the new report() API.
Removed Pike code for classes now implemented in the cmod.
Updated various metaprograms to know about the pos ==> context change.
Now compiles again.

Rev: src/modules/Tokenizer/module.pmod.in:1.27

2003-12-01

2003-12-01 14:33:03 by Henrik Grubbström (Grubba) <grubba@grubba.org>

Various fixes due to new naming conventions.
Started implementing TokenizingCompiler in C.

Rev: src/modules/Tokenizer/Tokenizer.cmod:1.11
Rev: src/modules/Tokenizer/module.pmod.in:1.26

2003-11-28

2003-11-28 16:49:38 by Henrik Grubbström (Grubba) <grubba@grubba.org>

{Literal,String,Integer,Float}Token are now implemented in C.

Rev: src/modules/Tokenizer/Tokenizer.cmod:1.9
Rev: src/modules/Tokenizer/module.pmod.in:1.25

2003-11-28 14:33:10 by Henrik Grubbström (Grubba) <grubba@grubba.org>

Moved some more stuff from Pike to C.

Rev: src/modules/Tokenizer/Tokenizer.cmod:1.8
Rev: src/modules/Tokenizer/module.pmod.in:1.24

2003-11-27

2003-11-27 16:50:03 by Henrik Grubbström (Grubba) <grubba@grubba.org>

Updated for the new basic token.

Rev: src/modules/Tokenizer/module.pmod.in:1.23

2003-10-31

2003-10-31 14:56:59 by Henrik Grubbström (Grubba) <grubba@grubba.org>

Added {insert,append}() to the TokenList iterator.
Fixed typo.
Added parsing of various literal types.
Added Break and BreakNoLabel.

Rev: src/modules/Tokenizer/module.pmod.in:1.22

2003-10-07

2003-10-07 14:43:19 by Henrik Grubbström (Grubba) <grubba@grubba.org>

Changed Scope interface.

Rev: src/modules/Tokenizer/module.pmod.in:1.21

2003-09-05

2003-09-05 16:44:02 by Henrik Grubbström (Grubba) <grubba@grubba.org>

Now utilizes the new foreach behaviour.

Rev: src/modules/Tokenizer/module.pmod.in:1.20

2003-09-05 16:25:51 by Henrik Grubbström (Grubba) <grubba@grubba.org>

Added parsing of namespace directives.

Rev: src/modules/Tokenizer/module.pmod.in:1.19

2003-09-05 12:41:47 by Henrik Grubbström (Grubba) <grubba@grubba.org>

Added lambda.

Rev: src/modules/Tokenizer/module.pmod.in:1.18

2003-07-23

2003-07-23 16:00:42 by Henrik Grubbström (Grubba) <grubba@grubba.org>

Added a few more @ignore.

Rev: src/modules/Tokenizer/module.pmod.in:1.17

2003-07-03

2003-07-03 14:57:47 by Henrik Grubbström (Grubba) <grubba@grubba.org>

Added parsing of if and else.
Removed some obsolete code.
Added some provisions for code generation.

Rev: src/modules/Tokenizer/module.pmod.in:1.16

2003-06-01

2003-06-01 15:33:43 by Henrik Grubbström (Grubba) <grubba@grubba.org>

Most operators are now mapped to calls of their predef variants.

Rev: src/modules/Tokenizer/module.pmod.in:1.15

2003-05-31

2003-05-31 17:10:49 by Henrik Grubbström (Grubba) <grubba@grubba.org>

Added parsing of foreach.

Rev: src/modules/Tokenizer/module.pmod.in:1.14

2003-05-31 15:55:23 by Henrik Grubbström (Grubba) <grubba@grubba.org>

Added support for implicit lambdas.
Fixed parsing of the ..-operator.

Rev: src/modules/Tokenizer/module.pmod.in:1.13

2003-05-30

2003-05-30 14:42:34 by Henrik Grubbström (Grubba) <grubba@grubba.org>

Added some support for scopes, and variable declarations.

Rev: src/modules/Tokenizer/module.pmod.in:1.12

2003-04-27

2003-04-27 15:44:01 by Henrik Grubbström (Grubba) <grubba@grubba.org>

Changed semantics of parse_decl() somewhat.

Rev: src/modules/Tokenizer/module.pmod.in:1.11

2003-04-26

2003-04-26 16:23:27 by Henrik Grubbström (Grubba) <grubba@grubba.org>

Some support for parsing of statements.

Rev: src/modules/Tokenizer/module.pmod.in:1.10

2003-04-20

2003-04-20 16:21:55 by Henrik Grubbström (Grubba) <grubba@grubba.org>

Large parts of the expression parser seems to work.

Rev: src/modules/Tokenizer/module.pmod.in:1.9

2003-04-17

2003-04-17 15:19:34 by Henrik Grubbström (Grubba) <grubba@grubba.org>

Begun support for parsing of expressions.

Rev: src/modules/Tokenizer/module.pmod.in:1.8

2003-04-13

2003-04-13 19:33:15 by Henrik Grubbström (Grubba) <grubba@grubba.org>

More token lookup fixes.

Rev: src/modules/Tokenizer/module.pmod.in:1.7

2003-04-12

2003-04-12 15:06:52 by Henrik Grubbström (Grubba) <grubba@grubba.org>

Now knows more about when reserved words showed up.

Rev: src/modules/Tokenizer/module.pmod.in:1.6

2003-04-12 13:03:11 by Henrik Grubbström (Grubba) <grubba@grubba.org>

Lots of changes. Now has the beginnings of a metaprogram based compiler.

Rev: src/modules/Tokenizer/module.pmod.in:1.5

2003-01-18

2003-01-18 15:07:52 by Henrik Grubbström (Grubba) <grubba@grubba.org>

Now uses new features in precompile.pike.
Added report() in basic_tokenizer to allow for easier overloading.

Rev: src/modules/Tokenizer/Tokenizer.cmod:1.4
Rev: src/modules/Tokenizer/module.pmod.in:1.4

2003-01-17

2003-01-17 16:46:28 by Henrik Grubbström (Grubba) <grubba@grubba.org>

Added extra level of indirection.
Now uses parent pointer to access the report function. Modified the API accordingly.

Rev: src/modules/Tokenizer/Tokenizer.cmod:1.2
Rev: src/modules/Tokenizer/module.pmod.in:1.3
Rev: src/modules/Tokenizer/toktest.pike:1.2

2002-10-17

2002-10-17 13:32:35 by Henrik Grubbström (Grubba) <grubba@grubba.org>

Fixed type.

Rev: src/modules/Tokenizer/module.pmod.in:1.2

2002-10-17 10:43:58 by Henrik Grubbström (Grubba) <grubba@grubba.org>

First version.

Rev: src/modules/Tokenizer/.cvsignore:1.1
Rev: src/modules/Tokenizer/Makefile.in:1.1
Rev: src/modules/Tokenizer/MetaProg.yacc:1.1
Rev: src/modules/Tokenizer/ParseTree.c:1.1
Rev: src/modules/Tokenizer/Pike.yacc:1.1
Rev: src/modules/Tokenizer/Pike2.yacc:1.1
Rev: src/modules/Tokenizer/Tokenizer.cmod:1.1
Rev: src/modules/Tokenizer/acconfig.h:1.1
Rev: src/modules/Tokenizer/configure.in:1.1
Rev: src/modules/Tokenizer/misc.c:1.1
Rev: src/modules/Tokenizer/module.pmod.in:1.1
Rev: src/modules/Tokenizer/testsuite.in:1.1
Rev: src/modules/Tokenizer/tokenizer.h:1.1
Rev: src/modules/Tokenizer/toktest.pike:1.1