Branch: Tag:

2013-04-21

2013-04-21 12:36:58 by Henrik Grubbström (Grubba) <grubba@grubba.org>

Merge pike-modules/Tokenizer into grubba/tokenizer

This merges the old experimental Tokenizer module into current Pike 7.9.

* /home/grubba/src/cvsmigration/Tokenizer: (72 commits)

2008-05-29

2008-05-29 10:13:57 by Henrik Grubbström (Grubba) <grubba@grubba.org>

Object subtype indexing mega patch.
Added inherit_number argument to the high-level object indexing functions.
Subtyped objects should now be usable in many cases in Pike code.
Updated several modules to the new API.

Rev: src/modules/Tokenizer/Tokenizer.cmod:1.41

2008-05-26

2008-05-26 09:07:36 by Henrik Grubbström (Grubba) <grubba@grubba.org>

Moved definition of yyerror macro so as to not conflict with the header files.

Rev: src/modules/Tokenizer/Tokenizer.cmod:1.40
Rev: src/modules/Tokenizer/acconfig.h:1.2

2008-05-17

2008-05-17 14:51:06 by Henrik Grubbström (Grubba) <grubba@grubba.org>

Updated to compile with low_sscanf() in Pike 7.7 taking flags.

Rev: src/modules/Tokenizer/Tokenizer.cmod:1.39

2008-04-26

2008-04-26 13:45:06 by Henrik Grubbström (Grubba) <grubba@grubba.org>

Updated to Pike 7.7's naming convention.
handle_pike_compat() ==> change_compiler_compatibility().

Rev: src/modules/Tokenizer/Tokenizer.cmod:1.38

2006-10-03

2006-10-03 16:47:34 by Henrik Grubbström (Grubba) <grubba@grubba.org>

Implemented BreakNoLabel in C.

Rev: src/modules/Tokenizer/Tokenizer.cmod:1.37
Rev: src/modules/Tokenizer/module.pmod.in:1.37

2006-10-03 16:29:59 by Henrik Grubbström (Grubba) <grubba@grubba.org>

Code cleanups and added diagnostics.

Rev: src/modules/Tokenizer/Tokenizer.cmod:1.36

2006-09-30

2006-09-30 14:33:47 by Henrik Grubbström (Grubba) <grubba@grubba.org>

Implemented more stuff in C. Removed some pike code that had already been implemented in C.

Rev: src/modules/Tokenizer/Tokenizer.cmod:1.35
Rev: src/modules/Tokenizer/module.pmod.in:1.36

2004-12-16

2004-12-16 16:05:56 by Henrik Grubbström (Grubba) <grubba@grubba.org>

Added get_{end_token,contents}().

Rev: src/modules/Tokenizer/Tokenizer.cmod:1.34

2004-12-10

2004-12-10 15:13:36 by Henrik Grubbström (Grubba) <grubba@grubba.org>

Fixed typo in BraceGroup.

Rev: src/modules/Tokenizer/Tokenizer.cmod:1.33

2004-12-09

2004-12-09 17:30:37 by Henrik Grubbström (Grubba) <grubba@grubba.org>

Renamed scan() to next() and updated return value accordingly.

Rev: src/modules/Tokenizer/Tokenizer.cmod:1.32

2004-12-03

2004-12-03 15:26:28 by Henrik Grubbström (Grubba) <grubba@grubba.org>

Changed all simple_add_constant(X, Y) to add_constant(MK_STRING(X), Y).
Made Group()->terminator a constant.
Added {Paren,Brace,Bracket}Group.
Fixed lookup of grouping_compiler_inh_report.

Rev: src/modules/Tokenizer/Tokenizer.cmod:1.31

2004-12-03 13:47:49 by Henrik Grubbström (Grubba) <grubba@grubba.org>

Group()->_sprintf() now shows the start and end token.

Rev: src/modules/Tokenizer/Tokenizer.cmod:1.30

2004-11-28

2004-11-28 19:38:34 by Henrik Grubbström (Grubba) <grubba@grubba.org>

Reduced verbosity of Token()->_sprintf().
Fixed initialization of Group().
GroupingCompiler()->group() now seems to work.

Rev: src/modules/Tokenizer/Tokenizer.cmod:1.29

2004-11-27

2004-11-27 16:33:22 by Henrik Grubbström (Grubba) <grubba@grubba.org>

Optimized GroupingCompiler()->group() somewhat by caching the lookup of the append function.

Rev: src/modules/Tokenizer/Tokenizer.cmod:1.28

2004-11-26

2004-11-26 19:19:20 by Henrik Grubbström (Grubba) <grubba@grubba.org>

Cleaned up grouping API.

Rev: src/modules/Tokenizer/Tokenizer.cmod:1.27

2004-11-25

2004-11-25 16:30:42 by Henrik Grubbström (Grubba) <grubba@grubba.org>

GroupingCompiler now implemented in C.

Rev: src/modules/Tokenizer/Tokenizer.cmod:1.26

2004-10-29

2004-10-29 15:51:39 by Henrik Grubbström (Grubba) <grubba@grubba.org>

Added class Group.

Rev: src/modules/Tokenizer/Tokenizer.cmod:1.25

2004-09-07

2004-09-07 15:43:55 by Henrik Grubbström (Grubba) <grubba@grubba.org>

Bugfix.

Rev: src/modules/Tokenizer/Tokenizer.cmod:1.24

2004-09-03

2004-09-03 12:46:43 by Henrik Grubbström (Grubba) <grubba@grubba.org>

Fixed bug in handle().

Rev: src/modules/Tokenizer/Tokenizer.cmod:1.23

2004-09-02

2004-09-02 14:35:28 by Henrik Grubbström (Grubba) <grubba@grubba.org>

Cleanups due to new precompile.pike.

Rev: src/modules/Tokenizer/Tokenizer.cmod:1.22

2004-07-08

2004-07-08 14:02:43 by Henrik Grubbström (Grubba) <grubba@grubba.org>

Improved f_werror() replacement.

Rev: src/modules/Tokenizer/Tokenizer.cmod:1.21

2004-07-08 13:08:27 by Henrik Grubbström (Grubba) <grubba@grubba.org>

f_werror() is no more.

Rev: src/modules/Tokenizer/Tokenizer.cmod:1.20

2004-04-01

2004-04-01 12:37:06 by Henrik Grubbström (Grubba) <grubba@grubba.org>

Fixed a few typos.

Rev: src/modules/Tokenizer/Tokenizer.cmod:1.19

2004-03-25

2004-03-25 15:34:20 by Henrik Grubbström (Grubba) <grubba@grubba.org>

Some OSes (WIN32) define ERROR, som the SL_CASE() macro doesn't work.
Fixed by expanding the macro.

Rev: src/modules/Tokenizer/Tokenizer.cmod:1.18

2003-12-28

2003-12-28 15:51:22 by Henrik Grubbström (Grubba) <grubba@grubba.org>

Now with some handling of pragmas.

Rev: src/modules/Tokenizer/Tokenizer.cmod:1.17

2003-12-21

2003-12-21 17:17:29 by Henrik Grubbström (Grubba) <grubba@grubba.org>

Various changes with regard to position and context handling.
* Joined Position and Context handling into a single mapping.
* As a consequence report() at all levels now takes a mapping
as the second argument.
* compiler()->basic_tokenizer()->line_no is no more.
* Added compiler()->basic_tokenizer()->context to hold context information.
* Added compiler()->basic_tokenizer()->{get,set}_context() to simplify
handling of the context mapping.
* Got rid of extraneous INIT and EXIT code.
* Shortened several variable names that were longer than 40 characters.
* Documented some inherits.
* TokenizingCompiler()->pike_tokenizer()->handle_pike_compat() is no longer
just a prototype. Allthough it currently just sets the context.

Rev: src/modules/Tokenizer/Tokenizer.cmod:1.16

2003-12-18

2003-12-18 17:46:18 by Henrik Grubbström (Grubba) <grubba@grubba.org>

Now uses MK_STRING().

Rev: src/modules/Tokenizer/Tokenizer.cmod:1.15

2003-12-13

2003-12-13 17:25:06 by Henrik Grubbström (Grubba) <grubba@grubba.org>

Added class Position.
Added C implementations of handle() and scan() in TokenizingCompiler()->pike_tokenizer.

Rev: src/modules/Tokenizer/Tokenizer.cmod:1.14

2003-12-12

2003-12-12 14:47:50 by Henrik Grubbström (Grubba) <grubba@grubba.org>

Added convenience macro APPLY_SELF().
Implemented yyerror(), handle_pragma(), parse_string() and preprocess() in TokenizingCompiler()->pike_tokenizer in C.
Named the SeverityLevel type.

Rev: src/modules/Tokenizer/Tokenizer.cmod:1.13

2003-12-02

2003-12-02 12:39:11 by Henrik Grubbström (Grubba) <grubba@grubba.org>

Fixed inherit of program in parent.

Rev: src/modules/Tokenizer/Tokenizer.cmod:1.12

2003-12-01

2003-12-01 14:33:03 by Henrik Grubbström (Grubba) <grubba@grubba.org>

Various fixes due to new naming conventions.
Started implementing TokenizingCompiler in C.

Rev: src/modules/Tokenizer/Tokenizer.cmod:1.11
Rev: src/modules/Tokenizer/module.pmod.in:1.26

2003-11-29

2003-11-29 13:23:23 by Henrik Grubbström (Grubba) <grubba@grubba.org>

Now uses the new PIKE_MODULE{INIT,EXIT} syntax.

Rev: src/modules/Tokenizer/Tokenizer.cmod:1.10

2003-11-28

2003-11-28 16:49:38 by Henrik Grubbström (Grubba) <grubba@grubba.org>

{Literal,String,Integer,Float}Token are now implemented in C.

Rev: src/modules/Tokenizer/Tokenizer.cmod:1.9
Rev: src/modules/Tokenizer/module.pmod.in:1.25

2003-11-28 14:33:10 by Henrik Grubbström (Grubba) <grubba@grubba.org>

Moved some more stuff from Pike to C.

Rev: src/modules/Tokenizer/Tokenizer.cmod:1.8
Rev: src/modules/Tokenizer/module.pmod.in:1.24

2003-11-27

2003-11-27 16:49:32 by Henrik Grubbström (Grubba) <grubba@grubba.org>

The basic token nolonger holds a value.
The field "token" in a token is now a constant.

Rev: src/modules/Tokenizer/Tokenizer.cmod:1.7

2003-07-24

2003-07-24 14:40:48 by Henrik Grubbström (Grubba) <grubba@grubba.org>

Fixed AutoDoc markup for SeverityLevel.

Rev: src/modules/Tokenizer/Tokenizer.cmod:1.6

2003-04-12

2003-04-12 13:02:08 by Henrik Grubbström (Grubba) <grubba@grubba.org>

Tokenizer.Token()->_sprintf() now looks at its first argument.

Rev: src/modules/Tokenizer/Tokenizer.cmod:1.5

2003-01-18

2003-01-18 15:07:52 by Henrik Grubbström (Grubba) <grubba@grubba.org>

Now uses new features in precompile.pike.
Added report() in basic_tokenizer to allow for easier overloading.

Rev: src/modules/Tokenizer/Tokenizer.cmod:1.4
Rev: src/modules/Tokenizer/module.pmod.in:1.4

2003-01-17

2003-01-17 17:36:09 by Henrik Grubbström (Grubba) <grubba@grubba.org>

Indentation fix.

Rev: src/modules/Tokenizer/Tokenizer.cmod:1.3

2003-01-17 16:46:28 by Henrik Grubbström (Grubba) <grubba@grubba.org>

Added extra level of indirection.
Now uses parent pointer to access the report function. Modified the API accordingly.

Rev: src/modules/Tokenizer/Tokenizer.cmod:1.2
Rev: src/modules/Tokenizer/module.pmod.in:1.3
Rev: src/modules/Tokenizer/toktest.pike:1.2

2002-10-17

2002-10-17 10:43:58 by Henrik Grubbström (Grubba) <grubba@grubba.org>

First version.

Rev: src/modules/Tokenizer/.cvsignore:1.1
Rev: src/modules/Tokenizer/Makefile.in:1.1
Rev: src/modules/Tokenizer/MetaProg.yacc:1.1
Rev: src/modules/Tokenizer/ParseTree.c:1.1
Rev: src/modules/Tokenizer/Pike.yacc:1.1
Rev: src/modules/Tokenizer/Pike2.yacc:1.1
Rev: src/modules/Tokenizer/Tokenizer.cmod:1.1
Rev: src/modules/Tokenizer/acconfig.h:1.1
Rev: src/modules/Tokenizer/configure.in:1.1
Rev: src/modules/Tokenizer/misc.c:1.1
Rev: src/modules/Tokenizer/module.pmod.in:1.1
Rev: src/modules/Tokenizer/testsuite.in:1.1
Rev: src/modules/Tokenizer/tokenizer.h:1.1
Rev: src/modules/Tokenizer/toktest.pike:1.1