Pike 8.1: Changes since Pike 8.0 (scratch area for future release notes) |
------------------------------------------------------------------------ |
|
New language features |
--------------------- |
|
o Unlimited character constant size. |
|
There is no longer any limit to the size of character constants, |
e.g. 'acdefghijk' creates the bignum 0x61636465666768696a6b. |
|
o 'auto' type added. |
|
This is much like a strict typed mixed. The actual type is deduced |
compile-time. |
|
Typical use is as the type for variables in foreach when looping over |
complexly typed values, or as the type in macro-defined functions and |
similar. |
|
auto can be used as return type for a function, it will be the |
type-union of all the types of the return statements in the |
function. |
|
o typeof(X) can now be used as a type. |
|
The main use case is in macros, but it might be useful for other |
things as well (as an example in a typedef). |
|
typedef typeof(Val.true)|typeof(Val.false) bool; |
|
typeof, when used like this, behaves very much like the C++ |
decltype() expression. |
|
o Shorthand for int(val..val) added: int(val). |
|
A syntax for when the range of an integer type only |
contains a single value. |
|
o Shorthand for string(val..val) added: string(val). |
|
A syntax for when the range of a string type only |
contains a single value. |
|
o Syntax for specifying the length of array and string types. |
|
Eg string(4bit: 8bit) is 8-bit strings of lengths 0-15. |
|
o ** operator added. It's exponentiation and works with most |
combination of numerical types (int,float,Gmp.mpq,Gmp.mpf,Gmp.mpz) |
|
`** and ``** operator overloading functions added. |
|
This introduces one incompatible change: Previously the "pow" |
function called a "_pow" function in the first argument if it was an |
object. It has now been changed to also use `** (or, rather, pow() |
is now implemented using predef::`**()). |
|
o Three pass compiler. |
|
An extra pass of the compiler has been added. It is used to properly |
resolve types in forward referring expressions, and is only run when |
needed. |
|
o #pragma disassemble |
|
This directive is now available even in pikes compiled --without-debug. |
|
The byte code output is now interleaved with the generated machine code |
on selected architectures. |
|
o Complain about redundant backslash escapes. |
|
o '__weak__' modifier added. |
|
It is now possible to declare object variables to have weak references. |
|
o Support for arguments to implicit lambdas added. |
|
Implicit lambdas now behave as if they were declared as |
lambda(mixed ... __ARGS__). |
|
Support for this feature can be detected with #ifdef __ARGS__. |
|
o Function local function declarations are now statements |
(and not expressions). This means that there is no longer |
any need to terminate them with a semicolon. |
|
o Anonymous class definitions are now always expressions. |
|
o Function local class definitions are now statements |
(and not expressions). This means that there is no longer |
any need to terminate them with a semicolon. |
|
o The inherit statement's argument is now a proper |
constant expression. This means that it is possible |
to dynamically (at compile-time) generate the value |
to be inherited, eg with Pike.ProxyFactory(). |
|
o Complain about shadowed variant functions. |
|
o Syntax for generator functions added. |
|
continue is now treated as a modifier, and causes such functions |
to become generator functions, where calling the function sets |
up a restartable function context. |
|
continue may also preceed return statements, which causes |
the generator function state to be saved before returning. |
|
Alternatively the "function" Pike.yield() may be used to return |
a value, and restart with a value from the restarter. |
|
Restartable functions get the type signature |
|
X fun(mixed|void value, function(mixed|void: void)|void resume_cb) |
|
where X is the declared return type of the generator function, |
value is the value to be returned by Pike.yield(), and resume_cb |
is a function that will be called at the yield point. resume_cb |
may be used to eg behave as if Pike.yield() had thrown an error. |
|
o Machine code support for more architectures. |
|
There's now machine code support for arm32 and arm64. |
|
o Fixed multiple integer over- and underflow bugs. |
|
o Extended sscanf %O |
|
sscanf() is now able to parse all base types that sprintf %O can |
output. It can now also parse most literal constants pike accepts, |
including integers prefixed with 0x and 0b, and floating point |
numbers in scientific notation. |
|
o Returning void |
|
It is now possible to return void responses from void functions. |
|
void a() { return; } |
void b() { return a(); } |
|
|
Bug fixes |
--------- |
|
o Runtime |
|
The runtime could get confused by PROGRAM_DESTRUCT_IMMEDIATE |
objects having destruct callbacks under some circumstances. |
|
o Operator functions |
|
Calling operator functions with more than two arguments will now |
work correctly for objects, where previously only the first two |
objects where added. |
|
When adding arrays, multisets and mappings, UNDEFINED will now |
always be ignored. Previously it was only ignored when first in the |
argument list, otherwise an exception would be thrown. |
|
The LFUNs will now be called with a consistent number of |
arguments. Pike implementations rarely implemented varargs on |
operator LFUNs, so this change should address many potential hidden |
errors. |
|
o Pike.Backend |
|
Backend callbacks throwing errors could cause backends to |
enter a state where they got stuck throwing "Backend already |
running - cannot reenter.". This was typically triggered via |
the Stdio.sendfile() result callback. |
|
o Protocols.HTTP.Server |
|
The server module will now read payloads for HTTP PUT requests, just |
as any other method. Previously it would stop reading the body and |
it was up to the caller to read enough data from the socket and |
combine with already read data. |
|
Setting "connection" header in the "extra_heads" to Request object |
method response_and_finish() will now control if keep-alive should be |
used or not. Otherwise it will be decided by the clients request |
headers, as previously. |
|
The headers "content-type", "content-range", "content-length", |
"server", "date" and "last-modified" will not be added or |
overwritten if provided in the "extra_heads". |
|
Header names in "extra_heads" will not have their case modified. |
|
|
Incompatible changes |
-------------------- |
|
o Sql.Sql is no longer a wrapper class, but a function. |
|
The wrapper class has been obsoleted by introduction of the |
new base class __builtin.Sql.Connection. Note that it is |
still possible to use Sql.Sql as the type of connection objects. |
|
o Named classes are always statements. |
|
As mentioned above, it is no longer possible to define a named |
class in an expression. Such code has been observed, but none |
where the name was actually used. Use an anonymous class instead. |
|
o Anonymous classes are always expressions. |
|
It used to be possible (but typically not useful) to define |
an anonymous class as a statement. Add a name for the class. |
|
o Gz.crc32 and Nettle.crc32c now only return positive results. |
|
o glob() has changed. |
|
The main incompatibilities are that [ and \ are now special |
characters in the pattern, and if you check the return value against |
1 instead of checking if it's true, and use an array as the first |
argument you will have to change your code. |
|
This is in order to extend the glob function to cover what |
'standard' glob functions do: |
|
glob() now accepts quotes (\* to match a single *, as an example) |
and handles ranges ([abc] for a, b or c, [a-z0-9] for a single |
character between a and z or 0-9 |
|
You can also negate a range using ^ or ! ([^a-zA-Z] as an example). |
|
When the first argument (the pattern) to glob is an array, glob now |
returns which pattern in the array matched. |
|
o hash() has been changed to use siphash. The old hash function is |
available under the name hash_8_0(). Note that this is not a |
cryptographic hash function. |
|
o Stdio.UDP()->send() no longer throws errors on EMSGSIZE and EWOULDBLOCK. |
|
o When a thread exits by throwing, the thrown value is propagated to |
wait(). Previously it was sent directly to master()->handle_error(). |
|
o String.trim_all_whites() renamed String.trim(). |
|
o Floats are no longer by default sorted before added. This may reduce |
the precision but increases the speed of adding large number of |
floats by a factor of 500. Applications handling floats with large |
differences in magnitude need to apply the appropriate sorting |
before arithmetics. As `+ was the only operator that performed |
sorting, and functions like Array.sum did not, this was already a |
concern. |
|
o Returning UNDEFINED from `+ and ``+ is not allowed and will cause an |
exception. |
|
o RegGetValue(), RegGetKeyNames(), RegGetKeyValues(), openlog(), |
syslog() and closelog() have been moved to System. |
|
|
New modules and functions |
------------------------- |
|
o predef::m_add() |
|
This function adds (as opposed to inserts) an element to a multiset. |
|
o ADT.Scheduler & ADT.TreeScheduler |
|
These are variants of ADT.Heap where elements typically aren't |
removed from the heap. |
|
o Apple.Keychain |
|
Parser for Apple Keychain format files (like the ones in |
/System/Library/Keychains/). |
|
o __builtin.Sql |
|
Generic base classes for Sql API modules. |
|
o Filesystem.Zip |
o Function.bind() |
|
Partially evaluate a function. Similar to Function.curry, but with |
better granularity. |
|
o Standards.HPack (RFC 7541) |
o Stdio.FakePipe |
|
A simulated Stdio.Pipe. |
|
o Parser.Markdown |
|
o Crypto.Checksum |
|
This module collect non-cryptographic checksums. Support for crc32, |
adler32 and Castagnoli CRC (CRC32C). |
|
NB: In the future these may be amended to support the Crypto.Hash API. |
|
o Parser.ECMAScript |
|
This module simply provides a token splitter for |
ECMAScript/JavaScript. |
|
o Pike.DestructImmediate |
|
Objects of classes that inherit Pike.DestructImmediate will be |
destructed immediately on their references reaching zero (rather |
than at some arbitrary time "soon"). This class should be used |
for any classes that are (or hold) locks or similar. |
|
o Pike.InhibitDestruct |
|
Objects of classes that inherit Pike.InhibitDestruct may |
control whether explicit destruct() calls should succeed |
(or be delayed until later). Note that this class is just |
a convenience API, and that this functionality may also |
be implemented by hand. The intended use is for C-classes |
that do not want their storage to be zapped eg during |
library calls. |
|
Note that it is not possible to inhibit destruction due |
to running out of references or by the gc. |
|
o Pike.LiveBacktraceFrame |
|
This is similar to Pike.BacktraceFrame, but refers two-way |
to an active (or not) execution context. It can be used |
to eg examine (and change) local variables, and is intended |
for debugger use. |
|
o Web.EngineIO & Web.SocketIO |
o Protocols.HTTP2 |
o Bittorrent.DHT |
o Standards.MsgPack |
o Web.Auth & Web.Api |
|
|
New features |
------------ |
|
o predef::backtrace() |
|
predef::backtrace() now takes an optional argument that causes it |
to return LiveBacktraceFrames (instead of BacktraceFrames), which |
may be used to access local variables for the frame while they are |
still in use. This is intended for debugger use. |
|
o predef::destruct() |
|
lfun::_destruct() may now inhibit explicit destruction. |
|
o predef::equal() |
|
equal() on functions now checks if their definitions are same identifier |
in the same program. |
|
o predef::gc() |
|
gc() called with a weak mapping as argument now removes weak references |
that are only held by that mapping. |
|
o ADT.Heap |
|
- An indirection object ADT.Heap.Element has been added to make it |
possible to optimize several heap operations. |
|
- Added low_pop(). |
|
o Crypto & Nettle |
|
- Added Curve25519 and EdDSA25519. |
|
o Filesystem.Monitor |
|
The filesystem monitoring system now uses accelleration via |
Inotify et al. |
|
o Gmp |
|
- mpf is now implemented using gmpf if the library is available. |
|
- Improved support for GMP 5.0 and later. |
|
o GTK2 |
|
Multiple runtime fixes. |
|
o JOSE (JSON Object Signing and Encryption) |
|
Some low-level API support has been added to the Crypto and Web |
modules to support parts of RFC 7515 - 7520. |
|
o MasterObject |
|
- Protect against the same file being compiled concurrently |
in multiple threads. |
|
- cast_to_program() and cast_to_object() should now be thread safe. |
|
o CompilerEnvironment()->lock() |
|
Access to the compiler lock. |
|
o Crypto.ECC.Curve.Point |
|
A point on an ECC curve. |
|
o Parser.Pike |
|
Support new language features. |
|
o Protocols.DNS |
|
- Protocols.DNS now supports encoding and decoding CAA RRs. |
|
- Classes derived from Protocols.DNS.server may now override |
report_decode_error() and handle_decode_error() to change how |
errors while decoding a DNS packet are reported and handled. |
|
o Protocols.WebSocket |
|
Multiple API changes. |
|
o Random rewrite |
|
The random functions have been rewritten to ensure security by |
default. random_string() and random() now get their data directly |
from the operating system random generator, i.e. /dev/urandom on |
most unixen. This is about half the speed compared with the |
random_string function in Pike 8.0, but is now as secure as the |
system random generator. |
|
For consumers of random data that have additional requirements, |
different random generators are exposed in the new module |
Random. The module has the following generators: |
|
- Random.Interface |
|
This is not actually a generator, but an interface class that is |
inherited into all the other generators. It contains code that can |
turn the output from the random_string method into random numbers |
with different limits without introducing bias. It also contains |
code for all the different variants of random() on different |
types. This is currently not possible to implement in Pike code, |
as the typing is too complicated and it is not possible to access |
private/protected _random methods in objects. |
|
- Random.System |
|
This generator maps directly on top of the system random |
generator. This is the default generator used for random() and |
random_string(). |
|
- Random.Deterministic |
|
This generator creates the same sequence of random numbers for a |
given seed, with good pseudo random properties. |
|
- Random.Hardware |
|
This generator accesses the hardware random generator, when |
available. |
|
- Random.Fast |
|
This generator takes entropy from the Random.System, but feeds |
it into a cryptographic pseudo random number generator to be |
able to output data fast. This is not the default random number |
generator to avoid loading crypto code on every startup. |
|
Comparing the different generators with each other gives the |
following approximate speeds on a Linux system with hardware |
random support: |
|
Random.System 1.0 |
Pike 8.0 random_string 0.45 |
Random.Hardware 0.25 |
Random.Fast 0.20 |
Random.Deterministic 0.20 |
|
Objects implementing the _random lfun now get two arguments, the |
current random_string() and random() functions. This is convenient |
for C-level functions that doesn't have to look up functions |
themselves. Note that it is possible for a user to replace these |
with non-conforming functions (returning values of the wrong type, |
strings of the wrong length or shift size, and values outside the |
given range) or even non-functions. |
|
All code in Pike that uses random now uses the current random |
functions (though in some cases fixed at object creation). This |
allows for repeatable results if the random functions are replaced |
with a deterministic random generator, such as |
Random.Deterministic. Example: |
|
Random.Deterministic rnd = Random.Deterministic( seed ); |
add_constant( "random_string", rnd->random_string ); |
add_constant( "random", rnd->random ); |
|
o Sql |
|
- Most Sql C-modules converted to cmod. |
|
- Added next_result(). |
|
- ODBC & tds: Support more datatypes. |
|
- ODBC: Support big_typed_query(). |
|
- pgsql: Lots of changes and fixes. |
|
o SSL |
|
- Support session tickets. |
|
- Support Negotiated FF-DHE. |
|
- Support client certificates. |
|
- Support ALPN. |
|
- Prefer AEAD suites to CBC suites. |
|
- SSL.File supports set_buffer_mode(). |
|
o Standards.PKCS |
|
Support PKCS#8 private keys. |
|
o String.Buffer & Stdio.Buffer |
|
Added _search(). |
|
o The self testing framework now supports *.test-files. |
|
o Thread |
|
- _sprintf() improvements: Thread.Mutex now prints the ID of the thread |
holding the lock, and thread IDs are shown as hexadecimal numbers. |
|
- Thread.Farm now supports a callback for thread creation and termination |
for the purpose of tracking thread names. |
|
o sprintf %x |
|
%X and %x can now be used on 8-bit wide strings to get a hexadecimal |
representation of their contents. Just calling sprintf("%x",data) is |
the same as calling String.string2hex(data). |
|
o Unicode 10.0.0. |
|
o Unicode.is_whitespace() |
|
This new function returns if a unicode character is a whitespace |
characer or not. |
|
|
Deprecated symbols and modules |
------------------------------ |
|
o Sql.mysql_result and Sql.mysqls_result have been deprecated. |
Use Sql.Result instead. |
|
o call_function() has been deprecated. Use `()() instead. |
|
|
Removed features and modules |
---------------------------- |
|
o Compatibility for Pike versions before 7.8 is no longer available. |
|
o GTK1 library is deprecated, so glue code is removed. |
|
|
C-level API changes |
------------------- |
|
o The contract for functions is now changed so that a function is no |
longer required to clean the stack. The topmost value of the stack |
will be regarded as the return value and the rest of the items on |
the stack, compared to before the function arguments were pushed, |
will be popped and discarded. Efuns still have to clean their stack |
as previously. |
|
o Object destructor callbacks may now run during pop_stack() and |
pop_n_elems(). This means that code must be prepared for arbitrary |
state changes after having called them. |
|
o Removed add_function, pike_add_function, pike_add_function2, |
simple_add_variable, map_variable and MAP_VARIABLE. This removes the |
remaining few API:s where text types were used. Use ADD_FUNCTION and |
PIKE_MAP_VARIABLE instead. |
|
o Removed the functions my_rand and my_srand. Use the random functions |
on the stack for _random lfuns, the push_random_string or look up |
the random function from get_builtin_constants(). For deterministic |
pseudo random, create a private Random.Deterministic object. |
|
o The preprocessor has been converted into a cmod, and been modified |
to use more standard Pike datatypes. |
|
o The preprocessor-specific hashtable implementation has been removed. |
|
o The gdb_backtraces() function is now available also --without-debug. |
|
o There's now support to block mapping hashtables from being shrunk |
on map_delete(). |
|
o guaranteed_memset() is replaced with secure_zero() which fills a |
buffer with zero. On x86 SSE2 is used to zero the memory without |
loading it into the CPU cache, as this function is typically used |
before calling free() on memory with cryptographic key data. |
|
o The GC marker hash table has been removed. For types which require |
GC markers, they are now allocated as part of the data type. Such |
structures must now start with GC_MARKER_MEMBERS (which includes |
eg the refs field). See "gc_header.h" for details. |
|
This change significantly improves GC performance (up to a factor |
of 2 in some situations). |
|
|
Documentation |
------------- |
|
o RFC references added. |
|
o Character encoding issues fixed. |
|
o Added @enum/@endenum markup. |
|
o Support undocumented enums with documented constants. |
|
|
Building and installing |
----------------------- |
|
o GMP 4.1 or later is now required. |
|
o C99 assumed |
|
The configure tests will not check for functions defined in C99 |
anymore and C99 compiler support is assumed. |
|