Debug: added functions to generate perf map files
Remote: Stricter handling of zero.
Crypto: Fix multiple warnings about passing of zero values.
Web.Api: Authenticate with the Authorization header by default. Sending OAuth authorization information via query variables has been deprecated by RFC6750 (and others), and some endpoints (eg GitHub) do no longer support the old method. Switch to using Authorization: Bearer <access_token> by default. Also updates Web.Api.Github accordingly.
Merge branch 'patches/support-mixed-in-count_memory' into 8.0 * patches/support-mixed-in-count_memory: GC: Support remaining types in Pike.count_memory().
GC: Support remaining types in Pike.count_memory(). Users of Pike.count_memory() typically do not want to bother filtering its arguments, so perform filtering in count_memory() rather than throwing errors. Also adds support for counting memory for objects referenced via functions.
System.TM: Support initialization with bignums. Fixes post-2038 support on ILP32 architectures with 64-bit time_t.
Backported fragment helper methods.
Merge branch 'security/decode_value' into 8.0 * security/decode_value: decode_value(): Allow to restrict decoding to simple types
decode_value(): Allow to restrict decoding to simple types The encoded value has quite a bit of control over our process. Allow to restrict decoding to primitive types by passing coding as -1.
Inotify: add_watch() now returns -1 on some failures. This is to simplify differentiation between failures due to the filesystem being in flux (eg races), and failures due to resource limits. Fixes race condition (time of check, time of use) in Filesystem.Monitor.basic.InotifyMonitor()->register_path(). Fixes #10059.
Documentation: fix markup errors
Documentation: Improved doc for search().
Gettext: Prevent argument of textdomain from being empty
strftime: Remove non-standard conversions %k and %l from doc and testsuite These are not part of any C standard, not even C2x. Thus libc can not be expected to implement them.
Concurrent.Future: Added try_get().
Concurrent.Future: Added query_{success,failure}_callbacks().
Thread: Mark simulated MutexKey objects for immediate destruction.
SSL.Context: Context commits to tickets via offers_tickets() In certain situations, i.e. after receiving an empty Session Ticket extension (client supports, but does not yet have a ticket), by including the Session Ticket extension in the server hello commits us to providing a ticket later. As SSL.Context would simply return 0 in encode_ticket() when use_cache was zero (not the default), we used to break the protocol on such occasions. Instead, we now use SSL.Context()->offers_tickets() to signal whether Session Tickets should be supported on the connection (and whether the Context commits to encoding a session as ticket later on).
Inotify: Improved handling of invalid event callbacks. The events are now buffered until the callback becomes valid. Fixes #8042.
Documentation [Standards.X509]: Fixed some typos. Fixes #10012.
Standard.X509: Make signature algorithms configurable
Merge branch 'grubba/pty-handling' into 8.0 * grubba/pty-handling: Process.Process [NT]: Added modifier "conpty".
Process.Process [NT]: Added modifier "conpty". This modifier associates a ConPTY with the new process without also setting any of stdin, stdout or stderr. Fixes #10008.
Merge branch 'grubba/fdlib' into grubba/pty-handling-new * grubba/fdlib: (55 commits) I/O [NT]: Use read-write locking for fd_busy and fd_to_{socket,handle()}. I/O [NT]: Fixed typo. I/O [NT]: Avoid blocking on read from pipes with data. Added some paranoia System [NT]: Normalize the drive letter explicitly in normalize_path(). System [NT]: More normalize_path() fixes. Configure [Solaris]: Fixed quoting level issue. System [NT]: Fixed trailing slashes from normalize_path(). I/O [Solaris]: Fix getcwd() on Solaris. I/O [NT]: Fixed a few warnings. I/O [NT]: Moved bulk of normalize_path() to fdlib. System [NT]: Update normalize_path() to use UTF8. I/O [NT]: Get rid of DO_NOT_WARN(). I/O [NT]: Added some more FD_DEBUG. I/O [NT]: Fixed typo in FD_DEBUG code. Build [NT]: Always use or own implementation of _dosmaperr(). Build [NT]: Improved support for MSVC 2010 (aka MSC 1600). I/O [NT]: Fixed deadlock. I/O [NT]: Updated FD_DEBUG diagnostics. I/O [NT]: Survive invalid UTF-16 surrogate pairs. ...
Stdio: Some pty-handling cleanup. Add wrappers emulating missing libc functions.
Merge branch 'grubba/fdlib' into grubba/pty-handling * grubba/fdlib: (55 commits) I/O [NT]: Use read-write locking for fd_busy and fd_to_{socket,handle()}. I/O [NT]: Fixed typo. I/O [NT]: Avoid blocking on read from pipes with data. Added some paranoia System [NT]: Normalize the drive letter explicitly in normalize_path(). System [NT]: More normalize_path() fixes. Configure [Solaris]: Fixed quoting level issue. System [NT]: Fixed trailing slashes from normalize_path(). I/O [Solaris]: Fix getcwd() on Solaris. I/O [NT]: Fixed a few warnings. I/O [NT]: Moved bulk of normalize_path() to fdlib. System [NT]: Update normalize_path() to use UTF8. I/O [NT]: Get rid of DO_NOT_WARN(). I/O [NT]: Added some more FD_DEBUG. I/O [NT]: Fixed typo in FD_DEBUG code. Build [NT]: Always use or own implementation of _dosmaperr(). Build [NT]: Improved support for MSVC 2010 (aka MSC 1600). I/O [NT]: Fixed deadlock. I/O [NT]: Updated FD_DEBUG diagnostics. I/O [NT]: Survive invalid UTF-16 surrogate pairs. ...
Fix documentation for next_prime
Fix function definition for next_prime and the documentation for next_prime and probably_prime_p
SSL.ServerConnection: Session tickets (RFC 4507 and RFC 5077). Server side support for session tickets. Note that the default ticket encoding is to use the session_id, it thus uses server side state. The ticket encoding can be changed by overriding {en,de}code_ticket() in SSL.Context. Implementation verified against OpenSSL's s_client. Backported from 372b2a05d05fa0d0e052e6634d2acf8d03629ed4 by Tobias Josefowitz.
Thread.ResourceCountKey: Survive being used with old pike binaries. Old pike 8.0 binaries do not have Pike.DestructImmediate, and it is not meaningful to attempt to kludge around it, as the corresponding program flag is not propagated with them anyway.
Pike: Move DestructImmediate to the Pike module.
Thread.ResourceCountKey: Make destruct immediate
Sprintf: Cleanup linebreak mode. The sprintf "%/s" and "%=s" modes now break at the column width (defaulting it to the field width) and not the field width. This makes it possible to use formats like eg "%;*-=s" (instead of "%*-=s") to not get every line padded to the full column width.
TURBO2-1398: Allow disabling of wrapping of describe_backtrace output. The wrapping can add lots of spaces to the end of lines to pad them, resulting in enormous backtrace files in some circumstances.
Documentation [Traditional]: Added Compression chapter.
Mysql.Result: Update the index counter when fetching rows. Call Sql.Result::increment_index() at relevant places. Fixes testsuite failure. Fixes PIKE-197 (#8197).
Dokumentation [Traditional]: Added chapter "Threads".
Documentation [Traditional]: Sectioned the DB access chapter.
Documentation [Traditional]: Added some structure to the I/O chapter.
Documentation [Traditional]: Move SQLite to the Database page.
Documentation: Move operators to the Operators chapter. They were hidden in the "The rest" chapter... Also adds some minimal documentation of the **-operator.
Documentation [Sybase]: Fixed broken AutoDoc markup. Fixes the inheritance graph for predef::...
Tools.Standalone.precompile: Fix storage offset handling Programs using their parent may store parent information in the beginning of ->storage, thus we need to add p->inherits[0].storage_offset when trying to reach an object's storage. The compiler/runtime correctly takes care of this in THIS, but the OBJ2_ and GET_..._STORAGE macros did not take this into account.
Locale: Reintroduce deprecated Charset alias This reverts commit f64cba1ff832383ac7a46b4bbe8a0264700de976. This reverts commit ffedb3787383db78beddce529eda0bde7e24d40b. In Pike 8.0.* and below, no deprecation warnings were issued by the compiler when accessing Locale.Charset, as the deprecated status of Locale.Charset was hidden from the compiler by a joinnode. In 8.1, the master will now issue deprecation warnings in such cases. Hence, re-adding the deprecated Locale.Charset alias seems like a good idea for users to have a good chance of noticing the deprecated status with an 8.1 release etc. before final removal of the alias in a future release.
Documentation [Web.Sass]: Make the doc appear in the correct place.
Documentation [Pike.InhibitDestruct]: Added some cross-references.
Sass: Create pike -x sass SCSS compilation tool.
HTTP.Server.Request: Restore backward compatibility and fixes to Shuffler. Changes to HTTP.Server.Request: - Support set_mode() in HTTP.Server.Request to optionally enable Shuffler to output the data on the wire. Without setting it, it will revert to the old buffered-output method. - Reinstate write-timeout on HTTP.Server.Request, make sure it works in both classic and Shuffler modes. Changes to Shuffler: - Add automatic TCP_CORK support. - Clean up code duplication between shuffler and sendfile. - Move wrap_callback from a separate member to an optional argument to add_source(). - Add support for Stdio.Buffer. - Adding empty sources works consistently now. - Fix broken cornercases when adding strings or memory regions. - Fix dataloss when using non-blocking io on input files. - Support starting in a file at the spot where the filepointer is. - Add start/length arguments to the main Shuffle object, to skip and limit in the entire concatenated output (needed for HTTP-range support).
Pipe.Gzip: Moved and renamed to a more generic spot.
Documentation [Locale]: Some minor adjustments.
Gz.Pipe.Uncompress/Transmogrify: Add parent class and Uncompress pipe.
Gz: Put classes in separate files to lower memory footprint.
Gz.Pipe.Compress: New non-blocking compression pipe.
Shuffler.Shuffle: Numerous fixes and add set_wrap_callback(). The fixes include: - Fixed broken end-of-stream detection for d_source_pikestream.c. - Reduce memory footprint of source-structs by half. - Fixed memory corruption issues because a memcpy was used on overlapping regions. - Eliminated all dynamic memory allocations in the shuffler. For file-descriptor based streams it still has a fixed overhead of 16KB per stream, for all other streams the overhead now is close to zero. Still TODO: - Reduce the memory overhead for fd-based streams by half; it uses double buffering now, it could trivially be reduced to use a single buffer.
Crypto.ECC.Curve25519: Added pkcs_eddsa_id(). This is useful for finding the correct curve to use, given an eddsa_id, as it differs from the corresponding curve_id. Also updates some RFC references to reference RFC 8410.
Documentation [Nettle]: Fixed typo in doc.
Stdio.File: Support openat with a single argument. Also fixes: - Avoid a SEGV when supplied with a single argument. - Synchronise low level implementation with pike level by setting the default mask to 0777 everywhere.
Standards.PKCS: Updated with references to RFC 8410.
Web.Api: Add and rewrite support for all Google APIs (past/present/future).
Tools.Standalone.precompile: Improved doc.
EFUNs: Add deprecated_typep() efun.
Compiler: export current_{line,file}
Compiler: Export state of no_deprecation_warnings ... in the active compilation to Pike.
EFUNs: Add typeof_identifier() typeof_identifier(prog, identifier) returns the type of a named identifier in the program.
Pike.ProxyFactory: Added new function. This is a factory for creating classes that act as a proxy for the specified program.
Master: Re-hook global constants on replace_master(). replace_master() sets the variable is_pike_master in the object that is to be the new master to 1. Catch this with a setter so that global functions implemented by the master are actually implemented by the active master. This mostly affects the test suite, which replaces and restores master objects several times.
Concurrent.Promise: Introduce delay() and optimise timeout().
Concurrent.Promis: Fix/optimise delay() and timeout().
Concurrent.Promise: Add delay() member to delay the future.
System.Memory: Stricter types for p{read,write}*(). Also fixes the typo in the type for pread{16,32}n().
EFUNs: get_iterator() now passes extra args to lfun::_get_iterator().
Testsuite: Protect lots of lfuns.
Pike.InhibitDestruct: New API for inhibiting destruction.
EFUNs: lfun::_destruct() may now inhibit explicit destruction. lfun::_destruct() returning 1 now causes the object to survive an explicit destruct() call. Note that the object will still be destructed if the destruct is for any other reason.
Documentation [Pike.Backend]: Improved markup. Pike.Backend and Pike.SmallBackend are now documented as classes (instead of as constants). This makes the documentation show up where people expect it.
Documentation [master]: Added some AutoDoc mk II markup.
Merge commit '722771973bd' into patches/lyslyskom22891031 * commit '722771973bd': (6177 commits) Verify that callablep responses are aligned with reality. ...
Merge branch 'grubba/thread-farm' * grubba/thread-farm: Thread.Farm: Added ResultWrapper.
Thread.Farm: Added ResultWrapper. This is a wrapper for the Result class to detect when the user loses its reference to it, in order to not lose reports of any errors from the worker threads.
Merge commit '2470270f500c728d10b8895314d8d8b07016e37b' into grubba/typechecker-automap * commit '2470270f500c728d10b8895314d8d8b07016e37b': (18681 commits) Removed the old typechecker. ...
Pike.Annotations: Added Override annotation. This annotation verifies that the annotated symbol also exists in one of the inherited programs.
Compiler: Call end_pass_identifier() for identifier annotations.
Concurrent.Future: Added wait(). This differs from get() in that it will not throw on failure.
Concurrent.Promise: Reduce number of complaints. Don't complain about dropping promises that have no corresponding future on the floor. As the promises have no future, nobody is affected by them not being fulfilled.
EFUNs: backtrace() now takes an optional argument. Calling backtrace() with an argument of 1 now causes it to return an array(LiveBacktraceFrame) instead of an array(BacktraceFrame).
Builtin.LiveBacktraceFrame: Initial implementation. NB: Some features are still missing.
Documentation [Concurrent]: Note where backend state isn't propagated.
Concurrent.Future: Added promise_factory(). This makes propagating state to new Promises easier. New Promises are now created with the same backend as the Promise they originated from.
Documentation [Concurrent]: Added note about backend disabled mode.
Concurrent.Future: Added {get,set}_backend(). This makes it possible to have a custom Pike.Backend handle the callbacks for the Future.
Concurrent.Future: Added internal function call_callback(). This makes it possible to adjust the way that callbacks are called via overloading. The typical use case is for wanting callbacks to be called by some other thread than the backend.
Merge branch 'marcus/exif_sanitization' into 8.0
Image.Dims: Revert EXIF flipping of dimensions in get_JPEG Like for JPEG._decode, add a specific exif_get_JPEG which does the flipping instead.
GSSAPI: Autodoc typo fixes
Compiler: Fix autodoc typo
Documentation: Moved doc for {base,dir}name().
Master: Support __HAVE_COMPILER_NO_HANDLERS__. Implement support for handlers by overloading the CompilerEnvironment.
Master: Mark Describer objects for immediate destruction.
Documentation [Master]: Added some more doc.
AutoDoc: Added some support for parsing annotations.
Documentation [Master]: Added some minimal AutoDoc markup.
LFUNs: Added magic lfun ::_annotations().
Merge branch 'patches/pike156' into 8.0 * patches/pike156: EFUNs: Hide mutex keys and crypto contexts from backtraces.
Merge branch 'patches/pike156' * patches/pike156: EFUNs: Hide mutex keys and crypto contexts from backtraces.
EFUNs: Hide mutex keys and crypto contexts from backtraces. Having unexpected references to mutex keys may cause hangs and complaints about attempting to lock mutexes recursively. Fixes PIKE-156 (#8156).
EFUNs: Return an array(multiset) from predef::annotations(). A multiset is a more natural datatype for the set of annotations. NB: The low-level code still uses arrays; this will most likely be changed later.
EFUNs: Added m_add(). This function adds (as opposed to inserts) an element to a multiset. Also adds corresponding (fake) lfun::_m_add().
Make werror() (and by extension exit()) respect current locale This makes it possible to output non-ASCII text and still have it legible in all environments. write() still outputs binary data, and so will werror() and exit() in 8.0 compat mode. Also, restore master.pike.in to ISO-8859-1 encoding, which is the only one supported for the master, and remove the (now superfluous and) aribitrary use of string_to_utf8() for the copyright message.
EFUNs: m_clear() now supports multisets and objects. Adds LFUN _m_clear().
Merge branch '8.1' * 8.1: Thread.Condition: Improve docs. pgsql: Avoid excessive latency on frequent ping()s.
EFUNs: m_delete() now supports operation on multisets.
Thread.Condition: Improve docs.
EFUNs: Added predef::annotations(). This function returns the annotations (if any) corresponding to the symbols returned by indices() on the same value. Also adds lfun::_annotations().
Pike.Annotations.Implements: Added initial implementation. This annotation causes the compiler to validate that the annotated class implements the specified API.
Compiler: Added Pike.Annotation. The compiler now calls end_pass() in any annotations on the program being compiled once each compiler pass.
Added program.h to module.h
sprintf: Support floats in %p.
Documentation [Threads]: Some notes about interrupt() and kill().
Threads.Mutex: Added cond(). This creates a Thread.Condition bound to the mutex.
Threads.Cond: Validate that the expected mutex is used.
Cpp: Added some optional parameters to define_macro(). Renumber the CPP_MACRO_* values, so that the ones useful for users (CPP_MACRO_VARARGS and CPP_MACRO_KEEP_NL) come first. define_macro() now accepts two additional optional arguments; numargs and flags.
EFUNs: Added m_clear().
Cpp: Macro expansion failure is now indicated by returning 0.
CHANGES: Added note about inherit of CompilerEnvironment.
Cpp: Added cpp_error().
Documentation [Cpp]: Fix documentation markup error. Fixes remainder of LysLysKOM 22790861.
Cpp: Simplified base implementation of report(). The base implementation of CompilerEnvironment.CPP::report() now just calls CompilerEnvironment::report() in its parent. The handling of handler and compat_handler in report() is moved to the implementation in the master.
Whitespace changes.
EFUNs: Added get_active_compiler(). This function could be used to implement get_active_error_handler() and get_active_compilation_handler(). API clean up in preparation for moving handling of handlers to code written in Pike.
CompilerEnvironment: Added handle_import().
Compiler: Cleaned up the resolv() calling conventions. CompilerEnvironment()->PikeCompiler()->resolv() now takes a single argument, and looks up the current file and handlers from the PikeCompiler object, and then calls CompilerEnvironment()->resolv(). The actual calling of resolv() in the handlers is now performed by CompilerEnvironment()->resolv().
Merge branch 'pike-modules/sass' into 8.0 * pike-modules/sass: Documentation [Sass]: Some clarifications regarding sass2scss(). Added method sass2scss to convert Sass syntax to SCSS syntax.
Merge branch 'pike-modules/sass' into 8.1 * pike-modules/sass: Documentation [Sass]: Some clarifications regarding sass2scss(). Added method sass2scss to convert Sass syntax to SCSS syntax.
Documentation [Sass]: Some clarifications regarding sass2scss().
Added method sass2scss to convert Sass syntax to SCSS syntax.
Merge branch 'pike-modules/sass' into 8.0 * pike-modules/sass: Added the option "sass_syntax". Fixed broken indentation
Added the option "sass_syntax". This option will parse the input as having indented Sass syntax. This option only has effect in compile_string(). Also wrapped an unused argument in UNUSED().
Pike.get_runtime_info: Report the size of time_t too. Adds the entry "time_size" to the result from Pike.get_runtime_info().
MasterObject: Validate utf8_string contents in -rt mode.
Merge branch 'patches/pike111' into 8.0 * patches/pike111: Sql.mysql / Mysql.SqlTable: Use UTF-8 encoded UTF-16 in a couple of other applicable cases. Sql.mysql: Use/support UTF-8 encoded UTF-16. Testsuite [utf8_to_string]: Fixed typo. EFUNs: Support encoding to UTF-8 encoded UTF-16 in string_to_utf8().
Merge branch 'patches/pike111' into 8.1 * patches/pike111: EFUNs: Support encoding to UTF-8 encoded UTF-16 in string_to_utf8().
EFUNs: Support encoding to UTF-8 encoded UTF-16 in string_to_utf8(). string_to_utf8() now supports the same extensions as utf8_to_string(). Fixes PIKE-111 (#8111).
CPP: fixed refdoc comment
Documentation: corrected documentation on set operators The operators |, &, - and ^ on arrays rely on the content having an ordering which is compatible with `==. Mention this in the documentation, as this will otherwise lead people to writte buggy code.
master: fixed refdoc comment
EFUNs: Rewrite of validate_utf8(). Changed validation approach in validate_utf8() to simplify the code, and make it easier to read/understand. Also adds some corresponding testsuite tests.
EFUNs: Added validate_utf8(). This efun checks whether a string is valid UTF-8 or not.
pgsql: Add streaming_typed_query() method to avoid emulate_bindings().
Debug.Rapidlog: New module for loggingfloods with minimal latency.
sprintf: Officially support %p to display addresses.
pgsql: Eliminate rare deadlock on heavy interleaved queries tuned.
Merge branch 'grubba/fdlib' into 8.0 * grubba/fdlib: System [NT]: Update normalize_path() to use UTF8.
Merge branch 'grubba/fdlib' into 8.1 * grubba/fdlib: System [NT]: Update normalize_path() to use UTF8.
System [NT]: Update normalize_path() to use UTF8.
Merge branch 'grubba/fdlib' into 8.0 * grubba/fdlib: (36 commits) I/O [NT]: Updated FD_DEBUG diagnostics. I/O [NT]: Moved FD_SETSIZE definition to global.h. I/O [NT]: Use new low-level functions from Process. I/O [NT]: Added some assertions. I/O [NT]: Added serialization of access to fd handles. I/O [NT]: Use the new low-level functions. I/O [NT]: Added some low-level fd handling functions. Build [NT]: Fix typo of EACCES. I/O [NT]: Document some internal data structures. Stdio: Cleanup File()->truncate(). Configure: Moved tests for truncate64() and ftruncate64(). Stdio [NT]: Fixed warning on NT. Stdio [NT]: Added detection constant __HAVE_UTF8_FS__. I/O [NT]: Added note about the intent of fdlib. Stdio [NT]: Update get_dir() to accept/return UTF8. Stdio [NT]: Cleanup filesystem_stat(). Stdio [NT]: Clean up getcwd(). I/O [NT]: Added fd_get_current_dir_name(). Configure: Check for getwd() and get_current_dir_name(). Stdio [NT]: Clean up cd(). ...
Sql: Tighten up documentation.
__builtin.Sql.Result: Added increment_index().
MIME: Setter for message boundary prefix.
Sql: Formalise blocking nature of affected_rows(), improve docs.
Merge branch 'pike-modules/sass' into 8.0 * pike-modules/sass: (91 commits) Sass: Use ONERROR to free sass contexts on error. Sass: Unlink stale JMPBUF. Sass: Fixed indentation. Added some tests for Sass compilation errors. Just some cleanup of a simple test. Transfered "ownership" of error message from libsass to Pike. Fixed stupid Emacs indentation.:wq Testsuite [Sass]: Fixed detection of Web.Sass some more. Testsuite [Sass]: Updated testsuite to the moved module. Sass: Move module to Web.Sass. Sass: Hide the module when not available. Install: Added Tools.Sass to the feature list. Testsuite [Sass]: Check that Tools.Sass is available. Sass: Hide the s8 symbol. Sass: Use HAVE_LIBSASS instead of HAVE_SASS_H. Sass: Clean up configure script. Sass: Clean up acheader.h. Sass: Removed redundant cast. Sass: Adjusted debug message. Sass: Fixed a few C99-isms. ...
pgsql: Make affected_rows() and status_command_complete() wait for a meaningful result.
I/O [NT]: Use new low-level functions from Process.
Efuns: Use utf8_string in string_to_utf8() and utf8_to_string().
Crypto: Added some support for generating JWK Thumbprints. Cf RFC 7638.
Merge branch 'pike-modules/sass' into 8.1 * pike-modules/sass: Sass: Move module to Web.Sass. Sass: Hide the module when not available.
Sass: Move module to Web.Sass.
Merge branch 'pike-modules/sass' into 8.1 * pike-modules/sass: (79 commits) Testsuite [Sass]: Check that Tools.Sass is available. Sass: Hide the s8 symbol. Sass: Use HAVE_LIBSASS instead of HAVE_SASS_H. Sass: Clean up configure script. Sass: Clean up acheader.h. Sass: Removed redundant cast. Sass: Adjusted debug message. Sass: Fixed a few C99-isms. Sass: Don't throw pike errors through the libsass layer. Sass: Error messages need to end with a line feed. Sass: Cleaned up documentation a bit. Sass: Fix building in a separate build tree. Sass: Generate config.h.in from acconfig.h. Sass: Removed obsolete stuff from Makefile.in. Sass: Removed quite a bit of stuff. ...
Sass: Cleaned up documentation a bit.
Merge github.com:poppa/pike-libsass into patches/pike89 * github.com:poppa/pike-libsass: (64 commits) [...]
Filesystem.Monitor: Add set_stable_time() function.
Filesystem.Monitor: Fix race condition on file deletion. This fixes a race that could occur on rapid exists => delete => exists transitions, where the directory monitor never noticed the change but the sub monitor was removed from the monitor lookup mapping. The directory monitor is now notified on sub-monitor release.
Added Thread.Farm()->set_thread_name_cb() to help applications that monitor thread creation and termination for the purpose of tracking thread names.
Added convenience function to decrypt private key.
Documentation [Cpp]: Added some minimal documentation.
Cpp: Added C-implementation of a stack. This class is API-compatible with ADT.Stack, and is intended to be used by the preprocessor.
Documentation [Crypto.ECC]: Added reference to rfc.
Documentation: Document some more of the master.
Stdio.File()->write: add documentation for buffer variant
Documentation [Crypto.Password]: Fixed typo.
Improved type of Array.everynth
Merge commit '75c9d1806f1a69ca21c27a2c2fe1b4a6ea38e77e' into patches/pike63 * commit '75c9d1806f1a69ca21c27a2c2fe1b4a6ea38e77e': (19587 commits) ...
Put cp() in the global scope, as rm() and mv() are already there.
Fix documentation.
Improve docs.
Fix typos and docs.
Filesystem.Monitor: Clean up rescheduling of backend_check().
Filesystem.Monitor: Remove co_id == 1 special case. The special case is no longer used as it was broken, so there's no need to keep the code around.
String.Buffer: Moved implementation. Moves implementation of String.Buffer from builtin.cmod to string_builder.cmod. Adds initialization of the module to happen before that of cpp. This will allow cpp.cmod to rely on String.Buffer to exist at compile time.
Deprecate RegGet*_76 and move RegGet* from efun to System.
Concurrent: Various improvements, bugfixes, backported from 8.1. Add map_with() as an alias for flat_map(). Reduce lock contention and cleanup superfluous catches. Clarify docs. Free callbacks after use, frees memory and references. Add convenience variant to zip() and extend testsuite. Transparent automatic backend enable/disable.
strptime/strftime: Added at global level; rip out special strptime/strftime in Val.*.
Sql.Connection: Fallback big_typed_query() to big_query(), and streamline docs.
pgsql: Readding primitive auto-reconnect logic.
Documentation [Nettle.BlockCipher]: Adjusted some references.
Sql.Connection: Fix docs and prototypes.
Sql: Fix several drivers to conform to new API.
Sql.Promise: Add support for eventbased mapping of resultrows.
HKDF rewrite. Should now be possible to use in TLS 1.3
MPI: Declare function type explicitly in refdoc comments not providing the function signature produced function(:void), which was misleading.
MPI: doc fix
MPI: Fix documentation for Init() and Finalize().
MPI: Add documentation.
MPI: Document MPI typed arrays.
MPI: mark callback lfuns as protected in MPI typed arrays.
Debug: Added find_all_clones().
Fix yarrow sources type.
Improved types.
Restore part of AmigaOS support.
Rip out final remains of OS2 and AmigaOS.
Equvivalent and equivalent are not equivalent.
Documentation: Added doc for Builtin._take_over_initial_predefines().
Val.Timestamp,Date: Support strftime()/strptime().
JSON5: add flags to control quotation mark usage and identifier encoding in keys
Val.Range: Salvage SQL syntax for use in SQL subsystems.
Move to ISO date format to avoid mm/dd, dd/mm confusion.
Val.Range: Add !, <<, >>, isempty(), merge() and contains().
mktime: Clarify docs.
JSON5: new module, not quite complete (missing proper symbolic numbers.)
Nettle: Fix AES128_CTR_DRBG on ILP32 architectures. 1L<<48 is zero on ILP32... Also some related ILP32 fixes for the reseed_interval getter/setter.
Convert Val to a directory and move the last pieces from builtin to it.
Moved out time-code from __builtin to Val. Reduces start overhead and prevents clobber of System.Time.
Documentation [Filesystem.Monitor]: Some clarifications.
TimeTZ: Add docs, finetune constructors.
Val.Timestamp: Polish docs.
Val.TimeTZ: More testsuite entries, docs and fixes.
Val.Time/Date: Small fixes, increased orthogonality of operators.
Enhance docs.
Change constants into inherited classes to make them show up in refdoc. Performance penalties?
Val.Timestamp et al: Moved the presence of the new types from the root to Val for now.
Timestamp/Date/Range/Inet: Operator orthogonality improved, testsuite expanded accordingly.
Timestamp/Date/Inet/Range: Documentation updates and fixes. Enhance orthogonality of the operators. Simplify the implementation of the operators (reduce codesize).
Timestamp/Date/Inet/Range: First stab at proper documentation.
Daylight 'savings' spelling corrections.
Added SHA512/224 and SHA512/256
Range: Reuse Math.inf etc; Use NetUtils.
Val: Cleanup references.
Val.NaN: Move definitions from __builtin to Val to avoid circularities.
Val.nan: Introduce new Val values and native lightweight Date/Time/Inet/Range types.
master: Added some minimal doc of describe_*().
Tumblr oauth URLs
Sql.Timebase: Adjust SQL-native types to support nanosecond resolution.
Sql: Add support for native Inet data type.
Concurrent: Transparent automatic backend enable/disable.
Sql: Add native support for Time, Date, Timestamp, Interval and TimeTZ types.
master: Document the resolver some more.
Back out the thin convenience wrapper hash and hash_hmac. Replacements: Crypto.hash(Crypto.SHA1, true)(x) -> Crypto.SHA1.hash(x) Crypto.hash(Crypto.SHA1) -> sprintf("%x", Crypto.SHA1.hash(x)) Crypto.hash_hmac(Crypto.SHA1,k)(x, true) -> Crypto.SHA1.HMAC(k)(x)
RC4 is alias for Arcfour
master: Added some documentation about the master filesystem.
Concurrent: Reduce stackdepth, minimise catch() breadth.
Concurrent: Slight optimisation and typo fix.
Promise: Update docs again.
Concurrent: Improve more docs.
Concurrent: Improve docs.
Concurrent: Add convenience variant to zip() and extend testsuite.
master: Added some documentation.
Stdio.Buffer: Support bignums in add_int{8,16,32}(). add_int8() and add_int16() now support getting called with a bignum. add_int32() with a bignum now ignores any bits outside the 32-bit range.
Crypto.Hash: Stricter string types.
Crypto.Hash.SCRAM: Moved class. Move Crypto.SCRAM to __builtin.Nettle.Hash()->SCRAM analogous to other similar APIs. Crypto.SCRAM(Crypto.SHA256) is now Crypto.SHA256.SCRAM(). Also updates the testsuite and Sql.pgsql_util accordingly.
Concurrent: Clarify docs.
Concurrent: Add use_backend() to allow promises without a running backend.
Concurrent.Promise: Reduce mutex contention, memory usage, simplify code, fix a few races.
Concurrent: Cleanup.
Concurrent.Promise: Harmonise method names (failed -> failures).
Concurrent.Promise: Improve docs.
Concurrent.Promise: depend() fixes. Rename apply_fold() to fold(). Add first_completed(). Add min_failed(), max_failed(), any_results().
Concurrent: Add map_with() as an alias for flat_map().
Added support for automatic entropy underflow management.
Added minimal AES128 CTR DRBG random generator from NIST SP800-90Ar1
Compiler: Replaced resolv_program() with resolv_type(). The function resolv_program() (not to be confused with resolve_program()) is no longer used, so remove it. Move some inlined code from language.yacc to a function called resolv_type().
Sql.FutureResult: Rename raw_data -> data, and data -> get().
Sql.Promise: Improve docs.
Sql.Promise: New & improved interface.
Concurrent.Future: depend(),fold() and apply_fold(); interfacechange.
Sql.FutureResult: Polish docs and internals; uses status_command_complete instead of exception.
Promise: Rename fold_finish() to apply_fold().
Promise: Fix docs.
Promise: Add depend(), fold() and fold_finish() methods.
Sql.FutureResult: Module was missing from the commits.
Sql.FutureResult: Add a Concurrent.Promise interface to Sql.Connection.
Sql.Connection: Added missing methods (amongst others: asynchronous callbacks).
Doc fixes.
Thread.ResourceCount: New module to implement resource counters.
Crypto.SCRAM: New module.
Sql.Sql: Support error(void|int clear).
Nettle.Hash.HMAC: Fix non-default block_sizes(). Fall back to the default (generic) HMAC implementation if a non-standard block_size() is requested. Fixes [LysLysKOM 22249286]. Thanks to Stephen R. van den Berg <srb@cuci.nl> for reporting the issue. Also adds some related documentation.
pgsql: Update docs.
Debug.Inspect: Allow forced dumps.
Crypto.SCRAM: Made robust against rogue input and fix caching mechanism.
Crypto.SCRAM: Streamline API and update docs.
Crypto.SCRAM: Update docs.
Crypto.SCRAM: API updated, implemented serverside and testsuite.
Spelling fixes.
Change destroy() to _destruct().
Merge branch 'grubba/rename_lfun_destroy' into 8.1 * grubba/rename_lfun_destroy: Modules: Fixed logts of warnings. Testsuite: Updated for LFUN::_destruct(). Compiler: Don't complain about LFUN::destroy() in compat mode. Fix multiple warnings. Runtime: LFUN::destroy() has been renamed to _destruct(). Compiler: Rename LFUN::destroy() to LFUN::_destruct().
Modules: Fixed logts of warnings. More fall out from the LFUN::destroy ==> LFUN::_destruct rename.
Fix multiple warnings. LFUN::destroy() has been renamed to _destruct().
Compiler: Rename LFUN::destroy() to LFUN::_destruct(). As decided at Pike Conference 2017.
Add GL.GLSL and GL.GLSLUtil. This was a project by Per Hedbor that he intended to prepare for inclusion in Pike and shared as a separate repository on the pike-git server, albeit yet in the form of an external module. We have now performed the integration into Pike as a post_module.
time: Clarify and correct documentation.
push_*: handle possible evaluation errors Always evaluate the macro arguments before incrementing the stack pointer.
Crypto.Checksum: Update docs.
crc32c: Add optional seed to chain invocations.
WhiteFish.Blobs: Minor code cleanup. Get rid of a cast by using a different function. Also some documentation cleanup.
Pike.Lazy: Improved typing. Also some documentation updates.
Documentation [Standards.PEM]: Some minor clarifications.
Have the #(#), #[#], #{#} documentation show up.
Added some helper methods to Crypto. string md5(string data, void|bool raw) string sha1(string data, void|bool raw) string sha256(string data, void|bool raw) These methods return a hexadecimal string unless `raw` is given. These methods are composed via the helper method `hash()`. string hmac_md5(string secret, string data) string hmac_sha1(string secret, string data) string hmac_sha256(string secret, string data) These methods return a hexadecimal string. These methods are composed via the helper method `hash_hmac()` which can be used to construct new hmac hashing functions.
Crypto.ECC.Curve25519.EdDSA: Fixed documentation.
Removed trailing spaces.
Crypto.ECC.Curve25519.EdDSA: Added jwk(). Support generating JWKs for Ed25519 keys.
Added missing documentation note of month_days
US date formatting, m/d/y.
Crypto.ECC.Curve.ECDSA: Support strings in set_private_key(). Convenience variant for setting the ECDSA private key from a binary string. Fixes testsuite failure.
__builtin.Nettle.Hash._HMAC.State: Implement jwk(). Web.encode_jwk() should now support generating JWKs for HMACs. Fixes remainder of PIKE-39 (#8039).
Crypto.ECC.Curve.ECDSA: Implemented jwk(). Support generation of JSON Web Keys (JWKs) from ECDSA objects. Fixes more of PIKE-39 (#8039).
Crypto.RSA: Implemented jwk(). Support generation of JSON Web Keys (JWKs) from RSA objects. Fixes some of PIKE-39 (#8039).
Web [Crypto]: Added encode_jwk(). This is the reverse of Web.decode_jwk(). Note that no actual encodings are generated yet. Fixes some of PIKE-39 (#8039).
Stdio.File(): close() fixes. Unify low level calls close_fd() and close_fd_quietly() to reduce code duplication. Fix error handling in close_fd() and do_close_fd(). POSIX says that fds are closed on all errors except EINTR, where it is unspecified. Don't complain about close(2) failing with EBADF if it previously failed with EINTR. Ignore close(2) failing with ECONNRESET. This happens on eg FreeBSD when there is still data pending to be sent and the peer has already closed the connection. Fix documentation for the return value of close(). Potential fix for testsuite failures on FreeBSD.
Compat [__builtin.Sql.Result]: Added getter for master_sql. Fixes testsuite failure.
Crypto.ECC.Curve25519.EdDSA: Fixed API for set_private_key(). It now returns the current object.
Add extensible error handling in Protocols.DNS.server Some servers might want to report errors during other mechanisms than werror. By moving the error reporting to the new report_decode_error() method, it is easy to override that method and do something else. Likewise, some servers might want to not reply to malformed requests. Move that code to the new handle_decode_error() method.
Standards.PKCS: Fixed decoding of PKCS#8 public keys. The public key data is for some reason kept in a BIT STRING (and not in an OCTET STRING)...
Parser.PKCS: Added some support for PKCS#10.
Abstract lazy resolving into the Pike.Lazy module.
Gmp.mpz: Support negative exponents in `**() and ``**().
Nettle [Blowfish]: Don't throw errors in make_key(). Let make_key() generate a new key if the generated key isn't accepted by set_encrypt_key(). Fixes issue where eg Nettle.Blowfish()->make_key() intermittently threw errors due to the generated key being weak.
Fix spelling of 'necessary' in comments and docs
Last major release of Irix was in 1998 with support updates until 2006 and declared unsupported 2013. Remove support.
Fixed digest_info.
Pikedoc fixes.
Added short module description.
There was an XML syntax error in this file.
Filesystem.Monitor: Added report(). This provides an API to simplify targeting tracing of the Filesystem.Monitor operation. Fixes PIKE-17 (#8017).
Rename Debug.Peek to Debug.Inspect.
Manual: Extended the data types chapter with void & mixed. Added some elaboration on the difference between compile-time and run-time types, as well as some about the types void & mixed and how they are used. Fixes [bug 1656 (#1656)].
Debug.Peek: New class for interactive debugging and live data inspection.
Filesystem.Monitor: Fixed some typos.
System.FSEvents: Fixed Autodoc mk II markup.
Stdio: filesystem_stat() on Linux should now set fstype.
Added Concurrent, and moved changes not in the next release to next beta.
Charset: Improved Autodoc mk II markup.
Debug.Peek: Introduce Debug.globals and fix some cosmetics.
Add note about the added optional fields.
Revert MAC changes and fix MIME instead.
Standards.X509: Reduce code duplication. Use a common function for adding certificates in load_authorities(). Also some minor documentation updates.
Documentation [Crypto]: Fixed some references.
Documentation: Minor clarification.
Documented HMAC block size argument.
Decoding of negative integers was broken. Remove pike implementation completely and always use the C version.
Backend.PollDeviceBackend: Make set_signal_event_callback() optional. Fixes "Warning: An expression of type function({ Backend = object(implements _static_modules.Builtin()->DefaultBackendClass) } : void) cannot be assigned to a variable of type function({ Backend = object(implements _static_modules.Builtin()->Backend) } : void)." Thanks to Per Cederqvist <cederp@opera.com> for the report. Fixes [LysLysKOM 22067696].
search: Fixed end of range condition. Fixes testsuite failure.
5x faster encode_base64url() and 50% faster decode_base64url().
Crypto.ECC.Curve25519: Implement more of the standard API.
Web.encode_jws(): Make the media_type argument optional. RFC 7515 4.1.9 says that the header field "typ" is optional, so make it possible to generate such JWSes.
Web: Added {en,de}code_jws(). These simplify handling of JWSes other than JWTs. Also cleans up {en,de}code_jwt() by using the new functions. Also fixes a typo in the type for decode_jwt().
Nettle.Curve25519: Improved support for JOSE. Implements some parts of RFC 8037.
Web.decode_jwk: Added support for decoding of symmetric keys. Decoding of JWK encoded keys for SHA256/HMAC, etc is now supported. Also adds some convenience variants.
Sql.sqlite: Added big_typed_query() et al. The SQLite glue should now use the same conventions as the others.
Documentation [MasterObject]: Documented some more stuff. Also restructures the markup a bit.
Sql.Sql: Sql.Sql() is now a function and not a class. Now that __builtin.Sql.Connection exists and is used, the wrapper class should no longer be needed. The wrapper class has moved to 8.0::Sql.Sql.
Documentation [SQLite]: Fixed AutoDoc markup typo.
SQLite: Use __builtin.Sql. Also some AutoDoc markup adjustments.
Oracle: Use __builtin.Sql.
Odbc: Use __builtin.Sql.
Documentation [Nettle]: Fixed cut'n'paste typo.
Added a global on_failure to capture errors not handled in individual promises.
search: Added end argument for the string and array cases.
Web: Added decode_jwk{,_set}(). These decode JOSE keys coded according to RFC 7515. Backported from Pike 8.1. Fixes PIKE-3 (#8003)
Web: Updated {en,de}code_jwt() to support HMAC signatures.
Support multiple instances of the same option.
Added Unicode.is_whitespace.
Crypto.ECC: Unified the APIs for Curve and Curve25519. new_scalar(), get_x() and get_y() now return coordinates in the preferred representation for the curve (ie either as a Gmp.mpz or as a string(8bit)). create(), set() and `*() accept that same value as argument. Added get_{x,y}_{num,str}() to get a specific representation. jose_name() now has a default implementation that returns UNDEFINED. Added default implementation of get_curve(). Curve25519 now has a Point subclass.
Documented finish.
Web: Added decode_jwk{,_set}(). These decode JOSE keys coded according to RFC 7515.
Documentation updates
Crypto.ECC.Curve.Point: Allow creation without arguments again. Points now default to 'g'. Fixes several testsuite failures.
__builtin.Nettle.ECC_Curve.Point: Require coordinate. Nettle doesn't provide a default coordinate on initialization, which means that Points created without arguments are likely to be off the Curve. Don't allow this for now. Possible alternatives would be to initialize to 'g' or infinity.
Crypto.ECC.Curve: Added some more support for JOSE. Added jose_name(), which returns the RFC 7518 name for the curve. It is now possible to initialize a Point from a JWK.
Crypto.RSA: Support initialization from a JWK mapping.
Add _sprintf() to Thread.Mutex() that prints current locking thread. Switch to hex-based thread IDs in _sprintf() for Thread.Thread() for easier reading.
Web.Auth.OAuth2: Cleaned up get_token_from_jwt() a bit. Use the convenience functions instead of doing lots of stuff by hand. As a side effect: Support DSA and ECDSA signatures in addition to RSA. As a side effect: Issued tokens now have the "jti" claim set.
Documentation: The DefaultCompilerEnvironment is a CompilerEnvironment.
Calling crypt() now crypts a random string. Useful as a quick way of generating passwords.
Have Charset.Encoder->set_replacement_callback return the object for call chaining.
Added Nettle.version()
Concurrent: Support creating promises from traditional methods with callbacks.
Concurrent: then() and thencatch() support smart mapping, improve docs.
EngineIO/SocketIO: Minor cosmetic fix and adjust reference in CHANGES.
SocketIO/EngineIO: Packetised queues, increment protocol versions for both. SocketIO/EngineIO: Drop id parameter, it's better solved using inheritance. EngineIO: Prioritise PING/PONG messages.
Arg: Moved handling of --help to a standard option. This allows for having overriding the name of the --help option, and for having multiple aliases for it (eg also -h). Also adds a default help description string for --help.
Documentation [__builtin.Nettle.HMAC]: Fixed some doc bugs.
Added example uses.
Expanded on failure condition on async_connect.
Concurrent: Doc fixes and simplify code.
Documentation [sscanf]: Fixed some typos and markup.
Fix some references.
Major redesign of the refdoc. Also removed some obsolete stuff.
SOAP: Use kebab-case for naming convention, clone arguments by value.
SOAP: SOAP glue functions including a limited wsdl parser.
Concurrent: Implement then() and other JavaScript APIs.
Fix doc prototype.
More narrow documentation.
More documentation.
Move Markdown code to Tools, as it is more of a tool than a module.
Revert sprintf %m.
program.c: autodoc fix
Link to RFCs in both places, not just one (oops)
Mention RFC 2047 as well as RFC 1522 (since the former obsoletes the latter)
Add note linking MIME headers with the way they're decoded
Revert "Add Unicode support to MIME.Message via a subclass." This reverts commit 5d65d4d56ddbfea827045c8aa690ea7a5654abaa.
Revert "MIME.UnicodeMessage: Decode the message body according to the charset" This reverts commit 94c5393bc0ebea8e6eb0701c5c04b6554d627159.
MIME.UnicodeMessage: Decode the message body according to the charset
Add Unicode support to MIME.Message via a subclass. * Minor refactoring that should have no visible external effect * One new constant, _is_unicode_message, which changes how create() parses the headers * An additional (optional) parameter to parse_headers() * New class MIME.UnicodeMessage, documented as being 21-bit Unicode everywhere, with deviations from such considered to be bugs.
Concurrent: Improved thread safety for Results and Fold.
Concurrent.Promise: Changed API somewhat. success() and failure() now throw errors if the Future isn't in the STATE_PENDING state. Add try_success() that is a noop if the Future isn't pending. Renamed maybe_failure() to try_failure() for symmetry with try_success(). Clean up FirstCompleted by using try_success() and try_failure().
Concurrent.Future: Support multiple concurrent callbacks. Multiple success and failure callbacks may now be registered concurrently. This is needed to avoid surprises when implicit callbacks are installed by eg flat_map(). Also some related documentation updates. Fixes [bug 7816 (#7816)].
Fix docs typo
Crypto [Nettle]: Inlined __builtin.Nettle.DH_Params into Crypto.DH. This is needed to break a circular Nettle-load time dependency. The problem was that __builtin.Nettle.DH_Params had a preprocessor dependency on Nettle.DH_Params, and __builtin.Nettle.DH_Params could sometimes get loaded while Nettle was still initializing.
The start parameter to search() has no effect on mappings.
Documentation: Documented optional further args to search(). Also fixes the types for search() and lfun::_search().
Documentation [Nettle]: Improved doc for Curve25519.
Nettle: Added support for Curve25519 and EdDSA25519. CAVEAT: API subject to change! Note that these have a different API than the other ECC curves due to a different low-level API.
Reinstate clone() documentation.
clone() is not referenced, so define it away for now.
Gz.deflate: Add clone() method; allows for testing compression. Gz.File: Support setting the window_size in setparams(). Gz: Support Z_BLOCK and Z_FULL_FLUSH.
Compiler: Added CompilerEnvironment()->lock(). This is a class that exposes the compiler internal lock. Fixes some of [bug 7783 (#7783)].
Added _sprintf
Correct encoding of arrays. Order must be sequential.
Fix typo in docs
Math: improved spelling of documentation
Debug: Added hexdump().
Allow for only specific tags to be parsed and the rest ignored.
Reuse the tag parsing loop.
Crypto.DH [SSL]: Updated with references to RFC 7919.
Documented some of the new Randomness subsystem.
Stdio: Added enums File{Mode,Property}Flags. Also adds corresponding documentation. Fixes some broken documentation references.
Tools.AutoDoc: Added support for @enum/@endenum. It should now be possible to document enums at the C-level.
Tools.AutoDoc: Keep undocumented enums with documented constants. Add an implementation of containsDoc() to the PikeObject base class, and cleanup the special case for classes a bit. Adds documentation for enum Standards.X509.CertFailure (and probably others).
Fixed autodoc
Even more specific types of crypt in documentation.
Calendar [FAQ]: Removed references to localization.h. localization.h is no more, so don't refer to it.
Additional documentation about forks
Added Int class used to case argument to integer. Made Default type agnostic.
More comment update.
Documentation improvement.
Efuns: Renamed hash_7_8() to hash_8_0() for consistency. Also made the hash_*() types somewhat stricter.
Efuns: Fixed multiple issues with the new hash(). It now follows the same API as the old hash() (ie the second argument is a modulo). It now also has the same value range as the old hash (eg it no longer returns negative values), and returns the same value on both 32-bit and 64-bit platforms. As a side-effect this fixes a buffer read overrun when the second argument was larger than the length of the string. It no longer leaves droppings on the stack. Updated the testsuite.
Renamed 'hash' to 'hash_7_8' Hopefully it will not be used now (this is triggered by yet another person thinking it's a good string hashing function). The function is not really suitable for use. The returned values are higly likely to be the same for similar strings, something that is not optimal for hashing. Also added comments about what to use instead (hash_value or one of the cryptographic hash functions). It might make sense to add a hash() that actually works, perhaps siphash?
Standards.PKCS.PFX: Added minimal AutoDoc markup.
Crypto.NTLM: Added some primitives from [MS-NLMP].
Concurrent.Promise: Fixed potential backtrace in destroy(). destroy() now uses unlocked_failure() to avoid potential recursive mutex locks. Also converted some comments to AutoDoc.
Gmp.mpz: Simplify `**(). Let ``**() convert its argument to a bignum and call `**() in it. Also adds some special cases. This fixes some remaining testsuite failures.
Documentation.
Gmp.mpq: Allow initialization with Gmp.mpz objects.
Several more ** cases implemented: int ** mpq -> mpq or float float ** mpq -> float mpq ** int -> mpq mpq ** float -> mpq or float mpq ** mpq -> mpq or float mpz ** mpf -> float I am open for suggestions on whether or not the type promoting going on is sane or not. It's at least slightly logical.
Concurrent.Future: Added timeout().
Concurrent.Promise: Added maybe_failure().
Documentation breakage When preparing a new Debian package of 8.0.240, I noticed that some documentation disappeared, apparently due to the following. The error in Nettle.Sign is already corrected but not the others, AFAICT, and I'm guessing that @exp should be @expr. Patch below, which I home someone can apply right away. ERROR: <Invalid error container: Tools.AutoDoc.AutoDocError(SourcePosition(File: /build/pike8.0-8.0.240/src/modules/Gmp/mpz_glue.c, lines: 668..668), "DocParser", "@exp cannot be used like this: @exp{ ... @}")> ERROR: <Invalid error container: Tools.AutoDoc.AutoDocError(SourcePosition(File: /build/pike8.0-8.0.240/lib/modules/__builtin.pmod/Nettle.pmod/Sign.pike, lines: 102..102), "DocParser", "@item is not allowed inside @array (allowed children are: @elem)")> ERROR: <Invalid error container: Tools.AutoDoc.AutoDocError(SourcePosition(File: /build/pike8.0-8.0.240/lib/modules/Crypto.pmod/ECC.pmod, lines: 286..286), "DocParser", "@item is not allowed inside @array (allowed children are: @elem)")> ERROR: <Invalid error container: Tools.AutoDoc.AutoDocError(SourcePosition(File: /build/pike8.0-8.0.240/lib/modules/Crypto.pmod/RSA.pmod, lines: 368..368), "DocParser", "@item is not allowed inside @array (allowed children are: @elem)")>
Documentation [Concurrent]: Fixed a few typos.
Tweaked the pow function in bignums/mpz o If the exponent is a float, the result will now be a float (ie, 1000000000000**0.5 is now 1000000.0, not 1) o Added ``** o Removed arbitrary size limit entirely. It's up to the programmer to avoid 999 ** 9999999999999999999999. You still get an error if the result is too large for your memory.
Filesystem.Monitor: Potential fix for [bug 7684 (#7684)]. Avoid recursive mutex locks from destroy() by only locking on explicit destruct().
Renamed pow lfun from pow to `**. This is somewhat incompatible, previously pow(X,3) called X->pow, now it will call X->`**.
Extended glob pattern syntax: o \ can now be used to quote special characters in the pattern o [ can be used for ranges of characters ([bx] [a-c0-9] [^a] etc). Also changed glob to return the matching glob instead of 1 when an array is passed as the first (pattern) argument. This can be used to remove some loops where you want to do different things depending on which pattern matched. Both these changes are incompatible.
Filexyxtem.Monitor: Got rid of update_monitor(). The function didn't work, and if it had worked it would've messed with the internal state of monitors and caused stuff to break.
Protocols.HTTP.Query: Fixed an old bug (https://bugzilla.roxen.com/bugzilla/show_bug.cgi?id=7676) where basically the timeout in timed_async_fetch wasn't reset each new data read. This timeout had it's own property (data_timeout) which now is "deprecated" since the property "timeout" pretty much serves the same purpose, but on connection. So now timeout is the default value to use unless data_timeout is explicitly set, in which case that value will be used. Since there was no proper way to set a max time for the entire operation (well, that was by accident and not intention how the data_timeout in timed_async_fetch worked) the new property "maxtime" has been added. If this is set (default is 0=indefinetly) the request will be aborted after maxtime second even if data is still being read. So in short: data_timeout = 0 // unless explicitly set timeout = 120 // connection timeout, and then data read timeout maxtime = 0 // 0 = off, otherwise the entire operation must be done within maxtime seconds or else the request is aborted Protocols.HTTP.Session: Added some documentation Protocols.HTTP.Promise: New module which utilise the new Concurrent.Promise/Future stuff for HTTP requests. Internally uses Protcols.HTTP.Session for the actual HTTP stuff. Web.Api.Api: Now fetches data asynchronously when async calls are made. Concurrent: on_success and on_fail now returns the object being called so that they can be chained. Parser.Markdown: Fixed the #require macro directive.
Don't pretend jwa is a variable, to be consistent with name(), digest_size() etc.
Compiler: this_function now obeys the inherit specifier.
Crypto.None: Added the plaintext algorithm.
Crypto.AE: Added class.
Nettle: Added __builtin.Nettle.AE. This is Authenticated Encryption without Authenticated Data.
__builtin.Nettle.Hash [HMAC]: Somewhat stricter type for `jwa(). Also adds some documentation.
__builtin.Nettle.MAC: Added support for HMAC-based JWS signatures.
Documentation [__builtin.Nettle.Sign]: Improved JOSE doc.
Implemented HMAC jwa with a little less code.
Auto bignum is always true
Documentation [__builtin.Sql.Connection]: Fixed minor typo.
Inotify: Simulate events for directory content at watch creation. This reduces the race window for tracking directory content. It also simplifies code that does such tracking. Also updates Filesystem.Monitor.basic accordingly.
Added documentation of array as second argument to map().
Documentation [Crypto.RSA]: Fixed cut and paste error.
__builtin.Nettle.Sign: Added JOSE default implementation. Fixes some warnings in SSL.Context. Adds default implementation of jose_{sign,decode}() for signature algorithms that JOSE doesn't support (eg Crypto.DSA).
Web: Added {en,de}code_jwt(). Fixes most of [bug 7671 (#7671)].
Crypto.ECC.Curve.ECDSA: Added support for jose_{sign,decode}().
Crypto.RSA: Some API changes for jose_sign(). Changed argument order and added defaults for the hash algorithm.
Documentation [RSA]: Minor doc change for jose_decode().
Crypto.RSA: Add some support for JOSE JWS signatures.
Crypto: Added algorithm identifiers from JWA (RFC 7518:3.1). Fixes some of [bug 7671 (#7671)].
Web.Api.Api: Fixed so that POST actions actually works. It now also tries to detect if a file is being uploaded. This works by looking at the parameter values and if a value contains "filename=..." a multipart message will be created. So in short this would look something like: string file = "the-file.png"; string data = Stdio.read_file(file); mapping params = ([ "the_file_param" : "filename=" + file ]); api->post("method/endpoint", params, data);
Apple.Keychain: Added initial parser of Keychain files. CAVEAT: This is a VERY limited implementation, and is only intended to be used to extract root certificates from "/Library/Keychains/X509Anchors".
Concurrent: Added some more functions.
Tools.Shoot: Add an optional prepare function to create test vectors This removes the need to put huge constants into the test program, greatly reducing the memory requirements to run the benchmarks.
Remove deprecated cast_to_int, cast_to_string and cast_to_float.
Fixed the early out when random limit is 2^x and !(x%8). Also some document fixes.
Crypto.RSA [SSL.Context]: Fixed some warnings.
Nettle.KW: This is an AE and not an AEAD.
__builtin.Nettle.__Hash: Some minor fixes.
Nettle.BlockCipher16.KW: Changed API to AEAD.
Nettle: Added BlockCipher16.KW. This is the AES Key Wrapping algorithm from RFC 3394.
Concurrent.Future: Added some common methods. Also adds some more documentation.
Concurrent: Added new module for handling asynchronous results.
Web.Api.Api: When running async the requests gets stored so that they can be forcefully closed if something goes wrong. The public method "close_connections()" will close all open connections.
__builtin.Sql.Connection: Simplified handle_extraargs() somewhat. handle_extraargs() is never called with any preexisting bindings, so there's no need to support that case.
__builtin.Sql.Connection: Use variants for big_query() et al. This simplifies overloading by the various drivers.
__builtin.Sql.Connection: Added emulate_bindings().
Documentation [Nettle.Hash]: Fixed typo.
Thread.Farm: Fix documentation typo.
__builtin.Sql.Result: Removed remains of master_res. This is a base class and not a wrapper...
Sql: Added __builtin.Sql.{Connection,Result}. These will be the new base classes for the Sql API.
Threads: Some minor documentation cheanges.
Doc fix.
Use random_string instead of Crypto.Random.random_string
Documentation fixes.
Random: Updated some types and AutoDoc.
Base Crypto.Random on Random.Fast. We should probably just deprecate Crypto.Random completely.
Documentation: Added note about some failure modes of rm().
Addressed code review issues
GC [arrays]: Added do_gc_weak_array(). It is now possible to request a quick gc of an array with weak references.
Documentation [GC]: Fixed typo.
GC [mappings]: Added do_gc_weak_mapping(). It is now possible to request a quick gc of a mapping with weak references.
Provide URL to documentation.
Documentation [cpp]: Move some doc to the correct place.
Added Nettle.version
Filesystem.Monitor: Improved thread safety. ADT.Heap is NOT thread-safe (especially not pop()), so use a Thread.Mutex to control access to the monitor_queue. Potential fix for [bug 7644 (#7644)] (where the top element becomes zero instead of an ADT.Heap()->Element).
Revert "Added the Markdown module and the standalone pike_to_html.pike from 8.1." This reverts commit 2706a62c3c72b296a0c77b418f1e186c682c7ac5.
a few autodoc fixes
Added creation method raw to put binary data directly in an image object without adaptation.
Documentation [HTTPAccept]: Fix all references to HTTPLoop. HTTPLoop installs itself as HTTPAccept, so fix the documentation to reflect this. Thanks to Eiichiro ITANI <emu@ceres.dti.ne.jp> for the report.
Git: Added convenience function try_git(). Also updates the documentation somewhat.
Added Fast random generator.
Added support for hardware random generator.
Merge branch 'new-pikedoc' into 8.1 * new-pikedoc: New doc layout done! (well, as far as I can tell anyway). It now also works on local modules with module_modref as build target. Start of new layout for the Pike refdoc
New doc layout done! (well, as far as I can tell anyway). It now also works on local modules with module_modref as build target.
Added the posibility to define the HTTP request timeout in async queries. Fixed a typo in pike_to_html.pike
Added the Markdown module and the standalone pike_to_html.pike from 8.1.
Removed f_random[string,seed] from module API.
Remove now unused random() implementations.
Make random_seed() a noop and deprecate.
Don't call f_random directly.
Added HMAC-based One-Time Password generator.
Sql: Added support for multiple results. Adds Sql.sql_result()->next_result().
strlen is for strings only these days
Make Testsuite objects iterators.
Documentation
Move pike compat handling into the Testsuite object.
Move M4Test into M4Testsuite
Moved testsuite parsing to M4Testsuite.
Documentation [Nettle]: Some minor cleanups.
Nettle: Added block cipher mode ABC (Accumulated Block Chaining).
Fixed the _sprintf code.
Documentation [Nettle]: Added note about IGE.
Nettle: Added block cipher mode IGE (Infinite Garble Extension).
On this branch portable bytecode generation is not normally done This actually noticeably speeds up the compiler and also saves about 15% of RAM when starting hilfe. If the decode_value code is modified to free the portable bytecode _TWICE_. This will not be merged to 8.1 until I figure out _why_ that is. It should only need to be freed once, but I guess it's in the string table twice?
Added Standards.MsgPack MsgPack is a binary serialization format. It supports encoding of the usual basic data types and additionally allows for representing arbitrary data types through extensions. It is designed to be more compact than JSON and can be considerably faster to encode and decode.
Documentation [Nettle]: Fixed some minor issues.
Documentation [Standards.X509]: Fixed typo.
Move compilation to the test object.
Gmp.mpf: Added quite a few functions in the MPFR case.
Moved parsing of tests to Testsuite objects.
Gmp.mpf: Use mpfr if available.
Added Standards.MsgPack
Standards.PKCS.RSA: Added oaep_algorithm_id(). Also adjusts the RFC reference for pss_signature_algorithm_id().
Crypto.RSA: Added support for RSAES-OAEP (RFC 3447:7.1).
__builtin.Nettle.Hash: Added eme_oeap_{encode,decode}(). These two functions implement the encoding used in RSAES-OAEP.
Crypto.RSA: Added pkcs_signature_algorithm_id() for PSS.
Documentation [Standards.URI]: Added some RFC markup.
Removed remnants of Pike.Security
Crypto.RSA: Fixed multiple typing errors. Values of type Crypto.RSA.LowState should be possible to assign to variables declared as Crypto.RSA.State without drowning in warnings... Also restores the default return value of name() to "RSA".
Standards.PKCS.RSA: Added pss_signature_algorithm_id().
Standards.PKCS.Identifiers: Fixed typo. "mfg1_id" ==> "mgf1_id". Also adds "pspecified_id" and some RFC references.
Added an example of how to use an OAuth2 derived module
Crypto.RSA: Cleaned up RSASSA-PSS API. It is now possible to get a PSS signature object from an RSA state by indexing it with "PSS". Eg Crypto.RSA.State rsa = ...; string(8bit) sign = rsa->PSS->pkcs_sign(message, hash); int(0..1) ok = rsa->PSS->pkcs_verify(message, hash, sign); Also adds a corresponding symbol "PKCS1_5" for the traditional (and default) PKCS#1 1.5 style signatures.
Added CRC32C
Macros without trailing newline/space got truncated by one char
Added a more natural place to find checksums (though not strictly cryptographic).
Expose adler32.
Added a tool for converting a Pike file into a syntax highlighted HTML file. Usage: pike -x pike_to_html source.pike > source.html This will create a full HTML page. The class also has a public method for converting a string of code into highlighted HTML. This will leave out the <html>...<body> parts in the result.
Crypto.RSA: Added pss_{sign,verify}() implementing RSASSA-PSS. Note: This is a temporary API as it is not compatible with the __builtin.Nettle.Sign API.
load_authorities takes hundreds of milliseconds. Include an optional cache.
__builtin.Nettle.Hash: Added emsa_pss_{encode,verify}(). These are algorithms specified in RFC 3447 section 9.1, and are primitives used in signatures with RSASSA-PSS.
Functions can not return void|string or void|object.
Updated documentation to mention that verify_certificate_chain also can return CERT_UNAUTHORIZED_CA and CERT_EXCEEDED_PATH_LENGTH.
Comment fixes.
Crypto.DH.Parameters: Added generate_keypair(). Also: Crypto.DH.DHParams now uses Nettle.DH_Params if available.
Crypto.DH: Added module. This module currently just contains the DH.Parameters class and the MODP groups from RFC 2409, RFC 3526 and RFC 5114. Actual Diffie-Hellman key-exchange is not implemented here (yet).
Documentation [Stdio.sendfile]: Document risk of SIGPIPE. Make sure that users of sendfile() are aware of [bug 7582 (#7582)] behavior.
Nettle: Added Galois Counter Mode (GCM) This is in large parts a rewrite of the Pike 8.0 implementation.
Change CAMELLIA to Camellia to be consistent with other non-acronym ciphers.
Crypto: Added SHA 224, 384 and 512. Also adjusts the ASN.1 identifier for SHA256 to make Pike 8.0 happy.
Crypto: Backported Crypto.[DR]SA()->pkcs_{sign,verify}() from Pike 8.0.
Documentation [JSON]: Added RFC markup.
Backported encode/decode base64url
Backport from 8.1