Open Dylan 2019.1

This document describes the 2019.1 release of Open Dylan, released 31 March, 2019. This release contains many enhancements and bug fixes, highlights of which are listed below. For complete details see the commit logs.

LLVM

The LLVM back-end, which uses LLVM 7.x or later for code generation, is now full-featured and mature on i386 and x86_64 Linux, FreeBSD, and macOS platforms. Features of the LLVM back-end include:

  • Full source-level debug information

  • Zero-cost nonlocal exit support

  • Arithmetic exception handling

Note that LLVM is not yet the default back end so for now you must add -back-end llvm to the command line when building if you want LLVM. We expect to make it the default in an upcoming release.

Note also that the binary releases do not assume that you have a sufficiently new release of the LLVM Clang compiler, and are configured to use the platform default C compiler instead. If you have installed Clang 7.0 or later, you can edit share/opendylan/build-scripts/config.jam and set the CC variable to point to your Clang executable:

CC                ?= clang-7 ;
CCFLAGS           ?= -w -g -O2 ;
C++               ?= clang++-7 ;
C++FLAGS          ?= $(CCFLAGS) ;

Compiler

  • The compiler emits warnings for issues with define sealed domain:

    • Incorrect number of type specializers provided. The number of type specializers must match the number of required parameters for the generic function.

    • Type specializers are not subtypes of the corresponding required argument of the generic function.

      For example, this is useful for catching an incorrect sealing of make:

      define sealed domain make (<my-class>); // Wrong
define sealed domain make (singleton(<my-class>)); // Correct

    • Previously, define domain was accepted without warning. This is not valid Dylan syntax as it should be define sealed domain.

  • Dylan functions that are marked as not-inline now take that into account when generating code in the LLVM and C back-ends.

  • The C back-end no longer generates invalid C when outputting a float with a value of infinity or NaN.

  • Some bugs that didn’t permit valid floating point literals to be used have been fixed. These are typically edge cases like +.5 or -3d3 rather than commonly used literal notations.

  • The C back-end correctly handles indirect C-functions (where a function pointer is given to be invoked rather than a direct function call).

  • Warnings and errors are now colorized when printing on supporting output devices.

  • The compiler progress messages are now less verbose unless the -verbose command-line option is supplied.

  • Warnings that refer to primitive and C functions are now clearer.

  • A bug in the C back-end that broke required return values combined with type-checked #rest return values has been fixed.

  • An erroneous validity check for the Base-Address: keyword in LID or HDP project files on Windows has been fixed.

  • The HARP back-end can now handle raw FFI calls that return <raw-single-float> or <raw-double-float> values.

  • The compiler command line, as well as the interactive build and link commands, accept a -jobs option to control the number of concurrent external build processes during the link stage. Setting this to the number of available CPU cores can provide speedups for large builds.

  • The experimental #name:foo syntax has been changed to #:name:foo. Requiring the initial colon will prevent conflicts with new hash literal syntax that may be added in the future. Several additional delimiter characters may now be used: `...`, '...', |...|

Run-time

  • Support for printing a backtrace (with demangled Dylan function names) when an unhandled error condition is signaled has been implemented. Note that this requires the use of the optional libunwind library.

  • Support for handling “invalid” floating point exceptions has been added. These are generated when taking the square root of a negative number and other similar situations where the numerical domain is invalid.

    When an invalid floating point exception occurs, <arithmetic-domain-error> will be signaled (much like the other <arithmetic-error> situations).

  • Limited integer instance checks have been optimized. This will have the most impact when using the LLVM compiler backend.

  • In the C runtime, primitive_sleep now functions correctly.

  • Two new primitives, primitive-read-cycle-counter and primitive-read-return-address, have been added. These are useful for writing profiling and event logging tools.

Debugging

  • There is a new dylan-lldb wrapper script which can be used to launch lldb and pre-load the Open Dylan LLDB integration scripts. lldb is the debugger that is part of the LLVM project. It is the default debugger on macOS.

    If you need it to launch a custom build of LLDB, you can set the OPEN_DYLAN_LLDB environment variable to point to an alternative lldb executable.

Documentation

  • The documentation generator included within the compiler has been updated to produce better and more accurate documentation that will require less manual editing. To use it, after compiling your library, use this command:

    export -format rst -file my-lib.rst interface-reference

Build System

  • The arguments to configure for specifying which garbage collector to use have changed. The arguments are now:

    --with-mps=DIR
--with-gc=DIR
--with-harp-collector=mps|boehm|malloc
--with-c-collector=boehm|malloc
--with-llvm-collector=boehm|malloc

    The --with-mps option (for pointing to the Memory Pool System source directory) is only needed when using the MPS collector with the HARP back-end on either 32-bit x86 FreeBSD or 32-bit x86 Linux.

    The system should be able to find the Boehm-Demers-Weiser collector automatically if it is in the standard system install directory; the --with-gc option should only be needed if it is installed in a non-standard location.

    The default collectors are MPS for the HARP back-end’s run-time library, and the Boehm-Demers-Weiser collector for the C and LLVM back-ends. These shouldn’t be changed unless you’re sure you know what you’re doing.

  • A new option for building C++ code, c++-source-files, has been added to LID files. This matches the c-source-files: keyword.

  • A failure to find shared libraries on Ubuntu 17 & 18 was fixed. See GitHub issues 1064 and 1197.

Library Changes

dylan Library

  • Symbol comparisons when using \= are now the same as using \== rather than being significantly more expensive. See issue #899.

  • The function dispatch implementation now makes a proper distinction between <single-method-engine-node> and <keyword-method> with the help of a new primitive-callable-as-engine-node? compiler primitive.

  • Integer and floating point literal syntax is changed to allow underscore between any two successive digits, for readability. See DEP-0011 for details.

common-dylan Library

  • The transcendentals module now has a sincos generic function.

  • The transcendentals module now has a hypot generic function.

  • The transcendentals module now has an ilog2 function that returns the integer value of the logarithm of a value in base 2.

  • The transcendental and hyperbolic functions are no longer sealing their domains on <real> parameters.

  • Again in the transcendentals module, many methods have been removed that converted from <real> to <float>. Callers should pass in values of the appropriate type and precision instead. This impacts:

    • ^, exp, log

    • sqrt

    • sin, cos, tan

    • asin, acos, atan

    • sinh, cosh, tanh

    • asinh, acosh, atanh

  • The common-extensions module now provides a classify-float method which determines whether the given float is #"normal", #"zero", #"infinite", #"nan", or #"subnormal".

  • The byte-vector module now provides hexstring and from-hexstring methods for fast conversion between <byte-vector> and hexadecimal strings.

  • The thread module has gained a current-thread-id function. The thread-id is also available for any <thread> object.

  • The simple-profiling module now exports start-profiling and stop-profiling rather than requiring that users directly invoke start-profiling-type and stop-profiling-type multiple times.

  • The machine-words module now knows how to count the bits set in a machine word via %count-ones. This is also available as a new compiler primitive, primitive-machine-word-count-ones.

  • Mismatches in the use of internal-use raw types have been resolved.

io Library

  • An implementation of indenting streams for handling indented text output has been added. See <indented-stream>, indent, and with-indentation in the streams module.

  • Some generic functions that apply to <buffered-stream> have had their signatures tightened.

  • Mismatches in the use of internal-use raw types have been resolved.

system Library

  • A new machine-concurrent-thread-count function, which returns the number of active CPU cores or execution threads, has been added to the operating-system module.

  • New specializations on as have been added for creating locators from strings for the <file-system-directory-locator> and <file-system-file-locator> classes. These aren’t typically used but their omission led to possible confusion for users.

  • A problem with constructing <date> objects for time_t values with more than 30 bits (i.e., any time after Sat Jan 10 13:37:04 UTC 2004) has been fixed.

  • Mismatches in the use of internal-use raw types have been resolved.

c-ffi Library

  • Nested C structs by value are now supported by the C backend. Previously, using this construct would result in a compilation error in the generated C code.

  • Using %call-c-function or %objc-msgsend with a void result type no longer requires specifying a result type of <raw-c-void>. Simply putting an empty value list is sufficient:

    %call-c-function ("SetLastError", c-modifiers: "__stdcall")
    (dwErrorCode :: <raw-c-unsigned-long>) => ()
  (integer-as-raw(0))
end;

    Previously, this required specifying the result type explicitly:

    %call-c-function ("SetLastError", c-modifiers: "__stdcall")
    (dwErrorCode :: <raw-c-unsigned-long>) => (nothing :: <raw-c-void>)
  (integer-as-raw(0))
end;

testworks Library

  • It is no longer necessary to use suites. By default run-test-application runs all defined tests.

  • Test results are now colorized when the output stream supports it. Green for success, red for failure, etc.

  • Test summaries are more concise and easier to read.

  • Added support for expected failures.

  • New assertion macros assert-instance? and assert-not-instance?.

  • Many other minor improvements including to the documentation.

collections Library

  • The bit-count operation from bit-vector (also used by <bit-set>) has been changed to use the new primitive-machine-word-count-ones that has been added in this release.

coloring-stream Library

variable-search Library

  • A bug that caused intermittent crashes on FreeBSD has been fixed.

Contributors

We’d like to thank all the people that made contributions to this release and to surrounding libraries in the Dylan ecosystem. This list is probably incomplete…

  • Bruce Mitchener

  • Peter S. Housel

  • Carl Gay

  • Peter Hull

  • Fernando Raya

  • Alfredo Beaumont

  • Wim Vander Schelden

  • Kamil Rytarowski

  • Ingo Albrecht

  • Dan Midwood