newLISP® v.10.0.2 Release Notes March 9th, 2009
Versions 10.0.1 and 10.0.2 are maintenance updates to release 10.0.0. Some minor feature enhancements and changes are noted in the next (search for 10.0.1 and 10.0.2).
newLISP version 10.0 marks a new generation of newLISP with new abilities, such as reference returns instead of copies for many built-in functions and a generalized method to modify lists, arrays and strings in-place. Version 10.0 is also a cleanup and streamlining of functionality added during the 8.x and 9.x series of releases.
Other details in the built-in functions API have been changed to make functions work better together. Use the conversion guide at the end of these release notes to help you convert existing files for use with newLISP 10.0.
The changes introduce incompatibilities with the previous versions of releases. Any production source running on the new generation of newLISP should be carefully checked against these release notes.
Many thanks to the community of newLISP users for their ideas and contributions.
Any built-in function returning a list, array or string, or an element of a list, array or string, will return a reference and not a copy if the list or array is bound to a symbol. Destructive functions can now work on the return value of other functions modifying the original list, array or string. The new setf can be used to modify an element of a list, array or string referencing it via one of the many available list, array and string accessors:(set 'lst '(a b c)) (setf (first lst) 99) lst → (99 b c) (setf (lst -1) 'C) lst → (99 b C) (set 'A '((a 1) (b 2) (c 3))) (push 'z (assoc 'c A)) A → ((a 1) (b 2) (z c 3)) (set 'L '(a b (c d e f g))) (= (replace 'f (nth (+ 1 1) L) 'z) '(c d e z g)) (= L '(a b (c d e z g))) (setq s "abc") (setf (s -1) "C") (setf (nth 1 s) "B") (setf (first s) "The A") s → "The ABC"
The following functions return a reference to the list, array or string or element of a list or array, which can be modified by other functions. Previously these functions returned a copy (with the exception of set):assoc, eval (index references v.10.0.2), first, last, lookup, ntn, replace, reverse, rotate, set, setf, setq, set-ref, set-ref-all, sort, swap
All control structures without local variables maintain a reference for the last returned value in the statement block, if that value was a reference:amb, begin, if, if-not, when, unless, while, until, do-while, do-until, cond and case.... and added in v.10.0.1:for, doargs, dolist, dostring, dotimes, dotree.
Now all built-in functions except let, letn, letx and local can return references.
To force the return of a copy instead of a reference, enclose the expression in the new copy function:; sort a copy not the original list (sort (copy myList))
Normally there are few places in program code that rely on returning copies. In fact, the new possibility to work directly on returned references will give many opportunities to write terser and speedier code.
Functions setf and copy
setf is a new function working not only on symbol references like setq but also on any reference returned from an expression. The function is well known under the same name in other Lisp dialects and works in a similar way. setq and the new setf both point to the same internal code and are interchangeable. It is recommended to use setq for setting symbol variables and use setf whenever a sublist, sub array or substring is referenced, Note that any list, array or string changed must be anchored in a symbol referencing it. If no reference is found, an error message is returned:(setq l1 '(a b c)) (setq l2 '(d e f)) (setf 1 (append l1 l2) 99) ERR: no symbol reference found in function setf
Allthough the partial lists are anchored in l1 and l2 the return value from the append function is not referenced by any symbol and cannot be used by setf.
To make destructive functions non-destructive, a new copy function has been introduced:(replace elmnt (copy aList) newElmnt)
In the example aList will not be changed although replace is destructive.
Anaphoric system variable $it
The new system variable $it is used when doing self-referential assignments in setf and hashes. It can also be used in many cases as an alternative choice where the system variable $0 is used.(setq lst '(1 2 3 4)) (setf (lst 2) (* $it 10)) lst → (1 2 30 4) (setq str "abc") (setf (str -1) (dup $it)) str → "abcc" (MyHash "var" "hello") (MyHash "var" (upper-case $it)) (MyHash → "HELLO")
The functions find-all, replace, set-ref and set-ref-all use $it as an alternative choice to $0.
The difference between $it and $0 is that $it will not retain the value beyond the evaluation of the self-referential expression it is part of, while $0 retains the value until used in another expression. $it is read-only, while $0 can be set by the user. Neither $it nor $0 - $15 are reentrant. Another subexpression using these variables will change them too.
The new setf function makes the functions set-assoc, assoc-set, set-nth, nth-set and ref-set obsolete. These functions have been removed. The integer function, deprecated since a few years, has been removed, and the shorter writing int should be used instead. The last section in this release notes gives examples of how to convert code to use the new or modified functions.
Changes in indexing syntax
The parenthesized syntax for assoc nth ref set-ref and set-ref-all has been eliminated. Only the traditional flat syntax is allowed. Note that the parenthesized syntax should not be confused with the syntax used in implicit indexing. In the abolished parenthesized syntax, the parentheses were syntactical sugar similar to parentheses used in the dotimes or for forms. The following example illustrates this:(nth (theList idx)) ; parenthesized syntax (setf (theList idx) value) ; implicit indexing syntax
In the first line the inner parentheses are part of a special form syntax. In the second example all parentheses are part of s-expressions.
When multiple indices or keys are used, a list is specified instead of a single index or key:(nth i myList) ; single index, flat form (myList i) ; single index implicit index form (nth idxList myList) ; multiple indices, flat form (myList idx1 idx2 ... ) ; multiple indices, implicit index, flat indices (myList idxList) ; multiple indices, inplicit index, index vector (assoc key aList) ; single key (assoc list-of-keys aList) ; multiple key
Index list vectors are the same as those returned by ref and ref-all. Index vectors have been supported by pop and push for several years. Index vectors tie together nth, ref, ref-all, push and pop. Using index vectors in impicit indexing and the syntax of 'nth' gives 'nth' a uniform parameter arity regardless of the number of indices involved.
Other changes in functions
- bits is a new function returning the binary representation of a number.
- dec does not need the quote before a symbol and also can decrement places like setf does.
- callback now can pass up to eight parameters (before four). Sixteen callback functions can be assigned (before eight) v.10.0.1.
- default returns the contents of a default functor when given a context. This function was not documented since v9.4.5. It was previously documented in version 9.3, but now works differently.
- env when given without any parameters, now returns an association list. Before, a flat list of environment strings, e.g. "KEY=value" was returned.
- eval-string has reversed the order of the context and error-handler parameters. The error-handler is now the last optional parameter.
- file-info takes an additional flag to force stats for the original file when the file-node checked is a link.
- inc does not need the quote before a symbol and also can increment places like 'setf' does.
- lookup can now take multiple keys similar to assoc.
- net-interface This new function allows the user to define the default network interface to be used by other network functions on multihomed machines with multiple IP numbers.
- net-receive no longer needs the quote before the buffer symbol. It also accepts a default functor specified by a context name as a buffer, for reference passing in and out of user-defined functions.
- (nth str 0) and (nth str -1) will return the empty string "" is str is the empty string "". Before an i index also returned the empty list. Now an out-of-bounds error will be thrown.
- push now returns the modified list instead of the element pushed. This modification makes more sense when having reference returns. It can be used to model queues: (pop (push tail Q -1)) => head
- read-expr has a revised syntax identical to the syntax pattern of eval-string. The workings of read-expr have changed too (see the Reference manual for details). Version 10.0.1 adds an additional offset parameter for the source.
- read-buffer no longer needs the quote before the buffer symbol. It also accepts a default functor specified by a context name as a buffer, for reference passing in and out of user-defined functions.
- read-utf8 reads UTF-8 multibyte characters from a file handle or stdin. New in UTF-8 enabled version 10.0.2.
- set and setq will not allow a missing value argument as was tolerated in previous versions.
- set-locale now returns a list of locale string and decimal point character, previously only the locale string was returned.
- sys-error now returns a list of error number and error text similar to net-error (v.10.0.2).
- unless is reintroduced, and works like (when (not ...) ...) with no alternate clause.
- write-buffer a never-used odd symbol syntax for the string to be written has been eliminated.
- write-line swapped parameters, which are now in the same order as the other write functions, with the device parameter coming first. If the second parameter is omitted the contents of (current-line) will be written to the device specified.
- Several new file extensions and media MIME types have been added to the HTTPD mode of newLISP in server mode. See the manual for details.
- The Status: response line in HTTPD mode now behaves closer to RFC spec (v.10.0.2).
- The initialization file in either $HOME/.init.lsp or $USERPROFILE/.init.lsp or $DOCUMENT_ROOT/.init is loaded if it exists, else if $NEWLISPDIR/init.lsp exists, it is loaded.
- A new environment variable NEWLISPLIB_INIT controls loading of an initialization file for newLISP shared libraries newlisp.so or newlisp.dll. Only if this variable is defined containing the full path name of an initialization file, the shared library will load it on startup.
- A new command line switch -n can be used to suppress loading any initialization file when starting newLISP.
- When using newLISP interactively in a UNIX shell (i.e. Mac OS X, Linux), TAB and double TAB work as keyword completion for built-in functions. On some UNIX systems running the Bash shell, keyboard macros can be defined in .inputrc or /etc/inputrc.
Changed Lisp source files
Although all old code should be thoroughly investigated for incompatibilities many newLISP source files will be compatible with version 10.0 without any changes. The following files needed minor changes.
exmples/client TCP/IP client example script exmples/finger TCP/IP finger example script exmples/server TCP/IP server example script exmples/sqlite.cgi SQLite3 CGI example script example/tcltk.lsp TclTk example example/udp-client.lsp UDP client example example/udp-server.lsp UDP server example example/upload.cgi CGI file upload example modules/ftp.lsp ftp module modules/postscript.lsp postscript module modules/stat.lsp statistics module (v.10.0.1) modules/xmlrpc-client.lsp XML RPC client module util/newlispdoc documentation generator (v.10.0.1) util/syntax.lsp syntax highlighter guiserver/newlisp-edit.lsp The editor in newLISP-GS guiserver/pinballs-demo.lsp A demo file in newLISP-GS
The following files needed more changes to conform to v.10.0 based versions. Changes were made in v.10.0.2.
modules/ftp.lsp ftp module modules/pop3.lsp pop3 module modules/smtp.lsp smtp module §
Conversion of eliminated and changed functions
set-nth, nth-set, set-assoc, assoc-set must be replaced by a combination of the new setf and nth, assoc.(nth-set (data idx) value) (nth-set idx data value) (set-nth (data idx) value) (set-nth idx data value) ; change to one of the following (setf (nth idx data) value) (setf (data idx) value) (set-assoc (aList key) value) ; change to (setf (assoc key aList) value)
data is either a list or array or a string. idx is either a single number or a list of numbers.
Multiple indices have to be put in a vector, and/or use implicit indexing:; old obsolete indexing (nth i j k data) (nth (data i j k)) ; change to one of the following (nth (list i j k) data) (data (list i j k)) (data i j k)
data is either a list or an array, because strings are one-dimensional.
The conversion of eliminated functions will typically not be overlooked, because programs will throw errors when running into undefined functions. More subtle are the following changes which could escape detection if the code is not tested carefully
Nested expressions of built-in destructive functions must not rely on functions to return a copy. E.g. in previous versions, after (pop (sort myList)), myList was sorted, but the pop of the first element happened on a copy:(set 'myList '(a b c d)) ; in 9.4.5 and earlier (pop (sort myList)) → a myList → (a b c d) ; in 10.0 (pop (sort myList)) → a myList → (b c d)
To restore the old behavior and make functions non-destructive, use copy:; restore old non-destructive behavior in nested expressions (set 'myList '(b d c a)) (pop (copy (sort myList))) L → '(b d c a)
dec take away the quote before the symbol.
env without any parameters now returns an association list. To get the old behavior, use (exec "env") returning a flat list of environment strings:; new return value (env) → (("PATH" "/usr/bin:/bin:/usr/sbin:/sbin") ...) ; return old format (exec "env") → ("PATH=/usr/bin:/bin:/usr/sbin:/sbin" ...)
eval-string has swapped the order of the error-handler and context parameters:; in 9.4.5 and earlier (eval-string theString error-handler the-context) ; in 10.0 (eval-string theString the-context error-handler)
Both the error-handler and context parameter are optional. The context parameter is usually the one used more frequently.
inc take away the quote before the symbol.
integer replace with the shorter written int.
net-receive take away the quote before the buffer symbol.
push now returns the modified list instead of the element pushed. Little code seems to rely on the element pushed as the return value. In many circumstances explicit list returns in user-defined functions can be eliminated:(define (foo ...) ... (push x aList) aList) ; can be modified to (define (foo ...) ... (push x aList))
set-locale now returns a list of locale string and decimal point character. Previously only the local string was returned. Code expecting the old return value can wrap the return value as in (first (set-locale ... )).
read-buffer take away the quote before the buffer symbol.
read-expr has changed the order of parameters and also changed functionality. Please consult the reference manual for details.
sys-error used to return an error number only. Now the function returns a list of error number and error text (v.10.0.2).
unless has lost the alternate clause. If there was no alternate clause, conversion is not necessary, but all instances of unless containing an alternate clause should be converted to if-not, which was introduced in version 9.4.5.
write-line has swapped parameters to (write-line <device> <string>) order, the same as other write- functions. The behavior has changed when the second parameter is missing, see the reference manual for details.
The order of loading $HOME/.init.lsp and $NEWLISPDIR/init.lsp has changed. The new version will look first for $HOME/.init.lsp and not load $NEWLISPDIR/init.lsp if $HOME/.init.lsp could be loaded. $NEWLISPDIR/init.lsp is only loaded is $HOME/.init.lsp does not exist.
Shared libraries newlisp.so or newlisp.dll only load a file specified with its full path in the new environment variable NEWLISPLIB_INIT.