| newLISP | ™ |
newLISP focuses on the core components of LISP: lists, symbols, and lambda expressions. To these, newLISP adds arrays, implicit indexing on lists and arrays, and dynamic and lexical scoping. Lexical scoping is implemented using separate namespaces called contexts.
The result is an easier-to-learn LISP that is even smaller than most Scheme implementations, but which still has about 350 built-in functions. Approximately 200k in size, newLISP is built for high portability using only the most common UNIX system C-libraries. It loads quickly and has a small memory footprint. newLISP is as fast or faster than other popular scripting languages and uses very few resources.
newLISP is dynamically scoped inside lexically separated contexts (namespaces). Contexts can be used to create isolated protected expansion packages and to write prototype-based object-oriented programs.
Both built-in and user-defined functions, along with variables, share the same namespace and are manipulated by the same functions. Lambda expressions and user-defined functions can be handled like any other list expression.
Contexts in newLISP facilitate the development of larger applications comprising independently developed modules with their own separate namespaces. They can be copied, dynamically assigned to variables, and passed by reference to functions as arguments. In this way, contexts can serve as dynamically created objects packaging symbols and methods. Lexical separation of namespaces also enables the definition of statically scoped functions.
newLISP's efficient red-black tree implementation can handle millions of symbols without degrading performance. Contexts can hold symbol-value pairs, allowing them to be used as hash-tables. Functions are also available to iteratively access symbols inside contexts.
newLISP allocates and reclaims memory automatically, without using traditional asynchronous garbage collection (except under error conditions). All objects — except for contexts, built-in primitives, and symbols — are passed by value and are referenced only once. When objects are no longer referenced, their memory is automatically deallocated. This results in predictable processing times, without the pauses found in traditional garbage collection. newLISP's unique automatic memory management makes it the fastest interactive LISP available.
Many of newLISP's built-in functions are polymorphic and accept a variety of data types and optional parameters. This greatly reduces the number of functions and syntactic forms it is necessary to learn and implement. High-level functions are available for distributed computing, financial math, statistics, and AI.
newLISP has functions to modify, insert, or delete elements inside complex nested lists or multidimensional array structures.
Since strings can contain null characters in newLISP, they can be used to process binary data.
newLISP can also be extended with a shared library interface to import functions that access data in foreign binary data structures. The distribution contains a module for importing popular database APIs.
newLISP's HTTP, TCP/IP, and UDP socket interfaces make it easy to write distributed networked applications. Its built-in XML interface, along with its text-processing features — Perl Compatible Regular Expressions (PCRE) and text-parsing functions — make newLISP a useful tool for CGI processing. The source distribution includes examples of HTML forms processing. newLISP can be run a as a CGI capable web server using its built-in http mode option.
The source distribution can be compiled for Linux, BSDs, Mac OS X/Darwin, Solaris, and Win32. On 64-bit Linux, SUN Solaris and True64Unix newLISP can be compiled as a 64-bit LP64 application for full 64-bit memory addressing.
newLISP-GS is a graphical user interface front-end written in newLISP and a Java based library server using the standard Java runtime environment installed on all Windows and Mac OS X platforms. Applications built with newLISP-GS can have the host operating system's native look and feel. Interfaces to GTK, Tcl/Tk and OpenGL graphics libraries are also available.
newLISP and Java are available for most operating systems. This makes newLISP-GS is a platform-independent solution for writing GUI applications.
For more information on newLISP-GS, see newLISP-GS.
newLISP and newLISP-GS are licensed under version 3 of the GPL (General Public License). Both the newLISP and other documentation packaged with newLISP are licensed under the GNU Free Documentation License.
The new set-assoc or pop-assoc should be used instead of replace-assoc, to replace or remove an association in an association list. The new set-assoc and pop-assoc handle multiple key expressions for nested association lists. The old replace-assoc will be removed in a future version.
When starting newLISP from the command line several switches and options and source files can be specified. The options and source files are executed. For options such as -p and -d, it makes sense to load source files first; other options, like -m and -s, should be specified before the source files. The -e switch is used to evaluate the program text and then exit; otherwise, evaluation continues interactively (unless an exit occurs while the files are loading).
newLISP will load and execute files specified on the command line. Files are specified with either their pathname on the local filesystem or with a http:// or file:// URL:
newlisp aprog.lsp bprog.lsp prog.lsp newlisp http://newlisp.org/example.lsp newlisp file:///usr/home/newlisp/demo.lsp
newlisp -s 4000 newlisp -s 100000 aprog bprog newlisp -s 6000 myprog newlisp -s 6000 http://asite.com/example.lsp
The above examples show starting newLISP with different stack sizes using the -s option, as well as loading one or more newLISP source files and loading files specified by an URL. When no stack size is specified, the stack defaults to 2048.
newlisp -m 128
This example limits LISP cell memory to 128 megabytes. In 32-bit newLISP, each LISP cell consumes 16 bytes, so the argument 128 would represent a maximum of 8,388,608 LISP cells. This information is returned by sys-info as the list's second element. Although LISP cell memory is not the only memory consumed by newLISP, it is a good estimate of overall memory usage.
The -w option specifies the initial working directory for newLISP after startup:
newlisp -w /usr/home/newlisp
All file requests without a directory path will now be directed to the path specified with the -w option.
The command-line prompt and initial copyright banner can be suppressed:
newlisp -c
Listen and connection messages are suppressed if logging is not enabled. The -c option is useful when controlling newLISP from other programs; it is mandatory when setting it up as a net-eval server.
The -c option also enables newLISP server nodes to answer HTTP GET, PUT, POST and DELETE requests, as well as perform CGI processing. Using the -c option, together with the -w and -d options, newLISP can serve as a standalone httpd webserver:
newlisp -c -d 8080 -w /usr/home/www
When running newLISP as a inetd or xinetd enabled server on UNIX machines, use:
newlisp -c -w /usr/home/www
In -c mode, newLISP processes command-line requests as well as HTTP and net-eval requests. Running newLISP in this mode is only recommended on a machine behind a firewall. This mode should not be run on machines open and accessible through the Internet. To suppress the processing of net-eval and command-line–like requests, use the safer -http option.
newLISP can be limited to HTTP processing using the -http option. With this mode, a secure httpd web server daemon can be configured:
newlisp -http -d 8080 -w /usr/home/www
When running newLISP as an inetd or xinetd-enabled server on UNIX machines, use:
newlisp -http -w /usr/home/www
To further enhance security and HTTP processing, load a program during startup when using this mode:
newlisp http-conf.lsp -http -w /usr/home/www
Defining a function named httpd-conf in a source file called httpd-conf.lsp can be used to customize HTTP request processing. The newLISP distribution contains an example httpd-conf.lsp file. This file shows how to configure redirects and filter unauthorized requests.
In the HTTP modes enabled by either -c or -http, the following file types are recognized, and a correctly formatted Content-Type: header is sent back:
file extension media type .jpg image/jpg .pgn image/png .gif image/gif application/pdf .mp3 image/mpeg .mov image/quicktime .mpg image/mpeg any other text/html
A capital C forces prompts when running newLISP in pipe I/O mode inside the Emacs editor:
newlisp -C
To suppress return values from evaluations, use silent.
newlisp some.lsp -p 9090
This example shows how newLISP can listen for commands on a TCP/IP socket connection. In this case, standard I/O is redirected to the port specified with the -p option. some.lsp is an optional file loaded during startup, before listening for a connection begins.
The -p option is also used to control newLISP from another application, such as a newLISP GUI front-end or a program written in another language.
A telnet application can be used to test running newLISP as a server. First enter:
newlisp -p 4711 &
The & indicates to a UNIX shell to run the process in the background. Now connect with a telnet application:
telnet localhost 4711
If connected, the newLISP sign-on banner and prompt appear. Instead of 4711, any other port number could be used.
When the client application closes the connection, newLISP will exit, too.
When the connection to the client is closed in -p mode, newLISP exits. To avoid this, use the -d option instead of the -p option:
newlisp -d 4711 &
This works like the -p option, but newLISP does not exit after a connection closes. Instead, it stays in memory, listening for a new connection and preserving its state. An exit issued from a client application closes the network connection, and the newLISP daemon remains resident, waiting for a new connection. Any port number could be used in place of 4711.
When running in -p or -d mode, the opening and closing tags [cmd] and [/cmd] must be used to enclose multiline statements. They must each appear on separate lines. This makes it possible to transfer larger portions of code from controlling applications.
The following variant of the -d mode is frequently used in a distributed computing environment, together with net-eval on the client side:
newlisp -c -d 4711 &
The -c spec suppresses prompts, making this mode suitable for receiving requests from the net-eval function.
newLISP server nodes running on UNIX like operating systems, will also answer HTTP GET, PUT and DELETE requests. This can be used to retrieve and store files with get-url, put-url, delet-url, read-file, write-file and append-file, or to load and save programs using load and save from and to remote server nodes. See the chapters for the -c and -http options for more details.
Instead of a port a local domain UNIX socket path can be specified in the -d or -p server modes.
newlisp -c -d /tmp/mysocket &
This mode will work together with local domain socket modes of net-connect, net-listen, and net-eval. Local domain sockets opened with with net-connect and net-listen can be served using net-accept, net-receive, and net-send. Local domain socket connections can be monitored using net-peek and net-select.
Local domain socket connections are much faster than normal Tcp/Ip network connections and preferred for communications between processes on the same local file system in distributed applications. This mode is not available on Win32.
The inetd server running on virtually all Linux/UNIX OSes can function as a proxy for newLISP. The server accepts TCP/IP or UDP connections and passes on requests via standard I/O to newLISP. inetd starts a newLISP process for each client connection. When a client disconnects, the connection is closed and the newLISP process exits.
inetd and newLISP together can handle multiple connections efficiently because of newLISP's small memory footprint, fast executable, and short program load times. When working with net-eval, this mode is preferred for efficiently handling multiple requests in a distributed computing environment.
Two files must be configured: services and inetd.conf. Both are ASCII-editable and can usually be found at /etc/services and /etc/inetd.conf.
Put one of the following lines into inetd.conf:
net-eval stream tcp nowait root /usr/bin/newlisp -c # as an alternative, a program can also be preloaded net-eval stream tcp nowait root /usr/bin/newlisp -c myprog.lsp
Instead of root, another user and optional group can be specified. For details, see the UNIX man page for inetd.
The following line is put into the services file:
net-eval 4711/tcp # newLISP net-eval requests
On Mac OS X and some UNIX systems, xinetd can be used instead of inetd. Save the following to a file named net-eval in the /etc/xinetd.d/ directory:
service net-eval
{
socket_type = stream
wait = no
user = root
server = /usr/bin/newlisp
port = 4711
server_args = -c
only_from = localhost
}
For security reasons, root should be changed to a different user, when traffic is accepted from other places than localhost. The only_from spec can be left out to permit remote access.
See the man pages for xinetd and xinetd.conf for other configuration options.
After configuring the daemon, inetd or xinetd must be restarted to allow the new or changed configuration files to be read:
kill -HUP <pid>
Replace <pid> with the process ID of the running xinetd process.
A number or network protocol other than 4711 or TCP can be specified.
newLISP handles everything as if the input were being entered on a newLISP command line without a prompt. To test the inetd setup, the telnet program can be used:
telnet localhost 4711
newLISP expressions can now be entered, and inetd will automatically handle the startup and communications of a newLISP process. Multiline expressions can be entered by bracketing them with [cmd] and [/cmd] tags, each on separate lines.
newLISP server nodes running on UNIX like operating systems, will also answer simple HTTP GET and PUT requests. This can be used to retrieve and store files with get-url, put-url, read-file, write-file and append-file, or to load and save programs using load and save from and to remote server nodes. On Win32 newLISP server nodes do not answer HTTP requests.
newlisp -e "(+ 3 4)" → 7
The expression enclosed in quotation marks is evaluated, and the result is printed to standard out (STDOUT). In most UNIX system shells, single quotes can also be used as command-line delimiters. Note that there is a space between -e and the quoted command string.
In any mode newLISP can write a log when started with the -l or -L option. Depending on the mode newLISP is running, different output is written to the logfile. Both options always must specify the path of a log-file. The path may be a relative path and can be either attached or detached to the -l or -L option.
newlisp -l./logfile.txt -c newlisp -L /usr/home/www/log.txt -http -w /usr/home/www/htpdocs
| logging mode | command line and net-eval with -c | HTTP server with -http |
|---|---|---|
| newlisp -l | log only input and network connections | log only net-work connections |
| newlisp -L | log also newLISP output (w/o prompts) | log also HTTP requests |
All logging output is written to the file specified after the -l or -L option.
The -h command-line switch prints a copyright notice and summary of options:
newlisp -h
On Linux and other UNIX systems, a newlisp man page can be found:
man newlisp
This will display a man page in the Linux/UNIX shell.
On Linux, BSDs, Mac OS X and other UNIXs the initialization file is installed and expected in /usr/share/newlisp/init.lsp. newLISP on Win32 compiled with MinGW looks for init.lsp in the same directory where newlisp.exe is installed. Along with any files specified on the command line, init.lsp is loaded before the banner and prompt are shown. When newLISP is executed or launched by a program or process other than a shell, the banner and prompt are not shown, and newLISP communicates by standard I/O. init.lsp, however, is still loaded and evaluated if present.
While newLISP does not require init.lsp to run, it is convenient for defining functions and systemwide variables.
The last part of init.lsp contains OS-specific code, which loads a second .init.lsp (starting with a dot). On Linux/UNIX, this file is expected in the directory specified by the HOME environment variable. On Win32, this file is expected in the directory specified by the USERPROFILE or DOCUMENT_ROOT environment variable.
The directory /usr/share/newlisp/modules contains modules with useful functions for a variety of tasks, such as database management with MySQL, procedures for statistics, POP3 mail, etc. The directory /usr/share/newlisp/guiserver contains sample programs for writing GUI aplications with newLISP-GS. The directory /usr/share/doc/newlisp/ contains documentation in HTML format.
On Win32 systems, all files are installed in the default directory $PROGRAMFILES\newlisp. $PROGRAMFILES is a Win32 environment variable that resolves to C:\Program files\newlisp\ in English language installations. The subdirectories $PROGRAMFILES\newlisp\modules and $PROGRAMFILES\newlisp\guiserver contain modules for interfacing to external libraries and sample programs written for newLISP-GS.
During startup newLISP sets the environment variable NEWLISPDIR, if it is not set already. On Linux, BSDs, Mac OS X and other UNIXs the variable is set to /usr/share/newlisp. On Win32 the variable is set to %PROGRAMFILES%/newlisp.
The environment variable NEWLISPDIR is useful when loading files installed with newLISP:
(load (append (env "NEWLISPDIR") "/guiserver.lsp")) (load (append (env "NEWLISPDIR") "/modules/mysql51.lsp"))
newLISP can be compiled as a UNIX shared library called newlisp.dylib on Mac OS X and as newlisp.so on Linux and BSDs. A newLISP shared library can be used like any other UNIX shared library.
To use newlisp.so or newlisp.dylib, import the function newlispEvalStr. Like eval-string, this function takes a string containing a newLISP expression and stores the result in a string address. The result can be converted using get-string. The returned string is formatted like output from a command-line session. It contains terminating line-feed characters, but without the prompt strings.
The first example shows how newlisp.so is imported from newLISP itself.
(import "/usr/lib/newlisp.so" "newlispEvalStr")
(get-string (newlispEvalStr "(+ 3 4)")) → "7\n"
The second example shows how to import newlisp.so into a program written in C:
/* libdemo.c - demo for importing newlisp.so
*
* compile using:
* gcc -ldl libdemo.c -o libdemo
*
* use:
*
* ./libdemo '(+ 3 4)'
* ./libdemo '(symbols)'
*
*/
#include <stdio.h>
#include <dlfcn.h>
int main(int argc, char * argv[])
{
void * hLibrary;
char * result;
char * (*func)(char *);
char * error;
if((hLibrary = dlopen("/usr/lib/newlisp.so",
RTLD_GLOBAL | RTLD_LAZY)) == 0)
{
printf("cannot import library\n");
exit(-1);
}
func = dlsym(hLibrary, "newlispEvalStr");
if((error = dlerror()) != NULL)
{
printf("error: %s\n", error);
exit(-1);
}
printf("%s\n", (*func)(argv[1]));
return(0);
}
/* eof */
This program will accept quoted newLISP expressions and print the evaluated results.
When calling newlisp.so's function newlispEvalStr, output normally directed to the console (e.g., return values or print statements) is returned in the form of an integer string pointer. The output can be accessed by passing this pointer to the get-string function. To silence the output from return values, use the silent function.
On the Win32 platforms, newLISP can be compiled as a DLL (Dynamic Link Library). In this way, newLISP functions can be made available to other programs (e.g., MS Excel, Visual Basic, Borland Delphi, or even newLISP itself).
When the DLL is loaded, it looks for the file init.lsp in the current directory of the calling process.
To access the functionality of the DLL, use newlispEvalStr, which takes a string containing a valid newLISP expression and returns a string of the result:
(import "newlisp.dll" "newlispEvalStr")
(get-string (newlispEvalStr "(+ 3 4)")) → "7"
The above example shows the loading of a DLL using newLISP. The get-string function is necessary to access the string being returned. Other applications running on Win32 allow the returned data type to be declared when importing the function.
When using newlisp.so, output normally directed to the console — like print statements or return values — will be returned in a string pointed to by the call to newlispEvalStr. To silence the output from return values, use the silent directive.
The following is a short introduction to LISP statement evaluation and the role of integer and floating point arithmetic in newLISP.
Top-level expressions are evaluated when using the load function or when entering expressions in console mode on the command line. As shown in the following snippet from an interactive session, multiline expressions can be entered by enclosing them between [cmd] and [/cmd] tags:
> [cmd] (define (foo x y) (+ x y)) [/cmd] (lambda (x y) (+ x y)) > (foo 3 4) 7 > _
Each [cmd] and [/cmd] tag is entered on a separate line. This mode is useful for pasting multiline code into the interactive console.
newLISP functions and operators accept integer and floating point numbers, converting them into the needed format. For example, a bit-manipulating operator converts a floating point number into an integer by omitting the fractional part. In the same fashion, a trigonometric function will internally convert an integer into a floating point number before performing its calculation.
The symbol operators (+ - * / % $ ~ | ^ << >>) return values of type integer. Functions and operators named with a word instead of a symbol (e.g., add rather than +) return floating point numbers. Integer operators truncate floating point numbers to integers, discarding the fractional parts.
newLISP has two types of basic arithmetic operators: integer (+ - * /) and floating point (add sub mul div). The arithmetic functions convert their arguments into types compatible with the function's own type: integer function arguments into integers, floating point function arguments into floating points. To make newLISP behave more like other scripting languages, the integer operators +, -, *, and / can be redefined to perform the floating point operators add, sub, mul, and div:
(constant '+ add) (constant '- sub) (constant '* mul) (constant '/ div) ;; or all 4 operators at once (constant '+ add '- sub '* mul '/ div)
Now the common arithmetic operators +, -, *, and / accept both integer and floating point numbers and return floating point results.
Note that the looping variables in dotimes and for, as well as the result of sequence, use floating point numbers for their values.
Care must be taken when importing from libraries that use functions expecting integers. After redefining +, -, *, and /, a double floating point number may be unintentionally passed to an imported function instead of an integer. In this case, floating point numbers can be converted into integers by using the function int. Likewise, integers can be transformed into floating point numbers using the float function:
(import "mylib.dll" "foo") ; importing int foo(int x) from C (foo (int x)) ; passed argument as integer (import "mylib.dll" "bar") ; importing C int bar(double y) (bar (float y)) ; force double float
Some of the modules shipping with newLISP are written assuming the default implementations of +, -, *, and /. This gives imported library functions maximum speed when performing address calculations.
The newLISP preference is to leave +, -, *, and / defined as integer operators and use add, sub, mul, and div when explicitly required. Since version 8.9.7 integer operations in newLISP are 64 bit operations, while 64 bit double floating point numbers only offer 52 bits of resolution in the integer part of the number.
Evaluate expressions by entering and editing them on the command line. More complicated programs can be entered using editors like Emacs and VI, which have modes to show matching parentheses while typing. Load a saved file back into a console session by using the load function.
A line comment begins with a ; (semicolon) or a # (number sign) and extends to the end of the line. newLISP ignores this line during evaluation. The # is useful when using newLISP as a scripting language in Linux/UNIX environments, where the # is commonly used as a line comment in scripts and shells.
When evaluation occurs from the command line, the result is printed to the console window.
The following examples can be entered on the command line by typing the code to the left of the → symbol. The result that appears on the next line should match the code to the right of the → symbol.
nil and true are boolean data types that evaluate to themselves:
nil → nil true → true
Integers and floating point numbers evaluate to themselves:
123 → 123 0xE8 → 232 ; hexadecimal prefixed by 0x 055 → 45 ; octal prefixed by 0 (zero) 1.23 → 1.23 123e-3 → 0.123 ; scientific notation
Integers are 64-bit numbers (including the sign bit, 32-bit before version 8.9.7). Valid integers are numbers between -9,223,372,036,854,775,808 and +9,223,372,036,854,775,807. Larger numbers converted from floating point numbers are truncated to one of the two limits. Integers internal to newLISP, which are limited to 32-bit numbers overflow to either +2,147,483,647 or -2,147,483,648. Floating point numbers are IEEE 754 64-bit doubles. Unsigned numbers up to 18,446,744,073,709,551,615 can be displayed using special formatting characters for format.
Strings may contain null characters and can have different delimiters. They evaluate to themselves.
"hello" →"hello" "\032\032\065\032" →" A " "\x20\x20\x41\x20" →" A " "\t\r\n" →"\t\r\n" "\x09\x0d\x0a" →"\t\r\n" ;; null characters are legal in strings: "\000\001\002" → "\000\001\002" {this "is" a string} → "this \"is\" a string" ;; use [text] tags for text longer than 2048 bytes: [text]this is a string, too[/text] → "this is a string, too"
Strings delimited by " (double quotes) will also process the following characters escaped with a \ (backslash):
| escaped character | description |
|---|---|
| \" | for a double quote inside a quoted string |
| \n | for a line-feed character (ASCII 10) |
| \r | for a return character (ASCII 13) |
| \t | for a TAB character (ASCII 9) |
| \nnn | for a three-digit ASCII number (nnn format between 000 and 255) |
| \xnn | for a two-hex-digit ASCII number (xnn format between x00 and xff) |
Quoted strings cannot exceed 2,048 characters. Longer strings should use the [text] and [/text] tag delimiters. newLISP automatically uses these tags for string output longer than 2,048 characters.
The { (left curly bracket), } (right curly bracket), and [text], [/text] delimiters do not perform escape character processing.
Lambda and lambda-macro expressions evaluate to themselves:
(lambda (x) (* x x)) → (lambda (x) (* x x)) (lambda-macro (a b) (set (eval a) b)) → (lambda (x) (* x x)) (fn (x) (* x x)) → (lambda (x) (* x x)) ; an alternative syntax
Symbols evaluate to their contents:
(set 'something 123) → 123 something → 123
Contexts evaluate to themselves:
(context 'CTX) → CTX CTX → CTX
Built-in functions also evaluate to themselves:
add → add <B845770D> (eval (eval add)) → add <B845770D> (constant '+ add) → add <B845770D> + → add <B845770D>
In the above example, the number between the < > (angle brackets) is the hexadecimal memory address (machine-dependent) of the add function. It is displayed when printing a built-in primitive.
Quoted expressions lose one ' (single quote) when evaluated:
'something → something ''''any → '''any '(a b c d) → (a b c d)
A single quote is often used to protect an expression from evaluation (e.g., when referring to the symbol itself instead of its contents or to a list representing data instead of a function).
In newLISP, a list's first element is evaluated before the rest of the expression (as in Scheme). The result of the evaluation is applied to the remaining elements in the list and must be one of the following: a lambda expression, lambda-macro expression, or primitive (built-in) function.
(+ 1 2 3 4) → 10 (define (double x) (+ x x)) → (lambda (x) (+ x x))
or
(set 'double (lambda (x) (+ x x))) (double 20) → 40 ((lambda (x) (* x x)) 5) → 25
For a user-defined lambda expression, newLISP evaluates the arguments from left to right and binds the results to the parameters (also from left to right), before using the results in the body of the expression.
Like Scheme, newLISP evaluates the functor (function object) part of an expression before applying the result to its arguments. For example:
((if (> X 10) * +) X Y)
Depending on the value of X, this expression applies the * (product) or + (sum) function to X and Y.
Because their arguments are not evaluated, lambda-macro expressions are useful for extending the syntax of the language. Most built-in functions evaluate their arguments from left to right (as needed) when executed. Some exceptions to this rule are indicated in the reference section of this manual. LISP functions that do not evaluate all or some of their arguments are called special forms.
Arrays evaluate to themselves:
Shell commands: If an ! (exclamation mark) is entered as the first character on the command line followed by a shell command, the command will be executed. For example, !ls on Unix or !dir on Win32 will display a listing of the present working directory. No spaces are permitted between the ! and the shell command. Symbols beginning with an ! are still allowed inside expressions or on the command line when preceded by a space. Note: This mode only works when running in the shell and does not work when controlling newLISP from another application.(set 'A (array 2 2 '(1 2 3 4))) → ((1 2) (3 4)) (eval A) → ((1 2) (3 4))
To exit the newLISP shell on Linux/UNIX, press Ctrl-D; on Win32, type (exit) or Ctrl-C, then the x key.
Use the exec function to access shell commands from other applications or to pass results back to newLISP.
Lambda expressions in newLISP evaluate to themselves and can be treated just like regular lists:
(set 'double (lambda (x) (+ x x))
(set 'double (fn (x) (+ x x)) ; alternative syntax
(last double) → (+ x x) ; treat lambda as a list
Note: No ' is necessary before the lambda expression since lambda expressions evaluate to themselves in newLISP.
The second line uses the keyword fn, an alternative syntax first suggested by Paul Graham for his Arc language project.
A lambda expression is a lambda list, a subtype of list, and its arguments can associate from left to right or right to left. When using append, for example, the arguments associate from left to right:
(append (lambda (x)) '((+ x x))) → (lambda (x) (+ x x))
cons, on the other hand, associates the arguments from right to left:
(cons '(x) (lambda (+ x x))) → (lambda (x) (+ x x))
Note that the lambda keyword is not a symbol in a list, but a designator of a special type of list: the lambda list.
(length (lambda (x) (+ x x))) → 2 (first (lambda (x) (+ x x))) → (x)
Lambda expressions can be mapped or applied onto arguments to work as user-defined, anonymous functions:
((lambda (x) (+ x x)) 123) → 246 (apply (lambda (x) (+ x x)) '(123)) → 246 (map (lambda (x) (+ x x)) '(1 2 3)) → (2 4 6)
A lambda expression can be assigned to a symbol, which in turn can be used as a function:
(set 'double (lambda (x) (+ x x))) → 246 (double 123) → (lambda (x) (+ x x))
The define function is just a shorter way of assigning a lambda expression to a symbol:
(define (double x) (+ x x))) → (lambda (x) (+ x x)) (double 123) → 246
In the above example, the expressions inside the lambda list are still accessible within double:
(set 'double (lambda (x) (+ x x))) → (lambda (x) (+ x x)) (last double) → (+ x x)
A lambda list can be manipulated as a first-class object using any function that operates on lists:
(set-nth (double 1) '(mul 2 x)) → (lambda (x) (mul 2 x)) double → (lambda (x) (mul 2 x)) (double 123) → 246
All arguments are optional when applying lambda expressions and default to nil when not supplied by the user. This makes it possible to write functions with multiple parameter signatures.
In newLISP, nil and true represent both the symbols and the boolean values true and false. Depending on their context, nil and true are treated differently. The following examples use nil, but they can be applied to true by simply reversing the logic.
Evaluation of nil yields a boolean false and is treated as such inside control flow expressions, such as if, unless, while, until, and not. Likewise, evaluating true yields true.
(set 'lst '(nil nil nil)) → (nil nil nil) (map symbol? lst) → (true true true)
In the above example, nil represents a symbol. In the following example, nil and true are evaluated and represent boolean values:
(if nil "no" "yes") → "yes" (if true "yes" "no") → "yes" (map not lst) → (true true true)
In newLISP, nil and the empty list () are not the same as in some other LISPs. Only in conditional expressions are they treated as a boolean false, as in and, or, if, while, unless, until, and cond.
The expression (list? '()) is true, but (list? nil) is not. This is because in newLISP, nil results in a boolean false when evaluated.
Evaluation of (cons x '()) yields (x), but (cons x nil) yields (x nil) because nil is treated as a boolean value when evaluated instead of as an empty list. The cons of two atoms in newLISP does not yield a dotted pair, but rather a two-element list. The predicate atom? is true for nil, but false for the empty list. The empty list in newLISP is only an empty list and not equal to nil.
A list in newLISP is a LISP cell of type list. It acts like a container for the linked list of elements making up the list cell's contents. There is no dotted pair in newLISP because the cdr (tail) part of a LISP cell always points to another LISP cell and never to a basic data type, such as a number or a symbol. Only the car (head) part may contain a basic data type. Early LISP implementations used car and cdr for the names head and tail.
newLISP's arrays enable fast element access within large lists. New arrays can be constructed and initialized with the contents of an existing list using the function array. Lists can be converted into arrays, and vice versa. Some of the same functions used for modifying and accessing lists can be applied to arrays, as well. Arrays can hold any type of data or combination thereof.
In particular, the following functions can be used for creating, accessing, and modifying arrays:
| function | description |
|---|---|
| append | appends arrays |
| array | creates and initializes an array with up to 16 dimensions |
| array-list | converts an array into a list |
| array? | checks if expression is an array |
| det | returns the determinant of a matrix |
| first | returns the first row of an array |
| invert | returns the inversion of a matrix |
| last | returns the last row of an array |
| mat | perform scalar operations on matrices |
| multiply | multiplies two matrices |
| nth | returns an element of and array |
| nth-set | changes the element, returning the old; significantly faster than set-nth |
| rest | returns all but the first row of an array |
| set-nth | changes the element and returns the changed array |
| slice | returns a slice of an array |
| sort | sort the elements in an array |
| transpose | transposes a matrix |
newLISP represents multidimensional arrays with an array of arrays (i.e., the elements of the array are themselves arrays).
When used interactively, newLISP prints and displays arrays as lists, with no way of distinguishing between them.
Use the source or save functions to serialize arrays (or the variables containing them). The array statement is included as part of the definition when serializing arrays.
Like lists, negative indices can be used to enumerate the elements of an array, starting from the last element.
An out-of-bounds index will cause an error message on an array. In contrast, lists pick the last or first element when an out-of-bounds occurs.
Arrays can be non-rectangular, but they are made rectangular during serialization when using source or save. The array function always constructs arrays in rectangular form.
The matrix functions det, transpose, multiply, and invert can be used on matrices built with nested lists or arrays built with array.
For more details, see array, array?, and array-list in the reference section of this manual.
newLISP has no built-in hash table data type. Instead, it uses symbols for associative memory access. Symbols in newLISP are implemented using an efficient red-black tree algorithm. This algorithm balances the binary symbol tree for faster symbol access. In newLISP, symbol trees are represented as namespaces called contexts, which are themselves part of the MAIN namespace.
For a more detailed introduction to namespaces, see the chapter on Contexts.
The context function can be used to make associations. It can also be used to create and switch contexts.
;; create a symbol and store data into it (context 'MyHash "John Doe" 123) → 123 (context 'MyHash "@#$%^" "hello world") → "hello world" ;; retrieve contents from the symbol (context 'MyHash "John Doe") → 123 (context 'MyHash "@#$%^") → "hello world"
The first two statements create the symbols "John Doe" and "@#$^", storing the values 123 and "hello world" into them. The hash context named MyHash is created in the first statement, while the second merely adds the new association to the existing one.
Note that hash symbols can contain spaces or other special characters not typically allowed in variable names.
Internally, context is just a shorter and faster form of:
;; create a symbol and store the data in it (set (sym "John Doe" 'MyHash) 123) → 123 ;; retrieve contents from the symbol (eval (sym "John Doe" MyHash)) → 123
The context default function can be used for a very short definition of a hash function:
(define (myhash:myhash key value)
(if value
(context myhash key value)
(context myhash key)))
;; create a dictionary key value pair
(myhash "hello" 123) → 123
;; retrieve the key value
(myhash "hello") → 123
Some functions take array, list, or string elements (characters) specified by one or more int-index (integer index). The positive indices run 0, 1, …, N-2, N-1, where N is the number of elements in the list. If int-index is negative, the sequence is -N, -N+1, …, -2, -1. Adding N to the negative index of an element yields the positive index. Unless a function does otherwise, an index greater than N-1 returns the last element in a list; it returns the first element for indices less than -N. An error message is produced for any indexing occurring outside an array's boundaries.
Implicit indexing can be used instead of nth to retrieve the elements of a list or array or the characters of a string:
(set 'lst '(a b c (d e) (f g))) (lst 0) → a ; same as (nth (lst 0)) (lst 3) → (d e) (lst 3 1) → e ; same as (nth (lst 3 1)) (lst -1) → (f g) (set 'myarray (array 3 2 (sequence 1 6))) (myarray 1) → (3 4) (myarray 1 0) → 3 (myarray 0 -1) → 2 ("newLISP" 3) → "L"
Indices may also be supplied from a list. In this way, implicit indexing works together with functions that take or produce index vectors, such as push, pop, ref and ref-all.
(lst '(3 1)) → e (set 'vec (ref 'e lst)) → (3 1) (lst vec) → e
Note that implicit indexing is not breaking Lisp syntax rules but is merely an expansion of existing rules to other data types in the functor position of an s-expression. In original Lisp the first element in an s-expression list is applied as a function to the rest elements as arguments. In newLISP a list in the functor position of an s-expression assumes self-indexing functionality using the index arguments following it.
Implicit indexing is faster than the explicit forms, but the explicit forms may be more readable depending on context.
Note that in the UTF-8–enabled version of newLISP, implicit indexing of strings using the nth function works on character rather than byte boundaries.
The default functor is a functor inside a context with the same name as the context itself. See The context default function chapter. A default functor can be used together with implicit indexing to serve as a mechanism for referencing lists:
(set 'MyList:MyList '(a b c d e f g)) (MyList 0) → a (MyList 3) → d (MyList -1) → g (3 2 MyList) → (d e) (-3 MyList) → (e f g) (set 'aList MyList) (aList 3) → d
In this example, aList references MyList:MyList, not a copy of it. For more information about contexts, see Programming with contexts.
The default functor can also be used with nht-set as shown in the following example:
(set 'MyList:MyList '(a b c d e f g)) (nth-set (MyList 3) 999) → d (MyList 3) → 999
Implicit forms of rest and slice can be created by prepending a list with one or two numbers for offset and length. If the length is negative it counts from the end of the list or string:
(set 'lst '(a b c d e f g)) ; or as array (set 'lst (array 7 '(a b c d e f g))) (1 lst) → (b c d e f g) (2 lst) → (c d e f g) (2 3 lst) → (c d e) (-3 2 lst) → (e f) (2 -2 lst) → (c d e) (set 'str "abcdefg") (1 str) → "bcdefg" (2 str) → "cdefg" (2 3 str) → "cde" (-3 2 str) → "ef" (2 -2 str) → "cde"
Implicit indexing for rest works on character rather than byte boundaries when using the UTF-8–enabled version of newLISP, whereas implicit indexing for slice will always work on byte boundaries and can be used for binary content.
(set 'aList '(a b c (d e (f g) h i) j k)) (nth-set (aList 0) 1) → a (nth-set (aList 3 2) '(1 2 3 4)) → (f g) (set 'i 3 'j 2 'k 2) (nth-set (aList i j k) 99) → 3 aList → (1 b c (d e (1 2 99 4) h i) j k) (set-nth (aList -3 -3 2) 999) → (1 b c (d e (1 2 999 4) h i) j k)
As with nth, indices can be supplied in a vector list to work together with functions handling index vectors such as push, pop, ref and ref-all.
(set 'aList '(a b c (d e (f g) h i) j k)) (set 'vec (ref 'f aList)) → (3 2 0) (set-nth (aList vec) 999) → (a b c (d e (999 g) h i) j k) (nth-set (aList vec) 'Z) → 999 ; old value aList → (a b c (d e (Z g) h i) j k)
Most of the primitives in newLISP are nondestructive (no side effects) and leave existing objects untouched, although they may create new ones. There are a few destructive functions, however, that do change the contents of a list, string, or variable:
| function | description |
|---|---|
| constant | sets the contents of a variable and protects it |
| dec | decrements the value in a variable |
| inc | increments the value in a variable |
| net-receive | reads into a buffer variable |
| pop | pops an element from a list or string |
| pop-assoc | remove an association from an association list |
| push | pushes a new element onto a list or string |
| push-assoc | remove an association from an association list |
| read-buffer | reads into a buffer variable |
| replace | replaces elements in a list or string |
| replace-assoc | replace an element in an association list |
| reverse | reverses a list or string |
| rotate | rotates the elements of a list or characters of a string |
| set, setq | sets the contents of a variable |
| set-assoc, assoc-set | replaces an element in an association list |
| set-nth, nth-set | changes an element in a list or string |
| set-ref, ref-set | searches for an element in a nested list and replaces it |
| set-ref-all | searches for an element in a nested list and replaces all instances |
| sort | sorts the elements of a list or array |
| swap | swaps two elements inside a list or string |
| write-buffer | writes to a string buffer or file |
| write-line | writes to a string buffer or file |
Note that the last two functions, write-buffer and write-line, are only destructive in one of their syntactic forms: when taking a string buffer instead of a file handle.
Some destructive functions can be made non-destruvtive by wrapping the target object into a begin block. A block returns a copy of the last evaluation in the block and the destructive function will work on the copy instead of the original.
(set 'aList '(a b c d e f)) (replace 'c (begin aList)) → (a b d e f) aList → (a b c d e f) (set 'str "newLISP") → "newLISP" (rotate (begin str)) → "PnewLIS" str → "newLISP"
newLISP uses dynamic scoping inside contexts. A context is a lexically closed namespace. In this way, parts of a newLISP program can live in different namespaces taking advantage of lexical scoping.
When the parameter symbols of a lambda expression are bound to its arguments, the old bindings are pushed on a stack. newLISP automatically restores the original variable bindings when leaving the lambda function.
The following example illustrates the dynamic scoping mechanism. The text in bold is the output from newLISP:
> (set 'x 1) 1 > (define (f) x) (lambda () x) > (f) 1 > (define (g x) (f)) (lambda (x) (f)) > (g 0) 0 > (f) 1 > _
The variable x is first set to 1, but when (g 0) is called x is bound to 0 and x is reported by (f) as 0 during execution of (g 0). After execution of (g 0) the call to (f) will report x as 1 again.
This is different from the lexical scoping mechanisms found in languages like C or Java, where the binding of local parameters occurs inside the function only. In lexically scoped languages like C, (f) would always print the global bindings of the symbol x with 1.
Be aware that passing quoted symbols to a user-defined function causes a name clash if the same variable name is used as a function parameter:
(define (inc-symbol x y) (inc x y)) (set 'y 200) (inc-symbol 'y 123) → 246 y → 200 ; y is still 200
Since 'y shares the same name as the function's second parameter, inc-symbol returns 246 (123 + 123), leaving 'y unaffected. Dynamic scoping's variable capture can be a disadvantage when passing symbol references to user-defined functions.
The problem is avoided entirely by grouping related user-defined functions into a context. A symbol name clash cannot occur when accessing symbols and calling functions from outside of the defining context.
Contexts should be used to group related functions when creating interfaces or function libraries. This surrounds the functions with a lexical "fence," thus avoiding variable name clashes with the calling functions.
newLISP uses contexts for different forms of lexical scoping. See the chapters Contexts and Programming with contexts, as well as the sections Lexical, static scoping in newLISP and default functions for more information.
What follows are methods of interrupting the control flow inside both loops and the begin expression.
The looping functions dolist and dotimes can take optional conditional expressions to leave the loop early. catch and throw are a more general form to break out of a loop body and are also applicable to other forms or statement blocks.
Because newLISP is a functional language, it uses no break or return statements to exit functions or iterations. Instead, a block or function can be exited at any point using the functions catch and throw:
(define (foo x)
(…)
(if condition (throw 123))
(…)
456)
;; if condition is true
(catch (foo p)) → 123
;; if condition is not true
(catch (foo p)) → 456
Breaking out of loops works in a similar way:
(catch
(dotimes (i N)
(if (= (foo i) 100) (throw i))))
→ value of i when foo(i) equals 100
The example shows how an iteration can be exited before executing N times.
Multiple points of return can be coded using throw:
(catch (begin
(foo1)
(foo2)
(if condition-A (throw 'x))
(foo3)
(if condition-B (throw 'y))
(foo4)
(foo5)))
If condition-A is true, x will be returned from the catch expression; if condition-B is true, the value returned is y. Otherwise, the result from foo5 will be used as the return value.
As an alternative to catch, the throw-error function can be used to catch errors caused by faulty code or user-initiated exceptions.
Using the logical functions and and or, blocks of statements can be built that are exited depending on the boolean result of the enclosed functions:
(and
(func-a)
(func-b)
(func-c)
(func-d))
The and expression will return as soon as one of the block's functions returns nil or an () (empty list). If none of the preceding functions causes an exit from the block, the result of the last function is returned.
or can be used in a similar fashion:
(or
(func-a)
(func-b)
(func-c)
(func-d))
The result of the or expression will be the first function that returns a value which is not nil or ().
In newLISP, symbols can be separated into namespaces called contexts. Each context has a private symbol table lexically separate from all other contexts. Symbols known in one context are unknown in others, so the same name may be used in different contexts without conflict.
Contexts are used to build modules of isolated variable and function definitions. They can also be copied and dynamically assigned to variables or passed as arguments. Because contexts in newLISP have lexically separated namespaces, they allow programming with lexical scoping and software object styles of programming.
Contexts are identified by symbols that are part of the root or MAIN context. While context symbols are uppercased in this chapter, lowercase symbols may also be used.
In addition to context names, MAIN contains the symbols for built-in functions and special symbols such as true and nil. The MAIN context is created automatically each time newLISP is run. To see all the symbols in MAIN, enter the following expression after starting newLISP:
(symbols)
The following rules should simplify the process of understanding contexts by identifying which ones the created symbols are being assigned to.
newLISP first parses and translates each top level expression. The symbols are created during the parsing and translation phase. After the expression is translated it gets evaluated.
A symbol is created when newLISP first sees it, when calling the load, sym, or eval-string functions. When newLISP reads a source file, symbols are created before evaluation occurs.
When an unknown symbol is encountered during code translation, a search for its definition begins inside the current context. Failing that, the search continues inside MAIN for a built-in function, context, or global symbol. If no definition is found, the symbol is created locally inside the current context.
Once a symbol is created and assigned to a specific context, it will belong to that context permanently.
A user-defined function is evaluated in the context of the symbol it is called with.
Special symbols like nil and true, as well as context and built-in function symbols, are global (visible to all contexts). Any symbol in the MAIN context can be made global by using the global function.
The following simulates a command-line session in newLISP:
> (context 'FOO) FOO FOO> _
If the FOO context already exists, newLISP switches to it. Otherwise, the context is created before the switch occurs. All symbols now read from the command line are created and known only within the context FOO. Note that the symbol used for the context name must be quoted ('FOO in this example) the first time a context is created. Subsequent uses of context do not require the quote. After the switch, the command-line prompt changes to FOO> :
FOO> (set 'x 123) 123 FOO> (set 'y 456) 456 FOO> (symbols) (x y) FOO> _
To switch back to the MAIN context, use:
FOO> (context MAIN) MAIN > _
A symbol can be referenced from outside its defining context by prepending a context name and a colon to it:
> FOO:x 123 > _
The same symbol may also be used in another context:
> (context 'FOO-B) FOO-B FOO-B> (set 'x 777) 777 FOO-B> FOO:x 123 > _
When quoting a fully qualified symbol (context:symbol), the quote precedes the context name:
> (set 'FOO-B:x 555) 555 > _
A context is implicitly created when referring to one that does not yet exist. Unlike the context function, the context is not switched. The following statements are all executed inside the MAIN context:
> (set 'ACTX:var "hello") "hello" > ACTX:var "hello" > _
The same symbol (x in this case) used in a context can also be used in MAIN. Now we have three versions of x, all in a different context:
> (set 'x "I belong to MAIN") "I belong to MAIN" > FOO:x 123 > FOO-B:x 555 > x "I belong to MAIN" > _
Symbols owned by a context (or MAIN) are not accessible unless prefixed by the context name:
FOO> MAIN:x "I belong to MAIN" FOO> FOO-B:x 555 FOO> x 123 > _
When loading source files on the command line with load, or when executing the functions eval-string or sym, the context function tells newLISP where to put all of the symbols and definitions:
;;; file MY_PROG.LSP
;;
;; everything from here on goes into GRAPH
(context 'GRAPH)
(define (draw-triangle x y z)
(…))
(define (draw-circle)
(…))
;; show the runtime context, which is GRAPH
(define (foo)
(context))
;; switch back to MAIN
(context 'MAIN)
;; end of file
The draw-triangle and draw-circle functions — along with their x, y, and z parameters — are now part of the GRAPH context. These symbols are known only to GRAPH. To call these functions from another context, prefix them with GRAPH:
(GRAPH:draw-triangle 1 2 3)
(GRAPH:foo) → GRAPH
The last statement shows how the runtime context has changed to GRAPH (foo's context).
A symbol's name and context are used when comparing symbols from different contexts. The name function can be used to extract the name part from a fully qualified symbol.
;; same symbol name, but different context name (= 'A:val 'B:val) → nil (= (name 'A:val) (name 'B:val)) → true
Note: The symbols are quoted with a ' (single quote) because we are interested in the symbol itself, not in the contents of the symbol.
By default, only built-in functions and symbols like nil and true are visible inside contexts other than MAIN. To make a symbol visible to every context, use the global function:
(set 'aVar 123) → 123 (global 'aVar) → aVar (context 'FOO) → FOO aVar → 123
Without the global statement, the second aVar would have returned nil instead of 123. If FOO had a previously defined symbol (aVar in this example) that symbol's value — and not the global's — would be returned instead. Note that only symbols from the MAIN context can be made global.
Once it is made visible to contexts through the global function, a symbol cannot be hidden from them again.
By using the constant function, symbols can be both set and protected from change at the same time:
> (constant 'aVar 123) → 123 > (set 'aVar 999) symbol is protected in function set : aVar >_
A symbol needing to be both a constant and a global can be defined simultaneously:
(constant (global 'aVar) 123)
In the current context, symbols protected by constant can be overwritten by using the constant function again. This protects the symbols from being overwritten by code in other contexts.
Global and built-in function symbols can be overwritten inside a context by prefixing them with their own context symbol:
(context 'Account)
(define (Account:new …)
(…))
(context 'MAIN)
In this example, the built-in function new is overwritten by Account:new, a different function that is private to the Account context.
Variables can be used to refer to contexts:
(set 'FOO:x 123) (set 'ctx FOO) → FOO ctx:x → 123 (set 'ctx:x 999) → 999 FOO:x → 999
Context variables are used when creating contexts with the new function (objects), as well as when writing functions for uninstantiated contexts.
They also allow for pass-by-reference of large data objects when contained inside contexts and passed to functions as context variables.
The sequence in which contexts are created or loaded can lead to unexpected results. Enter the following code into a file called demo:
;; demo - file for loading contexts
(context 'FOO)
(set 'ABC 123)
(context MAIN)
(context 'ABC)
(set 'FOO 456)
(context 'MAIN)
Now load the file into the newlisp shell:
> (load "demo")
symbol is protected in function set : FOO
> _
Loading the file causes an error message for FOO, but not for ABC. When the first context FOO is loaded, the context ABC does not exist yet, so a local variable FOO:ABC gets created. When ABC loads, FOO already exists as a global protected symbol and will be correctly flagged as protected.
FOO could still be used as a local variable in the ABC context by explicitly prefixing it, as in ABC:FOO.
The following pattern can be applied to avoid unexpected behavior when loading contexts being used as modules to build larger applications:
;; begin of file - MyModule.lsp (load "This.lsp") (load "That.lsp") (load "Other.lsp") (context 'MyModule) … (define (func x y z) (…)) … (context 'MAIN) (MyModule:func 1 2 3) (exit) ;; end of file
Always load the modules required by a context before the module's context statement. Always finish by switching back to the MAIN context, where the module's functions and values can be safely accessed.
Contexts are used in newLISP in different ways. Most frequently they are used for partioning code into modules, but they also can be used for simple object-oriented programming when using the : (colon) operator to handle polymorphism in function application.
Contexts in newLISP are mainly used for partitioning source into modules. Because each module lives in a different namespace, modules are lexically separated and the names of symbols cannot clash with identical names in other modules.
The modules, which are part of the newLISP distribution, are a good example how to put related functions into a module file, and how to document modules using the newLISPdoc utility.
For best programming practice a file should only contain one module and the filename should be similar if not identical to the context name used:
;; file db.lsp, commonly used database functions (context 'db) ;; Variables used throughout this namespace (define db:handle) (define db:host "http://loalhost) ;; Constants (constant 'Max_N 1000000) (constant 'Path "/usr/data/") ;; Functions (define (db:open ...) ...) (define (db:close ...) ...) (define (db:update ...) ... )
The example shows a good practice of predefining variables, which are global inside the namespace and defining as constants variables, which will not change.
If a file contains more then one context, then the end of the context should be marked with a switch back to MAIN:
;; Multi context file multi.lsp (context 'A-ctx) ... (context MAIN) (context 'B-ctx) ... (context MAIN) (context 'C-ctx) ... (context MAIN)
Contexts are frequently uses as data containers, e.g. for hash-like dictionaries and configuration data:
;; Config.lsp - configuration setup (context 'Config) (set 'user-name "admin") (set 'password "secret") (set 'db-name "/usr/data/db.lsp") ... ;; eof
Loading the Config namespace will now load a whole variable set into memory at once:
(load "Config.lsp") (set 'file (open Config:db-name "read")) ... ...
In a similar fashion a whole data set can be saved:
(save "Config.lsp" 'Config)
Read more about this in the section Serializing contexts.
Module files are loaded using the load function. If a programming project contains numerous modules which are referring to each other, they should be pre-declared to avoid problems due to context forward references before loading of that context.
;; predeclaring contexts (map context '(Utilities Config Aquisition Analysis SysLog)) ;; loading context module files (load "Utilities.lsp" "Aquisition.lsp") (load "http://192.168.1.34/Config.lsp") ; load module from remote location (load "Analysis.lsp" "SysLog.lsp") (define (run) ...) (run) ;; end of file
When pre-declaring and loading modules as shown in the example, the sequence of declaration or loading can be neglected. All forward references to variables and definitions in modules not loaded yet will be translated correctly.
Modules not starting with a context switch are loaded always into MAIN except when the load statement specifies a target context as the last parameter. The load function can take URLs to load modules from remote locations, via HTTP.
A default functor or default function is a symbol or user-defined function or macro with the same name as its namespace. When the context is used as the name of a function or in the functor position of an s-expression, newLISP executes the default function or refers to the contents of the default functor.
;; the default function (define (foo:foo a b c) (+ a b c)) (foo 1 2 3) → 6 ;; the default functor (define mylist:mylist '(a b c d e f g)) (mylist 3) → d (nth-set (mylist 3) 'D) → d ; returns old contents mylist:mylist → (a b c D e f g)
This allows the default function defined inside a context to be called whenever the context is applied as a function. A default function could update the lexically isolated static variables contained inside its context:
(define (gen:gen x)
(if gen:acc
(inc 'gen:acc x)
(set 'gen:acc x)))
(gen 1) → 1
(gen 1) → 2
(gen 2) → 4
(gen 3) → 7
gen:acc → 7
The first time the gen function is called, its accumulator is set to the value of the argument. Each successive call increments gen's accumulator by the argument's value.
If a default function is called from a context other than MAIN, the context must already exist or be declared with a forward declaration, which creates the context and the function symbol:
;; forward declaration of a default function
(define fubar:fubar)
(context 'foo)
(define (foo:foo a b c)
…
(fubar a b) ; forward reference
(…)) ; to default function
(context MAIN)
;; definition of previously declared default function
(context 'fubar)
(define (fubar:fubar x y)
(…))
(context MAIN)
Default functions work like global functions, but they are lexically separate from the context in which they are called. The arguments in a default function macro are safe from variable capture.
Like a lambda or lambda-macro function, default functions can be used with map or apply.
In newLISP, all parameters are passed by value. This poses a potential problem when passing large lists or strings to user-defined functions or macros. Symbols and context objects can also be passed by reference. This allows memory-intensive objects to be passed without the overhead of copying the entire list or string.
Any data can be passed by reference by passing the symbol holding it:
(define (change-list aList) (push 999 (eval aList))) (set 'data '(1 2 3 4 5)) ; note the quote ' in front of data (change-list 'data) → (999 1 2 3 4 5) data → (999 1 2 3 4 5)
Although this method is simple to understand and use, it poses the potential problem of variable capture when passing the same symbol as used in the function as a parameter:
;; pass data by symbol reference
> (set 'aList '(a b c d))
(a b c d)
> (change-list 'aList)
list or string expected : (eval aList)
called from user defined function change-list
>
Because of the danger of variable capture, passing an object by its symbol should only be used in small scripts or programs where each function and its parameters are well-known. A safer method would be to package the object in a context (namespace) and pass the context ID:
;; pass data by context reference
(set 'mydb:data (sequence 1 100000))
(define (change-db obj idx value)
(nth-set (obj:data idx) value))
(change-db mydb 1234 "abcdefg")
(nth (mydb:data 1234)) → "abcdefg"
This example shows how objects can be passed by reference to a user-defined function using context variables, without the overhead of passing them by value. String buffers or data objects enclosed in a context can also be passed using this technique.
As shown in the following variation, using default functor can further simplify the syntax:
;; pass a context containing a default functor
(set 'mydb:mydb (sequence 1 100000))
(define (change-db obj idx value)
(nth-set (obj idx) value))
(change-db mydb 1234 "abcdefg")
(mydb 1234) → "abcdefg"
The change-db function does not need to know the name of the variable inside the context. All functions which use a syntax (functor (L idx) ...) like: like assoc-set, nth, nth-set, pop-assoc, push, and pop and functions which use a syntax (functor (L key) ...) like: ref, ref-all, ref-set, set-assoc, set-nth, set-ref, and set-ref-all know how to extract the default functor from the context L. This technique works for arrays and strings, as well.
For destructive functions like replace, reverse, rotate, sort, and swap, which refer to the un-indexed list, the function default can be used to extract the default functor symbol:
(set 'foo:foo '( 6 3 5 8 6 7 4 1))
(define (my-sort clist)
(sort (eval (default clist))))
(my-sort foo)
foo:foo → (1 3 4 5 6 6 7 8)
In this example, the list in foo:foo is passed by context reference. default is used to acces the default functor symbol, the contents of which is access by eval.
Serialization makes a software object persistent by converting it into a character stream, which is then saved to a file or string in memory. In newLISP, anything referenced by a symbol can be serialized to a file by using the save function. Like other symbols, contexts are saved just by using their names:
(save "mycontext.lsp" 'MyCtx) ; save MyCtx to mycontext.lsp (load "mycontext.lsp") ; loads MyCtx into memory (save "mycontexts.lsp" 'Ctx1 'Ctx2 'Ctx3) ; save multiple contexts at once
For details, see the functions save (mentioned above) and source (for serializing to a newLISP string).
Any object-oriented programming (OOP) system built in newLISP is based on the following three principles:
The following paragraphs are a short introduction to FOOP: Functional Object-Oriented Programming as designed by Michael Michaels. This description covers only very basic elements. For more details see the Michael Michaels's FOOP training videos on newlisp.org or neglook.com.
Class attributes and methods are stored in the namespace of the object class. No object instance data is stored in this namespace/context. Data variables in the class namespace only desecribe the class of objects as a whole but don't contain any object specific information. A generic FOOP object constructor can be used as a template for specific object constructors when creating new object classes with new:
; generic FOOP object constructor (define (Class:Class) (cons (context) (args))) (new Class 'Rectangle) (new Class 'Circle)
Creating the namespace classes using new, reserves the class name as a context in newLISP and facilitates forward references. At the same time a simple constructor is defined for the new class for instantiating new objects. As a convention it is recommended to start class names in upper-case to signal that the name stands for a namespace.
In some cases it may be useful to overwrite the simple constructor, which was created during class creation with new:
; overwrite simple constructor
(define (Circle:Circle x y radius)
(list Circle x y radius))
A constructor can also specify defaults:
; constructor with defaults
(define (Circle:Circle (x 10) (y 10) (radius 3))
(list Circle x y radius))
In many cases the constructor as created when using new is sufficient and overwriting it is not necessary.
FOOP represents objects as lists. The first element of the list indicates the object's kind or class, while the remaining elements contain the data. The following statements define two objects using any of the contructors defined previously:
(set 'myrect (Rectangle 5 5 10 20)) → (Rectangle 5 5 10 20) (set 'mycircle (Circle 1 2 10)) → (Circle 1 2 10)
An object created is identical to the function necessary to create it. Nested objects can be created in a similar manner:
; create classes
(new Class 'Person)
(new Class 'Address)
(new Class 'City)
(new Class 'Street)
; create an object containing other objects
(set 'JohnDoe (Person (Address (City "Boston") (Street 123 "Main Street"))))
→ (Person (Address (City "Boston") (Street 123 "Main Street")))
Objects in FOOP not only resemble functions they also resemble associations. The assoc function can be used to acess object data by name:
(assoc (JohnDoe Address)) → (Address (City "Boston") (Street 123 "Main Street")) (assoc (JohnDoe Address Street)) → (Street 123 "Main Street")
In a similar manner set-assoc could be used to modify object data:
(set-assoc (JohnDoe Address Street) (Street 456 "Main Street"))
→ (Person (Address (City "Boston") (Street 456 "Main Street")))
The street number has been changes from 123 to 456.
In newLISP, the colon character : is primarily used to connect the context symbol with the symbol it is qualifying. Secondly, the colon function is used in OOP to resolve a function's application polymorphically.
The following code defines two functions called area, each belonging to a different namespace. Both functions could have been defined in different modules, but in this case they are defined in the same file and without bracketing context statements. Here, only the symbols rectangle:area and circle:area belong to different namespaces. The local parameters p, c, dx, and dy are all part of MAIN, but this is of no concern.
;; class methods for rectangles (define (Rectangle:area p) (mul (p 3) (p 4))) (define (Rectangle:move p dx dy) (list Rectangle (add (p 1) dx) (add (p 2) dy) (p 3) (p 4))) ;; class methods for circles (define (Circle:area c) (mul (pow (c 3) 2) (acos 0) 2)) (define (Circle:move p dx dy) (list Circle (add (p 1) dx) (add (p 2) dy) (p 3)))
By prefixing the area or move symbol with the : (colon), we can call these functions for each class of object. Although there is no space between the colon and the symbol following it, newLISP parses them as distinct entities. The colon works as a function that processes parameters:
(:area myrect) → 200 ; same as (Rectangle:area myrect) (:area mycircle) → 314.1592654 ; same as (Circle:area mycircle) (set 'myrect (:move myrect 2 3)) → (Rectangle 7 8 10 20) (set 'mycircle (:move mycircle 4 5)) → (Circle 5 7 10)
In this example, the correct qualified symbol (rectangle:area or circle:area) is constructed and applied to the object data based on the symbol following the colon and the context name (the first element of the object list).
Note that moving the shapes is done in a functional manner. Rather than changing the x and y coordinates directly in myrect and mycircle, newLISP constructs and then reassigns the moved shapes.
newLISP's built-in support for XML-encoded data or documents comprises three functions: xml-parse, xml-type-tags, and xml-error.
Use the xml-parse function to parse XML-encoded strings. When xml-parse encounters an error, nil is returned. To diagnose syntax errors caused by incorrectly formatted XML, use the function xml-error. The xml-type-tags function can be used to control or suppress the appearance of XML type tags. These tags classify XML into one of four categories: text, raw string data, comments, and element data.
XML source:
<?xml version="1.0"?>
<DATABASE name="example.xml">
<!--This is a database of fruits-->
<FRUIT>
<NAME>apple</NAME>
<COLOR>red</COLOR>
<PRICE>0.80</PRICE>
</FRUIT>
</DATABASE>
(xml-parse (read-file "example.xml"))
→ (("ELEMENT" "DATABASE" (("name" "example.xml")) (("TEXT" "\r\n")
("COMMENT" "This is a database of fruits")
("TEXT" "\r\n ")
("ELEMENT" "FRUIT" () (
("TEXT" "\r\n\t ")
("ELEMENT" "NAME" () (("TEXT" "apple")))
("TEXT" "\r\n\t\t")
("ELEMENT" "COLOR" () (("TEXT" "red")))
("TEXT" "\r\n\t\t")
("ELEMENT" "PRICE" () (("TEXT" "0.80")))
("TEXT" "\r\n\t")))
("TEXT" "\r\n"))))
S-XML can be generated directly from XML using xml-type-tags and the special option parameters of the xml-parse function:
(xml-type-tags nil nil nil nil)
(xml-parse (read-file "example.xml") (+ 1 2 4 8 16))
→ ((DATABASE (@ (name "example.xml"))
(FRUIT (NAME "apple")
(COLOR "red")
(PRICE "0.80"))))
S-XML is XML reformatted as LISP S-expressions. The @ (at symbol) denotes an XML attribute specification.
See xml-parse in the reference section of the manual for details on parsing and option numbers, as well as for a longer example.
The remote procedure calling protocol XML-RPC uses HTTP post requests as a transport and XML for the encoding of method names, parameters, and parameter types. XML-RPC client libraries and servers have been implemented for most popular compiled and scripting languages.
For more information about XML, visit www.xmlrpc.com.
XML-RPC clients and servers are easy to write using newLISP's built-in network and XML support. A stateless XML-RPC server implemented as a CGI service, can be found in the file examples/xmlrpc.cgi. This script can be used together with a web server, like Apache. This XML-RPC service scripts implement the following methods:
| method | description |
|---|---|
| system.listMethods | Returns a list of all method names |
| system.methodHelp | Returns help for a specific method |
| system.methodSignature | Returns a list of return/calling signatures for a specific method |
| newLISP.evalString | Evaluates a Base64 newLISP expression string |
The first three methods are discovery methods implemented by most XML-RPC servers. The last one is specific to the newLISP XML-RPC server script and implements remote evaluation of a Base64-encoded string of newLISP source code. newLISP's base64-enc and base64-dec functions can be used to encode and decode Base64-encoded information.
In the modules directory of the source distribution, the file xmlrpc-client.lsp implements a specific client interface for all of the above methods.
(load "xmlrpc-client.lsp") ; load XML-RPC client routines
(XMLRPC:newLISP.evalString
"http://localhost:8080"
"(+ 3 4)") → "7"
In a similar fashion, standard system.xxx calls can be issued.
All functions return either a result if successful, or nil if a request fails. In case of failure, XMLRPC:error can be evaluated to return an error message.
For more information, please consult the header of the file modules/xmlrpc-client.lsp.
All built-in primitives in newLISP can be easily renamed:
(constant 'plus +)
Now, plus is functionally equivalent to + and runs at the same speed. As with many scripting languages, this allows for double precision floating point arithmetic to be used throughout newLISP.
The constant function, rather than the set function, must be used to rename built-in primitive symbols. By default, all built-in function symbols are protected against accidental overwriting.
(constant '+ add) (constant '- sub) (constant '* mul) (constant '/ div)
All operations using +, -, *, and / are now performed as floating point operations.
Using the same mechanism, the names of built-in functions can be translated into languages other than English:
(constant 'wurzel sqrt) ; German for 'square-root' (constant 'imprime print) ; Spanish for 'print' …
newLISP can switch locales based on the platform and operating system. On startup, newLISP attempts to set the ISO C standard default POSIX locale, available for most platforms and locales. Use the set-locale function to switch to the default locale:
(set-locale "")
This switches to the default locale used on your platform/operating system and ensures character handling (e.g., upper-case) work correctly.
Many Unix systems have a variety of locales available. To find out which ones are available on a particular Linux/UNIX/BSD system, execute the following command in a system shell:
locale -a
This command prints a list of all the locales available on your system. Any of these may be used as arguments to set-locale:
(set-locale "es_US")
This would switch to a U.S. Spanish locale. Accents or other characters used in a U.S. Spanish environment would be correctly converted.
See the manual description for more details on the usage of set-locale.
Many countries use a comma instead of a period as a decimal separator in numbers. newLISP correctly parses numbers depending on the locale set:
;; switch to German locale on a Linux system
(set-locale "de_DE")
;; newLISP source and output use a decimal comma
(div 1,2 3) → 0,4
The default POSIX C locale, which is set when newLISP starts up, uses a period as a decimal separator.
The following countries use a period as a decimal separator:
Australia, Botswana, Canada (English-speaking), China, Costa Rica, Dominican Republic, El Salvador, Guatemala, Honduras, Hong Kong, India, Ireland, Israel, Japan, Korea (both North and South), Malaysia, Mexico, Nicaragua, New Zealand, Panama, Philippines, Puerto Rico, Saudi Arabia, Singapore, Thailand, United Kingdom, and United States
The following countries use a comma as a decimal separator:
Albania, Andorra, Argentina, Austria, Belarus, Belgium, Bolivia, Brazil, Bulgaria, Canada (French-speaking), Croatia, Cuba, Chile, Colombia, Czech Republic, Denmark, Ecuador, Estonia, Faroes, Finland, France, Germany, Greece, Greenland, Hungary, Indonesia, Iceland, Italy, Latvia, Lithuania, Luxembourg, Macedonia, Moldova, Netherlands, Norway, Paraguay, Peru, Poland, Portugal, Romania, Russia, Serbia, Slovakia, Slovenia, Spain, South Africa, Sweden, Switzerland, Ukraine, Uruguay, Venezuela, and Zimbabwe
Note that for many European languages, the set-locale mechanism is sufficient to display non-ASCII character sets, as long as each character is presented as one byte internally. UTF-8 encoding is only necessary for multibyte character sets as described in this chapter.
newLISP can be compiled as a UTF-8–enabled application. UTF-8 is a multibyte encoding of the international Unicode character set. A UTF-8–enabled newLISP running on an operating system with UTF-8 enabled can handle any character of the installed locale.
The following steps make UTF-8 work with newLISP on a specific operating system and platform:
(1) Use one of the makefiles ending in utf8 to compile newLISP as a UTF-8 application. If no UTF-8 makefile is available for your platform, the normal makefile for your operating system contains instructions on how to change it for UTF-8.
The Mac OS X binary installer contains a UTF-8–enabled version by default.
(2) Enable the UTF-8 locale on your operating system. Check and set a UTF-8 locale on Unix and Unix-like OSes by using the locale command or the set-locale function within newLISP. On Linux, the locale can be changed by setting the appropriate environment variable. The following example uses bash to set the U.S. locale:
export LC_CTYPE=en_US.UTF-8
(3) The UTF-8–enabled newLISP automatically switches to the locale found on the operating system. Make sure the command shell is UTF-8–enabled. When using the Tcl/Tk front-end on Linux/UNIX, Tcl/Tk will automatically switch to UTF-8 display as long as the UNIX environment variable is set correctly. The U.S. version of WinXP's notepad.exe can display Unicode UTF-8–encoded characters, but the command shell and the Tcl/Tk front-end cannot. On Linux and other UNIXes, the Xterm shell can be used when started as follows:
LC_CTYPE=en_US.UTF-8 xterm
The following procedure can now be used to check for UTF-8 support. After starting newLISP, type:
(println (char 937)) ; displays Greek uppercase omega (println (lower-case (char 937))) ; displays lowercase omega
While the uppercase omega (Ω) looks like a big O on two tiny legs, the lowercase omega (ω) has a shape similar to a small w in the Latin alphabet.
Note: Only the output of println will be displayed as a character; println's return value will appear on the console as a multibyte ASCII character.
When UTF-8–enabled newLISP is used on a non-UTF-8–enabled display, both the output and the return value will be two characters. These are the two bytes necessary to encode the omega character.
When UTF-8–enabled newLISP is used, the following string functions work on character rather than byte boundaries:
| function | description |
|---|---|
| bind | binds var |