README: library --Sexplib-- *************************** Copyright (C) 2005 Jane Street Capital, LLC (1) ====================================================== Author: Markus Mottl ===================== New York, 2005-11-07 ===================== 1 Directory contents *=*=*=*=*=*=*=*=*=*=* ------------------------------------------------------------------------ | CHANGES | History of code changes | ------------------------------------------------------------------------ | COPYRIGHT | Notes on copyright | ------------------------------------------------------------------------ | INSTALL | Short notes on compiling and | | | installing the library | ------------------------------------------------------------------------ | LICENSE | --GNU LESSER GENERAL PUBLIC LICENSE-- | ------------------------------------------------------------------------ | LICENSE.Tywith | License of Tywith, from which Sexplib is derived| ------------------------------------------------------------------------ | Makefile | Top Makefile | ------------------------------------------------------------------------ | OCamlMakefile | Generic Makefile for OCaml-projects | ------------------------------------------------------------------------ | OMakefile | Ignore this file | ------------------------------------------------------------------------ | README | This file | ------------------------------------------------------------------------ | VERSION | Current version | ------------------------------------------------------------------------ | lib/ | OCaml-library for S-expression conversions | ------------------------------------------------------------------------ | lib_test/ | Test applications for the Sexplib-library | ------------------------------------------------------------------------ 2 What is --Sexplib--? *=*=*=*=*=*=*=*=*=*=*=* This library contains functionality for parsing and pretty-printing S-expressions. In addition to that it contains an extremely useful preprocessing module for Camlp4, which can be used to automatically generate code from type definitions for efficiently converting OCaml-values to S-expressions and vice versa. In combination with the parsing and pretty-printing functionality this frees the user from having to write his own I/O-routines for datastructures he defines. Possible errors during automatic conversions from S-expressions to OCaml-values are reported in a very human-readable way. Another module in the library allows you to extract and replace sub-expressions in S-expressions. 3 How can you use it? *=*=*=*=*=*=*=*=*=*=*= The API (.mli-files) in the library directory is fully documented. Module --Sexp-- contains all I/O-functions for S-expressions, module --Conv-- helper functions for converting OCaml-values of standard types to S-expressions. Module --Path-- supports sub-expression extraction and substitution. The module --pa_sexp_conv.ml-- contains the extensions for the Camlp4-preprocessor. It adds the new construct --with sexp-- (and --with sexp_of-- and --with of_sexp--, which are implied by the first). When using this construct right after a type definition, function definitions will be generated automatically, which perform S-expression conversions. E.g. given the following type definition: << type t = A | B with sexp >> The above will generate the functions --sexp_of_t-- and --t_of_sexp--. See the file --lib_test/conv_test.ml-- for example usage. It also demonstrates how to extract and substitute sub-expressions. 4 Syntax Specification of S-expressions *=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*= 4.1 Lexical conventions of S-expression ======================================== Whitespace, which consists of space, newline, carriage return, horizontal tab and form feed, is ignored unless within an OCaml-string, where it is treated according to OCaml-conventions. The semicolon introduces comments. Comments are ignored, and range up to the next newline character. The left parenthesis opens a new list, the right parenthesis closes it again. Lists can be empty. The double quote denotes the beginning and end of a string following the lexical conventions of OCaml (see OCaml-manual for details). All characters other than double quotes, left- and right parentheses, and whitespace are considered part of a contiguous string. 4.2 Grammar of S-expressions ============================= S-expressions are either strings (= atoms) or lists. The lists can recursively contain further S-expressions or be empty, and must be balanced, i.e. parentheses must match. 4.3 Examples ============= << this_is_an_atom_123'&^%! ; this is a comment "another atom in an OCaml-string \"string in a string\" \123" ; empty list follows below () ; a more complex example ( ( list in a list ; comment within a list (list in a list in a list) 42 is the answer to all questions ) ) >> 4.4 Conversion of basic OCaml-values ===================================== Basic OCaml-values like the unit-value, integers (in all representations), floats, strings, and booleans are represented in S-exp syntax in the same way as in OCaml. Strings may also appear without quotes if this does not clash with the lexical conventions for S-expressions. 4.5 Conversion of OCaml-tuples =============================== OCaml-tuples are simple lists of values in the same order as in the tuple. E.g.: << (3.14, "foo", "bar bla", 27) <===> (3.14 foo "bar bla" 27) >> 4.6 Conversion of OCaml-records ================================ OCaml-records are represented as lists of pairs in S-expression syntax. Each pair consists of the name of the record field (first element), and its value (second element). E.g.: << { foo = 3; bar = "some string"; } <===> ( (foo 3) (bar "some string") ) >> 4.7 Conversion of sum types ============================ Constant constructors in sum types are represented as strings. Constructors with arguments are represented as lists, the first element being the constructor name, the rest being its arguments. Constructors may also be started in lowercase in S-expressions, but will always be converted to uppercase when converting from OCaml-values. For example: << type t = A | B of int * float * t with sexp B (42, 3.14, B (-1, 2.72, A)) <===> (B 42 3.14 (B -1 2.72 A)) >> The above example also demonstrates recursion in datastructures. 4.8 Conversion of variant types ================================ The conversion of polymorphic variants is almost the same as with sum types. The notable difference is that variant constructors must always start with an either lower- or uppercase character, matching the way it was specified in the type definition. This is because OCaml also distinguishes between upper- and lowercase variant constructors. Note that type specifications containing unions of variant types are also supported by the S-expression converter. 4.9 Conversion of OCaml-lists and arrays ========================================= OCaml-lists and arrays are straightforwardly represented as S-expression lists. 4.10 Conversion of optional values =================================== As a special case, values of option type are always represented as lists in S-expressions: << None <===> () Some value <===> (value) >> 4.11 Conversion of polymorphic values ====================================== There is nothing special about polymorphic values as long as there are conversion functions for the type parameters. E.g.: << type 'a t = A | B of 'a with sexp type foo = int t with sexp >> In the above case the conversion functions will behave as if `foo' had been defined as a monomorphic version of `t' with `'a' replaced by `int' on the right hand side. If a datastructure is indeed polymorphic, and you want to convert it, you will have to supply the conversion functions for the type parameters at runtime. E.g. in the above example, if you wanted to convert a value of type `'a t', you would have to write something like this: << sexp_of_t sexp_of_a v >> where `sexp_of_a', which may also be named differently in this particular case, is a function that converts values of type `'a' to an S-expression. Types with more than one parameter require passing conversion functions for those parameters in the order of their appearance on the left hand side of the type definition. 4.12 Conversion of abstract datatypes ====================================== Of course, if you want to convert an abstract datatype to an S-expression, you will have to roll your own conversion function, which should produce values of type `Sexp.t' directly. If, however, you want to make use of your abstract type within definitions of other types, make sure that you call your conversion function appropriately: it should be in the same scope as the typename, and must be named `sexp_of_{typename}'. 4.13 Conversion of hashtables ============================== Hashtables, which are abstract values in OCaml, are represented as association lists, i.e. lists of key-value pairs, e.g.: << ((foo 42) (bar 3)) >> Reading in the above S-expression as hashtable mapping strings to integers (`(string, int) Hashtbl.t') will map `"foo"' to 42 and `"bar"' to 3. Hashtables will be created with a default size (at time of writing: 1973). This size can be changed by setting the reference `Sexplib.Conv.hashtbl_size' to a different value. Note that the order of elements in the list may matter, because duplicates are kept: bindings will be inserted into the hashtable in order of appearence. Therefore, the last binding of a key will be the --visible-- one, the others are --hidden--. See the OCaml-documentation on hashtables for details. Note, too, that polymorphic equality may not hold between conversions. You will have to use a function implementing logical equality for that purpose. 5 Contact information *=*=*=*=*=*=*=*=*=*=*= In the case of bugs, feature requests and similar, you can contact us here: mmottl@janestcapital.com Up-to-date information concerning this library should be available here: http://www.janestcapital.com/ocaml Enjoy!! -------------------------------------- (1) http://www.janestcapital.com ----------------------------------------------------------------------------- This document was translated from LaTeX by HeVeA (2).