The basic wrapper is a good way to obtain a function of type String -> [token] from a lexer specification, with little fuss.

It provides definitions for AlexInput, alexGetChar and alexInputPrevChar that are suitable for lexing a String input. It also provides a function alexScanTokens which takes a String input and returns a list of the tokens it contains.

The basic wrapper provides no support for using startcodes; the initial startcode is always set to zero.

Here is the actual code included in the lexer when the basic wrapper is selected:

type AlexInput = (Char,     -- previous char
                  String)   -- current input string

alexGetChar :: AlexInput -> Maybe (Char,AlexInput)
alexGetChar (_, [])   = Nothing
alexGetChar (_, c:cs) = Just (c, (c,cs))

alexInputPrevChar :: AlexInput -> Char
alexInputPrevChar (c,_) = c

-- alexScanTokens :: String -> [token]
alexScanTokens str = go ('\n',str)
  where go inp@(_,str) =
          case alexScan inp 0 of
                AlexEOF -> []
                AlexError _ -> error "lexical error"
                AlexSkip  inp' len     -> go inp'
                AlexToken inp' len act -> act (take len str) : go inp'

The type signature for alexScanTokens is commented out, because the token type is unkonwn. All of the actions in your lexical specification should have type:

{ ... } :: String -> token

for some type token.

For an example of the use of the basic wrapper, see the file examples/Tokens_basic.x in the Alex distribution.

The posn wrapper provides slightly more functionality than the basic wrapper: it keeps track of line and column numbers of tokens in the input text.

The posn wrapper provides the following, in addition to the straightforward definitions of alexGetChar and alexInputPrevChar:

data AlexPosn = AlexPn !Int  -- absolute character offset
                       !Int  -- line number
                       !Int  -- column number

type AlexInput = (AlexPosn, -- current position,
                  Char,     -- previous char
                  String)   -- current input string

--alexScanTokens :: String -> [token]
alexScanTokens str = go (alexStartPos,'\n',str)
  where go inp@(pos,_,str) =
          case alexScan inp 0 of
                AlexEOF -> []
                AlexError _ -> error "lexical error"
                AlexSkip  inp' len     -> go inp'
                AlexToken inp' len act -> act pos (take len str) : go inp'

The types of the token actions should be:

{ ... } :: AlexPosn -> String -> token

For an example using the posn wrapper, see the file examples/Tokens_posn.x in the Alex distribution.

The monad wrapper is the most flexible of the wrappers provided with Alex. It includes a state monad which keeps track of the current input and text position, and the startcode. It is intended to be a template for building your own monads - feel free to copy the code and modify it to build a monad with the facilities you need.

data AlexState = AlexState {
        alex_pos :: !AlexPosn,  -- position at current input location
        alex_inp :: String,     -- the current input
        alex_chr :: !Char,      -- the character before the input
        alex_scd :: !Int        -- the current startcode
    }

newtype Alex a = Alex { unAlex :: AlexState
                               -> Either String (AlexState, a) }

runAlex          :: String -> Alex a -> Either String a

alexGetInput     :: Alex AlexInput
alexSetInput     :: AlexInput -> Alex ()

alexError        :: String -> Alex a

alexGetStartCode :: Alex Int
alexSetStartCode :: Int -> Alex ()

To invoke a scanner under the monad wrapper, use alexMonadScan:

alexMonadScan :: Alex result

The token actions should have the following type:

type AlexAction result = AlexInput -> Int -> Alex result
{ ... }  :: AlexAction result

The monad wrapper also provides some useful combinators for constructing token actions:

-- skip :: AlexAction result
skip input len = alexMonadScan

-- andBegin :: AlexAction result -> Int -> AlexAction result
(act `andBegin` code) input len = do alexSetStartCode code; act input len

-- begin :: Int -> AlexAction result
begin code = skip `andBegin` code

-- token :: (String -> Int -> token) -> AlexAction token
token t input len = return (t input len)

The gscan wrapper is provided mainly for historical reasons: it exposes an interface which is very similar to that provided by Alex version 1.x. The interface is intended to be very general, allowing actions to modify the startcode, and pass around an arbitrary state value.

alexGScan :: StopAction state result -> state -> String -> result

type StopAction state result 
         = AlexPosn -> Char -> String -> (Int,state) -> result

The token actions should all have this type:

{ ... }      :: AlexPosn                -- token position
             -> Char                    -- previous character
             -> String                  -- input string at token
             -> Int                     -- length of token
             -> ((Int,state) -> result) -- continuation
             -> (Int,state)             -- current (startcode,state)
             -> result