• Top
  • Documentation
  • Books
  • Recursion-and-induction
  • Boolean-reasoning
  • Projects
  • Debugging
  • Std
  • Proof-automation
  • Macro-libraries
  • ACL2
  • Interfacing-tools
    • Io
    • Defttag
    • Sys-call
    • Save-exec
    • Quicklisp
    • Oslib
    • Std/io
    • Bridge
    • Clex
      • Example-lexer
        • Token-p
        • Lex-punctuation
        • Lex-id/keyword
        • Lex-string
        • Lex-comment
        • Lex-whitespace
        • Lex1
        • Lex-main
        • Lex*
        • Tokenlist-p
        • Idtail-char-p
        • Letter-char-p
        • Whitespace-char-p
        • Number-char-p
        • Tokentype-p
        • Lex*-exec
        • Newline-string
      • Sin
      • Matching-functions
      • Def-sin-progress
    • Tshell
    • Unsound-eval
    • Hacker
    • Startup-banner
    • Command-line
  • Hardware-verification
  • Software-verification
  • Testing-utilities
  • Math

• Example-lexer

Lex*

Main lexer loop: completely tokenize the entire input stream.

Signature

(lex* sin) → (mv okp tokens sin)

Arguments

sin — The sin stobj.

Returns

okp — Did we lex everything successfully?.
    Type (booleanp okp).

tokens — The tokens we've collected.
    Type (tokenlist-p tokens).

Definitions and Theorems

Function: lex*

(defun lex* (sin)
       (declare (xargs :stobjs (sin)))
       (declare (xargs :guard t))
       (let ((__function__ 'lex*))
            (declare (ignorable __function__))
            (mbe :logic (b* (((when (sin-endp sin)) (mv t nil sin))
                             ((mv tok1 sin) (lex1 sin))
                             ((unless tok1) (mv nil nil sin))
                             ((mv okp rest-tokens sin) (lex* sin))
                             (ans (cons tok1 rest-tokens)))
                            (mv okp ans sin))
                 :exec (b* (((mv okp acc sin) (lex*-exec sin nil)))
                           (mv okp (reverse acc) sin)))))

Theorem: booleanp-of-lex*.okp

(defthm booleanp-of-lex*.okp
        (b* (((mv ?okp ?tokens ?sin) (lex* sin)))
            (booleanp okp))
        :rule-classes :type-prescription)

Theorem: tokenlist-p-of-lex*.tokens

(defthm tokenlist-p-of-lex*.tokens
        (b* (((mv ?okp ?tokens ?sin) (lex* sin)))
            (tokenlist-p tokens))
        :rule-classes :rewrite)

Theorem: lex*-exec-removal

(defthm lex*-exec-removal
        (b* (((mv okp tokens new-sin) (lex* sin)))
            (equal (lex*-exec sin acc)
                   (mv okp (revappend tokens acc)
                       new-sin))))

Theorem: true-listp-of-lex*-tokens

(defthm true-listp-of-lex*-tokens
        (true-listp (mv-nth 1 (lex* sin)))
        :rule-classes :type-prescription)

Theorem: tokenlist-all-text-of-lex*

(defthm tokenlist-all-text-of-lex*
        (b* (((mv okp tokens ?new-sin) (lex* sin)))
            (implies okp
                     (equal (tokenlist-all-text tokens)
                            (implode (strin-left sin))))))