A bit Documentation and a new more Tests for lawrence

.gitignore

@@ -7,3 +7,6 @@ pom.xml.asc
 *.class
 /.lein-*
 /.nrepl-port
+/.idea
+*.iml
+test/active/lawrence/calc_it_parser.clj

COPYING

@@ -0,0 +1,48 @@
+Copyright (c) 1999-2008 Michael Sperber and Peter Thiemann.
+
+Redistribution and use in source and binary forms, with or without
+modification, are permitted provided that the following conditions
+are met:
+1. Redistributions of source code must retain the above copyright
+   notice, this list of conditions and the following disclaimer.
+2. Redistributions in binary form must reproduce the above copyright
+   notice, this list of conditions and the following disclaimer in the
+   documentation and/or other materials provided with the distribution.
+3. All advertising materials mentioning features or use of this software
+   must display the following acknowledgement: 
+4. The names of the authors of this software may not be used to
+   endorse or promote products derived from this software
+   without specific prior written permission.
+
+THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND
+ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+ARE DISCLAIMED.  IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE
+FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
+OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
+HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
+LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
+OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
+SUCH DAMAGE.
+
+Distributing Autoconf Output
+****************************
+
+[excerpt from autoconf documentation]
+
+   The configuration scripts that Autoconf produces are covered by the
+GNU General Public License.  This is because they consist almost
+entirely of parts of Autoconf itself, rearranged somewhat, and Autoconf
+is distributed under the terms of the GPL.  As applied to Autoconf, the
+GPL just means that you need to distribute `configure.in' along with
+`configure'.
+
+   Programs that use Autoconf scripts to configure themselves do not
+automatically come under the GPL.  Distributing an Autoconf
+configuration script as part of a program is considered to be *mere
+aggregation* of that work with the Autoconf script.  Such programs are
+not derivative works based on Autoconf; only their configuration scripts
+are.  We still encourage software authors to distribute their work under
+terms like those of the GPL, but doing so is not required to use
+Autoconf.

README.md

@@ -4,11 +4,12 @@ An LR parser generator for Clojure.
 
 ## Usage
 
-FIXME
+For the specification and reference and more see REFERENCE.md 
+in the /doc directory of this project
 
 ## License
 
-Copyright © 2015 Active Group GmbH
+Copyright © 2015-2024 Active Group GmbH
 
-Distributed under the Eclipse Public License either version 1.0 or (at
+Distributed under the Eclipse Public License either version 2.0 or (at
 your option) any later version.

doc/REFERENCE.md

@@ -0,0 +1,95 @@
+## The parser and his functionality
+ - Remark if you like further explanation about left-to-right (shift - reduce parsers) look at
+the documentation of Essence:
+
+   [S48 Essence LR Parrser Generator Documentation](https://www.s48.org/essence/doc/html/essence.html)
+- in that document there is also an general explanation about the way LR parser generators work.
+### Parser and history
+
+Lawrence is a implementation of an 
+LR / SLR parser generator derived from the 
+Essence parser generator written in Scheme-48.
+
+### Definition of a context free grammer
+
+
+the same grammar in lawrence
+
+```scheme
+(define-grammar
+  calculator
+  (:plus :minus :mul :div :lparen :rparen
+    :not :decimal-symbol :power :whitespace)
+  expression
+  ((expression ((term) $1)
+     ((:$error) 0)
+     ((term :plus expression) (+ $1 $3))
+     ((term :minus expression) (- $1 $3)))
+    (term ((product) $1)
+      ((product :mul term) (* $1 $3))
+      ((product :div term) (/ $1 $3)))
+    (product ((:decimal-symbol) $1)
+      ((:lparen
+         expression :rparen) $2)
+      ((:lparen :$error :rparen) 0))
+
+    ))
+```
+
+### The explanation of the grammars above
+The expressions:
+
+- define terminals
+```scheme 
+(:+ :- :* :/ :**)
+```
+
+
+- The 'expression' in the line below the terminals 
+is the start symbol. Thjis symbol is needed by the generator to determine the start of the grammar
+- After that the rules for parsing follow:
+
+#### Something about placeholders (variables)
+- The $(n) where (n) is a integer defines a variable the (n) there defines the placement of the variable inside a term
+- Example:
+Lawrence (example above but as infix)
+  ```scheme 
+   ((term :plus expression) (+ $1 $3))
+  ```
+here a addition is defined (infix) code: 5 + 3 - so the first and the third placements defines variables -> $1 $3
+#### The structure of a rule
+
+```scheme
+1. ((expression ((term) $1) 
+2.      ((:$error) 0).....))
+```
+- 1: here the **_rule_** for building a **expression** out of a given **term** which is defined later
+- 2: here the **_rule_** what happens in case of a parser error is defined the **_rule_** tells us that we simply go on in case of a error
+
+
+ ```scheme 
+  3. ((term :plus expression) (+ $1 $3)) 
+  ```
+- 3: and now after we defined a **expression** by a term we define that a **term** and a **terminal**  
+
+```scheme
+ (term ((product) $1)... )
+```
+
+- 4: with this **_rule_** we define a **term** in final as a result of the definition of **product**
+```scheme
+ (product ((:decimal-symbol) $1)...)
+```
+
+- 5: what we need now is that we define the **product** by using the smallest unit 
+(in our case the **number**) to be sure that
+any rule we defined ends in a defined **_terminal/ token_**.
+
+### Some code examples how to call lawrence and test it direct or generate a specialized parser
+[Code to call lawrence parse](../test/active/lawrence/util/parse_lex_util.clj)
+
+#### **Remark :** 
+The definition of the terminals for this example is done in a scanner specification
+written with macros from ephemerol. See the `**_de.active-group/ephemerol_**` project
+on GitHub: [**_Ephemerol_**](https://github.com/active-group/ephemerol)
+

project.clj

@@ -1,8 +1,11 @@
-(defproject lawrence "0.12.0-SNAPSHOT"
+(defproject de.active-group/lawrence "0.12.0-SNAPSHOT"
   :description "Lawrence: LR parser generator"
   :url "https://github.com/active-group/lawrence"
   :license {:name "Eclipse Public License"
             :url "http://www.eclipse.org/legal/epl-v10.html"}
-  :dependencies [[org.clojure/clojure "1.8.0"]
-                 [active-clojure "0.12.0" :exclusions [org.clojure/clojure]]]
-  :profiles {:dev {:dependencies [[org.clojure/tools.trace "0.7.9"]]}})
+  :dependencies [[org.clojure/clojure "1.10.1"]
+                 [de.active-group/active-clojure "0.42.2"]
+                 [de.active-group/ephemerol "0.6.0-SNAPSHOT"]]
+  :profiles {:dev {:dependencies [[org.clojure/tools.trace "0.7.9"]]}}
+  )
+

test/active/lawrence/functional_test.clj

@@ -0,0 +1,83 @@
+(ns active.lawrence.functional-test
+  (:require [clojure.test :refer :all]
+            [active.lawrence.util.parse-lex-util :refer :all]
+            [active.ephemerol.char-set :refer :all]
+            [active.lawrence.grammar :as gr]
+            [active.ephemerol.regexp :refer :all]
+            [active.ephemerol.scanner :refer :all]
+            [active.ephemerol.scanner-run :refer :all]))
+
+
+(def lang-scan (scanner-spec
+                 ((+ char-set:digit)
+                  (fn [lexeme position input input-position]
+                     (make-scan-result
+                      [:decimal-symbol (Integer/parseInt lexeme)]
+                      input input-position)))
+                 (char-set:whitespace
+                   (fn [lexeme position input input-position]
+                     (make-scan-result [:whitespace]
+                                       input input-position)))
+                 ("*"
+                   (fn [lexeme position input input-position]
+                     (make-scan-result [:mul]
+                                       input input-position)))
+                 ("/"
+                   (fn [lexeme position input input-position]
+                     (make-scan-result [:div]
+                                       input input-position)))
+                 ("-"
+                   (fn [lexeme position input input-position]
+                     (make-scan-result [:minus]
+                                       input input-position)))
+                 ("+"
+                   (fn [lexeme position input input-position]
+                     (make-scan-result  [:plus]
+                                       input input-position)))
+                 ("**"
+                   (fn [lexeme position input input-position]
+                     (make-scan-result [:power]
+                                       input input-position)))
+                 ("("
+                   (fn [lexeme position input input-position]
+                     (make-scan-result [:lparen]
+                                       input input-position)))
+                 (")"
+                   (fn [lexeme position input input-position]
+                     (make-scan-result [:rparen]
+                                       input input-position)))
+                 ))
+(gr/define-grammar
+                    calculator
+                    (:plus :minus :mul :div :lparen :rparen
+                          :not :decimal-symbol)
+                    expression
+                    ((expression ((term) $1)
+                                 ((:$error) 0)
+                                 ((term :plus expression) (+ $1 $3))
+                                 ((term :minus expression) (- $1 $3)))
+                     (term ((product) $1)
+                           ((product :mul term) (* $1 $3))
+                           ((product :div term) (/ $1 $3)))
+                     (product ((:decimal-symbol) $1)
+                              ((:lparen
+                                 expression :rparen) $2)
+                              ((:lparen :$error :rparen) 0))
+
+                     ))
+
+(defn should-accept-string-data
+  [pkg-sym scan-spec grammar data res]
+  (is (= res (execute-direct pkg-sym
+                             scan-spec grammar
+                             data :lr)))
+  (is (= res (execute-direct pkg-sym
+                             scan-spec grammar
+                             data :slr))))
+(deftest do-direct
+  (should-accept-string-data 'active.lawrence.functional-test
+                 lang-scan calculator
+                 "(9*5)" 45)
+
+)
+

test/active/lawrence/generate_test.clj

@@ -1,10 +1,14 @@
 (ns active.lawrence.generate-test
-  (:require [active.lawrence.grammar-test :refer :all]
-            [active.lawrence.lr :as lr]))
+  (:require [active.lawrence.lr :as lr]
+            [active.lawrence.process-test :refer :all]
+            [clojure.test :refer :all]))
+
 
 (defn generate
   [grammar name method]
   (lr/write-ds-parse-ns grammar 1 method
                         (symbol (str "active.lawrence." name "-parser"))
                         '([active.lawrence.parser-runtime :refer :all])
-                        (str "test/active/lawrence/" name "_parser.clj")))
+                        (str "./" name "_parser.clj")))
+
+(generate toys-are-us "calc_it" :slr)

test/active/lawrence/process_test.clj

@@ -0,0 +1,94 @@
+(ns active.lawrence.process-test
+  (:require [clojure.test :refer :all]
+            [active.lawrence.util.parse-lex-util :refer :all]
+            [active.ephemerol.char-set :refer :all]
+            [active.lawrence.grammar :as gr]
+            [active.ephemerol.regexp :refer :all]
+            [active.ephemerol.scanner :refer :all]
+            [active.ephemerol.scanner-run :refer :all]))
+
+(def toys-scan (scanner-spec
+                 ((+ char-set:digit)
+                  (fn [lexeme position input input-position]
+                    (make-scan-result
+                      [:n (Integer/parseInt lexeme) ]
+                      input input-position)))
+                 (char-set:whitespace
+                   (fn [lexeme position input input-position]
+                     (make-scan-result [:ws]
+                                       input input-position)))
+                 ("+"
+                   (fn [lexeme position input input-position]
+                     (make-scan-result [:+]
+                                       input input-position)))
+                 ("-"
+                   (fn [lexeme position input input-position]
+                     (make-scan-result [:-]
+                                       input input-position)))
+                 ("*"
+                   (fn [lexeme position input input-position]
+                     (make-scan-result [:*]
+                                       input input-position)))
+                 ("/"
+                   (fn [lexeme position input input-position]
+                     (make-scan-result [:/]
+                                       input input-position)))
+                 ("("
+                   (fn [lexeme position input input-position]
+                     (make-scan-result [:l]
+                                       input input-position)))
+                 (")"
+                   (fn [lexeme position input input-position]
+                     (make-scan-result [:r]
+                                       input input-position)))
+
+                 ))
+
+(gr/define-grammar toys-are-us
+                   (:+ :- :* :/ :l :r :n)
+                   E
+                   ((E ((T) $1)
+                       ((:$error) 0)
+                       ((T :+ E) (+ $1 $3))
+                       ((T :- E) (- $1 $3)))
+                    (T ((P) $1)
+                       ((P :* T) (* $1 $3))
+                       ((P :/ T) (/ $1 $3)))
+                    (P ((:n) $1)
+                       ((:l E :r) $2)
+                       ((:l :$error :r) 0))))
+(gr/define-grammar toys-are-us-ext
+                   (:+ :- :* :/ :l :r :n :ws)
+                   E
+                   ((E ((T) $1)
+                       ((:$error) 0)
+                       ((T :+ E) (+ $1 $3))
+                       ((T :- E) (- $1 $3)))
+                    (T ((P) $1)
+                       ((P :* T) (* $1 $3))
+                       ((P :/ T) (/ $1 $3)))
+                    (P ((:n) $1)
+                       ((:l E :r) $2)
+                       ((:l :$error :r) 0))))
+(defn should-accept-list-data
+  [g vs res]
+  (is (= res (parse g :lr vs)))
+  (is (= res (parse g :slr vs))))
+
+(defn should-accept-string-data
+  [pkg-sym scan-spec grammar data res]
+  (is (= res (execute-direct pkg-sym
+                             scan-spec grammar
+                             data :lr)))
+  (is (= res (execute-direct pkg-sym
+                             scan-spec grammar
+                             data :slr))))
+(deftest use-parse
+  (let [expr (list [:l] [:n 5] [:+] [:n 9] [:r])]
+    (println expr)
+(should-accept-list-data toys-are-us  expr 14)))
+
+(deftest process-rest
+  (should-accept-string-data 'active.lawrence.process-test
+                  toys-scan toys-are-us-ext
+                  "(9*5+10+(3-1))" 57))