new rc relaease

README.md

@@ -22,13 +22,13 @@ npm install gpt-3-encoder
 
 
 
-[![JSDocs](https://img.shields.io/badge/JS%20Docs-Read%20them%20maybe-brightgreen)](https://latitudegames.github.io/GPT-3-Encoder/)
+[![JSDocs](https://img.shields.io/badge/JS%20Docs-Read%20them%20maybe-brightgreen)](https://syonfox.github.io/GPT-3-Encoder/)
 
 Also check out the browser demo [browser demo](https://latitudegames.github.io/GPT-3-Encoder/browser.html)
 
-[![GitHub last commit](https://img.shields.io/github/last-commit/latitudegames/GPT-3-Encoder)](https://github.com/latitudegames/GPT-3-Encoder/commits)
-[![example workflow](https://github.com/latitudegames/GPT-3-Encoder/actions/workflows/node.js.yml/badge.svg)](https://github.com/latitudegames/GPT-3-Encoder/actions)
-[![github](https://img.shields.io/badge/GitHub-100000?style=for-the-badge&logo=github&logoColor=white)](https://github.com/latitudegames/GPT-3-Encoder/)
+[![GitHub last commit](https://img.shields.io/github/last-commit/syonfox/GPT-3-Encoder)](https://github.com/syonfox/GPT-3-Encoder/commits)
+[![example workflow](https://github.com/syonfox/GPT-3-Encoder/actions/workflows/node.js.yml/badge.svg)](https://github.com/syonfox/GPT-3-Encoder/actions)
+[![github](https://img.shields.io/badge/GitHub-100000?style=for-the-badge&logo=github&logoColor=white)](https://github.com/syonfox/GPT-3-Encoder/)
 
 Compatible with Node >= 12
 
@@ -49,7 +49,7 @@ This function returns the number of tokens in the provided text, after encoding
 This function returns an object containing statistics about the tokens in the provided text, after encoding it using
 BPE. The returned object includes the following properties:
 
-- `total`: the total number of tokens in the text.
+- `count`: the total number of tokens in the text.
 - `unique`: the number of unique tokens in the text.
 - `frequencies`: an object containing the frequency of each token in the text.
 - `postions`: an object mapping tokens to positions in the encoded string
@@ -97,7 +97,7 @@ I have added som other examples to the examples folder.
 Please take a look at package.json for how to do stuff
 
 ```sh
-git clone https://github.com/latitudegames/GPT-3-Encoder.git
+git clone https://github.com/syonfox/GPT-3-Encoder.git
 
 cd GPT-3-Encoder
 

docs/Encoder.js.html

@@ -379,7 +379,7 @@ <h2><a href="index.html">Home</a></h2><h3>Global</h3><ul><li><a href="global.htm
 <br class="clear">
 
 <footer>
-    Documentation generated by <a href="https://github.com/jsdoc/jsdoc">JSDoc 4.0.0</a> on Wed Mar 08 2023 08:08:38 GMT-0800 (Pacific Standard Time)
+    Documentation generated by <a href="https://github.com/jsdoc/jsdoc">JSDoc 4.0.0</a> on Wed Mar 08 2023 08:25:29 GMT-0800 (Pacific Standard Time)
 </footer>
 
 <script> prettyPrint(); </script>

docs/global.html

@@ -1070,7 +1070,7 @@ <h2><a href="index.html">Home</a></h2><h3>Global</h3><ul><li><a href="global.htm
 <br class="clear">
 
 <footer>
-    Documentation generated by <a href="https://github.com/jsdoc/jsdoc">JSDoc 4.0.0</a> on Wed Mar 08 2023 08:08:38 GMT-0800 (Pacific Standard Time)
+    Documentation generated by <a href="https://github.com/jsdoc/jsdoc">JSDoc 4.0.0</a> on Wed Mar 08 2023 08:25:29 GMT-0800 (Pacific Standard Time)
 </footer>
 
 <script> prettyPrint(); </script>

docs/index.html

@@ -56,11 +56,11 @@ <h2>Usage</h2>
 <a href="https://www.npmjs.com/package/gpt-3-encoder">
   <img src="https://img.shields.io/npm/v/gpt-3-encoder.svg" alt="npm version">
 </a>
-<p><a href="https://latitudegames.github.io/GPT-3-Encoder/"><img src="https://img.shields.io/badge/JS%20Docs-Read%20them%20maybe-brightgreen" alt="JSDocs"></a></p>
+<p><a href="https://syonfox.github.io/GPT-3-Encoder/"><img src="https://img.shields.io/badge/JS%20Docs-Read%20them%20maybe-brightgreen" alt="JSDocs"></a></p>
 <p>Also check out the browser demo <a href="https://latitudegames.github.io/GPT-3-Encoder/browser.html">browser demo</a></p>
-<p><a href="https://github.com/latitudegames/GPT-3-Encoder/commits"><img src="https://img.shields.io/github/last-commit/latitudegames/GPT-3-Encoder" alt="GitHub last commit"></a>
-<a href="https://github.com/latitudegames/GPT-3-Encoder/actions"><img src="https://github.com/latitudegames/GPT-3-Encoder/actions/workflows/node.js.yml/badge.svg" alt="example workflow"></a>
-<a href="https://github.com/latitudegames/GPT-3-Encoder/"><img src="https://img.shields.io/badge/GitHub-100000?style=for-the-badge&amp;logo=github&amp;logoColor=white" alt="github"></a></p>
+<p><a href="https://github.com/syonfox/GPT-3-Encoder/commits"><img src="https://img.shields.io/github/last-commit/syonfox/GPT-3-Encoder" alt="GitHub last commit"></a>
+<a href="https://github.com/syonfox/GPT-3-Encoder/actions"><img src="https://github.com/syonfox/GPT-3-Encoder/actions/workflows/node.js.yml/badge.svg" alt="example workflow"></a>
+<a href="https://github.com/syonfox/GPT-3-Encoder/"><img src="https://img.shields.io/badge/GitHub-100000?style=for-the-badge&amp;logo=github&amp;logoColor=white" alt="github"></a></p>
 <p>Compatible with Node &gt;= 12</p>
 <p>To use the library in your project, import it as follows:</p>
 <pre class="prettyprint source lang-js"><code>const GPT3Encoder = require('gpt-3-encoder');
@@ -73,7 +73,7 @@ <h3>Additional Features</h3>
 <p>This function returns an object containing statistics about the tokens in the provided text, after encoding it using
 BPE. The returned object includes the following properties:</p>
 <ul>
-<li><code>total</code>: the total number of tokens in the text.</li>
+<li><code>count</code>: the total number of tokens in the text.</li>
 <li><code>unique</code>: the number of unique tokens in the text.</li>
 <li><code>frequencies</code>: an object containing the frequency of each token in the text.</li>
 <li><code>postions</code>: an object mapping tokens to positions in the encoded string</li>
@@ -88,7 +88,7 @@ <h2>Example</h2>
 Note you may need to include it from the appropriate place in node modules / npm package name</p>
 <pre class="prettyprint source lang-js"><code>
 import {encode, decode, countTokens, tokenStats} from &quot;gpt-3-encoder&quot;
-//or
+//or note you might need @syonfox/gpt-3-encoder if thats what you npm install
 const {encode, decode, countTokens, tokenStats} = require('gpt-3-encoder')
 
 const str = 'This is an example sentence to try encoding out on!'
@@ -112,7 +112,7 @@ <h2>Example</h2>
 <h2>Developers</h2>
 <p>I have added som other examples to the examples folder.
 Please take a look at package.json for how to do stuff</p>
-<pre class="prettyprint source lang-sh"><code>git clone https://github.com/latitudegames/GPT-3-Encoder.git
+<pre class="prettyprint source lang-sh"><code>git clone https://github.com/syonfox/GPT-3-Encoder.git
 
 cd GPT-3-Encoder
 
@@ -181,7 +181,7 @@ <h2><a href="index.html">Home</a></h2><h3>Global</h3><ul><li><a href="global.htm
 <br class="clear">
 
 <footer>
-    Documentation generated by <a href="https://github.com/jsdoc/jsdoc">JSDoc 4.0.0</a> on Wed Mar 08 2023 08:08:38 GMT-0800 (Pacific Standard Time)
+    Documentation generated by <a href="https://github.com/jsdoc/jsdoc">JSDoc 4.0.0</a> on Wed Mar 08 2023 08:25:29 GMT-0800 (Pacific Standard Time)
 </footer>
 
 <script> prettyPrint(); </script>

package.json

@@ -1,6 +1,6 @@
 {
-  "name": "gpt-3-encoder",
-  "version": "1.2.0-rc5",
+  "name": "@syonfox/gpt-3-encoder",
+  "version": "1.4.0-rc5",
   "description": "Javascript BPE Encoder Decoder for GPT-2 / GPT-3. The \"gpt-3-encoder\" module provides functions for encoding and decoding text using the Byte Pair Encoding (BPE) algorithm. It can be used to process text data for input into machine learning models, or to convert tokenized text back into human-readable format. It also includes functions for counting tokens in a given text and generating statistics about the tokens in a string or array.",
   "main": "index.js",
   "types": "./index.d.ts",