High speed Synchronous and Asynchronous access to InterSystems Cache/IRIS and YottaDB from Node.js.
Chris Munt cmunt@mgateway.com
21 May 2024, MGateway Ltd http://www.mgateway.com
- Verified to work with Node.js v8 to v22.
- Two connectivity models to the InterSystems or YottaDB database are provided: High performance via the local database API or network based.
- Release Notes can be found at the end of this document.
Contents
- Pre-requisites
- Installing mg-dbx
- Connecting to the database
- Invocation of database commands
- Invocation of database functions
- Cursor based data retrieval
- Transaction Processing
- Direct access to InterSystems classes (IRIS and Cache)
- Direct access to SQL: MGSQL and InterSystems SQL (IRIS and Cache)
- Working with binary data
- Using Node.js/V8 worker threads
- The Event Log
- License
mg-dbx is a Node.js addon written in C++. It is distributed as C++ source code and the NPM installation procedure will expect a C++ compiler to be present on the target system.
Linux systems can use the freely available GNU C++ compiler (g++) which can be installed as follows.
Ubuntu:
apt-get install g++
Red Hat and CentOS:
yum install gcc-c++
Apple OS X can use the freely available Xcode development environment.
There are two options for Windows, both of which are free:
- Microsoft Visual Studio Community: https://www.visualstudio.com/vs/community/
- MinGW: http://www.mingw.org/
If the Windows machine is not set up for systems development, building native Addon modules for this platform from C++ source can be quite arduous. There is some helpful advice available at:
Alternatively there are built Windows x64 binaries available from:
Assuming that Node.js is already installed and a C++ compiler is available to the installation process:
npm install mg-dbx
This command will create the mg-dbx addon (mg-dbx.node).
Before connecting to a local (relative to Node.js) InterSystems database via its API, check that the InterSystems 'Callin API' is enabled. From the InterSystems Database Management Portal select System Administration; then Security; then Services. Look for the row containing %Service_Callin and check that it is Enabled. Edit the row and mark this service as Enabled if necessary.
The M support routines are required for:
- Network based access to databases.
- Direct access to SQL (either via the API or via the network).
- The Merge command under YottaDB (either via the API or via the network).
If none of the above apply you do not need to install these routines - proceed to Connecting to the database.
Two M routines need to be installed (%zmgsi and %zmgsis). These can be found in the Service Integration Gateway (mgsi) GitHub source code repository (https://github.com/chrisemunt/mgsi). Note that it is not necessary to install the whole Service Integration Gateway, just the two M routines held in that repository.
Log in to the %SYS Namespace and install the zmgsi routines held in /isc/zmgsi_isc.ro.
do $system.OBJ.Load("/isc/zmgsi_isc.ro","ck")
Change to your development UCI and check the installation:
do ^%zmgsi
MGateway Ltd - Service Integration Gateway
Version: 4.5; Revision 31 (18 November 2023)
The instructions given here assume a standard 'out of the box' installation of YottaDB (version 1.38) deployed in the following location:
/usr/local/lib/yottadb/r138
The primary default location for routines:
/root/.yottadb/r1.38_x86_64/r
Copy all the routines (i.e. all files with an 'm' extension) held in the GitHub /yottadb directory to:
/root/.yottadb/r1.38_x86_64/r
Change directory to the following location and start a YottaDB command shell:
cd /usr/local/lib/yottadb/r138
./ydb
Link all the zmgsi routines and check the installation:
do ylink^%zmgsi
do ^%zmgsi
MGateway Ltd - Service Integration Gateway
Version: 4.5; Revision 31 (18 November 2023)
Note that the version of zmgsi is successfully displayed.
Finally, add the following lines to the interface file (zmgsi.ci in the example used in the db.open() method).
sqlemg: ydb_string_t * sqlemg^%zmgsis(I:ydb_string_t*, I:ydb_string_t *, I:ydb_string_t *)
sqlrow: ydb_string_t * sqlrow^%zmgsis(I:ydb_string_t*, I:ydb_string_t *, I:ydb_string_t *)
sqldel: ydb_string_t * sqldel^%zmgsis(I:ydb_string_t*, I:ydb_string_t *)
ifc_zmgsis: ydb_string_t * ifc^%zmgsis(I:ydb_string_t*, I:ydb_string_t *, I:ydb_string_t*)
A copy of this file can be downloaded from the /unix directory of the mgsi GitHub repository here
The default TCP server port for zmgsi is 7041. If you wish to use an alternative port then modify the following instructions accordingly.
- For InterSystems DB servers the concurrent TCP service should be started in the %SYS Namespace.
Start the DB Superserver using the following command:
do start^%zmgsi(0)
To use a server TCP port other than 7041, specify it in the start-up command (as opposed to using zero to indicate the default port of 7041).
- For YottaDB, as an alternative to starting the DB Superserver from the command prompt, Superserver processes can be started via the xinetd daemon. Instructions for configuring this option can be found in the mgsi repository here
Most mg-dbx methods are capable of operating either synchronously or asynchronously. For an operation to complete asynchronously, simply supply a suitable callback as the last argument in the call.
The first step is to add mg-dbx to your Node.js script
var dbx = require('mg-dbx').dbx;
And optionally (as required):
var mglobal = require('mg-dbx').mglobal;
var mcursor = require('mg-dbx').mcursor;
var mclass = require('mg-dbx').mclass;
var db = new dbx();
In the following examples, modify all paths (and any user names and passwords) to match those of your own installation.
Assuming Cache is installed under /opt/cache20181/
var open = db.open({
type: "Cache",
path:"/opt/cache20181/mgr",
username: "_SYSTEM",
password: "SYS",
namespace: "USER"
});
Assuming Cache is accessed via localhost listening on TCP port 7041
var open = db.open({
type: "Cache",
host: "localhost",
tcp_port: 7041,
username: "_SYSTEM",
password: "SYS",
namespace: "USER"
});
Assuming IRIS is installed under /opt/IRIS20181/
var open = db.open({
type: "IRIS",
path:"/opt/IRIS20181/mgr",
username: "_SYSTEM",
password: "SYS",
namespace: "USER"
});
Assuming IRIS is accessed via localhost listening on TCP port 7041
var open = db.open({
type: "IRIS",
host: "localhost",
tcp_port: 7041,
username: "_SYSTEM",
password: "SYS",
namespace: "USER"
});
Assuming an 'out of the box' YottaDB installation under /usr/local/lib/yottadb/r138.
var envvars = "";
envvars = envvars + "ydb_dir=/root/.yottadb\n"
envvars = envvars + "ydb_rel=r1.38_x86_64\n"
envvars = envvars + "ydb_gbldir=/root/.yottadb/r1.38_x86_64/g/yottadb.gld\n"
envvars = envvars + "ydb_routines=/root/.yottadb/r1.38_x86_64/o*(/root/.yottadb/r1.38_x86_64/r /root/.yottadb/r) /usr/local/lib/yottadb/r138/libyottadbutil.so\n"
envvars = envvars + "ydb_ci=/usr/local/lib/yottadb/r138/zmgsi.ci\n"
envvars = envvars + "\n"
var open = db.open({
type: "YottaDB",
path: "/usr/local/lib/yottadb/r138",
env_vars: envvars
});
Assuming YottaDB is accessed via localhost listening on TCP port 7041
var open = db.open({
type: "YottaDB",
host: "localhost",
tcp_port: 7041,
});
-
multithreaded: A boolean value to be set to 'true' or 'false' (default: multithreaded: true). Set this property to 'true' if the application uses multithreaded techniques in JavaScript (e.g. V8 worker threads).
-
timeout: The timeout (in seconds) to be applied to database operations invoked via network based connections. The default value is 10 seconds.
-
dberror_exceptions: A boolean value to be set to 'true' or 'false' (default: dberror_exceptions: false). Set this property to 'true' to instruct mg-dbx to throw Node.js exceptions if synchronous invocation of database operations result in an error condition. If this property is not set, any error condition resulting from the previous database operation can be retrieved using the db.geterrormessage() method.
var result = db.version();
Example:
console.log("\nmg-dbx Version: " + db.version());
current_namespace = db.namespace([<new_namespace>]);
Example 1 (Get the current Namespace):
var nspace = db.namespace();
- Note this will return the current Namespace for InterSystems databases and the value of the current global directory for YottaDB (i.e. $ZG).
Example 2 (Change the current Namespace):
var new_nspace = db.namespace("SAMPLES");
- If the operation is successful this method will echo back the new Namespace name. If not successful, the method will return the name of the current (unchanged) Namespace.
UTF-8 is the default character encoding for mg-dbx. The other option is the 8-bit ASCII character set (characters of the range ASCII 0 to ASCII 255). Native Unicode (as UTF-16) is supported for InterSystems DB Servers. The ASCII character set is a better option when exchanging single-byte binary data with the database.
current_charset = db.charset([<new_charset>]);
Example 1 (Get the current character set):
var charset = db.charset();
Example 2 (Change the current character set):
var new_charset = db.charset('ascii');
Example 3 (Native Unicode support for InterSystems DB Servers):
var new_charset = db.charset('utf-16');
- If the operation is successful this method will echo back the new character set name. If not successful, the method will return the name of the current (unchanged) character set.
- Currently supported character sets and encoding schemes: 'ascii', 'utf-8' and 'utf-16' for InterSystems DB Servers.
new_timeout = db.settimeout(<new_timeout>);
Specify a new timeout value (in seconds) for the connection. If the operation is successful this method will return the new value for the timeout.
Example (Set the timeout to 30 seconds):
var new_timeout = db.settimeout(30);
error_message = db.geterrormessage();
This method will return the error message (as a string) associated with the previous database operation. An empty string will be returned if the previous operation completed successfully.
db.close();
global = new mglobal(db, <global_name>[, <fixed_key>]);
Or:
global = db.mglobal(<global_name>[, <fixed_key>]);
Example (using a global named "Person"):
var person = db.mglobal("Person");
Synchronous:
var result = <global>.set(<key>, <data>);
Asynchronous:
<global>.set(<key>, <data>, callback(<error>, <result>));
Example:
person.set(1, "John Smith");
Synchronous:
var result = <global>.get(<key>);
Asynchronous:
<global>.get(<key>, callback(<error>, <result>));
Example:
var name = person.get(1);
- Note: use get_bx to receive the result as a Node.js Buffer.
Synchronous:
var result = <global>.delete(<key>);
Asynchronous:
<global>.delete(<key>, callback(<error>, <result>));
Example:
var name = person.delete(1);
Synchronous:
var result = <global>.defined(<key>);
Asynchronous:
<global>.defined(<key>, callback(<error>, <result>));
Example:
var name = person.defined(1);
Synchronous:
var result = <global>.next(<key>);
Asynchronous:
<global>.next(<key>, callback(<error>, <result>));
Example:
var key = "";
while ((key = person.next(key)) != "") {
console.log("\nPerson: " + key + ' : ' + person.get(key));
}
Synchronous:
var result = <global>.previous(<key>);
Asynchronous:
<global>.previous(<key>, callback(<error>, <result>));
Example:
var key = "";
while ((key = person.previous(key)) != "") {
console.log("\nPerson: " + key + ' : ' + person.get(key));
}
Synchronous:
var result = <global>.increment(<key>, <increment_value>);
Asynchronous:
<global>.increment(<key>, <increment_value>, callback(<error>, <result>));
Example (increment the value of the "counter" node by 1.5 and return the new value):
var result = person.increment("counter", 1.5);
Synchronous:
var result = <global>.lock(<key>, <timeout>);
Asynchronous:
<global>.lock(<key>, <timeout>, callback(<error>, <result>));
Example (lock global node '1' with a timeout of 30 seconds):
var result = person.lock(1, 30);
- Note: Specify the timeout value as '-1' for no timeout (i.e. wait until the global node becomes available to lock).
Synchronous:
var result = <global>.unlock(<key>);
Asynchronous:
<global>.unlock(<key>, callback(<error>, <result>));
Example (unlock global node '1'):
var result = person.unlock(1);
- Note: In order to use the 'Merge' facility with YottaDB the M support routines should be installed (%zmgsi and %zmgsis).
Synchronous (merge from global2 to global1):
var result = <global1>.merge([<key1>,] <global2> [, <key2>]);
Asynchronous (merge from global2 to global1):
<global1>.defined([<key1>,] <global2> [, <key2>], callback(<error>, <result>));
Example 1 (merge ^MyGlobal2 to ^MyGlobal1):
global1 = new mglobal(db, 'MyGlobal1');
global2 = new mglobal(db, 'MyGlobal2');
global1.merge(global2);
Example 2 (merge ^MyGlobal2(0) to ^MyGlobal1(1)):
global1 = new mglobal(db, 'MyGlobal1', 1);
global2 = new mglobal(db, 'MyGlobal2', 0);
global1.merge(global2);
Alternatively:
global1 = new mglobal(db, 'MyGlobal1');
global2 = new mglobal(db, 'MyGlobal2');
global1.merge(1, global2, 0);
<global>.reset(<global_name>[, <fixed_key>]);
Example:
// Process orders for customer #1
customer_orders = db.mglobal("Customer", 1, "orders")
do_work ...
// Process orders for customer #2
customer_orders.reset("Customer", 2, "orders");
do_work ...
This facility provides high-performance techniques for traversing records held in database globals.
The first task is to specify the 'query' for the global traverse.
query = new mcursor(db, {global: <global_name>, key: [<seed_key>]}[, {<options>}]);
Or:
query = db.mglobalquery({global: <global_name>, key: [<seed_key>]}[, {<options>}]);
The 'options' object can contain the following properties:
-
multilevel: A boolean value (default: multilevel: false). Set to 'true' to return all descendant nodes from the specified 'seed_key'.
-
getdata: A boolean value (default: getdata: false). Set to 'true' to return any data values associated with each global node returned.
-
format: Format for output (default: not specified). If the output consists of multiple data elements, the return value (by default) is a JavaScript object made up of a 'key' array and an associated 'data' value. Set to "url" to return such data as a single URL escaped string including all key values ('key[1->n]') and any associated 'data' value.
Example (return all keys and names from the 'Person' global):
query = db.mglobalquery({global: "Person", key: [""]}, {multilevel: false, getdata: true});
In key order:
result = query.next();
In reverse key order:
result = query.previous();
In all cases these methods will return 'null' when the end of the dataset is reached.
Example 1 (return all key values from the 'Person' global - returns a simple variable):
query = db.mglobalquery({global: "Person", key: [""]});
while ((result = query.next()) !== null) {
console.log("result: " + result);
}
Example 2 (return all key values and names from the 'Person' global - returns an object):
query = db.mglobalquery({global: "Person", key: [""]}, {multilevel: false, getdata: true});
while ((result = query.next()) !== null) {
console.log("result: " + JSON.stringify(result, null, '\t'));
}
Example 3 (return all key values and names from the 'Person' global - returns a string):
query = db.mglobalquery({global: "Person", key: [""]}, {multilevel: false, getdata: true, format: "url"});
while ((result = query.next()) !== null) {
console.log("result: " + result);
}
Example 4 (return all key values and names from the 'Person' global, including any descendant nodes):
query = db.mglobalquery({global: "Person", key: [""]}, {multilevel: true, getdata: true});
while ((result = query.next()) !== null) {
console.log("result: " + JSON.stringify(result, null, '\t'));
}
- M programmers will recognise this last example as the M $Query() command.
query = db.mglobalquery({global: <seed_global_name>}, {globaldirectory: true});
Example (return all global names held in the current directory)
query = db.mglobalquery({global: ""}, {globaldirectory: true});
while ((result = query.next()) !== null) {
console.log("result: " + result);
}
Synchronous:
result = db.function(<function>, <parameters>);
Asynchronous:
db.function(<function>, <parameters>, callback(<error>, <result>));
Example:
M routine called 'math':
add(a, b) ; Add two numbers together
quit (a+b)
JavaScript invocation:
result = db.function("add^math", 2, 3);
- Note: use function_bx to receive the result as a Node.js Buffer.
M DB Servers implement Transaction Processing by means of the methods described in this section. When implementing transactions, care should be taken with JavaScript operations that are invoked asynchronously. All the Transaction Processing methods describe here can only be invoked synchronously.
result = db.tstart(<parameters>);
- At this time, this method does not take any arguments.
- On successful completion this method will return zero, or an error code on failure.
Example:
result = db.tstart();
result = db.tlevel(<parameters>);
- At this time, this method does not take any arguments.
- Transactions can be nested and this method will return the level of nesting. If no Transaction is active this method will return zero. Otherwise a positive integer will be returned to represent the current depth of Transaction nesting.
Example:
tlevel = db.tlevel();
result = db.tcommit(<parameters>);
- At this time, this method does not take any arguments.
- On successful completion this method will return zero, or an error code on failure.
Example:
result = db.tcommit();
result = db.trollback(<parameters>);
- At this time, this method does not take any arguments.
- On successful completion this method will return zero, or an error code on failure.
Example:
result = db.trollback();
Synchronous:
result = new mclass(db, <class_name>, <classmethod_name>, <parameters>);
Or:
result = db.classmethod(<class_name>, <classmethod_name>, <parameters>);
Asynchronous:
db.classmethod(<class_name>, <classmethod_name>, <parameters>, callback(<error>, <result>));
Example (Encode a date to internal storage format):
result = db.classmethod("%Library.Date", "DisplayToLogical", "10/10/2019");
- Note: use classmethod_bx to receive the result as a Node.js Buffer.
The following simple class will be used to illustrate this facility.
Class User.Person Extends %Persistent
{
Property Number As %Integer;
Property Name As %String;
Property DateOfBirth As %Date;
Method Age(AtDate As %Integer) As %Integer
{
Quit (AtDate - ..DateOfBirth) \ 365.25
}
}
person = db.classmethod("User.Person", "%New");
Add Data:
result = person.setproperty("Number", 1);
result = person.setproperty("Name", "John Smith");
result = person.setproperty("DateOfBirth", "12/8/1995");
Save the object record:
result = person.method("%Save");
Retrieve data for object %Id of 1.
person = db.classmethod("User.Person", "%OpenId", 1);
Return properties:
var number = person.getproperty("Number");
var name = person.getproperty("Name");
var dob = person.getproperty("DateOfBirth");
Calculate person's age at a particular date:
today = db.classmethod("%Library.Date", "DisplayToLogical", "10/10/2019");
var age = person.method("Age", today);
- Note: use classmethod_bx, method_bx and getproperty_bx to receive data as a Node.js Buffer.
Once created, it is possible to reuse containers holding previously instantiated objects using the reset() method. Using this technique helps to reduce memory usage in the Node.js environment.
Example 1 Reset a container to hold a new instance:
person.reset("User.Person", "%New");
Example 2 Reset a container to hold an existing instance (object %Id of 2):
person.reset("User.Person", "%OpenId", 2);
mg-dbx provides direct access to the Open Source MGSQL engine (https://github.com/chrisemunt/mgsql) and InterSystems SQL (IRIS and Cache).
- Note: In order to use this facility the M support routines should be installed (%zmgsi and %zmgsis).
The first task is to specify the SQL query.
query = new mcursor(db, {sql: <sql_statement>[, type: <sql_engine>]});
Or:
query = db.sql({sql: <sql_statement>[, type: <sql_engine>]});
Example 1 (using MGSQL):
query = db.sql({sql: "select * from person"});
Example 2 (using InterSystems SQL):
query = db.sql({sql: "select * from SQLUser.person", type: "Cache"});
Synchronous:
var result = <query>.execute();
Asynchronous:
<query>.execute(callback(<error>, <result>));
The result of query execution is an object containing the return code and state and any associated error message. The familiar ODBC return and status codes are used.
Example 1 (successful execution):
{
"sqlcode": 0,
"sqlstate": "00000",
"columns": [
{
"name": "Number",
"type": "INTEGER"
},
"name": "Name",
"type": "VARCHAR"
},
"name": "DateOfBirth",
"type": "DATE"
}
]
}
Example 2 (unsuccessful execution):
{
"sqlcode": -1,
"sqlstate": "HY000",
"error": "no such table 'person'"
}
In result-set order:
result = query.next();
In reverse result-set order:
result = query.previous();
In all cases these methods will return 'null' when the end of the dataset is reached.
Example:
while ((row = query.next()) !== null) {
console.log("row: " + JSON.stringify(result, null, '\t'));
}
The output for each iteration is a row of the generated SQL result-set. For example:
{
"number": 1,
"name": "John Smith",
}
For 'select' queries that generate a result-set it is good practice to invoke the 'cleanup' method at the end to delete the result-set held in the database.
Synchronous:
var result = <query>.cleanup();
Asynchronous:
<query>.cleanup(callback(<error>, <result>));
Synchronous:
<query>.reset({sql: <sql_statement>[, type: <sql_engine>]);
Asynchronous:
<query>.reset({sql: <sql_statement>[, type: <sql_engine>], callback(<error>, <result>));
In mg-dbx the default character encoding scheme is UTF-8. When transmitting binary data between the database and Node.js there are two options.
- Switch to using the 8-bit ASCII character set.
- Receive the incoming data into Node.js Buffers.
On the input (to the database) side all mg-dbx function arguments can be presented as Node.js Buffers and mg-dbx will automatically detect that an argument is a Buffer and process it accordingly.
On the output side the following functions can be used to return the output as a Node.js Buffer.
-
dbx::function_bx
-
dbx::classmethod_bx
-
mglobal::get_bx
-
mclass::classmethod_bx
-
mclass::method_bx
-
mclass::getproperty_bx
These functions work the same way as their non '_bx' suffixed counterparts. The only difference is that they will return data as a Node.js Buffer as opposed to a type of String.
The following two examples illustrate the two schemes for receiving binary data from the database.
Example 1: Receive binary data from a DB function as a Node.js 8-bit character stream
<db>.charset('ascii');
var stream_str8 = <db>.function(<function>, <parameters>);
<db>.charset('utf-8'); // reset character encoding
Example 2: Receive binary data from a DB function as a Node.js Buffer
var stream_buffer = <db>.function_bx(<function>, <parameters>);
mg-dbx functionality can now be used with Node.js/V8 worker threads. This enhancement is available with Node.js v12 (and later).
- Note: be sure to include the property multithreaded: true in the open method when opening database connections to be used in multi-threaded applications.
Use the following constructs for instantiating mg-dbx objects in multi-threaded applications:
// Use:
var <global> = new mglobal(<db>, <global>);
// Instead of:
var <global> = <db>.mglobal(<global>);
// Use:
var <cursor> = new mcursor(<db>, <global_query>);
// Instead of:
var <cursor> = <db>.mglobalquery(<global_query>)
// Use:
var <class> = new mclass(<db>, <classmethod>);
// Instead of:
var <class> = <db>.classmethod(<classmethod>);
// Use:
var <sql> = new mcursor(<db>, <sqlquery>);
// Instead of:
var <sql> = <db>.sql(<sqlquery>);
The following scheme illustrates how mg-dbx should be used in threaded Node.js applications.
const { Worker, isMainThread, parentPort, threadId } = require('worker_threads');
if (isMainThread) {
// start the threads
const worker1 = new Worker(__filename);
const worker2 = new Worker(__filename);
// process messages received from threads
worker1.on('message', (message) => {
console.log(message);
});
worker2.on('message', (message) => {
console.log(message);
});
} else {
var dbx = require('mg-dbx').dbx;
// And as required ...
var mglobal = require('mg-dbx').mglobal;
var mcursor = require('mg-dbx').mcursor;
var mclass = require('mg-dbx').mclass;
var db = new dbx();
db.open(<parameters>);
var global = new mglobal(db, <global>);
// do some work
var result = db.close();
// tell the parent that we're done
parentPort.postMessage("threadId=" + threadId + " Done");
}
mg-dbx provides an Event Log facility for recording errors in a physical file and, as an aid to debugging, recording the mg-dbx functions called by the application. This Log facility can also be used by Node.js applications.
To use this facility, the Event Log file must be specified using the following function:
db.setloglevel(<log_file>, <Log_level>, <log_filter>);
Where:
- log_file: The name (and path to) the log file you wish to use. The default is c:/temp/mg-dbx.log (or /tmp/mg-dbx.log under UNIX).
- log_level: A set of characters to include one or more of the following:
- e - Log error conditions.
- f - Log all mg-dbx function calls (function name and arguments).
- t - Log the request data buffers to be transmitted from mg-dbx to the DB Server.
- r - Log the request data buffers to be transmitted from mg-dbx-bdb to the DB Server and the corresponding response data.
- log_filter: A comma-separated list of functions that you wish the log directive to be active for. This should be left empty to activate the log for all functions.
Examples:
db.setloglevel("c:/temp/mg-dbx.log", "e", "");
db.setloglevel("/tmp/mg-dbx.log", "ft", "dbx::set,mglobal::set,mcursor::execute");
Node.js applications can write their own messages to the Event Log using the following function:
db.logmessage(<message>, <title>);
Logging can be switched off by calling the setloglevel function without specifying a log level. For example:
db.setloglevel("c:/temp/mg-dbx.log");
Copyright (c) 2018-2024 MGateway Ltd,
Surrey UK.
All rights reserved.
http://www.mgateway.com
Email: cmunt@mgateway.com
Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.
- Initial Release
- Allow a global to be registered with a fixed leading key (i.e. leading fixed subscripts).
- Introduce a method to reset a global name (and any associated fixed keys).
- Introduce global 'increment()' and 'lock(); methods.
- Introduce cursor based data retrieval.
- Introduce outline support for multithreading in JavaScript - currently not stable!.
- Introduce support for direct access to InterSystems IRIS/Cache classes.
- Extend cursor based data retrieval to include an option for generating a global directory listing.
- Introduce a method to report and (optionally change) the current working global directory (or Namespace).
- Correct a fault that led to the timeout occasionally not being honoured in the lock() method.
- Correct a fault that led to Node.js exceptions not being processed correctly.
- Introduce support for direct access to InterSystems SQL and MGSQL.
- Correct a fault in the InterSystems Cache/IRIS API to globals that resulted in failures - notably in cases where there was a mix of string and numeric data in the global records.
- Correct a fault in the Global Increment method.
- Correct a fault that resulted in query.next() and query.previous() loops not terminating properly (with null) under YottaDB. This fault affected YottaDB releases after 1.30
- Modify the version() method so that it returns the version of YottaDB rather than the version of the underlying GT.M engine.
- Verify that mg-dbx will build and work with Node.js v13.x.x.
- Suppress a number of benign 'cast-function-type' compiler warnings when building on the Raspberry Pi.
- Verify that mg-dbx will build and work with Node.js v14.x.x.
- Introduce support for Node.js/V8 worker threads (for Node.js v12.x.x. and later).
- See the section on 'Using Node.js/V8 worker threads'.
- Introduce support for the M Merge command.
- Correct a fault in the mcursor 'Reset' method.
- Introduce a scheme for transmitting binary data between Node.js and the database.
- Correct a fault that led to some calls failing with incorrect data types after calls to the mglobal::increment method.
- mg-dbx will now pass arguments to YottaDB functions as ydb_string_t types and not ydb_char_t. Modify your YottaDB function interface file accordingly. See the section on 'Installing the M support routines'.
- Introduce the option to connect to a database over the network.
- Remove the 32K limit on the volume of data that can be sent to the database via the mg-dbx methods.
- Correct a fault that led to the failure of asynchronous calls to the dbx::function and mglobal::previous methods.
- Correct a fault in the processing of InterSystems Object References (orefs).
- This fault only affected applications using API-based connectivity to the database (as opposed to network-based connectivity).
- The fault could result in Node.js throwing 'Heap Corruption' errors after creating an instance of an InterSystems Object.
-
Extend the processing of InterSystems Object References (orefs) to cater for instances of an object embedded as a property in other objects. For example, consider two classes: Patient and Doctor where an instance of a Doctor may be embedded in a Patient record (On the Server: "Property MyDoctor As Doctor"). And on the Node.js side...
var patient = db.classmethod("User.Patient", "%OpenId", patient_id); var doctor = patient.getproperty("MyDoctor"); var doctor_name = doctor.getproperty("Name"); -
Correct a fault in the processing of output values returned from YottaDB functions that led to output string values not being terminated correctly. The result being unexpected characters appended to function outputs.
- Correct a fault that could lead to fatal error conditions when creating new JS objects in multithreaded Node.js applications (i.e. when using Node.js/V8 worker threads).
- Correct a fault that could lead to mg-dbx incorrectly reporting 'Database not open' errors when connecting to YottaDB via its API in multithreaded Node.js applications.
- Introduce a log facility to record error conditions and run-time information to assist with debugging.
- Change the default for the multihtreaded property to be true. This can be set to false (in the open() method) if you are sure that your application does not use Node.js/V8 threading and does not call mg-dbx functionality asynchronously. If in doubt, it is safer to leave this property set to true.
- A number of faults related to the use of mg-dbx functionality in Node.js/v8 worker threads have been corrected. In particular, it was noticed that callback functions were not being fired correctly for some asynchronous invocations of mg-dbx methods.
- Correct a fault that could lead to unpredictable behaviour and failures if more than one V8 worker thread concurrently requested a global directory listing.
- For example: query = new cursor(db, {global: ""}, {globaldirectory: true});
- For SQL SELECT queries, return the column names and their associated data types.
- This metadata is presented as a columns array within the object returned from the SQL Execute method.
- The columns array is created in SELECT order.
- Attempt to capture Windows OS exceptions in the event log.
- The default event log is c:\temp\mg-dbx.log under Windows and /tmp/mg-dbx.log under UNIX.
- Update the internal UNIX library names for InterSystems IRIS and Cache.
- For information, the Cache library was renamed from libcache to libisccache and the IRIS library from libirisdb to libiscirisdb
- This change does not affect Windows platforms.
- Verify that mg-dbx will build and work with Node.js v15.x.x.
- Note that files that were previously held under the /m, /yottadb, and /unix directories are now available from the mgsi GitHub repository. These files are common to a number of my Open Source projects.
- Correct a fault that occasionally led to failures in network-based connectivity between mg-dbx and DB Servers.
- Allow a DB Server response timeout to be set for network based connectivity.
- Specify the timeout property in the open() method.
- Use the db.settimeout() method to set or reset the timeout value.
- Introduce an option to throw Node.js exceptions if synchronous calls to database operations result in an error condition (for example an M "SUBSCRIPT" or "SYNTAX" error).
- Specify the dberror_exceptions property in the open() method (default is false).
- Introduce a method to return any error message associated with the previous database operation.
- var errormessage = db.geterrormessage()
- Extend the logging of request transmission data to include the corresponding response data.
- Include 'r' in the log level. For example: db.setloglevel("MyLog.log", "eftr", "");
- Correct a fault that occasionally led to failures when sending long strings (greater than 32K) to the DB Server.
- For example global.set('key1', 'key2', [string 2MB in length]);
- Correct a fault that occasionally led to failures when returning long strings to Node.js from the DB Server.
- This fault only affected network based connectivity to the DB Server.
- Introduce support for M Transaction Processing: tstart, $tlevel, tcommit, trollback.
- Correct a fault that resulted in a crash when loading the mg-dbx module in Node.js v10.
- This change only affects mg-dbx for Node.js v10.
- Introduce support for YottaDB Transaction Processing over API based connectivity.
- This functionality was previously only available over network-based connectivity to YottaDB.
- Verify that mg-dbx will build and work with Node.js v16.x.x.
- Introduce native Unicode support for InterSystems DB Servers - as character set/encoding UTF-16.
- db.charset('utf-16')
- For network-based connectivity, DB Superserver version 4.4.25 (or later) should be used.
- This update should be regarded as a beta-release. It is recommended that only those who need native support for Unicode on InterSystems DB Servers should upgrade to this release at this time.
- Verify that mg-dbx will build and work with Node.js v17.x.x.
- Verify that mg-dbx will build and work with Node.js v18.x.x.
- Verify that mg-dbx will build and work with Node.js v19.x.x.
- Verify that mg-dbx will build and work with Node.js v20.x.x.
- Documentation update.
- Correct a fault affecting the return of Unicode data to Node.js through SQL.
- Verify that mg-dbx will build and work with Node.js v22.x.x.
- Correct a fault in the InterSystems get and change namespace operations under network connections (db.namespace()).