Skip to content

Commit

Permalink
JSON schema importer
Browse files Browse the repository at this point in the history
Add ability to import schema defined in JSON format via file path or sirectly via JSON string.
This implementation doesn't migrate or update alreadfined schema elements. Neither it removes existing defined elements.
SchemaInitStrategy is defined in mind to be able to implement migration and more advance schema manage ev current JSON schema importer implementation isimple.
This work is made in hopes to simplify schema definition for beginers, speed-up prototypes development based on JanusGraph, and simplify testing.

Signed-off-by: Oleksandr Porunov <alexandr.porunov@gmail.com>
  • Loading branch information
porunov committed Oct 14, 2024
1 parent 9f09767 commit e2700c7
Show file tree
Hide file tree
Showing 43 changed files with 2,479 additions and 2 deletions.
23 changes: 23 additions & 0 deletions docs/configs/janusgraph-cfg.md
Original file line number Diff line number Diff line change
Expand Up @@ -384,6 +384,29 @@ Schema related configuration options
| schema.default | Configures the DefaultSchemaMaker to be used by this graph. Either one of the following shorthands can be used: <br> - `default` (a blueprints compatible schema maker with MULTI edge labels and SINGLE property keys),<br> - `tp3` (same as default, but has LIST property keys),<br> - `none` (automatic schema creation is disabled)<br> - `ignore-prop` (same as none, but simply ignore unknown properties rather than throw exceptions)<br> - or to the full package and classname of a custom/third-party implementing the interface `org.janusgraph.core.schema.DefaultSchemaMaker` | String | default | MASKABLE |
| schema.logging | Controls whether logging is enabled for schema makers. This only takes effect if you set `schema.default` to `default` or `ignore-prop`. For `default` schema maker, warning messages will be logged before schema types are created automatically. For `ignore-prop` schema maker, warning messages will be logged before unknown properties are ignored. | Boolean | false | MASKABLE |

### schema.init
Configuration options to configure schema initialization options.


| Name | Description | Datatype | Default Value | Mutability |
| ---- | ---- | ---- | ---- | ---- |
| schema.init.indices-activation | Indices activation type:<br>- `REINDEX_AND_ENABLE_UPDATED_ONLY` - Reindex process will be triggered for any updated index. After this all updated indexes will be enabled.<br>- `REINDEX_AND_ENABLE_NON_ENABLED` - Reindex process will be triggered for any index which is not enabled. After this all indexes will be enabled.<br>- `SKIP_ACTIVATION` - Skip reindex process for any updated indexes.<br>- `FORCE_ENABLE_UPDATED_ONLY` - Force enable all updated indexes without running any reindex process (previous data may not be available for such indices).<br>- `FORCE_ENABLE_NON_ENABLED` - Force enable all indexes (including previously created indexes) without running any reindex process (previous data may not be available for such indices).<br> | String | REINDEX_AND_ENABLE_NON_ENABLED | LOCAL |
| schema.init.schema-drop-before-startup | Drops the whole schema before JanusGraph schema initialization. Notice schema is dropped regardless of chosen initialization strategy (it will be dropped even if `none` schema-init-strategy is selected). | Boolean | false | LOCAL |
| schema.init.schema-init-strategy | Selects the strategy for schema initialization before JanusGraph is started. The full class path which implements `SchemaInitStrategy` interface and has no any constructor parameters must be provided. Also, the following shortcuts exist:<br>- `none` - Skip any schema initialization.<br>- `json` - Use provided JSON file for schema initialization.<br> | String | none | LOCAL |

### schema.init.json
Options for JSON schema initialization strategy.


| Name | Description | Datatype | Default Value | Mutability |
| ---- | ---- | ---- | ---- | ---- |
| schema.init.json.await-index-status-timeout | Timeout for awaiting index status operation defined in milliseconds. If the status await timeouts an exception will be thrown during schema initialization process. | Long | 180000 | LOCAL |
| schema.init.json.file | File path to JSON formated schema definition. | String | (no default value) | LOCAL |
| schema.init.json.force-close-other-instances | Force closes other JanusGraph instances before schema initialization regardless if they are active or not. This is a dangerous operation. This option exists to help people initialize schema who struggle with zombie JanusGraph instances. It's not recommended to be used unless you know what you are doing. Instead of this parameter, it's recommended to check `graph.unique-instance-id` and `graph.replace-instance-if-exists` options to not create zombie instances in the cluster. | Boolean | false | LOCAL |
| schema.init.json.skip-elements | Skip creation of VertexLabel, EdgeLabel, and PropertyKey. | Boolean | false | LOCAL |
| schema.init.json.skip-indices | Skip creation of indices. | Boolean | false | LOCAL |
| schema.init.json.string | JSON formated string of schema definition. This option takes precedence if both `file` and `string` are used. | String | (no default value) | LOCAL |

### storage
Configuration options for the storage backend. Some options are applicable only for certain backends.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@
import org.apache.tinkerpop.gremlin.structure.Graph;
import org.janusgraph.core.log.LogProcessorFramework;
import org.janusgraph.core.log.TransactionRecovery;
import org.janusgraph.core.schema.SchemaInitializationManager;
import org.janusgraph.diskstorage.Backend;
import org.janusgraph.diskstorage.BackendException;
import org.janusgraph.diskstorage.StandardStoreManager;
Expand Down Expand Up @@ -162,7 +163,7 @@ public static JanusGraph open(ReadConfiguration configuration, String backupName
final JanusGraphManager jgm = JanusGraphManagerUtility.getInstance();
if (null != graphName) {
Preconditions.checkNotNull(jgm, JANUS_GRAPH_MANAGER_EXPECTED_STATE_MSG);
return (JanusGraph) jgm.openGraph(graphName, gName -> new StandardJanusGraph(new GraphDatabaseConfigurationBuilder().build(configuration)));
return (JanusGraph) jgm.openGraph(graphName, gName -> defineSchemaAndStart(configuration));
} else {
if (jgm != null) {
log.warn("You should supply \"graph.graphname\" in your .properties file configuration if you are opening " +
Expand All @@ -173,10 +174,15 @@ public static JanusGraph open(ReadConfiguration configuration, String backupName
"\"graph.graphname\" so these graphs should be accessed dynamically by supplying a .properties file here " +
"or by using the ConfiguredGraphFactory.");
}
return new StandardJanusGraph(new GraphDatabaseConfigurationBuilder().build(configuration));
return defineSchemaAndStart(configuration);
}
}

private static JanusGraph defineSchemaAndStart(ReadConfiguration configuration) {
GraphDatabaseConfiguration graphDatabaseConfiguration = new GraphDatabaseConfigurationBuilder().build(configuration);
return SchemaInitializationManager.initializeSchemaAndStart(graphDatabaseConfiguration);
}

/**
* Return a Set of graph names stored in the {@link JanusGraphManager}
*
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
// Copyright 2024 JanusGraph Authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

package org.janusgraph.core.schema;

import org.janusgraph.graphdb.configuration.ConfigName;

public enum IndicesActivationType implements ConfigName {
SKIP_ACTIVATION("skip_activation"),
REINDEX_AND_ENABLE_UPDATED_ONLY("reindex_and_enable_updated_only"),
REINDEX_AND_ENABLE_NON_ENABLED("reindex_and_enable_non_enabled"),
FORCE_ENABLE_UPDATED_ONLY("force_enable_updated_only"),
FORCE_ENABLE_NON_ENABLED("force_enable_non_enabled");

private final String configurationOptionName;

IndicesActivationType(String configurationOptionName){
this.configurationOptionName = configurationOptionName;
}

@Override
public String getConfigName() {
return configurationOptionName;
}
}
Loading

0 comments on commit e2700c7

Please sign in to comment.