Harness doesn't create ES Index #224
Comments
|
I think it is because of this line |
|
Doesn't this line simply create an index with the name of the engineId? But why it returns null? |
@h3llj0ck3y please show your logs |
|
@h3llj0ck3y @HarrisJT I believe this may be a problem with 0.5.1 but is fixed in 0.5.2-SNAPSHOT currently in the develop branch. Can you check again to see if this is still a problem? |
|
@pferrel can the docker install be updated to 0.5.2? |
|
I believe this is still an issue on 0.5.2-SNAPSHOT. I just got this error on doing a simple users query: The engine was created without any issue, and i was able to add event items, this happens on a query. After doing some further testing, 0.5.1 and 0.5.2 both don't seem to perform any writes against ElasticSearch. Engines are created without error, and events are written, both (engines and events) show up in MongoDB just fine, but at no time is an index created in ES, nor after executing a training job anything is written to ES at all. Which begs the question, how does this even work? Reading through: it seems that at no time are ES operations are ever executed. Also making sure that: |
|
All operations involving writing to ES seem to be working for many other users. Is this still an issue? BTW in older versions of the Harness UR, only training causes a write to ES. In the latest 0.6.0-SNAPSHOT in the develop branch a $set event will also write to ES. So this is the only case where an event causes any operation to ES. |
|
@pferrel - i discovered the source of the issue i was having, and just neglected to update here, so here's what was the problem. engine creation, and event adding was fine, and when I would call the "train" endpoint, that was failing, and I simply failed to notice that step was failing, so the training failed, thus no index was ever created and the queries obviously would fail. the reason training was failing was that i didn't have sample data for a few of the indicators. i feel like this should be allowed (no data for an indicator, so that training can complete) but i do understand it. i have since worked around this factor and am now seeing everything work correctly. |
|
set "spark.es.nodes": "elasticsearch" -> http://localhost:9200 or ip address. |
|
I have same issue... someone fixed that ? events works without problem, only queries have error: |
|
OK... It seems like I must first add event with primary indicator, and then train engine... on train it create index and everything start works ;) |
|
I am having similar issue while training the model. I am doing it locally with docker-compose way. docker compose file {
"engineId": "test01",
"engineFactory": "com.actionml.engines.ur.UREngine",
"sparkConf": {
"master":"local",
"spark.serializer": "org.apache.spark.serializer.KryoSerializer",
"spark.kryo.registrator": "org.apache.mahout.sparkbindings.io.MahoutKryoRegistrator",
"spark.kryo.referenceTracking": "false",
"spark.kryoserializer.buffer": "300m",
"spark.executor.memory": "3g",
"spark.driver.memory": "3g",
"spark.es.index.auto.create": "true",
"spark.es.nodes": "elasticsearch",
"spark.es.nodes.wan.only": "true"
},
"algorithm": {
"indicators": [
{
"name": "like"
},
{
"name": "view"
}
]
}
}CURL:
Error: |
When I want to create an Engine from my engine-template, Harness doesnt create an Index in Elasticsearch. Does anyone know why?
My sample engin.json looks like
Further in the logs the output shows that
ES index name:isnullThe text was updated successfully, but these errors were encountered: