Issue with Mongo Cluster

[sjerman]

Hi, here is the first issue.

I am testing against a MongoDB cluster/replicaset, with the following env variables:

      containers:
        - name: orion
          image: fiware/orion-ld:1.1.2
          ports:
            - name: http
              containerPort: 1026
              protocol: TCP
          env:
            - name: ORIONLD_EXPERIMENTAL
              value: 'TRUE'
            - name: ORIONLD_MONGO_HOST
              value: >-
                kloudspot-mongodb-0.kloudspot-mongodb-headless:27017,kloudspot-mongodb-1.kloudspot-mongodb-headless:27017,kloudspot-mongodb-2.kloudspot-mongodb-headless:27017
            - name: ORIONLD_MONGO_DB
              value: orionld
            - name: ORIONLD_MONGO_USER
              value: root
            - name: ORIONLD_MONGO_PASSWORD
              valueFrom:
                secretKeyRef:
                  name: kloudspot-secret
                  key: mongodb-root-password
            - name: ORIONLD_MONGO_AUTH_MECH
              value: SCRAM-SHA-1
            - name: ORIONLD_LOG_LEVEL
              value: WARN
            - name: ORIONLD_TROE
              value: 'FALSE'

I am getting the following error:

time=Thursday 16 Mar 09:44:35 2023.078Z | lvl=TMP | corr=N/A | trans=N/A | from=N/A | srv=N/A | subsrv=N/A | comp=Orion | op=mongocInit.cpp[277]:mongocInit | msg=Connecting to mongo using the C driver (mongoc)
time=Thursday 16 Mar 09:44:35 2023.100Z | lvl=TMP | corr=N/A | trans=N/A | from=N/A | srv=N/A | subsrv=N/A | comp=Orion | op=mongoConnectionPool.cpp[320]:mongoConnectionPoolInit | msg=Connecting to mongo for the C++ legacy driver
terminate called after throwing an instance of 'mongo::UserException'
  what():  More than one ':' detected. If this is an ipv6 address, it needs to be surrounded by '[' and ']'; kloudspot-mongodb-0.kloudspot-mongodb-headless:27017,kloudspot-mongodb-1.kloudspot-mongodb-headless:27017,kloudspot-mongodb-2.kloudspot-mongodb-headless:27017

Note that this works if I add ORIONLD_MONGOCONLY

[sjerman]

OK, so I've been reading the code. I think this is the answer....

The 'old' mongoc driver doesn't work with newer Mongo so I have to use 'mongocOnly' flag with this cluster. Not sure if I also need ORIONLD_EXPERIMENTAL

When specifying the DB connection

dbUri/MONGO_URI can be used to specify the whole URI (with optional PWD substitution)

or you can specify the segments separately.

[kzangeli]

First a clarification:
the old driver is what is now called "the Legacy C++ mongo driver".
"mongoc", that's the new mongo driver written in "C".
(I imagine the new driver in C++ is called mongocxx, but, I haven't checked)

If you are using v6 of mongodb, then it might be that the legacy driver is no longer supported. It was deprecated about 5-6 years ago, so, that would not be a surprise at all.

[sjerman]

Thanks for the update. I am still working my way through..

My ultimate goal is to get the ultralight iot agent running...

So I've started with the IoT Agent tutorial...

So far :)

Following don't work

POST 'http://'"${CONTEXT_BROKER}"'/ngsi-ld/v1/entityOperations/upsert'

GET  /ngsi-ld/v1/entities?type=Device

PATCH","url":"http://orion:1026/ngsi-ld/v1/entities/urn:ngsi-ld:Device:filling001/attrs/add","headers":{"Content-Type":"application/json","NGSILD-Tenant":"openiot","NGSILD-Path":"/","fiware-service":"openiot","fiware-servicepath":"/","Link":"<http://context/ngsi-context.jsonld>; rel=\"http://www.w3.org/ns/json-ld#context\"; type=\"application/ld+json\""},"body":{"type":"Property","value":" "},"json":true}
2023-03-16T13:32:48.693Z tutorial:command-listener {
  type: 'https://uri.etsi.org/ngsi-ld/errors/OperationNotSupported',
  title: 'Not Implemented',
  detail: 'this request does not support the new mongoc driver'
}

I was able to replace the first with separate entity posts. The second may be related to a 'bad' model - doesn't look like the tutorial includes all the required data objects.

[kzangeli]

Ah, I was thinking ... but, I implemented that!!!
You are probably using something older that what I'm thinking about.
All requests are implemented of r mongoc, yiou just need to get a newer version.
New release planned for end-of-month, but, until then, you can use the newest one you find in dockerhub

[sjerman]

OK... I'll try with fiware/orion-ld:1.2.0-PRE-1296.

I'm going to try and get the tutorial sample to work 'properly' if I can.

[sjerman]

Looking better ... another issue with 1.2.0-PRE-1296:

time=2023-03-16T16:34:40.222Z | lvl=DEBUG | corr=6ce66b71-248c-49cc-a966-d8505c2aa253 | trans=6ce66b71-248c-49cc-a966-d8505c2aa253 | op=IoTAgentNGSI.Request | from=n/a | srv=openiot | subsrv=/ | msg=Options: {
    "url": "http://orion:1026/ngsi-ld/v1/csourceRegistrations/",
    "method": "POST",
    "json": {
        "type": "ContextSourceRegistration",
        "information": [
            {
                "entities": [
                    {
                        "type": "Device",
                        "id": "urn:ngsi-ld:Device:water001"
                    }
                ],
                "properties": [
                    "on",
                    "off"
                ]
            }
        ],
        "endpoint": "http://iot-agent:4041",
        "@context": "http://context/ngsi-context.jsonld"
    },
    "headers": {
        "fiware-service": "openiot",
        "fiware-servicepath": "/",
        "NGSILD-Tenant": "openiot",
        "NGSILD-Path": "/",
        "Content-Type": "application/ld+json"
    }
} | comp=IoTAgent
time=2023-03-16T16:34:40.227Z | lvl=DEBUG | corr=6ce66b71-248c-49cc-a966-d8505c2aa253 | trans=6ce66b71-248c-49cc-a966-d8505c2aa253 | op=IoTAgentNGSI.Request | from=n/a | srv=openiot | subsrv=/ | msg=Response {
    "type": "https://uri.etsi.org/ngsi-ld/errors/OperationNotSupported",
    "title": "Not Implemented",
    "detail": "this request does not support the new mongoc driver"
} | comp=IoTAgent

[kzangeli]

ok, that looks like a check that hasn't been taken away, and worse, error not captured by my functest suite ...
I will look into this right away

[kzangeli]

ok, found the problem:

1. Old check for when just a subset of the operations supported "mongoc" - I removed the check as it is no longer needed.
2. I have plenty of functests testing registrations with -experimental turned on (use mongoc where available), but no functest with -mongocOnly (use ONLY mongoc, not ever the old deprecated driver).

All explained and PR on its way (once I've compiled and run the test suite - some time before midnight)
Tomorrow morning you'll find a fresh docker image with that problem solved.
I'll be waiting for you to tell me about the next problem :)

[sjerman]

OK, heres another - which looks like a regression...

    environment:
      - ORIONLD_EXPERMENTAL=TRUE
      - ORIONLD_MONGOCONLY=TRUE

With: fiware/orion-ld:1.1.2

Steves-MacBook-Pro:fiware-test sjerman$  curl 'http://localhost:1026/ngsi-ld/v1/entities/urn:ngsi-ld:Building:barn002?options=keyValues'   -H 'Link: <http://context/ngsi-context.jsonld>; rel="http://www.w3.org/ns/json-ld#context"; type="application/ld+json"' | jq .
{
  "id": "urn:ngsi-ld:Building:barn002",
  "type": "Building",
  "category": "barn",
  "address": {
    "streetAddress": "Straße des 17. Juni",
    "addressRegion": "Berlin",
    "addressLocality": "Tiergarten",
    "postalCode": "10557"
  },
  "location": {
    "type": "Point",
    "coordinates": [
      13.3698,
      52.5163
    ]
  },
  "name": "Big Red Barn",
  "owner": "urn:ngsi-ld:Person:person001"
}

With fiware/orion-ld:1.2.0-PRE-1296:

Steves-MacBook-Pro:fiware-test sjerman$  curl 'http://localhost:1026/ngsi-ld/v1/entities/urn:ngsi-ld:Building:barn002?options=keyValues'   -H 'Link: <http://context/ngsi-context.jsonld>; rel="http://www.w3.org/ns/json-ld#context"; type="application/ld+json"' | jq .
{
  "id": "urn:ngsi-ld:Building:barn002",
  "type": "Building",
  "category": {
    "type": "Property",
    "value": "barn"
  },
  "address": {
    "type": "Property",
    "value": {
      "streetAddress": "Straße des 17. Juni",
      "addressRegion": "Berlin",
      "addressLocality": "Tiergarten",
      "postalCode": "10557"
    },
    "verified": {
      "type": "Property",
      "value": true
    }
  },
  "location": {
    "type": "GeoProperty",
    "value": {
      "type": "Point",
      "coordinates": [
        13.3698,
        52.5163
      ]
    }
  },
  "name": {
    "type": "Property",
    "value": "Big Red Barn"
  },
  "owner": {
    "type": "Relationship",
    "object": "urn:ngsi-ld:Person:person001"
  }
}

Note that keyValue doesn't seem to have an effect.

[kzangeli]

ok, that's weird.
I'll have a look.
FYI, the new name for "keyValues" in NGSI-LD is "simplified", however, Orion-LD supports both of them.
Perhaps I messed somewhere there ...

Let's not continue adding stuff to this issue though, but open a new issue for ... new issues :)

Thanks for reporting!

[kzangeli]

I moved the new issue to #1342

[sjerman]

OK - will do :)