Avoid iloop externalizing diagnostics for invalid references

[brson]

What

Don't store Vals with broken object references in the diagnostic event log.

Why

This avoids an infinite loop / recursion. There may be better solutions.

This is a followup to stellar/rs-soroban-sdk#1068 where I was experimenting with feeding the Env objects with invalid references.

The Host has a check_val_integrity function that is called on every Val it sees in order to prevent bad Vals from entering the environment, in particular it errors when it sees objects with broken references.

This function ends up calling visit_obj_untyped. When that function fails to find an object body it returns this

            Err(self.err(
                ScErrorType::Object,
                ScErrorCode::MissingValue,
                "unknown object reference",
                &[obj.to_val()],
            ))

When diagnostics are on, the effect of this is to log an "unknown object reference" event that contains the broken object, then immediately generate an error; while generating that error it attempts to externalize the diagnostic it just logged;
that diagnostic contains a broken object, and while externalizing it will generate more errors, etc. forever.

It is not clear to me why this doesn't smash the stack, but instead iloops (maybe the XDR DepthLimiter is helping). Here is a ~3k frame backtrace captured in gdb: https://gist.github.com/brson/aa2799208123895316228ca0cb425317

Known limitations

• This loses information about the broken object.
• There may be other ways to get into similar situations with the event log.

[brson]

I may not be seeing it smash the stack because it is potentially generating exponentially increasing numbers of diagnostics at each recursion level. Just taking a long time to hit the end of the stack.

[leighmcculloch]

👏🏻 Great find. One suggestion inline, but I don't feel strongly about it if others don't think that's a good move. Regardless, thanks for finding and reporting.