Remove unused schema definitions (e.g. --remove-unreferenced-schema )

[kirides]

To reduce the amount of generated Contracts, we could walk the paths and remove and non-referenced schema

As we already are able to filter paths, we can make use of that to clean up the spec and generate less code.

Example code for cleaning up the schema definitions

// Uses Microsoft.OpenApi.Readers & SwashBuckle.AspNetCore for Reading and Writing specs

void Main()
{
    var rdr = new Microsoft.OpenApi.Readers.OpenApiStreamReader();
    using var specFile = File.OpenRead(@"C:\spec.json");

    // doc is a "filtered" OpenApi-spec
    var doc = rdr.Read(specFile, out var diagnostic);

    var schemaCleaner = new SchemaCleaner();
    schemaCleaner.RemoveUnreferencedSchema(doc);

    using var tw = File.CreateText(@"C:\clean_spec.yaml");
    var writer = new Microsoft.OpenApi.Writers.OpenApiYamlWriter(tw);
    doc.SerializeAsV3(writer);
}


public class SchemaCleaner
{
    public void RemoveUnreferencedSchema(OpenApiDocument doc)
    {
        var usage = FindUsedSchema(doc);

        var unused = doc.Components.Schemas.Where(s => !usage.Contains(s.Key))
            .ToArray();
        foreach (var unusedSchema in unused)
        {
            doc.Components.Schemas.Remove(unusedSchema);
        }
    }

    HashSet<string> FindUsedSchema(OpenApiDocument doc)
    {
        var toProcess = new Stack<OpenApiSchema>();

        foreach (var (_, pathItem) in doc.Paths)
        {
            foreach (var p in pathItem.Parameters)
            {
                TryPush(p.Schema, toProcess);
            }

            foreach (var (_, op) in pathItem.Operations)
            {
                foreach (var (_, resp) in op.Responses)
                {
                    foreach (var (_, header) in resp.Headers)
                    {
                        TryPush(header.Schema, toProcess);
                    }
                    foreach (var (_, content) in resp.Content)
                    {
                        TryPush(content.Schema, toProcess);
                    }
                }
            }
        }

        var seen = new HashSet<string>();
        while (toProcess.TryPop(out var schema))
        {
            if (schema.Reference?.Id is { } refId)
            {
                if (!seen.Add(refId))
                {
                    // prevent recursion
                    continue;
                }
            }
            foreach (var subSchema in EnumerateSchema(schema))
            {
                TryPush(subSchema, toProcess);
            }
        }

        return seen;
    }

    private void TryPush(OpenApiSchema? schema, Stack<OpenApiSchema> stack)
    {
        if (schema == null)
        {
            return;
        }
        stack.Push(schema);
    }

    IEnumerable<OpenApiSchema> EnumerateSchema(OpenApiSchema? schema)
    {
        if (schema is null)
        {
            return Enumerable.Empty<OpenApiSchema>();
        }

        return EnumerateInternal(schema)
            .Where(x => x != null)
            .Select(x => x!);

        static IEnumerable<OpenApiSchema?> EnumerateInternal(OpenApiSchema schema)
        {
            yield return schema.AdditionalProperties;
            yield return schema.Items;
            yield return schema.Not;

            foreach (var subSchema in schema.AllOf)
            {
                yield return subSchema;
            }
            foreach (var subSchema in schema.AnyOf)
            {
                yield return subSchema;
            }
            foreach (var subSchema in schema.OneOf)
            {
                yield return subSchema;
            }
            foreach (var (_, subSchema) in schema.Properties)
            {
                yield return subSchema;
            }
        }

    }
}

[christianhelle]

@kirides Thanks for taking the time to investigate and suggest this

This could be really interesting as an option but definitely not as the default behavior. I have on multiple occasions used an OpenAPI spec for the sole purpose of generating contracts

Looks like you already have the code, if you create a pull request and all the tests pass then I'll approve so can get this in 😄

[kirides]

Will be looking at it.

This is mainly useful for people like me/company where we filter endpoints and only want the minimum required OpenApi spec parts in an assembly/module.

Reason being that breaking changes in contracts become easier to manage, due to less clutter after re-generating the endpoints and contracts.

[jods4]

Is excludedTypeNames a possible alternative?
I have crappy OpenAPI with unreferenced schemas. I tried to get rid of the types with ExcludedTypeNames but it doesn't seem to have any effect (I tried the schema name, namespace-prefixed class name... nothing worked).

@christianhelle
As we have includePathMatches, an equivalent includeSchemaMatches might be an option to include unreferenced schemas?

[kirides]

As we have includePathMatches, an equivalent includeSchemaMatches might be an option to include unreferenced schemas?

i like that suggestion, it's easy to implement and reason about

[christianhelle]

@all-contributors please add @jods4 for ideas

[allcontributors]

@christianhelle

I've put up a pull request to add @jods4! 🎉

[jods4]

I was gonna say don't bother, but you're too quick!
Thanks for the feature!