Server middleware changes in 0.56.x

[jdisanti]

The middleware system has been reworked as we push for a unified, simple, and consistent API. The following changes have been made in service of this goal:

• A ServiceShape trait has been added.
• The Plugin trait has been simplified.
• The HttpMarker and ModelMarker marker traits have been added to better distinguish when plugins run and what they have access to.
• The Operation structure has been removed.
• A Scoped Plugin has been added.

The Plugin trait has now been simplified and the Operation struct has been removed.

Addition of ServiceShape

Since the 0.52 release the OperationShape has existed.

/// Models the [Smithy Operation shape].
///
/// [Smithy Operation shape]: https://awslabs.github.io/smithy/1.0/spec/core/model.html#operation
pub trait OperationShape {
    /// The ID of the operation.
    const ID: ShapeId;

    /// The operation input.
    type Input;
    /// The operation output.
    type Output;
    /// The operation error. [`Infallible`](std::convert::Infallible) in the case where no error
    /// exists.
    type Error;
}

This allowed Plugin authors to access these associated types and constants. See the PrintPlugin as an example.

We continue with this approach and introduce the following trait:

/// Models the [Smithy Service shape].
///
/// [Smithy Service shape]: https://smithy.io/2.0/spec/service-types.html
pub trait ServiceShape {
    /// The [`ShapeId`] of the service.
    const ID: ShapeId;

    /// The version of the service.
    const VERSION: Option<&'static str>;

    /// The [Protocol] applied to this service.
    ///
    /// [Protocol]: https://smithy.io/2.0/spec/protocol-traits.html
    type Protocol;

    /// An enumeration of all operations contained in this service.
    type Operations;
}

With the changes to Plugin, described below, middleware authors now have access to this information at compile time.

Simplication of the Plugin trait

Previously,

trait Plugin<P, Op, S, L> {
    type Service;
    type Layer;

    fn map(&self, input: Operation<S, L>) -> Operation<Self::Service, Self::Layer>;
}

modified an Operation.

Now,

trait Plugin<Service, Operation, T> {
    type Output;

    fn apply(&self, input: T) -> Self::Output;
}

maps a tower::Service to a tower::Service. This is equivalent to tower::Layer with two extra type parameters: Service and Operation, which implement ServiceShape and OperationShape respectively.

Having both Service and Operation as type parameters also provides an even surface for advanced users to extend the codegenerator in a structured way. See this issue for more context.

The following middleware setup

pub struct PrintService<S> {
    inner: S,
    name: &'static str,
}

impl<R, S> Service<R> for PrintService<S>
where
    S: Service<R>,
{
    async fn call(&mut self, req: R) -> Self::Future {
        println!("Hi {}", self.name);
        self.inner.call(req)
    }
}

pub struct PrintLayer {
    name: &'static str,
}

impl<S> Layer<S> for PrintLayer {
    type Service = PrintService<S>;

    fn layer(&self, service: S) -> Self::Service {
        PrintService {
            inner: service,
            name: self.name,
        }
    }
}

pub struct PrintPlugin;

impl<P, Op, S, L> Plugin<P, Op, S, L> for PrintPlugin
where
    Op: OperationShape,
{
    type Service = S;
    type Layer = Stack<L, PrintLayer>;

    fn map(&self, input: Operation<S, L>) -> Operation<Self::Service, Self::Layer> {
        input.layer(PrintLayer { name: Op::NAME })
    }
}

now becomes

pub struct PrintService<S> {
    inner: S,
    name: &'static str,
}

impl<R, S> Service<R> for PrintService<S>
where
    S: Service<R>,
{
    async fn call(&mut self, req: R) -> Self::Future {
        println!("Hi {}", self.name);
        self.inner.call(req)
    }
}

pub struct PrintPlugin;

impl<Service, Operation, T> Plugin<Service, Operation, T> for PrintPlugin
where
    Operation: OperationShape,
{
    type Output = PrintService<T>;

    fn apply(&self, inner: T) -> Self::Output {
        PrintService { inner, name: Operation::ID.name() }
    }
}

impl HttpMarker for PrintPlugin { }

Alternatively, using the new ServiceShape, implemented on Ser:

impl<Service, Operation, T> Plugin<Service, Operation, T> for PrintPlugin
where
    Service: ServiceShape,
{
    type Service = PrintService<T>;

    fn apply(&self, inner: T) -> Self::Service {
        PrintService { inner, name: Service::ID.name() }
    }
}

A single Plugin can no longer apply a tower::Layer on HTTP requests/responses and modelled structures at the same time (see middleware positions C and D). Instead one Plugin must be specified for each and passed to the service builder constructor separately:

let app = PokemonService::builder_with_plugins(/* HTTP plugins */, /* model plugins */)
    /* setters */
    .build()
    .unwrap();

To better distinguish when a plugin runs and what it has access to, Plugins now have to additionally implement the HttpMarker marker trait, the ModelMarker marker trait, or both:

• A HTTP plugin acts on the HTTP request before it is deserialized, and acts on the HTTP response after it is serialized.
• A model plugin acts on the modeled operation input after it is deserialized, and acts on the modeled operation output or the modeled operation error before it is serialized.

The motivation behind this change is to simplify the job of middleware authors, separate concerns, accomodate common cases better, and to improve composition internally.

Because Plugin is now closer to tower::Layer we have two canonical converters:

use aws_smithy_http_server::plugin::{PluginLayer, LayerPlugin};

// Convert from `Layer` to `Plugin` which applies uniformly across all operations
let layer = /* some layer */;
let plugin = PluginLayer(layer);

// Convert from `Plugin` to `Layer` for some fixed protocol and operation
let plugin = /* some plugin */;
let layer = LayerPlugin::new::<SomeProtocol, SomeOperation>(plugin);

Removal of PluginPipeline

Since plugins now come in two flavors (those marked with HttpMarker and those marked with ModelMarker) that shouldn't be mixed in a collection of plugins, the primary way of concatenating plugins, PluginPipeline has been removed in favor of the HttpPlugins and ModelPlugins types, which eagerly check that whenever a plugin is pushed, it is of the expected type.

This worked before, but you wouldn't be able to do apply this collection of plugins anywhere; if you tried to, the compilation error messages would not be very helpful:

use aws_smithy_http_server::plugin::PluginPipeline;

let pipeline = PluginPipeline::new().push(http_plugin).push(model_plugin);

Now collections of plugins must contain plugins of the same flavor:

use aws_smithy_http_server::plugin::{HttpPlugins, ModelPlugins};

let http_plugins = HttpPlugins::new()
    .push(http_plugin)
    // .push(model_plugin) // This fails to compile with a helpful error message.
    .push(&http_and_model_plugin);
let model_plugins = ModelPlugins::new()
    .push(model_plugin)
    .push(&http_and_model_plugin);

In the above example, &http_and_model_plugin implements both HttpMarker and ModelMarker, so we can add it to both collections.

Removal of Operation

The aws_smithy_http_server::operation::Operation structure has now been removed. Previously, there existed a {operation_name}_operation setter on the service builder, which accepted an Operation. This allowed users to

let operation /* : Operation<_, _> */ = GetPokemonSpecies::from_service(/* tower::Service */);

let app = PokemonService::builder_without_plugins()
    .get_pokemon_species_operation(operation)
    /* other setters */
    .build()
    .unwrap();

to set an operation with a tower::Service, and

let operation /* : Operation<_, _> */ = GetPokemonSpecies::from_service(/* tower::Service */).layer(/* layer */);
let operation /* : Operation<_, _> */ = GetPokemonSpecies::from_handler(/* closure */).layer(/* layer */);

let app = PokemonService::builder_without_plugins()
    .get_pokemon_species_operation(operation)
    /* other setters */
    .build()
    .unwrap();

to add a tower::Layer (acting on HTTP requests/responses post-routing) to a single operation.

We have seen little adoption of this API and for this reason we have opted instead to introduce a new setter, accepting a tower::Service, on the service builder:

let app = PokemonService::builder_without_plugins()
    .get_pokemon_species_service(/* tower::Service */)
    /* other setters */
    .build()
    .unwrap();

Applying a tower::Layer to a subset of operations should now be done through the Plugin API via filter_by_operation_id

use aws_smithy_http_server::plugin::{PluginLayer, filter_by_operation_name, IdentityPlugin};

let plugin = PluginLayer(/* layer */);
let scoped_plugin = filter_by_operation_name(plugin, |id| id == GetPokemonSpecies::ID);

let app = PokemonService::builder_with_plugins(scoped_plugin, IdentityPlugin)
    .get_pokemon_species(/* handler */)
    /* other setters */
    .build()
    .unwrap();

or the new Scoped Plugin introduced below.

Addition of Scoped

Currently, users can selectively apply a Plugin via the filter_by_operation_id function

use aws_smithy_http_server::plugin::filter_by_operation_id;
// Only apply `plugin` to `CheckHealth` and `GetStorage` operation
let filtered_plugin = filter_by_operation_id(plugin, |name| name == CheckHealth::ID || name == GetStorage::ID);

In addition to this, we now provide Scoped, which selectively applies a Plugin at compiletime. Users should prefer this to filter_by_operation_id when applicable.

use aws_smithy_http_server::plugin::Scoped;
use pokemon_service_server_sdk::scoped;

scope! {
    /// Includes only the `CheckHealth` and `GetStorage` operation.
    struct SomeScope {
        includes: [CheckHealth, GetStorage]
    }
}
let scoped_plugin = Scoped::new::<SomeScope>(plugin);