OSC Support

Table of Contents OSC Usage Example Overview of ATF OSC Coverage IOscService Interface IOscService Elements OscMessageReceivedArgs Class OscServices Class Extension Methods OscService Component Setting C Sharp Properties from Messages Received Getting C Sharp Properties and Sending Messages Mapping an OSC Address onto a C Sharp Class Property OscAddressInfo Class IOscService Interface Implementation OscService Properties OscService Methods LemurOscService Component AddPropertyAddress Method Example OscCommandReceiver Component OscCommands Component OscDialog Class Using Lemur to Control an ATF Application

Open Sound Control, OSC, is an open, transport-independent, message-based protocol developed for communication among computers, sound synthesizers, and other multimedia devices, according to its specification. OSC allows for name/value pairs to be sent back and forth between different applications and/or devices. In ATF, the underlying network protocol is UDP.

For iOS devices, we recommend using the Lemur application from Liine. For Android devices, we recommend TouchOSC from Hexler.net. In either case, game designers can construct their own user-interfaces (and even macros in Lemur), which then allows the tablet computer to run commands or edit objects in an ATF application that incorporates ATF's OSC support.

The unit of transmission in OSC is the OSC Packet. Any application that sends OSC Packets is an OSC client; any application that receives OSC Packets is an OSC server. Some applications can be both, sending and receiving OSC packets. An ATF application can serve as either an OSC client or server, but generally operates as a server. That is, you can control the ATF application by sending it OSC messages, which include the capability of executing ATF commands. For details, see OscCommandReceiver Component.

A message to an OSC server contains an address, a description of its arguments, and the arguments. Addresses look like URLs. The address for an OSC server is a string of slash "/" separated strings, such as "/resonators/3/frequency". The address's last part specifies a Method, which causes the OSC server to do something. You can think of OSC as a mechanism for making Remote Procedure Calls to applications and devices.

Although the OSC standard addresses shortcomings of MIDI and can replace it, there is nothing in OSC limiting it to musical devices. In the future, the name Open Sound Control may change to Open Systems Control to reflect the fact that the standard is general purpose. For this reason, ATF class names simply use the prefix "OSC". This document also refers to OSC, rather than spelling out its full name.

Not all of the features of OSC 1.0 are supported in ATF. The '\' and '?' wildcard characters are not supported robustly in the OSC address; '\' only works at the end of an address, and '?' is not supported at all. Also, ATF does not support the extended features of OSC 1.1, such as using "//" as a wildcard for paths in the OSC Address.

For a general introduction to OSC on Wikipedia, see Open Sound Control. For the OSC 1.0 specification, see The Open Sound Control 1.0 Specification. For information on version 1.1, see Features and Future of Open Sound Control version 1.1 for NIME. For OSC in general, see http://opensoundcontrol.org/. For OSC best practices, see Best Practices for Open Sound Control.

OSC Usage Example

Guerrilla Games is using the Lemur iPad® application, developed by the Liine company, to communicate with their CoreText Editor. It has integrated its CoreText Editor with ATF's OSC support. To observe it in action, watch the video iPad® OSC Demo by Guerrilla Games at ATF Videos.

CoreText Editor can do two-way communication with Liine Lemur:

CoreText Editor's NodeTypePaletteItem from the DomNodeTypes can be used to populate the OSC addresses and then the properties can be filtered in various ways. This is all client-side.

Guerrilla Games prepared this video to show how they are using OSC:

Overview of ATF OSC Coverage

ATF's OSC support includes these items:

• IOscService: Interface for using the OSC protocol to communicate with OSC-compatible devices. This interface allows for sending and receiving name-value pairs, asynchronously. For a discussion, see IOscService Interface.
• OscService: Component that allows OSC devices to get and set properties on C# objects. This is the main component supporting OSC.
• LemurOscService: Component that provides OSC support compatible with the Lemur iPad® application. If you want to communicate with the Liine Lemur iPad® application, consider using LemurOscService (which derives from OscService) instead of OscService. For more information on using Lemur, see the Lemur site. For more details on this component, see LemurOscService Component.
• OscCommandReceiver: Component that listens for OSC messages and can trigger the execution of commands in the application.
• OscCommands: Component that provides menu commands that may be useful for users of OscService.
• OscDialog: Dialog to display information about OSC operation in the application.

IOscService Interface

IOscService is an interface for using the OSC protocol to communicate with OSC-compatible devices. This interface allows for sending and receiving name-value pairs, asynchronously. This interface allows an ATF application to send messages to and receive messages from an OSC application/device.

IOscService Elements

The interface contains a method and an event:

• void Send(IEnumerable<Tuple<string, object>> addressesAndData): Send the OSC addresses and data object pairs to each destination endpoint, asynchronously. Each pair is an OSC address string and a non-null data payload that is sent to each destination device. Valid types of data are 32-bit floats, arrays of 32-bit floats, 32-bit integers, strings, and arrays of bytes. This method returns immediately. If there is a network slowdown of some kind, it is possible that new values for a particular OSC address replace the old, unsent values. The order of the pairs can also be changed before they are sent. The pairs may not be sent all in one OSC bundle, if the size is too large.
• event EventHandler<OscMessageReceivedArgs> MessageReceived: Notify listeners that an OSC message has been received. The OSC address and data payload are provided in the OscMessageReceivedArgs, and listeners can set the OscMessageReceivedArgs.Handled property to true if no further processing should be done on the message.

OscMessageReceivedArgs Class

This event argument class derives from HandledEventArgs and provides the OSC address and data payload from an OSC message. Setting its Handled property (from HandledEventArgs) to true prevents other listeners from receiving this object. The IOscService.MessageReceived event uses this class to provide its event information.

OscMessageReceivedArgs has two fields:

• Address: OSC address that the message is directed to.
• Data: OSC data payload for the address.

OscServices Class Extension Methods

OscServices contains useful extension methods and utilities for working with IOscService objects.

• void Send(this IOscService service, string oscAddress, object data): Send a single OSC message to each destination endpoint with the given data, asynchronously.
• string FixPropertyAddress(string oscAddress): Fix an OSC address to make it compliant. It adds a leading '/' if it's missing and removes illegal characters: spaces, braces, and brackets.

OscService Component

OscService is the main item used in ATF OSC support.

OscService processes OSC messages by getting and setting properties on C# objects. OscService allows you to associate an OSC address with a particular property of a class. Objects with properties that have an OSC address associated with them are known as addressable. For details, see Mapping an OSC Address onto a C Sharp Class Property.

When the IInitializable.Initialize() method runs for this component, it starts a UDP server. UDP is the main protocol used with OSC.

If you want to communicate with the Liine Lemur iPad® application, you should probably use LemurOscService instead of OscService; for details, see LemurOscService Component. Consider also using the OscCommands component to process commands; for more information, see OscCommands Component. The OscCommandReceiver component can trigger the execution of commands, so it may also be useful in your application; for details, see OscCommandReceiver Component.

OscService uses the Bespoke Open Sound Control Library for a UDP server and to send and receive OSC messages. For information, see its description at Open Sound Control. Also see Bespoke Open Sound Control Library.

Setting C Sharp Properties from Messages Received

OscService receives OSC messages in a separate thread from the main application thread. OscService can set a C# property when it receives and processes an OSC message. The message's OSC address specifies the class and property. If the OSC address has no property associated with it, the message is ignored. The message's arguments specify the new value of the property. OscService attempts to convert the arguments to the right type of object for the property. If no arguments are supplied or the wrong type of arguments, the property is not set. You can specify a converter for these values; for a description, see Mapping an OSC Address onto a C Sharp Class Property.

When OscService receives a message from a device, the device is added to the listeners list, which is kept in the DestinationEndpoints property. A listener can also be specified in the OSC Dialog created by OscDialog. For information on what this dialog can do, see OscDialog Class.

Property value changes resulting from messages are done in a transaction context, so the changes can be undone or redone.

For example, an OSC device, such as an iPad® running Lemur, could have a slider control that sends an OSC message whenever the slider position changes. The address of this OSC message could be associated with the property FaderVolume1 in the SoundMixer class that controls sound use in a game application. Moving the slider on the iPad® changes the FaderVolume1 volume level in SoundMixer.

By using the OscCommandReceiver component, you can also send an OSC message to an ATF OSC-enabled application that executes an ATF command, rather than setting a C# property. For more information, see OscCommandReceiver Component.

Getting C Sharp Properties and Sending Messages

OscService sends messages in the main application thread. OscService sends property value data OSC messages to all listening OSC applications/devices in response to several context change events. OscService maintains contexts in these properties:

• SelectionContext for an ISelectionContext.
• ObservableContext for an IObservableContext.

These properties are typically set when the ActiveContextChanged event in the IContextRegistry occurs. OscService imports an IContextRegistry implementer, such as the ContextRegistry component.

OscService sends the value of a property in an OSC message when:

• The selection changes, that is, when the SelectionChanged event occurs on the ISelectionContext. Properties for the last selected object(s) are sent to all listeners.
• Contents change. This occurs when the ItemChanged, ItemInserted or ItemRemoved events occur in the IObservableContext. Properties for the changed object are sent to listeners if it was the last selected object.

Property values are sent only if the property has an OSC address associated with it. The OSC message sent has the OSC address of the property, followed by an argument description of the value and the value itself, as a set of one or more arguments.

This means that all OSC receivers broadcast to should be prepared to receive such messages. If these receivers include ATF applications implementing OscService, the application should map OSC addresses to the desired properties, if any, for which they would be receiving messages. Applications can map OSC addresses onto different properties and classes than the application/device sending the OSC messages. Keep in mind that receiving and processing OSC messages with OscService results in their associated properties changing. For more details on mapping OSC addresses, see Mapping an OSC Address onto a C Sharp Class Property.

If these selected objects can be adapted as DomNodes, sending these OSC messages has much better performance than otherwise.

Mapping an OSC Address onto a C Sharp Class Property

For OscService to get or set property values, a correspondence between an OSC address and a property in a class must be specified. This is an essential step for any OSC server.

Map addresses by calling one of OscService's AddPropertyAddress() methods, which take several forms:

• string AddPropertyAddress(PropertyInfo propertyInfo, string oscAddress)
• string AddPropertyAddress(Type classType, string propertyName, string oscAddress)
• string AddPropertyAddress(string typeName, string propertyName, string oscAddress)
• string AddPropertyAddress(PropertyDescriptor descriptor, string oscAddress)

All these methods take a string for the OSC address. The class and property can be specified in a variety of ways, depending on what's convenient. For instance, the application may have PropertyDescriptors handy, because they were used for another purpose.

The OSC address is "fixed" to remove any illegal characters by calling OscServices.FixPropertyAddress(). The name is also made unique by using a UniqueNamer object.

The OSC address and class property association is stored in an OscAddressInfo object.

If you use a PropertyDescriptor to specify the property, you can also specify a type converter for the property. This allows converting the arguments in the OSC message to the right type for the C# property. For an example of doing this, see AddPropertyAddress Method Example.

OscAddressInfo Class

OscAddressInfo contains information for mapping an OSC address to a property on a C# class. Its constructor takes either a PropertyInfo or PropertyDescriptor to specify the class and property:

• OscAddressInfo(string oscAddress, PropertyInfo propertyInfo)
• OscAddressInfo(string oscAddress, PropertyDescriptor descriptor)

OscAddressInfo has several useful fields, properties, and methods:

• string Address: Field for the OSC address.
• Type CompatibleType: Field for the type of the object that has the C# property.
• Type PropertyType: Field for the C# property's type.
• List<object> Addressable: Field containing a list of selected objects that have been converted into OSC-addressable objects of the type of the object that has the C# property, which is the same type as in CompatibleType.
• string PropertyName: Get the property's name.
• PropertyDescriptor PropertyDescriptor: Get the PropertyDescriptor, if one was used to create this OscAddressInfo. It may be null.
• object CommonToAddressable(object common): Attempt to convert a given object into an object that is compatible with the object holding the property, whose Type is in CompatibleType. The OscService.Addressable field contains objects returned from this method.
• object GetValue(object compatible): Get an object with the value of the C# property on the given object.
• void SetValue(object compatible, object data): Set the value of the associated C# property on the given object.

IOscService Interface Implementation

OscService implements IOscService:

• void Send(IEnumerable<Tuple<string, object>> addressesAndData): Send the OSC addresses and data objects to each destination endpoint in DestinationEndpoints, asynchronously. Each Tuple is a pairing of the OSC address and an object for the property value associated with that address. If there are multiple Tuples with the same OSC address, only the last one is sent. This avoids sending values unnecessarily for an object whose property is changing very quickly, as for a property whose value is set by a slider control, for example. This method returns immediately, queuing the data to be sent.
• event EventHandler<OscMessageReceivedArgs> MessageReceived: When a message has been received, delegates subscribing to this event are called with an OscMessageReceivedArgs providing message information, as described in OscMessageReceivedArgs Class. The OscCommandReceiver component uses this mechanism; for details, see OscCommandReceiver Component. Instead of using this, OscService subscribes to the MessageReceived event on the UDP server object

OscService Properties

OscService offers the following properties:

• int ReceivingPort: Get or set the port number that is used to receive UDP or TCP/IP messages. The default is 8000.
• IPAddress ReceivingIPAddress: Get this application's receiving IP address, which is only valid after IInitializable.Initialize() is called. When setting, you can use the static method GetLocalIPAddresses() to determine which valid receiving IP addresses are available.
• IPEndPoint ReceivingEndpoint: Get or set the local receiving endpoint, which is a combination of the ReceivingIPAddress and ReceivingPort.
• IPAddress PreferredReceivingIPAddress: Get or set the preferred receiving IP address. In the case where there are multiple valid local IP addresses to choose from, this is the one that is used. When setting, the value is ignored if it's not a currently valid local IP address.
• IPEndPoint DestinationEndpoint: Get or set the primary destination endpoint (IP address and port number). Setting this adds the value to the list in the DestinationEndpoints property. Additional connected devices can be communicated with if they initiate contact. This property cannot be null. If the IP address portion of the endpoint is IPAddress.None ("255.255.255.255"), no messages are sent, and a large amount of processing is possibly saved. This property can be set in the dialog created by the OscDialog class; for details, see OscDialog Class.
• IEnumerable<IPEndPoint> DestinationEndpoints: Get the current list of known IP endpoints (IP address and port number) to broadcast to. This always includes DestinationEndpoint, but may also include additional endpoints: when a message is received, its sender is automatically added to this list.
• IEnumerable<OscAddressInfo> AddressInfos: Get all the OscAddressInfos for OSC addresses that map onto C# properties.
• string StatusMessage: Get a user-readable status message that describes the current status of OscService. It includes the IP addresses broadcast to, the number of OSC messages received, and the number of OSC messages sent.

OscService Methods

OscService methods available include:

• IEnumerable<IPAddress> GetLocalIPAddresses(): A utility method for getting local IP addresses that could work for OSC communications. The first address in the enumeration is the best guess, but there can be multiple valid network adapters to choose from. You can use this method to get an address for the ReceivingIPAddress property.
• object ObservableToCommon(object observable): Convert a type of object reported by an event in the IObservableContext into a type of object that is compatible with an object in the selection context (after passing through SelectedToCommon()). The default is to assume that no conversion is necessary. For some applications, it may be necessary to convert the object, for example, adapting it to a DomNode:

protected override object ObservableToCommon(object observable)
{
    return observable.As<DomNode>();
}

• object SelectedToCommon(object selected): Convert a type of object from an ISelectionContext into a type of object that is compatible with an object from IObservableContext (after passing through ObservableToCommon()). The default is to assume that no conversion is necessary. For some applications, it may be necessary to convert the object, for example, adapting it to a DomNode.
• IEnumerable<Tuple<string, object>> GetCustomDataToSend(object common): Get a set of OSC addresses and associated data to send in messages, along with the other messages sent after a context change, as discussed in Getting C Sharp Properties and Sending Messages. The LemurOscService component overrides this method.
• object PrepareDataForSending(object data, object common, OscAddressInfo info): Transform the property data, if necessary, before sending it to a connected OSC device. The default is to not change the data; the LemurOscService component overrides this method.
• void SendSynchronously(IList<Tuple<string, object>> addressesAndData): Send pairs of OSC addresses and property data objects to the current destination endpoints immediately, on the current thread. The LemurOscService component overrides this method.
• void SendPacket(IList<Tuple<string, object>> addressesAndData, int first, int count): Send a single packet synchronously to each destination device in DestinationEndpoints, with the given pairs of OSC addresses and data objects.
• void SetValue(string oscAddress, object addressable, object value, OscAddressInfo info): Set a value on a property with an associated OSC address.
• IEnumerable<OscAddressInfo> GetInfos(object selected): Get the OscAddressInfos for the given object, indicating the properties that have OSC addresses associated with them.

LemurOscService Component

LemurOscService provides OSC support that is compatible with the Lemur iPad® application, from the Liine company. For more information on Lemur, see the Lemur site.

LemurOscService derives from OscService and overrides several of its methods to perform special processing:

• IEnumerable<Tuple<string, object>> GetCustomDataToSend(object common): This updates the "interface" value, which can switch GUI screens in Lemur.
• object PrepareDataForSending(object data, object common, OscAddressInfo info): This converts any Boolean values to floats, which is what Liine Lemur uses for Boolean values, even though OSC has a Boolean type.
• void SendSynchronously(IList<Tuple<string, object>> addressesAndData): This method limits the number of messages sent in a packet to 16 to deal with a Liine Lemur limitation.

For more information on using Lemur, see Using Lemur to Control an ATF Application.

AddPropertyAddress Method Example

The Add2DPointProperty() method shows how to use OscService.AddPropertyAddress():

public string Add2DPointProperty(ChildInfo childInfo, AttributePropertyDescriptor childAttributeDesc, string oscAddress)
{
    oscAddress = OscServices.FixPropertyAddress(oscAddress);

    var xCoordDesc = new ChildListFloatingPointArrayDesc(childInfo, childAttributeDesc, 0);
    AddPropertyAddress(xCoordDesc, oscAddress + "/x");

    var yCoordDesc = new ChildListFloatingPointArrayDesc(childInfo, childAttributeDesc, 1);
    AddPropertyAddress(yCoordDesc, oscAddress + "/y");

    return oscAddress;
}

Add2DPointProperty() adds OSC addresses (one for the x-coordinate and one for the y-coordinate) for a list of DOM children that have attributes that are arrays of floats. Each array of floats represents a 2D point where the first float is the x-coordinate and the second float is the y-coordinate.

ChildListFloatingPointArrayDesc is a private class derived from ChildAttributePropertyDescriptor:

private class ChildListFloatingPointArrayDesc : ChildAttributePropertyDescriptor
{
    public ChildListFloatingPointArrayDesc(ChildInfo childInfo, AttributePropertyDescriptor childAttributeDesc, int coordinateIndex)
        : base(
            childAttributeDesc.Name,
            childAttributeDesc.AttributeInfo,
            childInfo,
            childAttributeDesc.Category,
            childAttributeDesc.Description,
            childAttributeDesc.IsReadOnly,
            null, //editor
            childAttributeDesc.Converter)
    {
    ...

Note the last parameter in the call to the base method, which provides a converter for the type. This converter is used to convert the OSC message arguments into a float value for the coordinate.

OscCommandReceiver Component

OscCommandReceiver listens for OSC messages that can trigger the execution of commands in the application. All commands that are known to ICommandService can be triggered. This allows an OSC application/device to control an ATF application. The format for such an OSC address is:

{source} /{menu name}/{command name} {source}

with spaces and illegal characters removed.

In the application's TypeCatalog, the OscCommandReceiver component can only see commands that were previously registered with the ICommandService. Therefore, to see all commands, put OscCommandReceiver last in the TypeCatalog.

OscCommandReceiver uses the OscService.MessageReceived event to be notified when OSC messages arrive. OscCommandReceiver imports a ICommandService implementer, which it employs to execute commands received.

OscCommands Component

OscCommands is a component that provides menu commands that may be useful for users of OscService. It implements two commands:

• OSC Info: Display the status of the OSC service and list the available OSC addresses and associated properties in a dialog, using the OscDialog class. You can see all OSC address mappings this way. For an illustration and explanation of this dialog, see OscDialog Class.
• Copy OSC Address: Copy the OSC address of the clicked on property in the property editor to the clipboard. This menu item only appears when the property has an associated OSC address.

OscCommands implements IContextMenuCommandProvider so that context menus can display these commands where appropriate.

OscDialog Class

OscDialog derives from System.Windows.Forms.Form and creates the OSC information dialog, shown in this illustration:

The dialog's ListView shows all mapped OSC addresses along with their associated property name, data type, and C# class. It also provides information, such as the application's local IP address and port number that you might need to configure other OSC applications/devices.

The dialog performs several other functions:

• Modify the application's local IP address and port number.
• Modify the destination's local IP address and port number. Use this to send OSC messages to a device at this IP address. This adds this destination to the OscService.DestinationEndpoints, property, which holds the current list of known IP endpoints (IP address and port number) to broadcast to. For more information on this and other OscService properties, see OscService Properties.
• Clicking the To Clipboard button formats and copies information in the top part of the dialog, such as the application's local IP address, to the clipboard.
• Clicking the *OK* button sets the application's and destination's local IP addresses and port numbers that were entered in the dialog's fields.

This dialog allows you to perform any necessary configuration of the OSC-enabled ATF application. You may need to do other configuration, such as setting up firewalls to permit sending and receiving UDP packets on the appropriate port number.

Using Lemur to Control an ATF Application

The Lemur application allows constructing your own user-interface and macros, and then you can use an iPad® to run commands or edit objects in an OSC-enabled ATF application.

In addition to the configuration mentioned in OscDialog Class, you need to do the following to use Lemur with an OSC-enabled ATF application.

To create a button in Lemur that can execute commands in the ATF OSC-enabled application:

1. Set the button's behavior mode to "Pad".
2. For every ATF application command you want to use, set the OSC address to be the command name, which usually looks something like "/File/Save". If the command name has spaces in it, remove the spaces, because these are removed by OscService, as mentioned in Mapping an OSC Address onto a C Sharp Class Property.
3. Set the trigger condition to be when the value goes from 0 to positive. This prevents the command from being executed twice on each button click.