Protocol Specification
This is an overview of specification of the eXtensible Binary Universal Protocol as it is defined in prototype release version 0.2.0.
This should be usable for example to get idea how is the internal structure of protocol defined.
This document is not final and it's also incomplete.
For full specification, please see other parts of protocol specification in documentation.
Lowest protocol's level defines basic tree structure using two primitive types.
- Binary blob (sequence of bytes)
- Natural non-negative integer number
See complete specification of tree structure.
UBNumber is encoding used for representation of instance of countable infinite set stored as one or more bytes (similar to UTF-8 encoding).
First, non-zero bits are counted for length and then rest of bits is used as value while value is also incremented so that there is only one code for each number.
Most native encoding using UBNumber is UBNatural for representation of a natural non-negative integer number.
Examples of the UBNatural codes (sequence of bits = represented value):
<span style="color:green">0</span>0000000 = 0
<span style="color:green">0</span>0000001 = 1
<span style="color:green">0</span>0000010 = 2
<span style="color:green">0</span>0000011 = 3
...
<span style="color:green">0</span>1111111 = 7Fh = 127
<span style="color:green">10</span>000000 00000000 = 80h = 128
<span style="color:green">10</span>000000 00000001 = 81h = 129
...
<span style="color:green">10</span>111111 11111111 = 407Fh = 16511
<span style="color:green">110</span>00000 00000000 00000000 = 4080h = 16512
...
Various interpretations can be mapped on UBNumber encoding. For this level there are defined two:
- UBNatural encoding using directly value from UBNumber
- UBENatural where value 7Fh is reserved for infinity constant and higher values are shifted by one
Document starts with 6 bytes long blob called “Document Header” followed by a single block called “Root Block” and optional blob called “Extended Area”.
Header for current version of protocol is (hexadecimal):
FE 00 58 42 00 02
Each block starts with single value:
UBNatural attributePartSize
If attributePartSize = 0 then this block is called “Terminator” and block ends. Otherwise it is followed by value:
UBENatural dataPartSize
If attributePartSize = count of bytes used by dataPartSize then this block is called “Data Block” and binary blob follows which has length in bytes specified by dataPartSize value and block ends.
If attributePartSize > count of bytes used by dataPartSize then this block is called “Node Block” and a sequence of attributes follows until sum of count of bytes used by attributes = attributePartSize - count of bytes used by dataPartSize.
UBNumber attribute
After attributes, sequence of blocks follows until sum of block sizes = dataPartSize and block ends.
If dataPartSize = infinity for data block then binary blob is ended by a sequence of two zero bytes. A sequence of two bytes where first is zero followed by a non zero byte is considered a sequence of nonstoping zero bytes. The non zero byte defines the length of the sequence.
If dataPartSize = infinity for node block then sequence of data blocks is ended by the terminator.
See following picture for clarification:
Level 1 introduces block types and catalog. For this level all non-listed attributes in node blocks are considered zero.
See complete specification of block types.
First two attributes in node block are interpreted as follows:
UBNatural - TypeGroup
UBNatural - BlockType
Blocks types are organized into Groups. Type group with value 0 is called Basic Group and it's meaning is build-in and defined in the protocol itself.
TypeGroup value specifies to which group each block belongs. BlockType value specifies particular block in the corresponding group where meaning of single groups and block types is defined in document using basic blocks.
Basic group uses TypeGroup = 0. Basic blocks provides ability to store block type declarations, definitions and links to catalog.
This block is used for unknown block types or data padding.
Block: Basic (0) / Unknown (0)
Block: Basic (0) / Document Declaration (1)
Declaration block determines the allowed range of groups. This block should be located at the beginning of each file, if the application didn't provide any static/special meaning, but it might be used anywhere inside document as well.
Definition:
+Natural groupsCount - The number of allocated groups
+Natural preserveGroups - The number of groups to keep from previous declarations
FormatDeclaration formatDeclaration - Declaration of format
Any documentRoot - Root node of document
For subblocks of this block there is permitted range of values in the interval group preserveGroups + 1 .. preserveGroups + groupsCount + 1. preservedGroups + groupsCount + 1. If the value reserveGroups = 0, takes the highest not yet reserved group in the current or parental blocks + 1. For all values of zero and the application of rules of cutting the block of zeros coincides with the data block.
Format declaration allows you use either declaration from catalog or local format definition or both.
Block: Basic (0) / Format Declaration (2)
+CatalogFormatSpecPath catalogFormatSpecPath - Specification of format defined as path in catalog
+Natural formatSpecRevision - Specification's revision number
FormatDefinition formatDefinition
This block allows to specify the basic structure of format specification. Specifies the sequence of groups and their definition.
Block: Basic (0) / Format Definition (3)
Any[] formatParameters - Join or Consist format parameters
+RevisionDefinition[] revisions
Block: Basic (0) / Format Join Parameter (4)
+FormatDeclaration formatDeclaration
Block: Basic (0) / Format Consist Parameter (5)
+GroupDeclaration groupDeclaration
Group declaration allows you use either declaration from catalog or local group definition or both.
Block: Basic (0) / Group Declaration (6)
+CatalogGroupSpecPath catalogGroupSpecPath - Specification of format defined as path in catalog
+Natural groupSpecRevision - Specification's revision number
GroupDefinition groupDefinition
Block: Basic (0) / Group Definition (7)
Any[] groupParameters - Join or Consist group parameters
+RevisionDefinition[] revisions
Block: Basic (0) / Group Join Parameter (8)
+GroupDeclaration groupDeclaration
Block: Basic (0) / Group Consist Parameter (9)
+BlockDeclaration blockDeclaration
Block: Basic (0) / Group Declaration (10)
+CatalogBlockSpecPath catalogBlockSpecPath - Specification of format defined as path in catalog
+Natural blockSpecRevision - Specification's revision number
BlockDefinition blockDefinition
Block: Basic (0) / Block Definition (11)
Any[] blockParameters - Join or Consist or List Join or List Consist block parameters
+RevisionDefinition[] revisions
Block: Basic (0) / Block Join Parameter (12)
+BlockDeclaration blockDeclaration
Block: Basic (0) / Block Consist Parameter (13)
+BlockDeclaration blockDeclaration
Block: Basic (0) / Block List Join Parameter (14)
+BlockDeclaration blockDeclaration
Block: Basic (0) / Block List Consist Parameter (15)
+BlockDeclaration blockDeclaration
Block: Basic (0) / Revision Definition (16)
Block allows to define parameters count for particular specification definition.
+Natural parametersCount
XBUP protocol uses unified centralized catalog of type specifications. To access this centralized catalog, there is provided interface for catalog service, available on the xbup.org project server.
TODO
TODO