TRY ME

Try Valo for free

We want to show you something amazing.

We'll send you a link to download a fully functional Valo copy to play with.



Great! Check your email and enjoy Valo



Apologies, we seem to be having a problem processing your input, please try again

Schema

A schema describes the shape and format of a stream’s data. The schema is essential as it supports the following functionality:

  • Repository index mappings.
  • Runtime code generation.
  • Type resolution in queries and expression trees.
  • Data validation.

A schema has the following sections

  • A user defined schema version.
  • A config section.
  • A topDef section which defines the field hierarchy stored in the stream.

A schema defining no fields looks like this:

{
  "schema": {
    "version": "1.0.0",
    "config": {
      "key": []
    },
    "topDef": {}
  }
}

The topDef section can contain a simple or a complex type. A simple type may be just a simple individual double and so look like this:

{
    "topDef": { "type" : "double" }
}

and a complex type with a number fields of different types with key value pairs might look like this:

 {
     "topDef": {
         "type": "record",
         "properties": {
             "date": { "type": "date", "optional": "false" },
             "maxTemp": { "type": "double", "optional": "false" },
             "daysOfAccumulation": { "type": "byte", "optional": "true" },
             "airQuality": { "type": "boolean", "optional": "true" }
         }
     }
}

fields are mandatory by default, you can make them optional (i.e. capable of storing null) like this:

{
    "topDef": {
        "type" : "record",
        "properties" : {
            "host" : { "type" : "string", "optional": true }
        }
    }
}

A schema can be defined using any of the supported data types.

Annotations can also be added to individual fields to further refine behaviour.

Field Name Rules

Field names can be unicode characters, with the following restrictions:

  • Fields cannot start with __ (2 underscores) These are considered ‘private’
  • Cannot be an expressions constant: (true, false) in any case
  • Cannot be an operator: +, -, *, /, %, ||, &&, !, >=, <=, !=, ==, >, <, NOT, and, or
  • Cannot contain any operator symbol: +, -, *, /, %, ||, &&, !, >=, <=, !=, ==, >, <
  • Cannot contain spaces
  • All characters must be valid java identifier characters, where Character.isJavaIdentifierPart is true

Reserved Words

The following names are reserved and cannot be used as field identifiers.

  • true
  • false
  • from
  • historical
  • join
  • inner
  • on
  • order
  • by
  • ascending
  • descending
  • into
  • take
  • select
  • unique
  • emit
  • every
  • values
  • value
  • create
  • set
  • where
  • in
  • sliding
  • tumbling
  • group
  • window
  • of
  • many
  • asc
  • desc
  • fuzzy
  • view
  • model
  • stream
  • --

Full Example

{
   "schema":{
      "version": "1.0",
      "config": {},
      "topDef": {
         "type": "record",
         "properties": {
            "time": {
               "type": "datetime",
               "annotations": ["urn:itrs:default-timestamp"]
            },
            "contributor": {
               "type":"contributor", "definition":"probes"
            },
            "host":{ "type":"string" },
            "resource":{ "type":"string" },
            "status":{ "type":"int" },
            "size":{ "type":"int" },
            "domain":{ "type":"tags" },
            "category":{ "type":"tags" }
         }
      }
   }
}