TRY ME

Try Valo for free

We want to show you something amazing.

We'll send you a link to download a fully functional Valo copy to play with.



Great! Check your email and enjoy Valo



Apologies, we seem to be having a problem processing your input, please try again

Contributors

What is a Contributor?

A contributor is any external source which publishes data to Valo. The source could be a probe, an application, a domain, or a user manually publishing data. Contributors may have associated meta-data that is provided on registration with Valo. This meta-data allows us to filter contributors into dynamic sets across multiple different streams using the concept of Domains.

A domain can also be a contributor. In that case, the schema is exactly the same. The fields to include in the schema should be selected by the client.

A contributor is identified by a unique Contributor id which should be unique within the scope of the Contributor type. The contributor id is a string so it could be anything from a hostname, a mac address, or the name of an application.

A contributor can send data to Valo if it is authorized to do so. To ensure full functionality however, a contributor should register itself with Valo and then include its contributor id in any data payloads it sends to Valo. Without the contributor id in the data payloads there is no way for Valo to correlate data entries with contributors at query time.

See also the Contributors API and the Domains documentation.

Contributor type

Contributors are classified into types such as probes and apps. Contributors of the same type have common characteristics but with different metadata. Depending on the contributor it might have slightly different metadata compared to other contributors of the same type. For example, a contributor on a Windows server might be able to provide more metadata than a contributor on a Linux server.

A contributor type has a schema which must contain a Contributor id field.

{
  "uri": "/streams/eq/system/probes",
  "schema": {
    "type":"record", "properties": {
      "id": { "type": "contributor" }
    }
}

The schema can be extended to include common contributor fields. This is required if the contributor is to be used in a query.

Contributor id

A contributor id is externally defined and should be used as the identifier for the lifetime of the contributor. The id should be used when the contributor is registered in Valo and it should also be added to data payloads sent to Valo by the contributor.

The id is unique within the scope of the contributor type.

Contributor metadata document

A contributor is registered in Valo together with some contributor metadata. Whenever the metadata changes the contributor should make Valo aware by re-registering itself. If it does not, then Valo will not be able to update any domains that are depending on that contrbutor’s metadata.

The metadata describes the contributor and (as an example) for a probe running on a server the metadata could be the hardware and software specification.

An example of contributor metadata is:

{
    "id" : "9a2df62c-f97d-4971-932d-0099bc4efa49",
    "host" : {
      "name" : "ITRSPC173",
      "ip"   : "192.168.220.43"
    },
    "os" : {
      "name"          : "Windows",
      "version"       : "8.1",
      "manufacturer"  : "Microsoft"
    },
    "processors" : {
      "count" : 4,
      "items" : [
        { "model" : "..." }
      ]
    }
}

The metadata is stored in the KV-store in the cluster and is also published to a corresponding stream. The stream maps to the contributor uri in the following way;

  • Contributor Uri: /contributors/<tenant>/<type> (i.e. /contributors/eq/probes)
  • Stream Uri: /streams/<tenant>/contributors/<type> (i.e. /streams/eq/contributors/probes)

Metadata Examples

Contributor metadata provides a powerful mechanism for filtering that goes beyond the standard ‘where’ style query filtering of a single stream, and allows universal filters based on metadata critera. Examples of good metadata includes (non exhaustively):

  • Categorical information ( A probe might be on a Window or a Linux machine and may have specific software/hardware installed)
  • Geographical information ( IoT sensors might exist in multiple geographic locations for multiple different streams)

In the Categorical example, contributors provide specific self-describing categories which are totally up to the user to define. The example provides categories describing operating systems, hardware and software config. When such a domain was applied to a query, we could narrow our search space to:

  • Contributors with specific OS’s.
  • Contributors with specific hardware/software contributors
  • Any variation of the above

In the Geographical example above, a Valo user would be able to create a domain targeting sensors that came from specific geographic regions. So for example, when such a domain was applied to a query, it might only consider data points from:

  • Within specific regions ( Only consider data from Australia, UK & Spain)
  • Within a specific range of a GeoPoint (Only consider data withing 100km of US Capital Cities)
  • Any other classification you can make from the geographic meta data!

There are many more ways to use contributors to provide powerful domain filters that work across all your streams.