TRY ME

Try Valo for free

We want to show you something amazing.

We'll send you a link to download a fully functional Valo copy to play with.



Great! Check your email and enjoy Valo



Apologies, we seem to be having a problem processing your input, please try again

Queries

Queries are used to access and transform valo Streams for analysis. Valo queries are highly optimised and distributed.

SQL Like

Valo’s query language has a strong resemblance to SQL. The following query will be somewhat familiar to most SQL users:

from /streams/demo/infrastructure/cpu
select user, kernel
where user > 90

The most obvious difference from regular SQL is select comes after from. There is a good reason for this; A Valo query is actually a Transformation Pipeline

Real time and Historical

Valo queries can be executed in either historical or real time mode, simply by using (or omitting) the historical key word. A historical query gives you access to large amounts of data stored in one of the Repositories, real time queries give you access to the data as it happens.

Importantly, all the same query features can be used in both modes (See Query Language).

Transformation Pipeline

Valo queries works as a pipeline of stream transformations; Each instruction takes an input and produces an output via some form of transformation.

This is best explained using a simple example. Lets assume we have a stream /streams/demo/infrastructure/cpu that is stored in one of the Repositories and that it contains the following data:

[
  { "host": "c.itrs", "process": "Excel", "user": 50.0, "kernel": 75.0 },
  { "host": "a.itrs", "process": "Oracle", "user": 60.0, "kernel": 25.0 },
  { "host": "b.itrs", "process": "RiskEngine", "user": 70.0, "kernel": 99.0 }
]

Lets start by just getting the data from the repository. To do this, we simply use a from query with the historical key word:

from historical /streams/demo/infrastructure/cpu

Note

The same query would work on in real time, we are using a historical query here to keep the example simple!

This will give us the data as-is. Now we can modify the query and start performing some transformations. Lets add a where clause:

from historical /streams/demo/infrastructure/cpu
where kernel > 50

After this first transformation, we would get:

[
  { "host": "c.itrs", "process": "Excel", "user": 50.0, "kernel": 75.0 },
  { "host": "b.itrs", "process": "RiskEngine", "user": 70.0, "kernel": 99.0 }
]

Lets add a further transformation, ordering by the host field:

from historical /streams/demo/infrastructure/cpu
where kernel > 50
order by host

Now we have:

[
  { "host": "b.itrs", "process": "RiskEngine", "user": 70.0, "kernel": 99.0 },
  { "host": "c.itrs", "process": "Excel", "user": 50.0, "kernel": 75.0 }
]

Our final transformation will select some fields and alias them:

from historical /streams/demo/infrastructure/cpu
where kernel > 50
order by host
select process as p, user as u, kernel as k

Now we have:

[
  { "p": "RiskEngine", "u": 70.0, "k": 99.0 },
  { "p": "Excel", "u": 50.0, "k": 75.0 }
]

As each ‘instruction’ operates on the data emitted from the previous instruction, transformations can be specified in any order, the following query could be re-written like so and it would emit the same results:

from historical /streams/demo/infrastructure/cpu
select process as p, user as u, kernel as k
order by host
where kernel > 50

This is just a simple example, there are many other forms of transformation (see Query Language) and importantly there are also Functions

Functions

As well as the standard Query Language features that can be used to perform transformations, valo also includes a wide range of functions. These range from simple scalar operations (min, max, abs) to online operations (avg, bivariate, count) to domain specific functions (anomaly detection, moving kurtosis, histogram).

Using the example data above, we could use the max function in a select:

from historical /streams/demo/infrastructure/cpu
select max(kernel) as maxKernel
[
  { "maxKernel": 99.0 }
]