Thursday, 9 June 2011

Use meta with judiciousness

The common usage of meta in KDB is to provide the metadata of a table; however do not use it against a partioned table!

To empirically verify this for yourself, do this:

\ts meta partionedtable

The best way to retrieve the columns of a partioned table is to do something like this depending on how you are partioning:

columns:{[d]key exec from partionedtable where date=d, i=0}

Best way to learn Q / KDB programming

Developers with only an imperative programming background (C, C++, Java, etc.) have a learning curve with the highest gradient and to overcome, there are two programming paradigms that need to be understood:

1. Function programming: high-order functions, projections (currying), closures, etc.
2. Array Processing Language: right-to-left evaluation, vector algebra

Once a good grasp of these paradigms has been developed then the rest should be a nice, relaxing downhill descent.

Passing a table as an argument to a remote function in KDB

Executing queries against a historical (disk-based) KDB database can be computationally expensive - both in time and space. Rather than tautologically execute the query, it is best to cache the result if the query needs to executed again. I wanted to pass the result (kdb table) to a remote function on another Q process on another server. Passing the table to an IPC handle is trivial but to pass the table as an argument to a remote function is tricky.

In true blue peter style, here is one I made earlier:

h:hopen `someserver:3333
tbl:select avg price from ([]price:10?1.10)
h "add ",-3!`int$-8!tbl

This code serializes the table, then casts the serialized binary data to a list of integers to be passed to the remote function called add

On the other side of the IPC handle, the add function will look like:

add:{[data] tbl: -9!4h$data; ... }

Enjoy!