Read and Write to a Rudimentary Database from C++

[A_L_V_]

My goal is to be able to read and write to a database (Kdb) from C++ code.

This question seems to have been addressed numerous times on the forums, but never in an entirely satisfactory (read "do A B C and it will work" :D) manner.

1. I have the header file k.h (http://code.kx.com/wsvn/code/kx/kdb%2B/c/c/k.h).
2. I have the example file csv.c (http://code.kx.com/wsvn/code/contrib/wiki/csv.c).

(2) confuses me slightly, specifically the line (localhost?, username?, password?):

int row,col,nCols,nRows,handle=khpu("localhost",12001,"myusername:mypassword");

I have been playing with Kdb+, have a table "tabTrades" that contains a list (vector) "trades" with 680 entries. I haven't managed to save the database yet and have to create in the console every time I launch q.

I would like send a query "tabTrades" from C++ and get an output that shows the vector.

I have checked out the following links:
http://code.kx.com/wiki/Cookbook/InterfacingWithC
http://www.timestored.com/kdb-guides/kdb-c-api-example
http://code.kx.com/wiki/Startingkdbplus/contents
https://groups.google.com/forum/#!topic/personal-kdbplus/23KCuiCCkPA
http://kx.com/q/c/c/readme.txt
http://code.kx.com/wiki/Startingkdbplus/tables

Pointers and tips or even a full "recepie" would be much appreciated.

[charlie]

> int row,col,nCols,nRows,handle=khpu("localhost",12001,"myusername:mypassword");

is just multiple declarations and an init on one line. Could be written as

int row;

int col;

int nCols;

int nRows;

int handle;

handle=khpu("localhost",12001,"myusername:mypassword");

but if that's confusing you, you are in for some fun with c++ programming 😉

This example connects to kdb+ via tcp, so you specify the hostname, port and username and password.

To start kdb+ listening on port 5000, start q as

q -p 5000

to save your small tabTrades table to disk, you can do

q)`:tabTrades set tabTrades

to load that from disk into variable tabTrades, just do

q)tabTrades:get`:tabTrades

You can still use the csv.c as an example

  result=k(handle,"tabTrades",(K)0);

Hope this helps,

Charlie

[A_L_V_]

Thank you, Charles. The C++ example requires a username and a passsword.

Is there a way to launch Kdb to create them in addition to the port number, e.g. something like
q -p 5001 -u name:password (unfortunately this dummy syntax does not work).

[charlie]

the username:password supplied in the khpu call is relevant only if using authentication. By default, kdb+ does not use authentication, so the username:password could just be chosen to be something to identify your client app.

you can setup username:password pairs in a text file, e.g. u.txt, as

verminburger:verminspassword

one entry per line. This file can then be referenced via the cmd line -u option

q -u u.txt -p 5000

the password can also be stored as an md5 digest

http://code.kx.com/wiki/Reference/md5

q)md5"verminspassword"

0xb7096af841afbf9c1b8e7c4b706cff8c

so the entry in u.txt would then be

verminburger:b7096af841afbf9c1b8e7c4b706cff8c

[A_L_V_]

Hi Charles. This is excellent information! If the username:password are not mandatory I would prefer keep things simple for the time being and change the C++ code to
int row,col,nCols,nRows,handle=khpu("localhost",5000); to reflect that.

[charlie]

ok, then you should use khp not khpu.

handle=khp("localhost",5000); t

[effbiae]

Just don't use authentication - use khp():

  handle=khp("localhost",5001)

If you want to use authentication later...

I couldn't find info on the password file on the wiki, but here's a start:

https://groups.google.com/d/topic/personal-kdbplus/gO6wN_UP1_s/discussion

Ta, Jack.

[charlie]

it's kind of a red-herring. The username:password will be ignored by the kdb+ process if it is not using authentication. For good practice, maybe for consideration later, client applications should always provide a username that easily identifies that client application from the server side (visible with .z.u).

khp is just a wrapper for khpu that defaults u to "". So this is equivalent:

handle=khpu("localhost",5000,"");

and acts as a reminder for later that username and password are missing. 

[A_L_V_]

I will keep the khpu. I am having problems compiling the csv.c though -- "an undefiened reference to" khpu (and a number of other functions k, kclose etc). In the header file they are labelled external, which would suggest they are defined in a separate *.cpp. I searched http://code.kx.com/wsvn/code/kx/kdb%2B/c/c/?#a13bd25ee24546baceff884a3804f0ecc but did not really see any definitions of the above mentioned functions.

[charlie]

the comments at the top of the csv.c show which file to link with. Those functions are present in c.o/c.dll. Choose whichever one for your target platform.

// obtain k.h from http://kx.com/q/c/c/k.h
// linux:
//  compile with gcc -m64 -DKXVER=3 csv.c c.o
//  obtain c.o from http://kx.com/q/l64/c.o for linux
// windows:
//  start the x86 or 64 bit version of build environment, then: cl -DKXVER=3 /MD csv.c c.obj ws2_32.lib
//  obtain c.obj from http://kx.com/q/w32/ or w64/

[A_L_V_]

I am working with a NetBeans IDE; was perplexed a little by the c.o file mentioned in the comment (as it is in between a c.cpp and a c.exe). Will try placing all three files in a folder and compiling from the terminal. Thank you.

[A_L_V_]

Moved all the files (k.h, csv.c and c.o) in a single folder. Ran gcc -m64 -DKXVER=3 csv.c c.o , but still got a number of undefined functions: sem_post, ..., pthread_rwlock_rdlock etc. 😞 

[charlie]

link with the pthread library

gcc -m64 -DKXVER=3 csv.c c.o -lpthread

[A_L_V_]

Thank you, Charles. That worked. When running the executable for a query tabTrades in a.out I got a type 0 output. Then tried running a different query for an example given in http://code.kx.com/wiki/A_Brief_Introduction_to_kdb%2B. It gave me type 16 not supported by this client,banana,80,chicago. So some of the information can be retrieved.

[mark_rooney]

Hi,

The type 16 is a timestamp which is not supported in that code. The switch statement below in the csv.c
code shows which types are supported:

switch(obj->t)
{
    case(0):{ // handle a list of char vectors
     K x=kK(obj)[row];
      if(10==x->t){int i;for(i=0;i<xn;i++)printf("%c",kG(x)[i]);}
       else printf("type %d not supported by this client",obj->t);
    }break;
    case(1):{printf("%d",kG(obj)[row]);}break;
    case(4):{printf("%d",kG(obj)[row]);}break;
    case(5):{printf("%d",kH(obj)[row]);}break;
    case(6):{printf("%d",kI(obj)[row]);}break;
    case(7):{printf("%lld",kJ(obj)[row]);}break;
    case(8):{printf("%f",kE(obj)[row]);}break;
    case(9):{printf("%f",kF(obj)[row]);}break;
    case(11):{printf("%s",kS(obj)[row]);}break;
    default:{printf("type %d not supported by this client",obj->t);}break;
       }

Thanks
--
Mark Rooney
Financial Software Developer
AQUAQ Analytics

[A_L_V_]

Thanks Mark! Added this case(16):{printf("%lld",kJ(obj)[row]);}break; to the code, which solved the second problem. Will see if I can solve the type 0 problem.

N.B. This is a great active and knowledgeable google group.

[mark_rooney]

This document may help you parse the different types of K objects from your C code:

http://www.aquaq.co.uk/wp-content/uploads/2015/01/AquaQ-Analytics-Transferring-Data-between-kdb+-and...

Thanks
--
Mark Rooney
Financial Software Developer
AQUAQ Analytics