Q to C compiler

[quintanar401]

Hi,

I've recently added to github a Q to C (so lib) compiler - https://github.com/quintanar401/q2c . It can compile any Q code (small adjustments may be required) into a C lib that can be loaded into Q and then the compiled functions will work as before. It also supports C-like extention to Q so you may tranform Q values into C values and back and call C functions, all this in a safe manner without memory leaks or core dumps.

It can be used to create proxy functions for C libraries (see examples) with no effort at all, also you can use it to hide/obfuscate your Q code or code critical functions in C. I should warn that the compiled Q functions will not run faster than Q funcs themselves, this is because Q calls are done via dot func (similar to .) and it has a significant overhead.

To give an idea of how C-DSL looks like here is an example of the auto generated proxy for mmap function:

{[x1;x2;x3;x4;x5;x6]  c.void_p.res:mmap[`c.void_p$`c.ij$x1; `c.size_t$x2; `c.i$x3; `c.i$x4; `c.i$x5; `c.off_t$x6]; if[-1=`c.ij$c.res;'"mmap: ",.string.strerror[C.toK C.errno]]; (C.toK `c.ij$c.res; x2)}

Expressions like `c.xx$ tranform Q values into C values (they check that the type is correct), variables like c.var are local C variables and C.toK function is used to create K values from C values (it is overloaded on C type). Also the expression -1=`c.ij$c.res inside "if" is pure C - C-DSL supports almost all C operators + abs, not, neg funcs. The above function will be compiled into:

K shmemex_mman_mmap__q(K _x1,K _x2,K _x3,K _x4,K _x5,K _x6){
  K __st[1]; int __stc=0; K __v=(K)1;
  void* _c_res; J _vc_j1; I _vc_i2; I _vc_i3; I _vc_i4; J _vc_j5; J _vc_j6; _Bool _vc_bool1;

  __v=__K2j_cast(&_vc_j1,r1(_x6)); if(0==__v) goto __return; r0(__v);   // `c.j$<val>
  __v=__K2i_cast(&_vc_i2,r1(_x5)); if(0==__v) goto __return; r0(__v);   // `c.i$<val>
  __v=__K2i_cast(&_vc_i3,r1(_x4)); if(0==__v) goto __return; r0(__v);   // `c.i$<val>
  __v=__K2i_cast(&_vc_i4,r1(_x3)); if(0==__v) goto __return; r0(__v);   // `c.i$<val>
  __v=__K2j_cast(&_vc_j5,r1(_x2)); if(0==__v) goto __return; r0(__v);   // `c.j$<val>
  __v=__K2j_cast(&_vc_j6,r1(_x1)); if(0==__v) goto __return; r0(__v);   // `c.j$<val>
  _c_res=mmap((void*)(shmemexc_ij)_vc_j6,(size_t)_vc_j5,_vc_i4,_vc_i3,_vc_i2,(off_t)_vc_j1);  // mmap[<val>;<val>;_vc_i4;_vc_i3;_vc_i2;<val>]
  // c.res[()]::<val> - no rval, global

  // end of statement
  _vc_bool1=-1LL==(shmemexc_ij)_c_res;  // =[-1LL;<val>]
  if(0==_vc_bool1) goto __l2;  // if/while check condition
  __v=ki(errno); if(0==__v) goto __return;   // C.toK[C.errno]
  __v=shmemex_string_strerror__q(__v); if(0==__v) goto __return;   // .string.strerror[<val>]
  __v=kdot(r1(__consts[51]),knk(2,r1(__consts[127]),__v)); if(0==__v) goto __return;   // ,["mmap: ";<val>]
  __st[__stc++]=__v;  if(__v->t==-KS) __v=krr(__v->s); else if(__v->t==KC)  __v=krr(sn(kC(__v),__v->n)); else __v=krr("stype"); goto __return;  // raise an exception
  r0(__v);
  // end of statement

 __l2:
  __v=kj((shmemexc_ij)_c_res); if(0==__v) goto __return;   // C.toK[<val>]
  __v=__enlist(knk(2,__v,r1(_x2))); if(0==__v) goto __return;   // enlist[<val>;x2]
  goto __return;  // return cmd

 __return:
  r0(_x1); r0(_x2); r0(_x3); r0(_x4); r0(_x5); r0(_x6);
  while(__stc>0) r0(__st[--__stc]);
  return __v;
 };

A bit verbose but all K vars are freed and all errors are checked.

I could check the compiler only with gcc 32/64 bit but the generated file doesn't contain any unusual code and it will probably compile with other c compilers as well (Visual Studio will require different include files I guess).

WBR, Andrey Kozyrev.
 

[rspa9428]

Very cool. Will definitely check it out

Cheers

Ryan

[Alexander_Belop]

This is awesome.  BTW, I love your work ever since I discovered your q debugger!

I wonder if you considered compiling q code to LLVM IR?

[quintanar401]

Thanks,

Yes, I did and made the result of q2b look like llvm IR for this reason. But I think llvm is hard to use (you need to compile it first) and then it is unlikely that someone will be compiling something from Q very often. Bindings to llvm and translation to IR require a lot of work and as I couldn't find a useful usecase I didn't do anything yet.

WBR,
Andrey Kozyrev.

[Alexander_Belop]

On Wednesday, October 11, 2017 at 4:17:50 PM UTC-4, Andrew Kozyrev wrote:

 Bindings to llvm and translation to IR require a lot of work and as I couldn't find a useful usecase I didn't do anything yet.

Right, but I could use it in my "Julia for kdb+" project.