cancel
Showing results for 
Search instead for 
Did you mean: 

Sometimes getting empty columns when loading data

A_L_V_
New Contributor
Loading a number of *.mat files into Kdb. Some of the files load with no problems, some (probably less than a half) frow a bunch of 'type 'rank errors and leave consistently four columns in the table blank. The failed *.mat files seem OK.
6 REPLIES 6

kuentang
New Contributor

Not sure what you mean by loading a matlab file into kdb+.

 

Normally you

 

·       load c.jar in matlab,

·       make a connection to kdb+ using kx.c,

·       send a command and\or data to kdb+ using kx.c.k or kx.c.ks

 

What you need to make sure is that the matlab object can be cast to native java object. See here: http://uk.mathworks.com/help/matlab/matlab_external/passing-data-to-a-java-method.html

 

HTH,

 

Kim

 

 

Von: personal-kdbplus@googlegroups.com [mailto:personal-kdbplus@googlegroups.com] Im Auftrag von A.L.V.
Gesendet: Samstag, 28. Februar 2015 03:34
An: personal-kdbplus@googlegroups.com
Betreff: [personal kdb+] Sometimes getting empty columns when loading data

 

Loading a number of *.mat files into Kdb. Some of the files load with no problems, some (probably less than a half) frow a bunch of 'type 'rank errors and leave consistently four columns in the table blank. The failed *.mat files seem OK.

--
You received this message because you are subscribed to the Google Groups "Kdb+ Personal Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to personal-kdbplus+unsubscribe@googlegroups.com.
To post to this group, send email to personal-kdbplus@googlegroups.com.
Visit this group at http://groups.google.com/group/personal-kdbplus.
For more options, visit https://groups.google.com/d/optout.

It is more to do with the q syntax I am using:

exec(q, ['tempTable:([] name; number1; number2;'...
    'number3; number4; number5; number6; number7; number8l)']);

exec(q,'`:Table/ upsert .Q.en[`:.; tempTable]');

I have a temp table for memory management purposes.

This works for loading individual entries (each entry corrresponds to a matlab file), but as the number of entries grows I start to get empty values in number6, number7 and number8.



Again, you need to make sure that number6, number7, number8 are correctly transferred to kdb from matlab.

What kind of values are stored in number6,7 and 8. I remember that cells cannot be transferred to kdb+.

 

And I have seen tables with more than 200 columns.

 

 

HTH,

 

Kim

 

Von: personal-kdbplus@googlegroups.com [mailto:personal-kdbplus@googlegroups.com] Im Auftrag von A.L.V.
Gesendet: Samstag, 28. Februar 2015 22:45
An: personal-kdbplus@googlegroups.com
Betreff: Re: [personal kdb+] Sometimes getting empty columns when loading data

 

It is more to do with the q syntax I am using:

exec(q, ['tempTable:([] name; number1; number2;'...
    'number3; number4; number5; number6; number7; number8l)']);

exec(q,'`:Table/ upsert .Q.en[`:.; tempTable]');

I have a temp table for memory management purposes.

This works for loading individual entries (each entry corrresponds to a matlab file), but as the number of entries grows I start to get empty values in number6, number7 and number8.



double and int64. Some double variables actually could be int, as they contain no decimal places.

Is there an alternative syntax I could try? The thing that worries me that this problem only occurs when I append the table; it does not happen for loading individual entries.

It is the number of rows that is high (milions) rather than columns.

 

Works as expected,

 

HTH,

 

Kim

 

javaaddpath('C:\q\listbox\26matlab\c.jar')

q=kx.c('localhost',8888);

x = random('Normal',0,1,50,1500000);

for k=1:50;q.k('{(`$"a",string x) set y}',k,x(k,:));end

q.ks('t:flip t!value each t:`$"a",\:string 1+til 50')

 

Von: personal-kdbplus@googlegroups.com [mailto:personal-kdbplus@googlegroups.com] Im Auftrag von A.L.V.
Gesendet: Montag, 2. März 2015 02:27
An: personal-kdbplus@googlegroups.com
Betreff: Re: [personal kdb+] Sometimes getting empty columns when loading data

 

double and int64. Some double variables actually could be int, as they contain no decimal places.

Is there an alternative syntax I could try? The thing that worries me that this problem only occurs when I append the table; it does not happen for loading individual entries.

It is the number of rows that is high (milions) rather than columns.

Fixed. The problen was I was not specifying data types when creating lists to be used in the table kdb woudl assign different data types sometimes when reading which created problems with upserting.