How to traverse all of the text files

[huangyuanfei14]

Hi,

suppose:

q)dir1:`:/test;

q)key dir1

`20141201`20141202`....

q)dir2:` sv dir,(key dir1) 0

q)key dir2

`IBM.txt`AMD.txt`....

I should how to iterate through all the text files and generate partition hdb?

Best regards,

Sky.

[sohagan]

Here is a very good twiki on loading from a list of csvs and creating as hdb...

http://code.kx.com/wiki/Cookbook/LoadingFromLargeFiles#Data_Loading_Example

just modify your read to read in *.txt files

hope it goes well,

Sean

[huangyuanfei14]

Hi Sean,

Thanks.

Maybe you misunderstood my meaning.I'm not reading large file.

I want to read many files in different folders.

Best Regards,

Sky

在 2015年4月13日星期一 UTC+8下午5:04:24，Sean O'Hagan写道：

Here is a very good twiki on loading from a list of csvs and creating as hdb...

http://code.kx.com/wiki/Cookbook/LoadingFromLargeFiles#Data_Loading_Example

just modify your read to read in *.txt files

hope it goes well,

Sean

[LamHinYan]

.util.tree   tree:{ $[ x ~ k:key x; x; 11h = type k; raze (. z.s ` sv x,) each k;()]} The tree function accepts a file path as its only parameter and returns all the files found within the directory or any sub-directory.

Psaris, Nick (2015-03-19). Q Tips: Fast, Scalable and Maintainable Kdb+ (Kindle Locations 5861-5866). Vector Sigma. Kindle Edition. 

[sohagan]

Hi,

I provided that link because it has the ability to load a list of files from an inputted directory. You just need to modify it to accept multiple dirs.

Yes the link says "Loading From Large Files" , but the actual script is just a loader...http://code.kx.com/wsvn/code/cookbook_code/dataloader/loader.q.

The script also generates a partitioned hdb...which is what your question was.

Whilst we are here, lets see what we can do...

//create 2 dirs for files

cd /var/tmp

for j in seano seano2 ; do for i in file1 file2; do touch $j/$i ; done ; done

//pass in dirs into a cmd line param

q -dir /var/tmp/seano /var/tmp/seano2

//create file list from these two dirs

` sv/: raze foo,''key each foo:hsym each `$.Q.opt[.z.x]`dir

Or focusing on the script...

currently it has 

inputdir:`:examplecsv

loadallfiles[inputdir;1b]

Change it to

inputdir:`:examplecsv`:examplecsv2

loadallfiles[;1b] each inputdir

Thanks,

Sean

[huangyuanfei14]

在 2015年4月15日星期三 UTC+8下午9:58:52，Sean O'Hagan写道：

Hi,

I provided that link because it has the ability to load a list of files from an inputted directory. You just need to modify it to accept multiple dirs.

Yes the link says "Loading From Large Files" , but the actual script is just a loader...http://code.kx.com/wsvn/code/cookbook_code/dataloader/loader.q.

The script also generates a partitioned hdb...which is what your question was.

Whilst we are here, lets see what we can do...

//create 2 dirs for files

cd /var/tmp

for j in seano seano2 ; do for i in file1 file2; do touch $j/$i ; done ; done

//pass in dirs into a cmd line param

q -dir /var/tmp/seano /var/tmp/seano2

//create file list from these two dirs

` sv/: raze foo,''key each foo:hsym each `$.Q.opt[.z.x]`dir

Or focusing on the script...

currently it has 

inputdir:`:examplecsv

loadallfiles[inputdir;1b]

Change it to

inputdir:`:examplecsv`:examplecsv2

loadallfiles[;1b] each inputdir

Thanks,

Sean

On Monday, April 13, 2015 at 9:16:58 AM UTC+1, huangyu...@gmail.com wrote:

Hi,

suppose:

q)dir1:`:/test;

q)key dir1

`20141201`20141202`....

q)dir2:` sv dir,(key dir1) 0

q)key dir2

`IBM.txt`AMD.txt`....

I should how to iterate through all the text files and generate partition hdb?

Best regards,

Sky.

Hi sean,Thank you very much.