reading output of `ls`

[effbiae]

hi,

can someone help me with this one:

q)show x:system"find ~/q -exec ls -lda {} \\; ";

"drwxrwxr-x 3 tb tb 4096 Apr  4 19:56 /home/tb/q"

"-rw-r--r-- 1 tb tb 821 Jan 21 11:17 /home/tb/q/sp.q"

"-rw-r--r-- 1 tb tb 228 Jan 21 11:17 /home/tb/q/q.q"

"-rw-r--r-- 1 tb tb 150 Jan 21 11:17 /home/tb/q/trade.q"

"-rw-r--r-- 1 tb tb 19402 Mar 24 18:29 /home/tb/q/q.k"

"-rw-r--r-- 1 tb tb 5421 Jan 15  2014 /home/tb/q/s.k"

"-rw-r--r-- 1 tb tb 3078 Jan 21 11:17 /home/tb/q/README.txt"

"drwxrwxr-x 2 tb tb 4096 Apr  4 19:56 /home/tb/q/l32"

"-rwxr-xr-x 1 tb tb 557124 Mar 24 18:29 /home/tb/q/l32/q"

q)("SSSSSSSSSS";" ")0: x

drwxrwxr-x -rw-r--r--      -rw-r--r--     -rw-r--r--         -rw-r--r--     -..

3          1               1              1                  1              1..

tb         tb              tb             tb                 tb             t..

tb         tb              tb             tb                 tb             t..

4096       821             228            150                19402          5..

Apr        Jan             Jan            Jan                Mar            J..

           21              21             21                 24             1..

4          11:17           11:17          11:17              18:29           ..

1. 19:56      /home/tb/q/sp.q /home/tb/q/q.q /home/tb/q/trade.q /home/tb/q/q.k 2..
   
2. /home/tb/q                                                                  /..
   

q)

notice where there's two spaces, 0: doesn't treat them as one.

the next problem i will have is the filenames with spaces in them - i'm trying to analyse all the material i've gathered over the  years over a few disks.  .

maybe there are options to `ls` that makes columns fixed width but i was hoping to learn more about q.

many thanks,

jack.

[LamHinYan]

You could replace the first 8 blocks of spaces with commas to get a 9 column csv. You may perform the search and replace in shell/perl/kdb.

$ ls ~/q/ -l | perl -pne 'for($a=0; $a<8; ++$a) { s/ +/,/; }'

total,1359

-rw-rwxr--+,1,yan,None,3078,Feb,25,20:21,README.txt

d---rwx---+,1,yan,None,0,Mar,24,00:21,echoclnt

----rwx---+,1,yan,None,16224,Mar,22,00:11,help.q

drwxrwxr-x+,1,yan,None,0,Feb,25,20:21,l32

-rw-rwxr--+,1,yan,None,19402,Feb,25,20:21,q.k

-rw-rwxr--+,1,yan,None,228,Mar,25,19:58,q.q

-rw-rwxr--+,1,yan,None,103,Apr,4,09:37,qprofile

-rw-rwxr--+,1,yan,None,5421,Feb,25,20:21,s.k

[pauljloughran]

Hi Jack,

You could remove those double spaces using something like this:

q)fileinfo:1 _ system "ls -l"
q)ssr[;"  ";" "]/'[fileinfo]
"-rw-r--r-- 1 paul staff 3078 21 Jan 10:16 README.txt"
"drwxr-xr-x@ 4 paul staff 136 1 Apr 16:08 m32"
"-rw-r--r--@ 1 paul staff 19402 10 Feb 15:44 q.k"
"-rw-r--r--@ 1 paul staff 229 28 Feb 16:29 q.q"
"-rw-r--r--@ 1 paul staff 5421 21 Jan 10:16 s.k"
"-rw-r--r--@ 1 paul staff 821 21 Jan 10:16 sp.q"
"-rw-r--r--@ 1 paul staff 150 21 Jan 10:16 trade.q"

q)("********"; " ") 0: ssr[;"  ";" "]/'[fileinfo]
"-rw-r--r--" "drwxr-xr-x@" "-rw-r--r--@" "-rw-r--r--@" "-rw-r--r--@" "-rw-r--..
,"1"         ,"4"          ,"1"          ,"1"          ,"1"          ,"1"    ..
"paul"       "paul"        "paul"        "paul"        "paul"        "paul"  ..
"staff"      "staff"       "staff"       "staff"       "staff"       "staff" ..
"3078"       "136"         "19402"       "229"         "5421"        "821"   ..
"21"         ,"1"          "10"          "28"          "21"          "21"    ..
"Jan"        "Apr"         "Feb"         "Feb"         "Jan"         "Jan"   ..
"10:16"      "16:08"       "15:44"       "16:29"       "10:16"       "10:16" ..

One possible way to deal with file names with spaces could be making another system call with the -m flag set.  This will return a comma delimited list of file names:

q)`quot;," vs first system"ls -m"
`README.txt`m32`q.k`q.q`s.k`sp.q`trade.q

Regards,

Paul
AquaQ Analytics

[sohagan]

personally I like to use tr for formatting... ls -lda | tr ' ' ','

q)2#system"find /q/ -exec ls -lda {} \\; | tr ' ' ','"

"drwxr-xr-x,3,root,root,4096,Mar,16,22:39,/q/"

"-rw-r--r--,1,root,root,3078,Aug,30,,2014,/q/README.txt"

if you want it within q, and want to avoid using tr above, you can remove spaces like so...

f:system"find /q/ -exec ls -lda {} \\;"

\ts:1000 {x where(or)':[not null x]}each f

98 8096

btw, if you want simple file list in q, use key

key `:/q

to avoid hidden files...

{foo where not (foo:key x) like ".*"}`:/q

or a full directory structure (without the additional ls details)...

https://groups.google.com/forum/#!searchin/personal-kdbplus/directory$20structure/personal-kdbplus/QkfA9hAoVz8/s0NA76ClwFIJ

HTH,

Sean

[effbiae]

thank you all,

yes, the key is to give simple input to q and i found this to be best for my needs:

$ find . -type f -fprintf a '%i\t%y\t%s\t%A@\t%C@\t%T@\t%p\n'

which prints TDF (tab delimited) info to ./a  

(the ls utility isn't designed to give easily readable output)

then loading into q:

q)d:{14h$(x%8.64e4)-10957}

q)select inode:n, ty:y, len:s, access:d a, change:d c, modify:d t, path:50_'p from (flip `n`y`s`a`c`t`p!("JSJFFF*";"\t")0:`a)

inode    ty len     access     change     modify     path

-----------------------------------------------------------------------------------------------------------------------

89784415 f  8886088 2015.04.14 2015.04.14 2008.12.16 "Music/Soundtrack/High Fidelity/13 Inside Game.mp3"

89784422 f  373     2015.04.14 2015.04.14 2013.06.17 "Music/Soundtrack/High Fidelity/desktop.ini"

89784421 f  13829   2015.04.14 2015.04.14 2013.06.17 "Music/Soundtrack/High Fidelity/Folder.jpg"

89784405 f  8666668 2015.04.14 2015.04.14 2008.12.16 "Music/Soundtrack/High Fidelity/03 I'm Wrong About Everything.mp3"

89784403 f  5943653 2015.04.14 2015.04.14 2008.12.16 "Music/Soundtrack/High Fidelity/01 You're Gonna Miss Me.mp3"

..