How to load partitioned tables?

[James1]

Hi,

I have a partitioned db, the structure of files is like this:

/db

 2015.01

    t1

      c1

      c2

    t2

      c3

      c4

 2015.02

    t1

      c1

      c2

    t2

      c3

      c4

  2015.03

    t1

      c1

      c2

    t2

      c3

      c4

...

  2015.11

    t1

      c1

      c2

    t2

      c3

      c4

  sym

We often use "\l db" to load partitioned db entirely, however,

1. How to load partitioned db with specific months? just as:

1) 2015.01 - 2015.03

2) 2015.01, 2015.03, 2015.05

2. How to load partitioned db with specific months and tables? Like this scenario:

The data ONLY contains table t1 within 2015.01 - 2015.03

3. How to load partitioned db with specific months, tables and columns? Like this scenario:

Within 2015.01 - 2015.03, table t1 with column c1 ONLY.

4. CAUTION: the partitioned db is include symbol enumeration(sym) 

BTW, not use SQL, but with loading script.

Thanks,

Roy

[manni_patel]

you could use view

http://code.kx.com/wiki/Reference/DotQDotview

You can't do by table with that though.

One option might be keep you actual data separate from the loading directory and have symbolic links in the loading directory to define the data set.

[david_demner]

To: Manish Patel , personal-kdbplus@googlegroups.com

For 2 you could delete a table after startup.

\l db

.Q.view 2015.01m+til 3

delete t2 from `.

=E2=80=8E3 can't be done within kdb+ afaik

If you want ultimate flexibility, at the cost of maintainability, you could create a new hdb root with symlinks to the sym file + months + tables + columns (with a non-symlinked .d file)=E2=80=8E you want. Nasty but should work.

From: Manish Patel

Sent: Monday, November 23, 2015 06:10

To: personal-kdbplus@googlegroups.com

Reply To: personal-kdbplus@googlegroups.com

Subject: Re: [personal kdb+] How to load partitioned tables?

you could use view

http://code.kx.com/wiki/Reference/DotQDotview

You can't do by table with that though.

One option might be keep you actual data separate from the loading directory and have symbolic links in the loading directory to define the data set.

[Aaron_Davies]

To: personal-kdbplus@googlegroups.com
X-Mailer: Apple Mail (2.2104)

>> 3. How to load partitioned db with specific months, tables and =
columns? Like this scenario:
>> Within 2015.01 - 2015.03, table t1 with column c1 ONLY.

>

> =E2=80=8E3 can't be done within kdb+ afaik

modify the definition of the table after it's loaded to only reference =
the column(s) you want

q)`:/tmp/db/2015.01/t1/.d set`c1`c2`c3
`:/tmp/db/2015.01/t1/.d
q)`:/tmp/db/2015.01/t1/c1 set 1 2 3
`:/tmp/db/2015.01/t1/c1
q)`:/tmp/db/2015.01/t1/c2 set 1 2 3
`:/tmp/db/2015.01/t1/c2
q)`:/tmp/db/2015.01/t1/c3 set 1 2 3
`:/tmp/db/2015.01/t1/c3
q)\l /tmp/db
q)t1
month c1 c2 c3
----------------
2015.01 1 1 1
2015.01 2 2 2
2015.01 3 3 3
q)0N!t1;
+`c1`c2`c3!`t1
q)t1:flip(enlist`c1)!`t1
q)t1
month c1
----------
2015.01 1
2015.01 2
2015.01 3
q)select month,c1 from t1
month c1
----------
2015.01 1
2015.01 2
2015.01 3
q)select c2 from t1
k){0!(?).@[x;0;p1[;y;z]]}
'c2
.
?
(+(,`c1)!`:./2015.01/t1;();0b;(,`c2)!,`c2)
q.Q))

this could be implemented in a .q file in the root of the hdb if you =
wanted

(N.B. such files are skipped when you reload with \l ., so you'll have =
to use the full path (or generate it with system"l ",system"cd" or =
something)

q)\l .
q)t1
month c1 c2 c3
----------------
2015.01 1 1 1
2015.01 2 2 2
2015.01 3 3 3
q)`:/tmp/db/q.q 0:enlist"t1:flip(enlist`c1)!`t1";
q)\l /tmp/db
q)t1
month c1
----------
2015.01 1
2015.01 2
2015.01 3
q)

note also this will have to be (re)applied after a .Q.view, since that =
re-reads the .d files and resets the columns=