cancel
Showing results for 
Search instead for 
Did you mean: 

How can i understand the code below

chan_chenyanj
New Contributor II

.quantQ.IO.SAVEsPLAYtAB:{[tabPath;pvar;table]

  // tabPath -- path where to store the table

  // pvar -- variable to sort and index on for fast querying

// table -- name of the table to save

:@[;pvar; `p#] pvar xasc (`  sv (tabPath; ` ; table; ` ) ) set .Q.en[tabPath]

    get table

};

how should i understand this function? In particular, what is :@ mean, what is `p#, why ` before sv? the question might be simple but new to q not sure i follow

 

3 ACCEPTED SOLUTIONS

awilson
New Contributor
New Contributor

Code reads right to left

.Q.en[tabPath] get table

// Fetch table (defined by sym/hsym 'table') and enumerate any symbol columns (to sym file located in directory tabPath)

(` sv (tabPath; ` ; table; ` ) ) set ...

// Save the table (splayed) to a directory defined by '/'-separated path (tabPath/table/)

@[;pvar; `p#] pvar xasc ...

// Sort by pvar and apply the parted attribute to this column in the table

: ...

// Return the result (the path to the saved table)


` sv (scalar from vector) joins elements of a filepath.
e.g.
q)` sv (`:path/to/dir; `filename)
`:path/to/dir/filename
q)` sv (`:path/to/dir; `filename; `)
`:path/to/dir/filename/

It is explained here https://code.kx.com/q/ref/sv/#filepath-components

`p# asserts that the array/column is parted (i.e. common values are adjacent)

It is explained here https://code.kx.com/q/ref/set-attribute/#grouped-and-parted

View solution in original post

SJT
Contributor III
Contributor III

A lot to unpack here but let’s break it down. 

.quantQ.IO.SAVEsPLAYtAB:{[tabPath;pvar;table]
  // tabPath -- path where to store the table
  // pvar -- variable to sort and index on for fast querying
  // table -- name of the table to save
  @[;pvar;`p#] pvar xasc (`sv (tabPath; ` ; table; `)) set .Q.en[tabPath] get table}

The three arguments are all symbols: respectively a file path, a table column name, and a table name. (It were best to state that in the comments.) I’ll come to how we see that.

Let’s set up a small table so we can watch what happens.

q)atable:([]ab:til 3;cd:`x`y`z;de:string`tom`**bleep**`harry)
q)get `atable
ab cd de
-------------
0 x "tom"
1 y "**bleep**"
2 z "harry"
q)tabPath:`:path/to/table

The argument to get is the name of the table; the result is the table, which becomes the argument to .Q.en[tabPath].

That is, binary .Q.en is projected onto tabPath to make a unary. The effect is the same as if we had assigned the result of get table to t and then called .Q.en[tabPath;t]. What .Q.en does is too extensive to discuss here but you can read up on it. The work it does is side effects; its result is the table. 

Which then gets passed as the right argument to set, which is quite a workhorse. Its syntax is overloaded, so we examine its left argument (` sv (tabPath; ` ; table; ` ) ) to see how set is used here. 

The sv keyword is being called in this form, which is how we know that tabPath and table must both be symbols. It returns a filehandle symbol as the left argument of set.

From this we see that set is being called with syntax til set y // serialize y to fil. It returns its left argument: the symbol filepath to which it has written the table; this becomes the right argument to xasc, which will sort the table on disc by the column named by pvar

The result is again a reference to the table, which becomes the argument to @[;pvar;`p#], a projection of Apply At onto second and third arguments pvar and `p#. The effect is to set the partitioned attribute on the column named by pvar. The result of Apply At is the table reference. Your code uses Explicit return : to make that the result of .quantQ.IO.SAVEsPLAYtAB. That is unnecessary: a function’s result is the result of evaluating its last (here the single) expression.

View solution in original post

eohara
New Contributor III
New Contributor III

Hi,

 

The : in :@ is an explicit return (see Function notation – Basics – kdb+ and q documentation - Kdb+ and q documentation (kx.com)).

 The @ is an amend operator (see Amend, Amend At – Reference – kdb+ and q documentation - Kdb+ and q documentation (kx.com)).

`p# is the parted attribute (see Tables | Q For Mortals | A textbook forkdb+ and the q programming language - Q for Mortals (kx.com)).

` is the first argument of sv here as we wish to build a filepath to a splayed table (see sv – scalar from vector | Reference | kdb+ and q documentation - Kdb+ and q documentation (kx.com)).

To bring that altogether, what is happening in your function is:

  1. get table
    1. This returns the table data for the given table name
  2. .Q.en[tabPath] get table

    1. This enumerates the output of 1) against a sym file in "tabPath", and returns the enumerated table
  3. (`  sv (tabPath; ` ; table; ` ) ) set .Q.en[tabPath] get table

    1. Writes the table from 2) to the filepath, and returns the filepath if successful e.g. if tabPath is `:/home/kx/db and table is `trade, then (`  sv (tabPath; ` ; table; ` ) ) will return `:/home/kx/db/trade/
      1. The trailing slash after trade means the table will be saved splayed
  4. pvar xasc (`  sv (tabPath; ` ; table; ` ) ) set .Q.en[tabPath] get table

    1. Sorts the written table in 3) on the column in the pvar variable
  5. :@[;pvar; `p#] pvar xasc (`  sv (tabPath; ` ; table; ` ) ) set .Q.en[tabPath] get table

    1. Amends the sorted table on disk, adding a parted attribute to the column in the pvar variable

 

View solution in original post

4 REPLIES 4

awilson
New Contributor
New Contributor

Code reads right to left

.Q.en[tabPath] get table

// Fetch table (defined by sym/hsym 'table') and enumerate any symbol columns (to sym file located in directory tabPath)

(` sv (tabPath; ` ; table; ` ) ) set ...

// Save the table (splayed) to a directory defined by '/'-separated path (tabPath/table/)

@[;pvar; `p#] pvar xasc ...

// Sort by pvar and apply the parted attribute to this column in the table

: ...

// Return the result (the path to the saved table)


` sv (scalar from vector) joins elements of a filepath.
e.g.
q)` sv (`:path/to/dir; `filename)
`:path/to/dir/filename
q)` sv (`:path/to/dir; `filename; `)
`:path/to/dir/filename/

It is explained here https://code.kx.com/q/ref/sv/#filepath-components

`p# asserts that the array/column is parted (i.e. common values are adjacent)

It is explained here https://code.kx.com/q/ref/set-attribute/#grouped-and-parted

SJT
Contributor III
Contributor III

A lot to unpack here but let’s break it down. 

.quantQ.IO.SAVEsPLAYtAB:{[tabPath;pvar;table]
  // tabPath -- path where to store the table
  // pvar -- variable to sort and index on for fast querying
  // table -- name of the table to save
  @[;pvar;`p#] pvar xasc (`sv (tabPath; ` ; table; `)) set .Q.en[tabPath] get table}

The three arguments are all symbols: respectively a file path, a table column name, and a table name. (It were best to state that in the comments.) I’ll come to how we see that.

Let’s set up a small table so we can watch what happens.

q)atable:([]ab:til 3;cd:`x`y`z;de:string`tom`**bleep**`harry)
q)get `atable
ab cd de
-------------
0 x "tom"
1 y "**bleep**"
2 z "harry"
q)tabPath:`:path/to/table

The argument to get is the name of the table; the result is the table, which becomes the argument to .Q.en[tabPath].

That is, binary .Q.en is projected onto tabPath to make a unary. The effect is the same as if we had assigned the result of get table to t and then called .Q.en[tabPath;t]. What .Q.en does is too extensive to discuss here but you can read up on it. The work it does is side effects; its result is the table. 

Which then gets passed as the right argument to set, which is quite a workhorse. Its syntax is overloaded, so we examine its left argument (` sv (tabPath; ` ; table; ` ) ) to see how set is used here. 

The sv keyword is being called in this form, which is how we know that tabPath and table must both be symbols. It returns a filehandle symbol as the left argument of set.

From this we see that set is being called with syntax til set y // serialize y to fil. It returns its left argument: the symbol filepath to which it has written the table; this becomes the right argument to xasc, which will sort the table on disc by the column named by pvar

The result is again a reference to the table, which becomes the argument to @[;pvar;`p#], a projection of Apply At onto second and third arguments pvar and `p#. The effect is to set the partitioned attribute on the column named by pvar. The result of Apply At is the table reference. Your code uses Explicit return : to make that the result of .quantQ.IO.SAVEsPLAYtAB. That is unnecessary: a function’s result is the result of evaluating its last (here the single) expression.

eohara
New Contributor III
New Contributor III

Hi,

 

The : in :@ is an explicit return (see Function notation – Basics – kdb+ and q documentation - Kdb+ and q documentation (kx.com)).

 The @ is an amend operator (see Amend, Amend At – Reference – kdb+ and q documentation - Kdb+ and q documentation (kx.com)).

`p# is the parted attribute (see Tables | Q For Mortals | A textbook forkdb+ and the q programming language - Q for Mortals (kx.com)).

` is the first argument of sv here as we wish to build a filepath to a splayed table (see sv – scalar from vector | Reference | kdb+ and q documentation - Kdb+ and q documentation (kx.com)).

To bring that altogether, what is happening in your function is:

  1. get table
    1. This returns the table data for the given table name
  2. .Q.en[tabPath] get table

    1. This enumerates the output of 1) against a sym file in "tabPath", and returns the enumerated table
  3. (`  sv (tabPath; ` ; table; ` ) ) set .Q.en[tabPath] get table

    1. Writes the table from 2) to the filepath, and returns the filepath if successful e.g. if tabPath is `:/home/kx/db and table is `trade, then (`  sv (tabPath; ` ; table; ` ) ) will return `:/home/kx/db/trade/
      1. The trailing slash after trade means the table will be saved splayed
  4. pvar xasc (`  sv (tabPath; ` ; table; ` ) ) set .Q.en[tabPath] get table

    1. Sorts the written table in 3) on the column in the pvar variable
  5. :@[;pvar; `p#] pvar xasc (`  sv (tabPath; ` ; table; ` ) ) set .Q.en[tabPath] get table

    1. Amends the sorted table on disk, adding a parted attribute to the column in the pvar variable

 

chan_chenyanj
New Contributor II

thanks