cancel
Showing results for 
Search instead for 
Did you mean: 

insert into global table in peach

Xiaocheng
New Contributor

Suppose we have table tblAAPL, tblIBM already, can I insert data into each table by using peach to accelerate the process? If tables do not affect each other.

for example:

insert[; data] peach `tblAAPL`tblIBM

 

I tested, got noupdate error.

1 ACCEPTED SOLUTION

rocuinneagain
New Contributor III
New Contributor III

 

No, it is not possible to update a global variable from any thread other the main thread.

The 'noupdate' error is detailed on: https://code.kx.com/q/basics/errors/

Full list of blocked operations within peach: https://code.kx.com/q/ref/each/#blocked-within-peach

 

 

View solution in original post

6 REPLIES 6

rocuinneagain
New Contributor III
New Contributor III

 

No, it is not possible to update a global variable from any thread other the main thread.

The 'noupdate' error is detailed on: https://code.kx.com/q/basics/errors/

Full list of blocked operations within peach: https://code.kx.com/q/ref/each/#blocked-within-peach

 

 

View solution in original post

sstantoncook
New Contributor II
New Contributor II

What is the use case that you want to insert the same data into two different tables?

I can see the topic names resemble stocks but there is no filter on the data so would appear all records go into each.

 

The only use case I can think of having the same schema and upserting to two tables is where you maintain two versions the same schema but one with keyed columns to maintain both the full trail and current "state".

E.g. all FX quotes in one table and the other keyed by Currency Pair and LP to display the current quoted depth.

Cheers,

Sam

darrenwsun
New Contributor

Is it the bottleneck that needs optimization? insert is quite cheap.

The q insert is highly optimised, it can handle millions of rows a second per core.

The insert must be performed in the main thread to ensure data consistency and sequencing. This is why it cannot by run with peach, even though in this example it is appending to different tables.

What use case is there to insert identical data into two tables? There may be a more appropriate process architecture to manage these tables or mechanism to achieve the same result.

Thanks. insert is a simple example here. Complex case might be, I need to apply sort rule to each table of tickers. Then, sort one by one is not efficient and tables will not affect each other, so peach can be helpful. I tried, but found cannot update the global tables in peach.

If that's what you really have to do, one way is to split those tables into multiple instances and use .z.pd (slave processes) for parallel computation. This adds to the complexity though so it's not necessarily worth the cost.