cancel
Showing results for 
Search instead for 
Did you mean: 

How to update/delete certain rows from a splayed table on disk?

Richard_Zwz
New Contributor II
Received: by 10.100.45.5 with SMTP id s5mr599989ans.10.1233983229998; Fri, 06 Feb 2009 21:07:09 -0800 (PST)Date: Fri, 6 Feb 2009 21:07:09 -0800 (PST)X-IP: 119.122.241.10User-Agent: G2/1.0X-Google-Token: 8ZEDiAwAAAANlbU470GZnuiHuoxQzDDbX-HTTP-UserAgent: Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.0; SLCC1; .NET CLR 2.0.50727; .NET CLR 3.0.04506; .NET CLR 3.5.21022; CIBA; .NET CLR 1.1.4322),gzip(gfe),gzip(gfe)Message-ID: Subject: How to update/delete certain rows from a splayed table on disk?From: "Richard Z." To: "Kdb+ Personal Developers" X-Google-Approved: charlie@kx.com via web at 2009-02-07 08:54:14For example,1) I builded a test table and saves(splays) it to disk:/start q.exeq) n:10000000; /increase n to increase the table sizeq) t:([]id:til n; c1:n?1000j; c2:n?1000j; c3:n?1000j; c4:n?1000j); /generate the test table tq) `:d:/qdata/t/ set t; /splays to directory d:/qdata/q , directorycan be changedq) \\ / exit q2)update certain rows:q d:/qdata/t /start q ,load directory d:/qdata/tq) update c1:0j, c2:0j, c3:0j, c4:0j from `t where id<5 /updatesome rowsq) `:d:/qdata/t/ set t /saves to diskIt seems that the q process will load all the data of t into memorybefore updating (I found the memory used by q process increased byabout table t's size) .My question is: how to update/delete the splayed table on disk withoutloading all rows into memory?Any ideas? Thanks in advance.Richard Z.
2 REPLIES 2

trieder
New Contributor
If you want to update just one column or one column basedon others, you can map the columns you'd like to modify in with get, change it/them and then flush back to disk set. When deleting data you have to read/modify/write every column to keep the vectors the same size so you may as well let q load it in for you. If for some reason memory is an issue (i.e. not partitioned), you can follow the same approach as the update, but in a loop for all columns. A huge splayed table that is not part'ed in some way is probably not recommended though.

Richard_Zwz
New Contributor II
Thanks!On Feb 8, 12:51�am, Tim Rieder wrote:> If you want to *update *just one column or one column based on others, you> can map the columns you'd like to modify in with get, change it/them and> then flush back to disk set. When *deleting *data you have to> read/modify/write every column to keep the vectors the same size so you may> as well let q load it in for you. If for some reason memory is an issue> (i.e. not partitioned), you can follow the same approach as the update, but> in a loop for all columns. A huge splayed table that is not part'ed in some> way is probably not recommended though.>>>> On Sat, Feb 7, 2009 at 12:07 AM, Richard Z. wrote:>> > For example,> > 1) I builded a test table and saves(splays) it to disk:>> > /start q.exe> > q) �n:10000000; � /increase n to increase the table size> > q) �t:([]id:til n; c1:n?1000j; c2:n?1000j; c3:n?1000j; c4:n?1000j); �/> > generate the test table t> > q) �`:d:/qdata/t/ set t; �/splays to directory d:/qdata/q , directory> > can be changed> > q) \\ � � / exit q>> > 2)update certain rows:>> > q d:/qdata/t � � � � /start q ,load directory d:/qdata/t> > q) update c1:0j, c2:0j, c3:0j, c4:0j from `t where id<5 � � � �/update> > some rows> > q) `:d:/qdata/t/ set t � � � � �/saves to disk>> > It seems that the q process will load all the data of t into memory> > before updating (I found the memory used by q process increased by> > about table t's size) .>> > My question is: how to update/delete the splayed table on disk without> > loading all rows into memory?>> > Any ideas? � Thanks in advance.>> > Richard Z.- Hide quoted text ->> - Show quoted text -