Limitations On Non-Commercial Version

[Mike_Thompson]

I'd like to confirm the limitations of the non-commercial version onwindows.>From what I can see: 1. There's no slaves 2. Work space size is limited to 2GigI've tried starting like this: q.exe -s4 -w4096But this seems to make no difference. For a start, the applicationblows up (suddenly disappears) as the 2Gig size is reached (slightlyless), so the command line setting given above does not seems to havean effect (test machine has 4Gig of Ram).Also, using 'peach' does not seem to improve runtime speed asexpected, so I'm assuming there's no slaves. I compared theperformance of this: {sum exp x?1.0} peach 2#50000000to the 'each' version. The peach version ran about 20% faster but Icould only see activity on one CPU core. I think I should have seen a,say, 3.5 times performance improvement if four slaves were on the job.Are there any other significant limitations, other those associatedwith 32bit (file size < 4Gig)?Also, on other matters completely, I'm interested to experiment withthe 'partioning' of tables. Is there any documentation available onhow to do that?

[Mike_Thompson]

>> Also, using 'peach' does not seem to improve runtime speed as> expected, so I'm assuming there's no slaves. I compared the> performance of this:> {sum exp x?1.0} peach 2#50000000Sorry, that was a bad test because only a maximum of two CPUs could beused.This is better: {sum exp x?1.0} peach 4#50000000But 'peach' is only 25% faster than 'each'. I wouldn't have thoughtthis micro benchmark was dominated by IPC overhead.Anyway, so peach/slaves question is answered. The other questionsremain.

[niall_dalton]

> This is better:>> � � � {sum exp x?1.0} peach 4#50000000>> But 'peach' is only 25% faster than 'each'. �I wouldn't have thought> this micro benchmark was dominated by IPC overhead.Hmm.. on my box:one threadq)\t {sum exp x?1.0} peach 4#500000009205q)\t {sum exp x?1.0} peach 4#500000008826q)\\four threadsq)\t {sum exp x?1.0} peach 4#500000002897q)\t {sum exp x?1.0} peach 4#500000002452can you tell me what hardware you are running on?

[Mike_Thompson]

On Apr 10, 5:12 am, niall wrote:> > This is better:>> > {sum exp x?1.0} peach 4#50000000>> > But 'peach' is only 25% faster than 'each'. I wouldn't have thought> > this micro benchmark was dominated by IPC overhead.>> Hmm.. on my box:>> one thread> q)\t {sum exp x?1.0} peach 4#50000000> 9205> q)\t {sum exp x?1.0} peach 4#50000000> 8826> q)\\>> four threads> q)\t {sum exp x?1.0} peach 4#50000000> 2897> q)\t {sum exp x?1.0} peach 4#50000000> 2452>> can you tell me what hardware you are running on?Dell Precision 390 running Windows XP, about a year old. CPU isIntel Core 2 Extreme QX6700. Recently upgraded to 4Gig of Ram.But I'm seeing very odd results, first with 4 slaves, look at whathappens switchng between 'each' and 'peach' ...D:\Qtest>q.exe -s 4KDB+ 2.4 2008.03.31 Copyright (C) 1993-2008 Kx Systemsw32/ 4()core 2047MB ......q)\t {sum exp x?1.0} each 4#5000000 / each1359q)\t {sum exp x?1.0} each 4#5000000 / again each1343q)\t {sum exp x?1.0} peach 4#5000000 / now peach4750q)\t {sum exp x?1.0} peach 4#5000000 /again peach4828q)peach is taking about four times longer than each!!!!!Then with no slaves ...D:\Qtest>q.exeKDB+ 2.4 2008.03.31 Copyright (C) 1993-2008 Kx Systemsw32/ 4()core 2047MB Mike ...q)\t {sum exp x?1.0} each 4#50000001343q)\t {sum exp x?1.0} each 4#50000001328q)\t {sum exp x?1.0} peach 4#50000001015q)\t {sum exp x?1.0} peach 4#50000001015q)Now 'peach' runs 30% fast, despite there being only one thread.--Mike

[simon]

no, that's not correcttry starting it withq -s 4 -p 4096(note spaces)and retry your peach examplewhether you have 2G or 3G depends on how you have windows setup(google the /3G flag) but that limit is a feature of 32bit OSswhen the address space limit is reached you'll get a wsfull error andq will exitpartitions, see the "abridged kdb+ manual" http://kx.com/q/d/kdb+.htmOn Apr 8, 10:17�pm, Mike Thompson wrote:> I'd like to confirm the limitations of the non-commercial version on> windows.>> From what I can see:> � �1. �There's no slaves> � �2. �Work space size is limited to 2Gig>> I've tried starting like this:> � �q.exe -s4 -w4096>> But this seems to make no difference. �For a start, the application> blows up (suddenly disappears) as the 2Gig size is reached (slightly> less), so the command line setting given above does not seems to have> an effect (test machine has 4Gig of Ram).>> Also, using 'peach' does not seem to improve runtime speed as> expected, so I'm assuming there's no slaves. �I compared the> performance of this:> � � {sum exp x?1.0} peach 2#50000000>> to the 'each' version. �The peach version ran about 20% faster but I> could only see activity on one CPU core. I think I should have seen a,> say, 3.5 times performance improvement if four slaves were on the job.>> Are there any other significant limitations, other those associated> with 32bit (file size < 4Gig)?>> Also, on other matters completely, I'm interested to experiment with> the 'partioning' of tables. �Is there any documentation available on> how to do that?

[simon]

sorry, misread your q parameters (hence the -p 4096 reference insteadof -w 4096)-w 4096 doesn't increase the amount of memory available - you'd use itif you wanted to restrict a task to _less_ memory than is availableOn Apr 9, 9:07�am, simon wrote:> no, that's not correct> try starting it with> q -s 4 -p 4096> (note spaces)> and retry your peach example>> whether you have 2G or 3G depends on how you have windows setup> (google the /3G flag) but that limit is a feature of 32bit OSs> when the address space limit is reached you'll get a wsfull error and> q will exit>> partitions, see the "abridged kdb+ manual"http://kx.com/q/d/kdb+.htm>> On Apr 8, 10:17�pm, Mike Thompson wrote:>> > I'd like to confirm the limitations of the non-commercial version on> > windows.>> > From what I can see:> > � �1. �There's no slaves> > � �2. �Work space size is limited to 2Gig>> > I've tried starting like this:> > � �q.exe -s4 -w4096>> > But this seems to make no difference. �For a start, the application> > blows up (suddenly disappears) as the 2Gig size is reached (slightly> > less), so the command line setting given above does not seems to have> > an effect (test machine has 4Gig of Ram).>> > Also, using 'peach' does not seem to improve runtime speed as> > expected, so I'm assuming there's no slaves. �I compared the> > performance of this:> > � � {sum exp x?1.0} peach 2#50000000>> > to the 'each' version. �The peach version ran about 20% faster but I> > could only see activity on one CPU core. I think I should have seen a,> > say, 3.5 times performance improvement if four slaves were on the job.>> > Are there any other significant limitations, other those associated> > with 32bit (file size < 4Gig)?>> > Also, on other matters completely, I'm interested to experiment with> > the 'partioning' of tables. �Is there any documentation available on> > how to do that?