I'm receiving the message 'Not enough storage is available to process this command.' when attempting to load a large database with slave processes.
I'm attempting to use sub-processes to query a large database on the 32-bit version of kdb. I've modified buildhdb.q to us 5 segments. I've also tried it with 15 segments, which puts each segment slightly over 500M
I've started up each of the slaves on different ports and then in the master I execute
It looks like there's a difference in how segmented databases get initialized compared to partitioned databases. Running under strace, it looks like the segmented database tries to mmap each of the partitions. This will will exceed the address space with a sufficiently large db. When I strictly use partitions and no segments, k will read the table definition (.d file), but will mmap the file only when executing the query and then munmap later.
I have been able to create a hdb that is over 5gb with a partition per month. I still have the issue where I run into wsfull doing various calculations, but that's to be expected