KX Community

erichards

Our access patterns are heavily based upon accessing one sym for a long history, rather than accessing everything for a given date. As such it would be massively more efficient for us to partition by sym rather than by date. Or even to maintain two H...

erichards

I am trying to come up with an efficient way to load data from disk, for multiple instruments over different date ranges. The actual problem we're trying to solve is loading rolled series for futures, forwards, and other derivatives. Say we want to l...

erichards

Thank you David, it is re-assuring to know that we are not following an anti-pattern or something too bespoke. With a small demo we have found a sym-partitioned db to be much more performant for our access patterns. I had not considered the partition...

erichards

Thank you - the syntax for inspecting the thread timings is very useful. If we had our data segmented over 2+ disks and were not using peach, surely we wouldn't utilise the available I/O throughput? Whereas if we were to use peach, with the thread nu...

erichards

Thank you for the prompt and thorough response. This is undoubtedly useful for loading various instruments with different ranges. In our specific example of rolled series there would only be one day of overlap between contracts. I am wondering if the...

KX Community

User Statistics

User Activity

HDB - map sym to int for partitioning

Loading data for combinations of instruments and date ranges (rolled series)

Re: HDB - map sym to int for partitioning

Re: Loading data for combinations of instruments and date ranges (rolled series)

Re: Loading data for combinations of instruments and date ranges (rolled series)