cancel
Showing results for 
Search instead for 
Did you mean: 

Each Parallel issue

k-mambo
New Contributor III

Follows https://code.kx.com/q/basics/peach/,

The result of m':[x] is exactly the same as m'[x]. If no secondary tasks are available, performance is the same as well.

but m':[x] more fast and less memory used. Is not m'[x] parallel execuation??...

1 ACCEPTED SOLUTION

rocuinneagain
Valued Contributor
Valued Contributor

m'[x] is not parallel execution

 

There can be cases where the overhead of using multiple threads can make execution slower than single threaded. (.i.e performing many small operations)

.Q.fc is available to help in some of these cases.

https://code.kx.com/q/ref/dotq/#qfc-parallel-on-cut

 

If you measure memory usage with \ts note that it only sees usage in main thread. It does not sum usage from all threads. So you cannot compare results directly.

 

 

q)\ts {til 10000000;x}'[til 1000] // ts shows real memory usage
5395 134250848
q)\ts {til 10000000;x}':[til 1000] // ts only sees memory usage in main thread - not the sum of threads
5612 33408

 

 

 

q)\ts {x}'[til 100000] // Single core
8 4746288
q)\ts {x}':[til 100000] // Slower due to parallel overhead
13 4194864
q)\ts .Q.fc[{x}][til 100000] //Faster way to use multiple cores
1 3146512

 

 

View solution in original post

1 REPLY 1

rocuinneagain
Valued Contributor
Valued Contributor

m'[x] is not parallel execution

 

There can be cases where the overhead of using multiple threads can make execution slower than single threaded. (.i.e performing many small operations)

.Q.fc is available to help in some of these cases.

https://code.kx.com/q/ref/dotq/#qfc-parallel-on-cut

 

If you measure memory usage with \ts note that it only sees usage in main thread. It does not sum usage from all threads. So you cannot compare results directly.

 

 

q)\ts {til 10000000;x}'[til 1000] // ts shows real memory usage
5395 134250848
q)\ts {til 10000000;x}':[til 1000] // ts only sees memory usage in main thread - not the sum of threads
5612 33408

 

 

 

q)\ts {x}'[til 100000] // Single core
8 4746288
q)\ts {x}':[til 100000] // Slower due to parallel overhead
13 4194864
q)\ts .Q.fc[{x}][til 100000] //Faster way to use multiple cores
1 3146512