Question about query that stuck the KDB process

[carfield]

HI all

We’ve run below query

select minute, date, sums(size)/sum(size) from select sum(size) by 1 xbar time.minute, date from table where filtering

We understand we actually should use “%” instead of “/” for divide, but I believe it just return something not correct or throw an error. However, somehow this query hang the process and even click “SIGINT interrupt signal” it doesn’t interrupt, will you have any idea about why?

Regards,

Carfield

[SJT · Accepted solution]

Looks like you have already spotted part of the problem. The / does not denote Divide % but the iteration operator Over. Depending on the values in the size column, you might have inadvertently specified a Converge iteration, and one that might not actually converge. Iterations defined this way run very tight and can be difficult to interrupt.

The result column you want is perhaps sums[size]%sum[size], or, avoiding computing both sum and sums,
.[%]1 last\sums size.

q)show size:5?10
6 6 1 8 5
q)sums[size]%sum[size]
0.2307692 0.4615385 0.5 0.8076923 1
q).[%]1 last\sums size
0.2307692 0.4615385 0.5 0.8076923 1

The use above of 1 last\ is an example of the Zen monks idiom.  

[rocuinneagain · Accepted solution]

https://code.kx.com/q4m3/6_Functions/#677-more-over-iteration

The final overload of / is equivalent to a “while” loop in imperative programming. It provides a declarative way to specify a test to end the iteration. Thinking functionally, we provide a predicate function that is applied to the result at each step. The iteration continues as long as the predicate result is 1b and stops otherwise. For example, we stop the computation of the Fibonacci sequence once it exceeds 1000 as follows.

q)fib:{x,sum -2#x}
q)fib/[{1000>last x}; 1 1]
1 1 2 3 5 8 13 21 34 55 89 144 233 377 610 987 1597

Your code accidentally boiled down to an infinite loop as the result never moves to 0b

1 1/[sums;1]

/Each iteration it runs
1 1[1] / Which returns 1
/Then
sums 1
/This returns 1
/This is a non zero value which gets treated as 1b so the loop starts again
/Infinite loop

[SJT · Accepted solution]

Looks like you have already spotted part of the problem. The / does not denote Divide % but the iteration operator Over. Depending on the values in the size column, you might have inadvertently specified a Converge iteration, and one that might not actually converge. Iterations defined this way run very tight and can be difficult to interrupt.

The result column you want is perhaps sums[size]%sum[size], or, avoiding computing both sum and sums,
.[%]1 last\sums size.

q)show size:5?10
6 6 1 8 5
q)sums[size]%sum[size]
0.2307692 0.4615385 0.5 0.8076923 1
q).[%]1 last\sums size
0.2307692 0.4615385 0.5 0.8076923 1

The use above of 1 last\ is an example of the Zen monks idiom.  

[carfield]

Thanks a lot, not sure if you have more idea about why this is difficult to interrupt?

[rocuinneagain · Accepted solution]

https://code.kx.com/q4m3/6_Functions/#677-more-over-iteration

The final overload of / is equivalent to a “while” loop in imperative programming. It provides a declarative way to specify a test to end the iteration. Thinking functionally, we provide a predicate function that is applied to the result at each step. The iteration continues as long as the predicate result is 1b and stops otherwise. For example, we stop the computation of the Fibonacci sequence once it exceeds 1000 as follows.

q)fib:{x,sum -2#x}
q)fib/[{1000>last x}; 1 1]
1 1 2 3 5 8 13 21 34 55 89 144 233 377 610 987 1597

Your code accidentally boiled down to an infinite loop as the result never moves to 0b

1 1/[sums;1]

/Each iteration it runs
1 1[1] / Which returns 1
/Then
sums 1
/This returns 1
/This is a non zero value which gets treated as 1b so the loop starts again
/Infinite loop

[rocuinneagain · Accepted solution]

Using parse helps to see what is happening.

Parsing a simplified version of your query helped to show the order of execution in the parse tree:

q)-1 .Q.s1 last value last parse "select sums(size)/sum(size) from tab";
((/;`size);+\;(sum;`size))

The q equivalent can then be shown to have the same parse tree

q)-1 .Q.s1 parse"size/[sums;sum size]";
((/;`size);+\;(sum;`size))