unexpected behavior for div

[Andy]

q)floor 15%2.5

6

q)15 div 2.5

5

seems like kdb is casting the rhs to the type of lhs before doing the divide? this also affects xbar

q)1.1 xbar 5

5.5

[Kimmo_Linna]

Hi Andy,

According to documentation div is for integer division. I assume 15 div 2.5 equals 15 div 3. If you cast 2.5 to integer you will get 3. If you try 6h$2.5 q returns 3i.

Best regards,

Kimmo

--

Kimmo Linna
Nihtisalontie 3 as 1

02630  ESPOO
kimmo.linna@gmail.com
+358 40 590 1074

From: Andy
Sent: maanantai 5. lokakuuta 2020 17.31
To: Kdb+ Personal Developers
Subject: [personal kdb+] unexpected behavior for div

q)floor 15%2.5

6

q)15 div 2.5

5

seems like kdb is casting the rhs to the type of lhs before doing the divide? this also affects xbar

q)1.1 xbar 5

5.5

--
You received this message because you are subscribed to the Google Groups "Kdb+ Personal Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to personal-kdbplus+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/personal-kdbplus/41cb1970-101e-4249-9f24-b691f6492711n%40googlegro....

[Andy]

this is not true. integer division does not mean that the input need to be integer. 

in fact, on the reference page they have examples of using div on non integers such as

q)-7 7 div/:\:-2.5 -2 2 2.5 
2 3 -4 -3 

-3 -4 3 2 

[Kimmo_Linna]

Hi Andy,

What do you mean? Which is not true?

div is for integer division. https://code.kx.com/q/ref/div/

q)-7 7 div/:\:-2.5 -2 2 2.5

2  3  -4 -3

-3 -4 3  2

q)-7 7 div/:\:-3 -2 2 3

2  3  -4 -3

-3 -4 3  2

If your input is something else it is cast for integer for you.

Xbar is working also according documentation. https://code.kx.com/q/ref/xbar/

Best regards,

Kimmo

--

Kimmo Linna
Nihtisalontie 3 as 1

02630  ESPOO
kimmo.linna@gmail.com
+358 40 590 1074

From: Andy
Sent: maanantai 5. lokakuuta 2020 20.41
To: Kdb+ Personal Developers
Subject: Re: [personal kdb+] unexpected behavior for div

this is not true. integer division does not mean that the input need to be integer. 

in fact, on the reference page they have examples of using div on non integers such as

q)-7 7 div/:\:-2.5 -2 2 2.5 
2 3 -4 -3 

-3 -4 3 2 

On Monday, October 5, 2020 at 12:58:26 PM UTC-4 Kimmo Linna wrote:

Hi Andy,

According to documentation div is for integer division. I assume 15 div 2.5 equals 15 div 3. If you cast 2.5 to integer you will get 3. If you try 6h$2.5 q returns 3i.

Best regards,

Kimmo

--

Kimmo Linna
Nihtisalontie 3 as 1

02630  ESPOO
kimmo...@gmail.com
+358 40 590 1074

From: Andy
Sent: maanantai 5. lokakuuta 2020 17.31
To: Kdb+ Personal Developers
Subject: [personal kdb+] unexpected behavior for div

q)floor 15%2.5

6

q)15 div 2.5

5

seems like kdb is casting the rhs to the type of lhs before doing the divide? this also affects xbar

q)1.1 xbar 5

5.5

--
You received this message because you are subscribed to the Google Groups "Kdb+ Personal Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to personal-kdbpl...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/personal-kdbplus/41cb1970-101e-4249-9f24-b691f6492711n%40googlegro....

--
You received this message because you are subscribed to the Google Groups "Kdb+ Personal Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to personal-kdbplus+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/personal-kdbplus/8a019342-ca91-4ff0-a780-05e6246c6e16n%40googlegro....

[Andy]

xbar is working? so you're saying that

the result of 1.1 xbar 5  is expected to be 5.5? that's ridiculous.

the kdb documentation is very clear, it says "returns y rounded down to the nearest multiple of x."

[Kimmo_Linna]

Hi Andy,

Xbar is working according to documentation.

1. xbar 5 should give you 5.5

And

q)1.1 xbar til 16

0 1.1 2.2 3.3 4.4 5.5 6.6 7.7 8.8 9.9 11 12.1 13.2 14.3 15.4 16.5

Compared to documentation

q)2.5 xbar til 16

0 0 0 2.5 2.5 2.5 5 5 5 7.5 7.5 7.5 10 10 10 12.5

Best regards,

Kimmo

--

Kimmo Linna
Nihtisalontie 3 as 1

02630  ESPOO
kimmo.linna@gmail.com
+358 40 590 1074

From: Andy
Sent: maanantai 5. lokakuuta 2020 21.12
To: Kdb+ Personal Developers
Subject: Re: [personal kdb+] unexpected behavior for div

xbar is working? so you're saying that

the result of 1.1 xbar 5  is expected to be 5.5? that's ridiculous.

the kdb documentation is very clear, it says "returns y rounded down to the nearest multiple of x."

On Monday, October 5, 2020 at 1:56:37 PM UTC-4 Kimmo Linna wrote:

Hi Andy,

What do you mean? Which is not true?

div is for integer division. https://code.kx.com/q/ref/div/

q)-7 7 div/:\:-2.5 -2 2 2.5

2  3  -4 -3

-3 -4 3  2

q)-7 7 div/:\:-3 -2 2 3

2  3  -4 -3

-3 -4 3  2

If your input is something else it is cast for integer for you.

Xbar is working also according documentation. https://code.kx.com/q/ref/xbar/

Best regards,

Kimmo

--

Kimmo Linna
Nihtisalontie 3 as 1

02630  ESPOO
kimmo...@gmail.com
+358 40 590 1074

From: Andy
Sent: maanantai 5. lokakuuta 2020 20.41
To: Kdb+ Personal Developers
Subject: Re: [personal kdb+] unexpected behavior for div

this is not true. integer division does not mean that the input need to be integer. 

in fact, on the reference page they have examples of using div on non integers such as

q)-7 7 div/:\:-2.5 -2 2 2.5 
2 3 -4 -3 

-3 -4 3 2 

On Monday, October 5, 2020 at 12:58:26 PM UTC-4 Kimmo Linna wrote:

Hi Andy,

According to documentation div is for integer division. I assume 15 div 2.5 equals 15 div 3. If you cast 2.5 to integer you will get 3. If you try 6h$2.5 q returns 3i.

Best regards,

Kimmo

--

Kimmo Linna
Nihtisalontie 3 as 1

02630  ESPOO
kimmo...@gmail.com
+358 40 590 1074

From: Andy
Sent: maanantai 5. lokakuuta 2020 17.31
To: Kdb+ Personal Developers
Subject: [personal kdb+] unexpected behavior for div

q)floor 15%2.5

6

q)15 div 2.5

5

seems like kdb is casting the rhs to the type of lhs before doing the divide? this also affects xbar

q)1.1 xbar 5

5.5

--
You received this message because you are subscribed to the Google Groups "Kdb+ Personal Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to personal-kdbpl...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/personal-kdbplus/41cb1970-101e-4249-9f24-b691f6492711n%40googlegro....

--
You received this message because you are subscribed to the Google Groups "Kdb+ Personal Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to personal-kdbpl...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/personal-kdbplus/8a019342-ca91-4ff0-a780-05e6246c6e16n%40googlegro....

--
You received this message because you are subscribed to the Google Groups "Kdb+ Personal Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to personal-kdbplus+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/personal-kdbplus/17da8baf-1641-4a9b-8ede-498801a8fd36n%40googlegro....

[Andy]

i'm aware that the example in the documentation matches the behavior. but you're missing the point. no reasonable kdb developer would expect this behavior.

the expected output of of 1.1 xbar 5 is 4. the purpose of my post is not to argue with some stranger on the internet, it's to alert kx of a possible bug.

[Kimmo_Linna]

Hi Andy,

If I have understood right one of the main purpose of xbar is to use it with dates. To get proper intervals for dates and times. Thus one could say that no reasonable kdb developer would use decimals with xbar.

If you really want to look the logic of xbar for decimals you have to look a longer series. For me the following is quite consistent series but I’m just a student, not a developer.

1. xbar 5            0
2. xbar 5            0
3. xbar 5            0
4. xbar 5            0
5. xbar 5            2.5
6. xbar 5            3.0
7. xbar 5            3.5
8. xbar 5            4.0
9. xbar 5            4.5
0. xbar 5            5
1. xbar 5            5.5
2. xbar 5            6.0
3. xbar 5            6.5  

1.4 xbar 5            7.0

1.5 xbar 5            3

1.6 xbar 5            3.2

1.7 xbar 5            3.4

1.8 xbar 5            3.6

1.9 xbar 5            3.8

2.0 xbar 5            4.0

2.1 xbar 5            4.2

2.2 xbar 5            4.4

2.3 xbar 5            4.6

2.4 xbar 5            4.8

2.5 xbar 5            5.0

2.6 xbar 5            2.6

2.7 xbar 5            2.7

2.8 xbar 5            2.8

2.9 xbar 5            2.9

3.0 xbar 5            3.0

3.1 xbar 5            3.1

3.2 xbar 5            3.2

3.3 xbar 5            3.3

3.4 xbar 5            3.4

3.5 xbar 5            3.5

3.6 xbar 5            3.6

3.7 xbar 5            3.7

…

4.8 xbar 5            4.8

4.9 xbar 5            4.9

5.0 xbar 5            5

Best regards,

Kimmo

--

Kimmo Linna
Nihtisalontie 3 as 1

02630  ESPOO
kimmo.linna@gmail.com
+358 40 xbar 590 1074

From: Andy
Sent: maanantai xbar 5. lokakuuta 2020 21.45
To: Kdb+ Personal Developers
Subject: Re: [personal kdb+] unexpected behavior for div

i'm aware that the example in the documentation matches the behavior. but you're missing the point. no reasonable kdb developer would expect this behavior.

the expected output of of 1.1 xbar xbar 5 is 4. the purpose of my post is not to argue with some stranger on the internet, it's to alert kx of a possible bug.

On Monday, October xbar 5, 2020 at 2:37:50 PM UTC-4 Kimmo Linna wrote:

Hi Andy,

Xbar is working according to documentation.

1. xbar xbar 5 should give you xbar 5.5

And

q)1.1 xbar til 16

0 1.1 2.2 3.3 4.4 xbar 5.5 6.6 7.7 8.8 9.9 11 12.1 13.2 14.3 15.4 16.5

Compared to documentation

q)2.5 xbar til 16

0 0 0 2.5 2.5 2.5 xbar 5 xbar 5 xbar 5 7.5 7.5 7.5 10 10 10 12.5

Best regards,

Kimmo

--

Kimmo Linna
Nihtisalontie 3 as 1

02630  ESPOO
kimmo...@gmail.com
+358 40 xbar 590 1074

From: Andy
Sent: maanantai xbar 5. lokakuuta 2020 21.12
To: Kdb+ Personal Developers
Subject: Re: [personal kdb+] unexpected behavior for div

xbar is working? so you're saying that

the result of 1.1 xbar xbar 5  is expected to be xbar 5.5? that's ridiculous.

the kdb documentation is very clear, it says "returns y rounded down to the nearest multiple of x."

On Monday, October xbar 5, 2020 at 1:56:37 PM UTC-4 Kimmo Linna wrote:

Hi Andy,

What do you mean? Which is not true?

div is for integer division. https://code.kx.com/q/ref/div/

q)-7 7 div/:\:-2.5 -2 2 2.5

2  3  -4 -3

-3 -4 3  2

q)-7 7 div/:\:-3 -2 2 3

2  3  -4 -3

-3 -4 3  2

If your input is something else it is cast for integer for you.

Xbar is working also according documentation. https://code.kx.com/q/ref/xbar/

Best regards,

Kimmo

--

Kimmo Linna
Nihtisalontie 3 as 1

02630  ESPOO
kimmo...@gmail.com
+358 40 xbar 590 1074

From: Andy
Sent: maanantai xbar 5. lokakuuta 2020 20.41
To: Kdb+ Personal Developers
Subject: Re: [personal kdb+] unexpected behavior for div

this is not true. integer division does not mean that the input need to be integer. 

in fact, on the reference page they have examples of using div on non integers such as

q)-7 7 div/:\:-2.5 -2 2 2.5 
2 3 -4 -3 

-3 -4 3 2 

On Monday, October xbar 5, 2020 at 12:58:26 PM UTC-4 Kimmo Linna wrote:

Hi Andy,

According to documentation div is for integer division. I assume 15 div 2.5 equals 15 div 3. If you cast 2.5 to integer you will get 3. If you try 6h$2.5 q returns 3i.

Best regards,

Kimmo

--

Kimmo Linna
Nihtisalontie 3 as 1

02630  ESPOO
kimmo...@gmail.com
+358 40 xbar 590 1074

From: Andy
Sent: maanantai xbar 5. lokakuuta 2020 17.31
To: Kdb+ Personal Developers
Subject: [personal kdb+] unexpected behavior for div

q)floor 15%2.5

6

q)15 div 2.5

5

seems like kdb is casting the rhs to the type of lhs before doing the divide? this also affects xbar

q)1.1 xbar xbar 5

5.5

--
You received this message because you are subscribed to the Google Groups "Kdb+ Personal Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to personal-kdbpl...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/personal-kdbplus/41cb1970-101e-4249-9f24-b691f6492711n%40googlegro....

--
You received this message because you are subscribed to the Google Groups "Kdb+ Personal Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to personal-kdbpl...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/personal-kdbplus/8a019342-ca91-4ff0-a780-05e6246c6e16n%40googlegro....

--
You received this message because you are subscribed to the Google Groups "Kdb+ Personal Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to personal-kdbpl...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/personal-kdbplus/17da8baf-1641-4a9b-8ede-498801a8fd36n%40googlegro....

--
You received this message because you are subscribed to the Google Groups "Kdb+ Personal Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to personal-kdbplus+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/personal-kdbplus/c20b60a5-4c56-42e5-8201-d7f611e17c0en%40googlegro....

[DevonMcC]

I was going to agree with Kimmo that xbar works as documented as I have my own rounding function in J that works in much the same fashion but then I noticed this:

q)0.05 xbar 0 1 2 3 +/:\: 0.04 0.05 0.06   # Round prices to nearest nickel

0 0.05 0.05

1 1.05 1.05

2 2    2.05

3 3    3.05

It's unclear why 1.05<->0.05 xbar 1.05 but 2<->0.05 xbar 2.05 but I suspect floating point imprecision.

However, the case of 5.5<->1.1 xbar 5.5 seems fine since (the integer 5)*1.1->5.5.

[Kimmo_Linna]

Hi Devon,

That’s true and it’s strange that after 7.05 xbar returns n.05.

Best regards,

Kimmo

--

Kimmo Linna
Nihtisalontie 3 as 1

02630  ESPOO
kimmo.linna@gmail.com
+358 40 590 1074

From: Devon McCormick
Sent: tiistai 6. lokakuuta 2020 10.44
To: personal-kdbplus@googlegroups.com
Subject: Re: [personal kdb+] unexpected behavior for div

I was going to agree with Kimmo that xbar works as documented as I have my own rounding function in J that works in much the same fashion but then I noticed this:

q)0.05 xbar 0 1 2 3 +/:\: 0.04 0.05 0.06    # Round prices to nearest nickel

0 0.05 0.05

1 1.05 1.05

2 2    2.05

3 3    3.05

It's unclear why 1.05<->0.05 xbar 1.05 but 2<->0.05 xbar 2.05 but I suspect floating point imprecision.

However, the case of 5.5<->1.1 xbar 5.5 seems fine since (the integer 5)*1.1->5.5.

On Mon, Oct 5, 2020 at 3:50 PM Kimmo Linna <kimmo.linna@gmail.com> wrote:

Hi Andy,

If I have understood right one of the main purpose of xbar is to use it with dates. To get proper intervals for dates and times. Thus one could say that no reasonable kdb developer would use decimals with xbar.

If you really want to look the logic of xbar for decimals you have to look a longer series. For me the following is quite consistent series but I’m just a student, not a developer.

1. xbar 5            0
2. xbar 5            0
3. xbar 5            0
4. xbar 5            0
5. xbar 5            2.5
6. xbar 5            3.0
7. xbar 5            3.5
8. xbar 5            4.0
9. xbar 5            4.5

2. xbar 5            5

1. xbar 5            5.5
2. xbar 5            6.0
3. xbar 5            6.5  

1.4 xbar 5            7.0

1.5 xbar 5            3

1.6 xbar 5            3.2

1.7 xbar 5            3.4

1.8 xbar 5            3.6

1.9 xbar 5            3.8

2.0 xbar 5            4.0

2.1 xbar 5            4.2

2.2 xbar 5            4.4

2.3 xbar 5            4.6

2.4 xbar 5            4.8

2.5 xbar 5            5.0

2.6 xbar 5            2.6

2.7 xbar 5            2.7

2.8 xbar 5            2.8

2.9 xbar 5            2.9

3.0 xbar 5            3.0

3.1 xbar 5            3.1

3.2 xbar 5            3.2

3.3 xbar 5            3.3

3.4 xbar 5            3.4

3.5 xbar 5            3.5

3.6 xbar 5            3.6

3.7 xbar 5            3.7

…

4.8 xbar 5            4.8

4.9 xbar 5            4.9

5.0 xbar 5            5

Best regards,

Kimmo

--

Kimmo Linna
Nihtisalontie 3 as 1

02630  ESPOO
kimmo.linna@gmail.com
+358 40 xbar 590 1074

From: Andy
Sent: maanantai xbar 5. lokakuuta 2020 21.45
To: Kdb+ Personal Developers
Subject: Re: [personal kdb+] unexpected behavior for div

i'm aware that the example in the documentation matches the behavior. but you're missing the point. no reasonable kdb developer would expect this behavior.

the expected output of of 1.1 xbar xbar 5 is 4. the purpose of my post is not to argue with some stranger on the internet, it's to alert kx of a possible bug.

On Monday, October xbar 5, 2020 at 2:37:50 PM UTC-4 Kimmo Linna wrote:

Hi Andy,

Xbar is working according to documentation.

1. xbar xbar 5 should give you xbar 5.5

And

q)1.1 xbar til 16

0 1.1 2.2 3.3 4.4 xbar 5.5 6.6 7.7 8.8 9.9 11 12.1 13.2 14.3 15.4 16.5

Compared to documentation

q)2.5 xbar til 16

0 0 0 2.5 2.5 2.5 xbar 5 xbar 5 xbar 5 7.5 7.5 7.5 10 10 10 12.5

Best regards,

Kimmo

--

Kimmo Linna
Nihtisalontie 3 as 1

02630  ESPOO
kimmo...@gmail.com
+358 40 xbar 590 1074

From: Andy
Sent: maanantai xbar 5. lokakuuta 2020 21.12
To: Kdb+ Personal Developers
Subject: Re: [personal kdb+] unexpected behavior for div

xbar is working? so you're saying that

the result of 1.1 xbar xbar 5  is expected to be xbar 5.5? that's ridiculous.

the kdb documentation is very clear, it says "returns y rounded down to the nearest multiple of x."

On Monday, October xbar 5, 2020 at 1:56:37 PM UTC-4 Kimmo Linna wrote:

Hi Andy,

What do you mean? Which is not true?

div is for integer division. https://code.kx.com/q/ref/div/

q)-7 7 div/:\:-2.5 -2 2 2.5

2  3  -4 -3

-3 -4 3  2

q)-7 7 div/:\:-3 -2 2 3

2  3  -4 -3

-3 -4 3  2

If your input is something else it is cast for integer for you.

Xbar is working also according documentation. https://code.kx.com/q/ref/xbar/

Best regards,

Kimmo

--

Kimmo Linna
Nihtisalontie 3 as 1

02630  ESPOO
kimmo...@gmail.com
+358 40 xbar 590 1074

From: Andy
Sent: maanantai xbar 5. lokakuuta 2020 20.41
To: Kdb+ Personal Developers
Subject: Re: [personal kdb+] unexpected behavior for div

this is not true. integer division does not mean that the input need to be integer. 

in fact, on the reference page they have examples of using div on non integers such as

q)-7 7 div/:\:-2.5 -2 2 2.5 
2 3 -4 -3 

-3 -4 3 2 

On Monday, October xbar 5, 2020 at 12:58:26 PM UTC-4 Kimmo Linna wrote:

Hi Andy,

According to documentation div is for integer division. I assume 15 div 2.5 equals 15 div 3. If you cast 2.5 to integer you will get 3. If you try 6h$2.5 q returns 3i.

Best regards,

Kimmo

--

Kimmo Linna
Nihtisalontie 3 as 1

02630  ESPOO
kimmo...@gmail.com
+358 40 xbar 590 1074

From: Andy
Sent: maanantai xbar 5. lokakuuta 2020 17.31
To: Kdb+ Personal Developers
Subject: [personal kdb+] unexpected behavior for div

q)floor 15%2.5

6

q)15 div 2.5

5

seems like kdb is casting the rhs to the type of lhs before doing the divide? this also affects xbar

q)1.1 xbar xbar 5

5.5

--
You received this message because you are subscribed to the Google Groups "Kdb+ Personal Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to personal-kdbpl...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/personal-kdbplus/41cb1970-101e-4249-9f24-b691f6492711n%40googlegro....

--
You received this message because you are subscribed to the Google Groups "Kdb+ Personal Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to personal-kdbpl...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/personal-kdbplus/8a019342-ca91-4ff0-a780-05e6246c6e16n%40googlegro....

--
You received this message because you are subscribed to the Google Groups "Kdb+ Personal Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to personal-kdbpl...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/personal-kdbplus/17da8baf-1641-4a9b-8ede-498801a8fd36n%40googlegro....

--
You received this message because you are subscribed to the Google Groups "Kdb+ Personal Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to personal-kdbplus+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/personal-kdbplus/c20b60a5-4c56-42e5-8201-d7f611e17c0en%40googlegro....

--
You received this message because you are subscribed to the Google Groups "Kdb+ Personal Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to personal-kdbplus+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/personal-kdbplus/13413708-9781-4093-9C68-6D69919648B4%40hxcore.ol.

--

Devon McCormick, CFA

Quantitative Consultant

--
You received this message because you are subscribed to the Google Groups "Kdb+ Personal Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to personal-kdbplus+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/personal-kdbplus/CAGdEmpH4jUacotCqbLm6yqTkC7DNh5xy3BZHTL5EcZmE3y2A....

[Kevin_Smyth]

Yes it is a floating point precision issue

Switch on max precision to examine the stored values

\P 0

q)1.05
1.05 (this is fine)

q)2.05
2.0499999999999998 (slightly smaller, rounds down to 2)

[Andy]

the question was not regarding 

1.1 xbar 5.5 being 5.5

it was 

1.1 xbar 5 being 5.5

On Oct 6, 2020, at 3:45 AM, Devon McCormick <devonmcc@gmail.com> wrote:

I was going to agree with Kimmo that xbar works as documented as I have my own rounding function in J that works in much the same fashion but then I noticed this:

q)0.05 xbar 0 1 2 3 +/:\: 0.04 0.05 0.06   # Round prices to nearest nickel

0 0.05 0.05

1 1.05 1.05

2 2    2.05

3 3    3.05

It's unclear why 1.05<->0.05 xbar 1.05 but 2<->0.05 xbar 2.05 but I suspect floating point imprecision.

However, the case of 5.5<->1.1 xbar 5.5 seems fine since (the integer 5)*1.1->5.5.

[ajay]

I agree with Andy, the "issue" seems to be with div, not xbar

q)5 div 1.1
5
q)5 % 1.1
4.545455

The result of div contradicts Kx's documentation :-  "Integer division: returns the greatest whole number that does not exceed x%y"

I would suggest not mixing the types, either use floats or long/ints

q)1.1 xbar 5.0
4.4

[Andy]

xbar is implemented using div so it affects both. Ajay brings up a good point. when both the lhs and rhs are float, there are no issues. the issue only arises when types are mixed. I'm quite certain this is NOT the intended behavior.

q)5 div 1.1

5

q)5.0 div 1.1

4f

q)1.1 xbar 5

5.5

q)1.1 xbar 5.0

4.4

[alvi_kabir919]

You can override div:

q).q.div:{floor x%y}
q)5 div 1.1
4

[Kimmo_Linna]

Hi,

Could it be so that implementation is according the smallest common factor? If one is integer then those  are used(including casting) for div and then multiplication is done by using given x (number). If both numbers are doubles then those numbers are used.

BR,

Kimmo Linna

Nihtisalontie 3 as 1

02630 ESPOO

+358 40 590 1074

kimmo.linna@gmail.com

Andy <iluvhyori@gmail.com> kirjoitti 6.10.2020 kello 18.57:

xbar is implemented using div so it affects both. Ajay brings up a good point. when both the lhs and rhs are float, there are no issues. the issue only arises when types are mixed. I'm quite certain this is NOT the intended behavior.

q)5 div 1.1

5

q)5.0 div 1.1

4f

q)1.1 xbar 5

5.5

q)1.1 xbar 5.0

4.4

[alvi_kabir919]

Looks like xbar's implementation uses the original 'div' function in k namespace which cannot be changed, so here's the (cleanest imo) workaround if you're always wanting div/xbar to use floor division:

q).q.div:{floor x%y}

q).q.xbar:{x*y div x:$[16h=abs[type x];"j"$x;x]}

q)5 div 1.1

4

q)1.1 xbar 5

4.4

[charlie]

andy is right that it's an oversight(/bug) that div casts rhs to lhs type under some circumstances; good advice not to mix types. We'll look into fixing this to something more sensible.

The fp precision issues are compounded by the fact that floor does not use comparison tolerance asof v3.0. The best workaround for xbar with fp input, to my knowledge, is:

q).1 xbar .1 .2 .3 .4 .5 .6 / demonstrate the problem
0.1 0.2 0.2 0.4 0.5 0.5
q)k).q.xbar:{$[9h=t:abs[@x];x*r+y=x*1+r:y div x;x*y div x:$[16h=t;"j"$x;x]]}
q).1 xbar .1 .2 .3 .4 .5 .6

0.1 0.2 0.3 0.4 0.5 0.6 

[SJT]

Just about to write to you about this… want anything on code.kx.com/q/ref/div in the interim?

Stephen Taylor | Librarian | Kx | +44 7713 400852 | stephen@kx.com

[ajay]

q)k).q.xbar:{$[9h=t:abs[@x];x*r+y=x*1+r:y div x;x*y div x:$[16h=t;"j"$x;x]]}
q)1.4 xbar 6
8.4
q)
q)k).q.xbar:{$[9h=t:abs[@x];x*$[t;y] div x;x*y div x:$[16h=t;"j"$x;x]]}  // with explicit cast
q)1.4 xbar 6
5.6
q)1.6 xbar 6
4.8
q)1.9 xbar 6
5.7
q)1.1 xbar 6
5.5

[Andy]

the purpose of xbar is to round down. from the kdb docs:

xbar

Round down

x xbar y xbar[x;y]

Where

• x is a non-negative numeric atom
• y is numeric or temporal

returns y rounded down to the nearest multiple of x.

although applying it on time columns is a common usage pattern, the point of xbar is to round down. 1.1 xbar 5 does not round down, it cannot be the intended behavior.