Using IPC and peach

[simon_watson_sj]

Hey All,

I have tried to set up my KDB HDB to use a set of 4 separate processes as 1 secondary process and 3 peach processes.

However, after assigning the 3 processes to peach, I get an error in the Developer window which says:

"could not lint the specified item(s). 10 is not an IPC handle"

thing is, it was an IPC handle but passing it to peach seems to have released it.

Before using .z.pd

10"1+1" 

gives 2

What am I doing wrong?

I am on Pop_OS. I use this function in my .bashrc to start up the q sessions:

KDB(){
echo "start developer with 4 slave processes.";
q -p 20001 &
q -p 20002 &
q -p 20003 &
q -p 20004 &
source /home/simon/developer/config/config.profile; q /home/simon/developer/launcher.q_ -s -4 &
}

First thing from the command line I execute that function and can see from the usual q startup notifications it works. I can also see the processes have started. After that, I open KDB Developer.

In Developer I then capture the processes as follows:

// capture the other processes. 
{(`$".procs.h.",(string x[0])) set hopen x[1]} each flip (`tom`deric`jane`nessie;("J"$("20001";"20002";"20003";"20004"))); 

// assign process names to each of the processes as `self. 
{(neg `.procs[`h][x[0]])x[1];(neg `.procs[`h][x[0]])[];} each flip (`tom`deric`jane`nessie; raze each "`self set `",/:string `tom`deric`jane`nessie);
                                   
// assign 3 processes to peach.
.z.pd: `u#`.procs.h[`tom`deric`jane]                                                                                                

Sometimes it seems like this works until I change developer window, alter a file and save. I don't know if that just coincidence though.

Keen to hear if anyone can see any obvious schoolboy errors.

Simon

[LeahS]

Great content here Simon! Thanks for sharing with the community. 

We're working on feedback for you! In the meantime, looking forward to the contributions by fellow community members. 

Kind regards,

Leah 

[LeahS · Accepted solution]

Hello Simon,

I'm reverting with feedback shared by our internal pros. Kudos to Benjamin Jeffery here!

There appears to be two identifiable issues:

1. Analyst was not designed to work with .z.pd. As these secondary processes don't have internal Analyst functions or other state defined, behaviour when using .z.pd is undefined. This limitation will be added to the documentation for the next Analyst release. 

2. The handle to the secondary process is closing whenever peach is called with a locked function. An internal ticket has been raised acknowledging this. 

Hope this helps. Thanks again for sharing with the community 🙌

Kind regards,

Leah

[simon_watson_sj]

Thanks Leah. Good to know - so best practice if you are going to use peach with Developer/Analyst is  to use the standard threaded approach (starting q with -s set to a positive integer denoting the number of secondary threads used in Peach. Then if I need an additional secondary process, I can just use standard IPC.

Alternatively, if I really want to use .z.pd, which (if any) of the following might you think worthwhile:

• I try using the KDB dashboard to run previously defined functions?
• I try using JupyterQ and running processes from a Jupyter notebook written in Q
• I build a manual interface via websockets as per this white paper: https://code.kx.com/q/wp/websockets/

Really appreciate your having a look and answering this. Let me know if I'm not right about anything above, otherwise I'm happy and have marked this issue as resolved.

Also, pass on thanks to Benjamin Jeffery.

Simon