-
Is this similar to function alias?
I am studying the official example code from here: https://github.com/kxcontrib/websocket/blob/master/AppendixB/pubsub.q . After playing around for a while, I can get the publisher/subscriber model work but still I don’t understand what exactly this line (https://github.com/kxcontrib/websocket/blob/ad2f0b268afaee1fc5f4dda2fc2467440c7e2f0c/AppendixB/pubsub.q#L8) is doing.
After checking the Reference card (https://code.kx.com/q/ref/), I know that insert is a built-in function and Overloaded glyphs (https://code.kx.com/q/ref/overloads/) says that : (colon) only has two possible meanings (https://code.kx.com/q/ref/overloads/#colon).
So here comes the question, what does
upd:insert;
do? Does it mean that we declare upd as an alias of insert, so calling upd will be exactly the same as calling insert? (But I did try it myself, eliminating this line and directly call insert here: https://github.com/kxcontrib/websocket/blob/ad2f0b268afaee1fc5f4dda2fc2467440c7e2f0c/AppendixB/fh.q#L18-L19, no, it doesn’t work lol)
-
6 Replies
-
Yes defining upd in this way means it behaves the same as insert (mostly)
q)upd:insert q)tab:([] a:1 2) q)insert[`tab;enlist 3] ,2 q)tab a - 1 2 3 q)upd[`tab;enlist 4] ,3 q)tab a - 1 2 3 4
But there are differences. ‘insert’ is a built in operator which cannot be passed as the first item by reference over a handle.
(This is causing the issue you are seeing)
q)value(`upd;`tab;enlist 5) //Pass by reference succeeds for user defined function ,4 q)value(`insert;`tab;enlist 6) //Pass by reference fails for operator 'insert [0] value(`insert;`tab;enlist 5) ^ q)value("insert";`tab;enlist 6) //Pass as parse string succeeds ,5 q)value(insert;`tab;enlist 6) //Pass by value succeeds ,6User defined functions can only use prefix notation whereas operators can be used prefix or infix.
q)`tab insert enlist 7 //Infix with operator succeeds ,7 q)`tab upd enlist 8 //Infix with user defined function fails 'type [0] `tab upd enlist 8 ^ q)insert[`tab;enlist 8] //Prefix with operator succeeds ,8 q)upd[`tab;enlist 9] //Prefix with user defined function succeeds ,9
-
.z.ts is the timer function which is evaluated on intervals of the timer variable set by system command t
In your example ‘pub’ is called for each subscriber in ‘subs’ once per second.
To start subs has count 0 but when a new subscriber connects and subscribes they will be added.
How the subscribers are added to that table in that demo is:
- websockets.html calls connect() on load of the page (code)
- websockets.js holds the connect() definition which shows it sends loadPage[] to the q process (code)
- pubsub.q holds the definition of loadPage which shows that it calls ‘sub’ (code)
- ‘sub’ then adds your subscriber to ‘subs’
-
Thanks @rocuinneagain for taking a look at my lengthy question! Let’s say we examine the flow this way:
- information is generated by feedhandler fh.q (https://github.com/kxcontrib/websocket/blob/master/AppendixB/fh.q)
- information is passed to tickerplant subpub.q (https://github.com/kxcontrib/websocket/blob/master/AppendixB/pubsub.q) (please correct me if this is not a tickerplant)
- information is received by a JavaScript-based subscriber (https://github.com/kxcontrib/websocket/blob/master/AppendixB/websockets.js) In this particular example, I understand that information is randomly generated by fh.q; and fh.q interact with pubsub.q via these two function calls (https://github.com/kxcontrib/websocket/blob/ad2f0b268afaee1fc5f4dda2fc2467440c7e2f0c/AppendixB/fh.q#L18-L19) .
My confusion starts here: in lines 18 and 19 of fh.q, `upd is an alias to insert; `quote and `trade are table names, and the rest are parameters randomly generated–so which line in pubsub.q will be invoked when these two function calls are made (please correct me if the term “function call” is inaccurate)?
Thanks!
-
TP = Tickerplant (Relays data to subscribers + recovers from logfile after crashes)
RDB = Realtime Database (Stores data for query)
RTE = Realtime engine (Performs streaming calculations and stores caches or publishes results)
(Any process can be customised)
1. Yes
2. It is acting more like a mixture between a TP,RDB,RTE
a) It does not store a logfile to recover in case of a crash (tp-logfile – a TP normally does this)
b) It stores data indefinitely instead of acting only as a relay. (Unlike a TP, more like and RDB, although an RDB will clear once every 24hrs)
c) It does not relay data untouched instead only specific data is forwarded (similar to an RTE)
getSyms – sends like of unique symbols across tables
getQuotes – sends last row by sym from quote table
getTrades – sends last row by sym from trade table
3. Yes
The execution flow is:
1. FH sends messages to PubSub (lines 18/19) every 100ms
2. The messages arrive to PubSub and .z.pg evaluates them. This mean upd/insert (pubsub.q line 8 will save the incoming data to quote/trade. PubSub now has some data cached.
3. The next time the PubSub timer (.z.ts) is triggered (every second) the ‘pub’ function will trigger and send data to subscriptions.
This code is a basic demo so it may have some holes in it’s logic (like never clearing data in PubSub so eventually memory will run out)
-
Hi @rocuinneagain , thanks for the detailed reply and some informative clarification!
But I still have some confusion…the crux is this sentence you wrote
2. The messages arrive to PubSub and .z.pg evaluates them. This mean upd/insert (pubsub.q line 8 will save the incoming data to quote/trade. PubSub now has some data cached.
I understand that .z.pg is a function under the .z namespace, my questions are:
- Can I say that it is like a callback function which will be called by q when some events (i.e. synchronous request in this case) happen?
- In the caller side (https://github.com/kxcontrib/websocket/blob/ad2f0b268afaee1fc5f4dda2fc2467440c7e2f0c/AppendixB/fh.q#L18-L19), since we called the handler this way: h(`upd;`quote;(n#.z.N;s;getbid'[s];getask'[s]));, does it mean that .z.pg will evaluate the parameters, i.e., `upd, `quote and (n#.z.N;s;getbid'[s];getask'[s]) and .z.pg will interpret these parameters as: “execute the function upd and pass two parameters (i.e., `quote and (n#.z.N;s;getbid'[s];getask'[s])) to it”?
- Regarding this statement: “This code is a basic demo so it may have some holes in it’s logic (like never clearing data in PubSub so eventually memory will run out)”, do you mean that if feedhandler constantly inserts new rows by calling h(`upd;`quote;(n#.z.N;s;getbid'[s];getask'[s]));, tables `quote and `trade will grow larger and larger and the memory will be filled by these two tables?
- Regarding this function (https://github.com/kxcontrib/websocket/blob/ad2f0b268afaee1fc5f4dda2fc2467440c7e2f0c/AppendixB/pubsub.q#L33-L36), due to brevity of kdb/q, I am still decoding what exactly it is doing. But after reading you last reply, my vague understanding is that subscribers registered their handlers in the table `subs. So the pub function is essentially enumerating the `subs table and call all subscribers’ handlers one by one. This series of function calls is what we say to “publish data” to subscribers. Is this understanding basically correct?Thanks!
-
Yes your understanding is correct for those 4 items.
These tutorial videos may be useful:
Log in to reply.