cancel
Showing results for 
Search instead for 
Did you mean: 

websocket client in v3.2

charlie
New Contributor II
New Contributor II
For those of you interested in websockets, kdb+ 3.2 can now act as a client, in addition to as a server. Documented here

http://code.kx.com/wiki/Cookbook/Websocket

This is an interesting way to connect to websocket services, e.g. pusher.com or bitcoin exchanges etc.

Added websocket client functionality. .z.ws must be defined before opening a websocket, e.g.
 q).z.ws:{0N!x;} / print incoming msgs to console, no echo.
 Then open a websocket, e.g.
 q)r:(`quot;:ws://host:port")""GET / HTTP/1.1\r\nHost: host:port\r\n\r\n"
 If successful it will return a 2 element list of (handle;http response), e.g.
 (3i;"HTTP/1.1 101 Switching Protocols\r\nConnection: Upgrade\r\nUpgrade: websocket\r\nSec-WebSocket-Accept: HSmrc0sMlYUkAGmm5OPpG2HaGWk=\r\nSec-WebSocket-Extensions: permessage-deflate\r\n\r\n")
 and from that point on will callback via .z.ws when msgs arrive. To send msgs, use neg[handle]"text" or neg[handle]byteVector.
 If the protocol upgrade from http to websocket failed, it returns the 2 element list, with the handle as 0Ni, e.g.
 (0Ni;"HTTP/1.1 400 Bad Request\r\nContent-Type: text/html; charset=UTF-8...")
 The response text is returned for debug purposes only; ideally, you need only be concerned whether the handle is valid.
 Any other error is thrown as usual, e.g.
 'www.nonexist.badcom: No route to host
 Should you need to use websockets over ssl, e.g. wss://host:port, consider stunnel, and open from kdb+ to that stunnel with ws://.
 Basic Authentication can be passed in the char vector on opening, along with any other necessary fields such as cookies etc.

An example session -

q).z.ws:{0N!x;}

q)`:ws://ws.blockchain.info:80 "GET /inv HTTP/1.1\r\nHost: ws.blockchain.info\r\n\r\n"
3i
"HTTP/1.1 101 Switching Protocols\r\nServer: nginx\r\nDate: Sat, 26 Jul 2014 09:53:02 GMT\r\nConnection: upgrade\r\nUpgrade: websocket\r\nSec-WebSocket-Accept: HSmrc0sMlYUkAGmm5OPpG2HaGWk=\r\n\r\n"

q)neg[3]"{\"op\":\"unconfirmed_sub\"}"

then, after a few secs, callback prints
"{\"op\":\"utx\",\"x\":{\"hash\":\"0b3dc1108477539c3b635b30e4e67630b7e038c65a013efb62653276216b8267\",\"vin_sz\":1,\"vout_sz\":2,\"lock_time\":\"Unavailable\",\"size\":226,\"relayed_by\":\"188.226.161.239\",\"tx_index\":61038847,\"time\":1406368531,\"inputs\":[{\"prev_out\":{\"value\":5192700,\"addr\":\"166wZYCWrBwj1eppfbV8K4aJV2pRtsaY4d\",\"type\":0}}],\"out\":[{\"value\":5082700,\"addr\":\"1PXZ8fpGKnWH9tUucbXGciiJ7kMW4yiACq\",\"type\":0},{\"value\":100000,\"addr\":\"1LuckyR1fFHEsXYyx5QK4UFzv3PEAepPMK\",\"addr_tag\":\"LuckyBit red\",\"addr_tag_link\":\"http://luckyb.it/\",\"type\":0}]}}"
5 REPLIES 5

Francisco
New Contributor
Hello, Can this new feature be used to access a streaming API like this?

Using curl:

C:\>curl -k -H "Authorization: Bearer 137529d301a65re3a32b00ee2c94b8f3-72ffdf45bb72dacy72w40fc67b6b9fbfa" "https://stream-fxpractice.oanda.com/v1/prices?accountId=1234567&instruments=AUD_CAD%2CAUD_CHF"
{"tick":{"instrument":"AUD_CAD","time":"2014-10-03T02:44:45.688156Z","bid":0.98166,"ask":0.98188}}
{"tick":{"instrument":"AUD_CHF","time":"2014-10-03T02:44:42.806837Z","bid":0.83898,"ask":0.83932}}
{"tick":{"instrument":"AUD_CAD","time":"2014-10-03T02:44:47.293618Z","bid":0.98164,"ask":0.98188}}
{"heartbeat":{"time":"2014-10-03T02:44:47.744523Z"}}
{"heartbeat":{"time":"2014-10-03T02:44:50.112267Z"}}
{"heartbeat":{"time":"2014-10-03T02:44:52.744526Z"}}
{"tick":{"instrument":"AUD_CAD","time":"2014-10-03T02:44:53.304949Z","bid":0.98164,"ask":0.98189}}
{"tick":{"instrument":"AUD_CAD","time":"2014-10-03T02:44:53.441623Z","bid":0.98164,"ask":0.98188}}
{"tick":{"instrument":"AUD_CAD","time":"2014-10-03T02:44:54.442378Z","bid":0.98166,"ask":0.98188}}
{"heartbeat":{"time":"2014-10-03T02:44:55.091689Z"}}
{"heartbeat":{"time":"2014-10-03T02:44:57.744556Z"}}
{"tick":{"instrument":"AUD_CAD","time":"2014-10-03T02:44:59.854973Z","bid":0.98165,"ask":0.98189}}
{"heartbeat":{"time":"2014-10-03T02:45:00.122216Z"}}
{"tick":{"instrument":"AUD_CAD","time":"2014-10-03T02:45:01.241442Z","bid":0.98168,"ask":0.98189}}
{"tick":{"instrument":"AUD_CAD","time":"2014-10-03T02:45:01.345852Z","bid":0.98168,"ask":0.98191}}
{"tick":{"instrument":"AUD_CAD","time":"2014-10-03T02:45:02.240562Z","bid":0.9817,"ask":0.98189}}
^C

I don't know how to setup q to access the streaming API.

Any help please?

Cheers

Francisco

charlie
New Contributor II
New Contributor II
That particular interface doesn't look like it supports websockets.

They list a websocket api, at least experimental, here


but their streaming demo


doesn't appear to work at the moment; maybe their test server is down. There's a couple of questions on their forums about websockets, but I didn't find any news since Feb2014 when they stated they are working on a http streaming api.

Which OS are you using?

Francisco
New Contributor
Thanks for your answer.

I'm using MSwindows 8.1 

Oanda has 3 different environments: 1 for real trading and 2 for testing purposes. The fx-practice environment that I used in my prior post is for testing purposes but simulates the real trading machine so it requires authentication.  The sandbox environment is also for testing purposes, doesn't require autentication and it's working ok:

C:\>curl -k "http://stream-sandbox.oanda.com/v1/prices?accountId=1125870&instruments=AUD_CAD%2CAUD_CHF"
{"tick":{"instrument":"AUD_CAD","time":"2014-10-03T09:11:08.569581Z","bid":0.93847,"ask":0.93898}}
{"tick":{"instrument":"AUD_CHF","time":"2014-10-03T09:11:08.573002Z","bid":0.97679,"ask":0.97716}}
{"heartbeat":{"time":"2014-10-03T09:11:09.783949Z"}}
{"tick":{"instrument":"AUD_CHF","time":"2014-10-03T09:11:09.939337Z","bid":0.97691,"ask":0.97728}}
{"tick":{"instrument":"AUD_CAD","time":"2014-10-03T09:11:10.202582Z","bid":0.9387,"ask":0.93921}}
{"tick":{"instrument":"AUD_CHF","time":"2014-10-03T09:11:10.205480Z","bid":0.97698,"ask":0.97735}}
{"tick":{"instrument":"AUD_CHF","time":"2014-10-03T09:11:10.311909Z","bid":0.97702,"ask":0.97739}}
{"tick":{"instrument":"AUD_CAD","time":"2014-10-03T09:11:10.397303Z","bid":0.93876,"ask":0.93927}}
{"tick":{"instrument":"AUD_CAD","time":"2014-10-03T09:11:10.738665Z","bid":0.9388,"ask":0.93931}}
{"tick":{"instrument":"AUD_CHF","time":"2014-10-03T09:11:10.749701Z","bid":0.97706,"ask":0.97743}}
{"tick":{"instrument":"AUD_CHF","time":"2014-10-03T09:11:11.317733Z","bid":0.9771,"ask":0.97747}}
{"tick":{"instrument":"AUD_CAD","time":"2014-10-03T09:11:11.961576Z","bid":0.93886,"ask":0.93937}}
{"heartbeat":{"time":"2014-10-03T09:11:12.308776Z"}}
{"tick":{"instrument":"AUD_CHF","time":"2014-10-03T09:11:13.276997Z","bid":0.97706,"ask":0.97743}}
{"tick":{"instrument":"AUD_CAD","time":"2014-10-03T09:11:14.229094Z","bid":0.93883,"ask":0.93934}}
{"heartbeat":{"time":"2014-10-03T09:11:14.783964Z"}}
^C

Cheers

Francisco 

charlie
New Contributor II
New Contributor II
kdb+ currently doesn't support http streaming. Aside from using curl to save to a file and then import it, you can hook up a live feed via java very easily, using


and the following java

import java.net.*;
import java.io.*;
import kx.c;

public class URLConnectionReader {
  public static void main(String[] args) throws Exception {
    c c= new c("127.0.0.1",5000,"username:password");
    URLConnection yc=url.openConnection();
    BufferedReader in=new BufferedReader(new InputStreamReader(yc.getInputStream()));
    String inputLine;
    while((inputLine=in.readLine())!=null)
      c.ks("upd",inputLine.toCharArray());
    in.close();
    c.close();
  }
}

with kdb+ listening on locahost, port 5000

q)upd:{0N!.j.k x}

then prints each line as it is received

(,`tick)!+`instrument`time`bid`ask!(,"AUD_CAD";,"2014-10-03T18:37:56.449661Z";,0.93852;,0.93903)
(,`tick)!+`instrument`time`bid`ask!(,"AUD_CHF";,"2014-10-03T18:37:52.225257Z";,0.97927;,0.97964)
(,`heartbeat)!+(,`time)!,,"2014-10-03T20:43:47.355207Z"

and if you define
upd:{x:.j.k x;(first key x) upsert value x}

then it will insert each row into the respective table

q)upd:{x:.j.k x;(first key x) upsert value x}
q)tick
instrument time                          bid     ask    
--------------------------------------------------------
"AUD_CAD"  "2014-10-03T18:37:56.449661Z" 0.93852 0.93903
"AUD_CHF"  "2014-10-03T18:37:52.225257Z" 0.97927 0.97964
q)heartbeat
time                         
-----------------------------
"2014-10-03T20:51:22.431392Z"
"2014-10-03T20:51:27.431396Z"

you can cast the incoming data as necessary

q)upd:{k upsert update "P"$-1_'time from $[`tick=k:first key x:.j.k x;x:update`$instrument from value x;value x];}
q)tick
instrument time                          bid     ask    
--------------------------------------------------------
AUD_CAD    2014.10.03D18:37:56.449661000 0.93852 0.93903
AUD_CHF    2014.10.03D18:37:52.225257000 0.97927 0.97964
q)heartbeat
time                         
-----------------------------
2014.10.03D21:01:12.495310000
2014.10.03D21:01:17.495362000
2014.10.03D21:01:22.505329000

this is fine for an example, but for real world use, aside from adding error handling, for efficiency you should bulk up data to multiple rows per event/msg. Don't forget to look at kdb+tick if you want to flesh this out


Maybe oanda will finish their websocket api - post into their forums to let them know if you're interested.

Francisco
New Contributor
Thanks a lot for your help.

Francisco