cancel
Showing results for 
Search instead for 
Did you mean: 

Web Function and Data Parsing Question

DuoCentillion
New Contributor


I am trying to get this q code to work

yahoo: {[offset;stocks]
tbl:();i:0;zs:(ze:.z.d)-offset;
parms:"&d=",(string -1+`mm$ze),"&e=",(string`dd$ze),"&f=",
(string`year$ze),"&g=d&a=",(string -1+`mm$zs),"&b=",(string`dd
$zs),"&c=",(string`year$zs),"&ignore=.csv";
do[count stocks:distinct stocks,();
txt:`:http://ichart.finance.yahoo.com "GET /table.csv?s=",
(string stock:stocks[i]),parms," http/1.1\r
\nhost:ichart.finance.yahoo.com\r\n\r\n";
tbl,:update Sym:stock from select from ("DEEEEI ";enlist",")0:(txt
ss"Date,Open")_ txt;i+:1];
(lower cols tbl)xcol`Date`Sym xasc select from tbl where not null
Volume}

from

https://code.kx.com/svn/cookbook_code/yahoo_compact.q


I cannot seem to get it to work. Any suggestions?

3 REPLIES 3

Swee_Heng
New Contributor


Make sure you do not have connectivity problems to Yahoo. Try this
link: <>. You should
get a CSV file. If you do, then this should work:

q)yahoo[7;`GOOG`AAPL]
date open high low close volume sym
----------------------------------------------------
2008.07.28 162.34 162.47 154.02 154.4 27882600 AAPL
2008.07.28 492.09 492.09 475.13 477.12 3160000 GOOG
2008.07.29 155.41 159.45 153.65 157.08 24431100 AAPL
2008.07.29 479.3 487.26 478 483.11 2802800 GOOG
2008.07.30 157.78 160.49 156.08 159.88 25899400 AAPL
2008.07.30 485.5 486.02 472.81 482.7 3490700 GOOG
2008.07.31 157.54 162.2 156.98 158.95 22744600 AAPL
2008.07.31 474.56 480.89 471.44 473.75 2865100 GOOG
2008.08.01 159.9 159.99 155.75 156.66 19451400 AAPL
2008.08.01 472.51 473.22 462.5 467.86 3007900 GOOG
2008.08.04 156.6 157.9 152.91 153.23 21161700 AAPL
2008.08.04 468.12 473.01 461.9 463 2487000 GOOG

Swee Heng

I am having same issue. Yahoo connectivity is fine. Have saved code athttps://code.kx.com/svn/cookbook_code/yahoo_compact.q to a file, andloaded using the \l command. Then tried yahoo[7;`GOOG`AAPL], output isbelowq)\l c:/yahoo.qq)yahoo[7;`GOOG`AAPL]{[offset;stocks] tbl:();i:0;zs:(ze:.z.d)-offset; parms:"&d=",(string -1+`mm$ze),"&e=",(string`dd$ze),"&f=",(string`year$ze),"&g=d&a=",(string -1+`mm$zs),"&b=",(string`dd$zs),"&c=",(string`year$zs),"&ignore=.csv"; do[count stocks:distinct stocks,(); txt:`:http://ichart.finance.yahoo.com "GET /table.csv?s=",(string stock:stocks[i]),parms," http/1.1\r\nhost:ichart.finance.yahoo.com\r\n\r\n"; tbl,:update Sym:stock from select from ("DEEEEI ";enlist",")0:(txt ss"Date,Open")_ txt;i+:1]; (lower cols tbl)xcol`Date`Sym xasc select from tbl where not nullVolume}'length0:("DEEEEI ";,",")()q))Suggestions?On Aug 4, 7:56�pm, Swee Heng wrote:> Make sure you do not have connectivity problems to Yahoo. Try this> link: <>. You should> get a CSV file. If you do, then this should work:>> q)yahoo[7;`GOOG`AAPL]> date � � � open � high � low � �close �volume � sym> ----------------------------------------------------> 2008.07.28 162.34 162.47 154.02 154.4 �27882600 AAPL> 2008.07.28 492.09 492.09 475.13 477.12 3160000 �GOOG> 2008.07.29 155.41 159.45 153.65 157.08 24431100 AAPL> 2008.07.29 479.3 �487.26 478 � �483.11 2802800 �GOOG> 2008.07.30 157.78 160.49 156.08 159.88 25899400 AAPL> 2008.07.30 485.5 �486.02 472.81 482.7 �3490700 �GOOG> 2008.07.31 157.54 162.2 �156.98 158.95 22744600 AAPL> 2008.07.31 474.56 480.89 471.44 473.75 2865100 �GOOG> 2008.08.01 159.9 �159.99 155.75 156.66 19451400 AAPL> 2008.08.01 472.51 473.22 462.5 �467.86 3007900 �GOOG> 2008.08.04 156.6 �157.9 �152.91 153.23 21161700 AAPL> 2008.08.04 468.12 473.01 461.9 �463 � �2487000 �GOOG>> Swee Heng>> On Aug 4, 1:48 pm, DuoCentillion wrote:>> > I am trying to get this q code to work>> > yahoo: {[offset;stocks]> > � � � � tbl:();i:0;zs:(ze:.z.d)-offset;> > � � parms:"&d=",(string -1+`mm$ze),"&e=",(string`dd$ze),"&f=",> > (string`year$ze),"&g=d&a=",(string -1+`mm$zs),"&b=",(string`dd> > $zs),"&c=",(string`year$zs),"&ignore=.csv";> > � � do[count stocks:distinct stocks,();> > � � � � txt:`:http://ichart.finance.yahoo.com"GET /table.csv?s=",> > (string stock:stocks[i]),parms," http/1.1\r> > \nhost:ichart.finance.yahoo.com\r\n\r\n";> > � � � � � � � � tbl,:update Sym:stock from select from ("DEEEEI ";enlist",")0:(txt> > ss"Date,Open")_ txt;i+:1];> > � � (lower cols tbl)xcol`Date`Sym xasc select from tbl where not null> > Volume}>> > from>> >https://code.kx.com/svn/cookbook_code/yahoo_compact.q>> > I cannot seem to get it to work. �Any suggestions?



I get your error if I use an offset of 0. That is:
q)yahoo[0;`GOOG`AAPL]
... (output snipped)...
'length
0:
("DEEEEI ";,",")
()

What I'd suggest is that you modify yahoo[] to show "txt" and where
the string "Date,Open" appears in it. E.g.
ya: {[offset;stocks]
tbl:();i:0;zs:(ze:.z.d)-offset;
parms:"&d=",(string -1+`mm$ze),"&e=",(string`dd$ze),
"&f=",(string`year$ze),"&g=d&a=",(string -1+`mm$zs),
"&b=",(string`dd$zs),"&c=",(string`year$zs),"&ignore=.csv";
do[count stocks:distinct stocks,();
txt:`:http://ichart.finance.yahoo.com "GET /table.csv?s=",
(string stock:stocks[i]),parms,
" http/1.1\r\nhost:ichart.finance.yahoo.com\r\n\r\n";
show txt; show txt ss "Date,Open";i+:1]}

Then try:
q)ya[0;`GOOG`AAPL]
"HTTP/1.1 404 Not Found\r\nProxy-Connection: Keep-Alive\r\nConnection:
Keep-A..
`int$()
"HTTP/1.1 404 Not Found\r\nDate: Tue, 05 Aug 2008 17:54:59 GMT\r\nP3P:
policy..
`int$()
q)ya[7;`GOOG`AAPL]
"HTTP/1.1 200 OK\r\nDate: Tue, 05 Aug 2008 17:55:03 GMT\r\nP3P:
policyref=\"h..
,291
"HTTP/1.1 200 OK\r\nProxy-Connection: Keep-Alive\r\nConnection: Keep-
Alive\r\..
,382

In the first case, Yahoo returned a HTTP 404 Error and so the content
does not contain the expected "Date,Open" string. That seems to be the
error in yahoo[] function when 0: parsed the records. You can also use
a sniffer like Wireshark to inspect the traffic Yahoo returns you if
the problem persists.

Swee Heng