cancel
Showing results for 
Search instead for 
Did you mean: 

How to replay csv file one line by one line

hzadonis
New Contributor
Hi, Masters:
  I have some csv files, the format such as:
    timestamp, bid, ask, volumn
    09:00:00,2,3,100
    09:00:01,3,4,200
    09:00:02,3,5,300
    ......

  How to replay the data one line by one line into kdb? Or do you have suggestions? Thanks so much!

Zheng
6 REPLIES 6

collin_ola
New Contributor
Depending on the size of your file, you could read it all into memory at once, as a table. http://code.kx.com/q/cookbook/faq/#how-do-i-import-a-csv-file-into-a-table

Yes, you're right. I can read the whole csv file into kdb+ without problem.
I just want to replay it line by line based on the recorded timestamp.

User
Not applicable
Why exactly do you want to replay the .csv file line by line? When you say replay do you mean load into kdb+?

Starting with this table: 
table:([]time:asc 1000000?.z.P;sym:1000000?`abc`def`ghi;price:1000000?100f)
and saving it down: 
`:table.csv 0: csv 0: table
The following options are available to you: 

If you just want to load the csv file into kdb you can do: 
("SSF";enlist",")0:`table.csv

If your .csv file is really big you can use .Q.fs or .Q.fsn 
.Q.fs[{table::("SSF";enlist",")0:x}]`:table.csv

If you really want to load it line by line, even though I wouldn't recommend this, you can use over and do it this way: 
{x:x,y}/[0#("SSF";enlist",")0:`:table.csv;("SSF";enlist",")0:`table.csv]

Let me know if this helps you! 
Regards!
Alex

hzadonis
New Contributor
Alex, thanks!
Yes, the process of your method is working. It can be load into kdb+.

You asked "Why you want to replay the .csv file line by line?". It is because I want to implement a replay service to playback what I had recorded based on the market real tick-by-tick data. Actually, I can load the whole csv file into kdb+, but my purpose is replay the data line-by-line based on the timestamp, such as: 09:00:00, replay line 1; 09:00:01, replay line 2; 09:00:05, replay line 3; and so on. I prefer to have another table to save the streaming data.

Flying
New Contributor III
You could either read the csv into memory in one go, or read through it trunk by trunk. (Both of these methods are described in the documentations.)

Once you get the data, may it be all in one go or trunk by trunk, you can replay it whichever way you prefer.

Flying, thanks!
Do you mean the document that Collin mentioned?