Showing results for 
Search instead for 
Did you mean: 

Fixed length binary table column

New Contributor

I have a table in which I need to store unique 32 byte cryptographic signatures for each row. I receive the column inputs via JSON as a 64 byte hex string. What's the best column definition for that data and how would I convert the string to that type? Are there any hex to byte vector and byte vector to hex routines built in to kdb?

Thanks for any help!


New Contributor II
Returns a byte vector from hex string: "X"$2 cut "deadbeef" To do the inverse transform, you can just use 'string' Sean

New Contributor
I'd use a compound byte column to store crypto signatures, i.e. list of lists of bytes.

You can use these two functions to convert hex strings to byte vectors and the other way around:

hex2bin: { "X"$'2 cut x }

bin2hex: { "" sv string x }

Hope this helps.

Sean and Igor, very helpful, thank you very much!

One additional question: what is the storage overhead of a list of lists of bytes? I'm assuming that a list of GUID's has less overhead, as the fixed length is built in to the data type?  

If your table is splayed the overhead will be 8 bytes (sizeof(long)) per crypto signature, here's why:
There will be some performance degradation, too. You can use GUID's, but since a GUID is 16 bytes long you will need to split a crypto hash into 2 halves and store them in two separate GUID columns (and combine them during retrieval). I doesn't look like a very elegant solution to me unless you really need to save some space.

Thanks for the explanation Igor, much appreciated!

New Contributor III
Yes, less overhead for GUIDs.

List of lists is structured as a two-level index in q. The first level serves as an index to the second level, where the latter stores the flatten list of all bytes. You may consider the first level data as storage overhead in this sense.