DC_ARCHIVE directory format

[leades]

Hello,

For the DC_ARCHIVE template, you must put in a directory for the analytic to act on. What if the directory path includes a date? For example, .../hdb/{date}/data. 

Any help is greatly appreciated

Many Thanks

Luke

[Laura]

Hi Luke,

Thanks for your question!

In the instance of archiving older dates in a HDB using the DC_ARCHIVE template, you would have to act upon the entire date partition anyway (as removing specific columns only within some partitions would break your HDB)

So in your example, your directory would be .../hdb/

and you would use the numDaysOlderThan parameter to archive dates older than X days.

Hope this helps!

Laura

[damholda]

Hi Laura,

Thanks for response, actually Luke from support was asking on my behalf. What we are trying to do is to have temporary table within HDB that only keeps records for 7 days. We dont care about old data in that "schema" so we want to purge it to save space. We were told by support that "archive" functionality can do that but from what you saying that is not feasible. Do you have any alternatives where we can discard data for specific schema or maybe limit it so it overwrites oldest data ../hdb/{data}/tempTable ? 

Thanks

Damian

[LeahS · Accepted solution]

Hello Damian, 

Thanks for reaching out here.

DC_ARCHIVE is an inbuilt file maintenance mechanism in KX Control. It allows you to archive and delete files in a Unix sense. 

However, it seems clearer now (and correct me if I am wrong) that you are continuously 'upserting' to a flat table within the HDB daily, and your intention is to delete older rows in that table. 

In this case, the archive solution is not appropriate as that is for files only. 

In order to delete rows older than 7 days in that flat table, you can avail of another inbuilt process template in Control called DS_EXEC_ANALYTIC. The process template allows for scheduled execution. I have documented the implementation steps here: DS_EXEC_ANALYTIC Template  

In your case, your target process will be the relevant HDB and the supplied analytic should work to delete the rows in that flat table. 

Useful links for your analytic logic and for dealing with flat tables:

https://code.kx.com/q/ref/get/

https://code.kx.com/q/ref/get/#set

https://code.kx.com/q/ref/delete/#table-rows

In summary, the archiving process template is great for managing file maintenance. However, when maintaining a flat table, we recommend the use of DS_EXEC_ANALYTIC here for scheduled execution of row removal. 

Please let me know if you encounter any issues with the above or have any further questions. 

Kind regards,

Leah