back to forum.

Topic: Would like to use but missing some key functionality

Topic by
bsrexroth

2011-03-08
22:12

Would like to use but missing some key functionality

I would really love to be able to use DataCleaner for some work we are currently doing however it is missing a couple of features that leaves us unable to use it. We need the ability to set up a generic profiling job and then feed it the connection and table name (or sql statement) at runtime. In other words, the generic job would know the database connection would be to a particular database type and would know that we want to profile all the columns in a given table (the table itself being variable). Then at runtime, we pass connection information along with a table name (and output information) and the job runs the appropriate profiling on each column based on the datatype in the database.

We have an automated system right now where data comes in, is automatically restored to the appropriate database or file location and then profiling is automatically run. Thus, no human interaction is necessary. This is key to use based on the amount of data we are receiving (and the different kinds of data). We already have a proprietary data profiling tool that can perform this task but were trying to look for a possible replacement (this tool uses the idea of job variables which can be defined at run time). Unless someone can tell me that I missed something, it appears that DataCleaner will not be a suitable tool for us.

Thanks!

Brian

Reply by
beno

2011-03-09
08:07
Did you have a look at the xml file format that datacleaner saves it files into? I see that it contains the names of the columns in a <source> tag. If you can handle a bit of XML generation then I think you would be able to generate a job file automatically by reading your database and just putting the names of your columns into something like a prototype job file.

You need to be logged in to participate

In order to post your own comments on this topic, you need to be logged in.

Username:

Log in by clicking the login link at the top of the screen

 

Go back to forum.