How To Evaluate ETL Testing Tools
C
CGMIMM Import
June 23, 2022
data validation and observabilityETL testing tools
Why ETL Testing is ImportantToday, Data has become the core of any business, and ETL processes are important for every data-related project. Be it Data Warehouse or Data Lake every business builds and runs thousands of ETL processes for its data movement. Data validation and ETL Testing will be of prime importance for all these data movements.
ETL Process and ETL Testing should work hand in hand for a successfully validated data transfer from data source to destination. Be it customer data, financial data, marketing data or any data which plays a crucial part in decision making must be rightly validated.A tough task for project owners is to choose which ETL Testing Tools best suit their domain or process.
We have put together a core checklist for evaluating ETL Testing Tools and Data Validation Tools that can be tailored to your ETL Testing requirements.
Category | Feature/CheckList | Yes/No | Additional information |
General | Architecture | NA | Provide a high-level overview of the tool architecture. |
License Model | NA | What is the license model? | |
Trial Version – Evaluation | Is a free trial available? What is the trial period? | ||
In-house deployment | Does the tool support in-house deployment | ||
Hardware Requirements | NA | What are the hardware requirements? | |
Web-based access | Does the tool provide web-based access? | ||
Data Sources | Relational DBs(SQL Server,etc) | <Specify required JDBC data sources here> | |
Cloud Technologies(Azure,Snowflate,Salesforce,etc) | <Specify required cloud data sources here> | ||
Flat Files Data Sources (XML,Excel,txt,etc) | <Specify required file formats here> | ||
NoSQL Data Sources (MongoDB, Elastic Search, etc.) | <Specify required NoSQL DataSources here> | ||
REST API, SOAP API | <Specify required API Data sources here> | ||
BI Reporting Data Sources(Tableau, Power Bi) | <Specify required BI tool support here> | ||
Source to Target Data Comparison | Metadata Validation | Can the tool help perform metadata validation across source and target? | |
Data Transformation Testing | Does the tool provide support testing of transformations? Please specify. | ||
Big Data testing | What are the maximum data volumes that can be tested in a single test case? Is there any limit on the number of columns in a single test case? | ||
SCD Type-2 dimension testing | Can the tool be used to test SCD Type-2 dimensions? Please explain how. | ||
Incremental ETL Testing | Does the tool support testing of incremental ETL/ELT processes? | ||
Parameterizing of tests | Is there support for parameterizing test cases? Can these parameters be set at run time? | ||
Support for ignoring minor differences | Does the tool provide options to compare data while ignoring minor differences that are within the acceptable range? | ||
Query Builder | Does the tool provide a Query Builder than can be used to form complex queries joining multiple tables? | ||
Automated mapping of columns | Is there any support to automate the mapping of source and target dataset columns? Please specify | ||
Support for different data types | Are there any unsupported data types during data comparison? | ||
Duplicate checks | Does the tool identify duplicates in the data being compared? | ||
Data Quality Testing | Data Profile Testing | Does the tool support data profiling and comparison of data profiles? | |
Flat file testing | Can the tool test the quality of flat-file by providing options for defined configurable data quality rules? | ||
Data Quality Dimensions | Does the tool support testing of data quality rules for different data quality dimensions such as completeness, accuracy, consistency, etc. | ||
Data Quality Scoring | Is there any option in the tool to automatically compute data quality scores? | ||
Data Observability & Anomaly detection | Does the tool support automated anomaly detection? | ||
Reporting | Exporting of results | Are there any options to export the data differences in a spreadsheet? | |
Column wise data differences | Can the tool generate a column-wise data differences report so that users can easily identify issues with each column? | ||
Emailing of test results | Can the test results be emailed to the users that are not licensed in the tool? | ||
Dashboards & reports | Are there any canned dashboards and reports? | ||
Custom reports | Does the tool support creating custom reports and emailing them? | ||
Reporting from external BI tools | Does the vendor expose a schema that can be used to create custom reports on test cases and results in external BI tools such as Power BI, Tableau, etc? | ||
Automation | Scheduling of test runs | Does the tool come with an inbuilt feature to schedule test case runs? | |
Automated CI/CD (Jenkins etc) | Can tests be run using CI/CD such as Jenkins? | ||
Version Control (GIT support) | Does the tool support version control of test scripts using a GIT repository? | ||
Moving tests between environments | Can test cases be migrated from one environment to another? Can this be automated using GIT and CI/CD tools? | ||
API Access | Is there an API for creating and executing test cases, test plans, and executing them? Does the tool offer API to create test cases? | ||
Command-line Execution | Is there a CLI tool for executing test cases and test plans? | ||
Automated Test Generation | Does the tool offer options to automatically generate test scripts? | ||
Integrations | Project Management Tools (JIRA) | Can the tool automatically raise an incident when there is a failure in <specify your incident management system> | |
Test Management Tools (ALM, Zephyr, X-ray) | Does the tool integrate with <Specify your test management tools>? | ||
Security & Audit | SSO Support | Does the tool support SSO with <specify your SSO provider>? | |
Role-based access control | Can new roles be dynamically configured? | ||
Configurable permissions on test cases | Are permissions on test cases configurable such that other users cannot modify them? | ||
Encryption of data | Is the data encrypted in transit and rest? Are passwords and access keys protected? | ||
Vulnerability assessments | Is the software scanned on a regular basis for vulnerabilities? Can the latest vulnerability report be shared? | ||
Masking of PII data | Does the tool provide options to automatically mask sensitive data such as PII? | ||
Support for multiple teams | Can the tool support multiple teams/projects in the same installation? | ||
Data security | Is any data moved out of the customer environment? | ||
Auditing of events | Does the tool provide a detailed audit of user actions such as logins, login failures, reads, writes, deletions, and unauthorized access attempts? Can the audit data to exported? | ||
Training and Support | Training | Does the vendor provide any training? Is it free or paid? | |
Certification | Are there any options to get certified in the tool? | ||
Ticketed Support | Is ticketed support available? What are the SLAs? | ||
Help documentation/ Videos | Is there help documentation and videos? |
