Revolutionizing Test Data Automation for Financial Institutions

In our modern, data-centric world, organizations encounter numerous challenges when testing critical financial systems and applications. Managing vast amounts of data for testing these applications while ensuring data privacy, consistency and integrity can appear daunting. However, with the right tools and strategies, these challenges can become opportunities for innovation and success.

In a new series of “Solution Spotlights,” we delve into how GenRocket’s Test Data Automation platform has enabled financial services companies to tackle their most significant data challenges, enhancing efficiency, quality, and agility in their testing processes.

Following are 4 success stories that illustrate the power and versatility of GenRocket’s Test Data Automation platform in meeting these challenges.

Automating Event Data Provisioning for Comprehensive Software Testing

GenRocket Drives Efficiency and Quality Gains for Global Payment Processor

Background

A global payment processor with over 300 million customers worldwide chose GenRocket to address their test data challenges. The Payment Quality Architecture Team was the first to adopt GenRocket within the company, focusing on two use cases in the Payment Services domain.

One of the key use cases involved generating event data to simulate the process of authenticating credit card customers and adjusting their account balances in response to card payments.

This event data, critical for testing the payment processing system’s accuracy and reliability, is generated daily by the company’s internal teams responsible for managing customer authentication and payment disputes.

The data is stored in an AWS S3 bucket in the form of Parquet files, which contain the event data, and metadata files, which include event counts and total transaction amounts for corresponding parties.

Challenges

The team faced several challenges in managing this event data:

• Generating metadata files that accurately sum up amounts in each corresponding Parquet file
• Ensuring Parquet and metadata files match precisely
• Lacking a dynamic utility for converting JSON data to Parquet format
• Needing a solution that integrates seamlessly with their automated testing processes

Dependent on other teams and bound by strict file naming conventions, the Payment Quality Architecture Team had to choose between developing a custom Java utility for JSON to Parquet conversion or leveraging GenRocket’s Test Data Automation capabilities.

GenRocket Solution

After evaluating their options, the team chose GenRocket to address their event data provisioning needs. GenRocket’s platform provided key capabilities that solved the team’s challenges:

1. Dynamic File Generation

   GenRocket’s flexible synthetic data generation engine enabled the team to create Parquet and metadata files with the required naming conventions, including current date, time, and seconds. This ensured files matched precisely and could be easily identified and managed.

2. Automated Data Conversion

   Leveraging GenRocket’s 700+ data generators and 100+ data formatters, the team automated the conversion of JSON event data into Parquet format. This eliminated the manual effort previously required and accelerated their testing processes.

3. Seamless Test Automation Integration

   The team integrated GenRocket into their automated testing suite, including their Jenkins CI/CD pipeline. This enabled them to efficiently provision the exact test data needed for daily regression runs, saving significant time and effort.

Benefits and Achievements

By implementing GenRocket, the Payment Quality Architecture Team realized substantial benefits:

• Reduced Dependency on External Teams

  GenRocket empowered the team to independently generate the event data required for testing, eliminating their reliance on the company’s internal teams responsible for managing customer authentication and payment disputes. This increased the team’s agility and control over the testing process, enabling them to efficiently validate the payment processing system’s accuracy and reliability.

• Significant Time Savings

  Automating the JSON to Parquet conversion and file generation processes saved the team approximately one hour per day in their regression testing runs. Over the course of a year, this equates to over 250 hours saved, allowing the team to focus on higher-value tasks.

• Streamlined CI/CD Pipeline

  Integrating GenRocket into their Jenkins CI/CD pipeline empowered the team with on-demand test data provisioning. They can now dynamically generate event data to match any testing scenario, enabling more comprehensive testing and higher quality software releases.

• Reusability and Scalability

  The GenRocket solution provides a reusable and scalable framework for test data generation. The team can efficiently model additional event types and scenarios, provisioning billions of rows of data in minutes. This positions them to handle evolving testing needs and increasing data volumes with ease.

Conclusion

GenRocket’s Test Data Automation platform enabled the Payment Quality Architecture Team to overcome their event data provisioning challenges. By dynamically generating Parquet and metadata files, automating data conversion, and integrating with their CI/CD pipeline, the team significantly streamlined their regression testing processes.

The time savings and productivity gains achieved in this use case demonstrate the value GenRocket provides in enabling efficient, high-quality software testing. The extensible nature of the solution ensures the team can effectively handle new event types and data scenarios, positioning them for continued success.

As adoption of GenRocket expands within the organization, the platform’s capabilities for automated test data generation, masking, subsetting, and synthetic data generation will continue to drive efficiency and quality improvements across software development and testing functions.

Overcoming Test Data Challenges in Legacy System Modernization

Conquering Complex Data Challenges for Successful Transformation

Background

A global payment processor faced the challenge of modernizing a legacy data platform that had been in use for over 20 years. The platform, a Daily Batch Data Collection and Settlement product, is responsible for reading files from various partners, processing them into a fixed file format, and storing them in a database. As part of the modernization effort, the platform is being upgraded to accept both fixed data formats and JSON, offering more flexibility and compatibility.

Problem Statement

The legacy data platform’s reliance on fixed file formats made it challenging to generate synthetic test data for lower environments. The traditional approach involved manually scrubbing sensitive data from raw files, which was time-consuming, error-prone, and lacked scalability. Moreover, maintaining data consistency and integrity across various data segments within the platform posed a significant challenge.

GenRocket Solution

GenRocket emerged as a robust solution to address these challenges head-on. With its ability to generate synthetic test data efficiently while adhering to business logic and referential integrity requirements, GenRocket has proven invaluable in several key use cases:

1. Generating Test Data for Modernization

   GenRocket’s versatile platform generated test data in both fixed file and JSON formats, aligning perfectly with the modernization project’s requirements. The tool automated the data generation process, reducing the time needed to create test data from several days to hours. This efficiency gain allowed the team to focus on other critical aspects of the project.

2. Testing and Troubleshooting with Historical Data

   GenRocket’s ability to generate historical test data has been a game-changer for the team. Users can easily go back in time and generate files for specific periods, such as last month, enabling them to replicate past scenarios and analyze system behavior. This functionality has proven invaluable for troubleshooting and testing, saving countless hours and resources.

3. Handling Large Data Volumes with Ease

   With some files reaching sizes of two gigabytes and containing 100,000 to 200,000 records, the application dealt with substantial data volumes. GenRocket demonstrated its ability to efficiently generate these large datasets, facilitating comprehensive testing without compromising performance.

4. Ensuring Referential Integrity Across Complex Data Structures

   Maintaining referential integrity between data segments across multiple domains and files was a complex challenge. GenRocket’s ability to model these intricate relationships and generate data that adheres to the required constraints has been a significant achievement. The tool ensured that all generated data maintained accurate data relationships, a critical aspect of the platform’s functionality.

Implementation Process

1. Project Specification: The team outlined the project’s specifications and requirements, providing a clear roadmap for the implementation.
2. Design and Modeling: The data model was accurately designed and modeled by importing metadata from the target data environment into the GenRocket platform, allowing for precise control over data generation.
3. Fixed File Generation: GenRocket was used to generate synthetic data in the fixed file format, aligning with the legacy platform’s requirements and providing a seamless transition to the modernization project.
4. JSON File Conversion: As part of the modernization effort, GenRocket was utilized to generate the same data in a consolidated JSON format, ensuring data compatibility and marking a significant milestone in the project.

Benefits and Achievements

• Successfully implemented GenRocket for generating test data in fixed and JSON formats, streamlining the modernization process.
• Reduced test data generation time from days to hours, resulting in significant efficiency gains.
• Enabled efficient troubleshooting and testing through historical data generation, saving time and resources.
• Demonstrated scalability by generating large data volumes (up to 300,000 transactions) in less than an hour.
• Maintained referential integrity for complex data relationships across multiple domains and files.
• Seamlessly modernized its legacy application with the help of GenRocket’s flexible output formatting capabilities.

Conclusion

GenRocket has played a pivotal role in modernizing the legacy data platform for this global payment processor. By providing an efficient, flexible, and scalable test data generation solution, GenRocket has empowered the team to overcome complex challenges, meet project specifications, and ensure data consistency and integrity.

The successful implementation of GenRocket has not only streamlined the modernization process but also laid the foundation for future enhancements and expansions. As the organization continues to evolve its data platform, GenRocket’s innovative capabilities will remain a key enabler of success.

Conquering Performance Testing at Scale with GenRocket

Generating 20 Million Synthetic Users for Comprehensive API Testing

Background

A global financial services company annually needed to conduct performance testing and synthetic data generation for an internal API. The scenario involved assessing the functionality and scalability of an API where transaction records were stored in a large-scale Postgres database. The database held a staggering 7 billion records, but the sensitive financial data and personally identifiable information (PII) contained in these records made typical production data masking untenable due to potential security risks. Creating an ad hoc script to mask the data was also deemed too time-consuming.

GenRocket’s Synthetic Test Data Automation platform was chosen to efficiently generate the large volume of synthetic test data required, while ensuring data security and referential integrity.

Challenges

The financial services company faced several challenges in this performance testing initiative:

• Generating millions of synthetic test data records to adequately stress-test the API
• Ensuring a wide variety of conditional data for comprehensive testing
• Provisioning data that could not be pulled from production environments due to PII concerns
• Enabling flexibility to modify data attributes and logic for future testing needs
• Avoiding the time-consuming process of manual test data creation or complex scripting

GenRocket Solution

GenRocket addressed these challenges through its powerful test data generation capabilities:

1. Domain Modeling with Custom Generation Logic
   • Test Data Cases were designed using the GenRocket platform to specify the data generation rules for various attributes such as transaction dates, IDs, company names, and compliance ratings.
   • GenRocket’s flexible platform allowed for precise definition of data for testing business rules and data variations for both positive and negative test scenarios.
2. High-Speed Data Generation with Scenario Thread Engine
   • To expedite the creation of the 20 million user records, GenRocket’s Scenario Thread Engine was leveraged to perform high speed parallel data generation.
   • Four concurrent threads were used to generate data across four scenarios, with the output organized into six-month intervals per the client’s requirements.
   • This multi-threaded approach dramatically reduced the time needed to create the test data.
3. Streamlined Execution and Database Integration
   • The entire data generation process was executed with a single command, greatly simplifying the client’s workflow.
   • Integration with the client’s Postgres database was seamlessly handled through a GenRocket properties configuration file specifying the database connection details.
   • This enabled the direct insertion of the generated test data into the database, establishing an end-to-end testing process.

Benefits and Results

By implementing GenRocket, the financial company realized significant benefits:

• Massive Scale and Speed

  GenRocket generated test data for 20 million users across four tables, totaling 1.2 billion attribute values. The same process that previously took 2 days to generate 1 million users was reduced to just 13 hours for 20 million users—a 74x speed improvement.

• Comprehensive Test Coverage

  The synthetic data provided by GenRocket enabled thorough testing of the API across a wide range of scenarios and edge cases, ensuring robustness and reliability.

• Seamless Database Integration

  Direct insertion of test data into the Postgres database was made possible through GenRocket’s flexible configuration options, establishing an efficient end-to-end testing process.

• Time and Effort Savings

  GenRocket’s Test Data Automation platform dramatically reduced the time and manual effort required compared to traditional data provisioning approaches, allowing the testing team to focus on higher-value tasks.

Conclusion

GenRocket proved to be instrumental in enabling the global financial services company to conduct extensive performance testing of their internal API. The platform’s ability to rapidly generate huge volumes of synthetic test data while maintaining referential integrity and complex data relationships showcased its versatility and effectiveness.

By leveraging GenRocket, the company significantly reduced their test data generation time, achieved comprehensive test coverage, and established an efficient end-to-end testing process integrated with their database. The successful execution of this testing initiative demonstrated GenRocket’s value in supporting the performance and scalability needs of enterprise-scale applications.

Revolutionizing Test Data Generation for Loan Processing and Vendor Payments

GenRocket Automates Fixed File Generation, Saving Time and Ensuring Accuracy

Background

A global payment processing company serving over 300 million customers annually faced significant challenges in testing two critical applications: Loan Boarding (the process of setting up a new loan in the system) and Vendor Payments. These applications, managed by the data services team, relied on manual processes for generating fixed file formats. The data had to be extracted from the financial tables with their Finacle application environment and meticulously formatted into specific fixed file layouts. For Loan Boarding, the team needed approximately 20,000 records, with each record taking up to 10 minutes to prepare manually using macros. Similarly, the Vendor Payments application required the generation of around 7,000 records, a process that was equally time-consuming and prone to errors due to the need for manual data conditioning.

Challenges

The data services team encountered several obstacles in their data generation processes:

1. Manual extraction and formatting of data from Finacle tables into fixed file formats for Loan Boarding and Vendor Payments.
2. Excessive time and effort required to process individual records, with each record taking 5-10 minutes to prepare.
3. Complexity in manually setting up and configuring criteria for data extraction.
4. High risk of errors and inconsistencies in the manually generated data.
5. Lack of a scalable and reusable solution to accommodate changing criteria and future data requirements.

GenRocket Solution

GenRocket’s Synthetic Test Data Automation platform provided a comprehensive solution to automate the generation of fixed file formats for Loan Boarding and Vendor Payments. The platform addressed the challenges faced by the data services team through the following key features:

1. Seamless Integration with Finacle
   • GenRocket seamlessly integrated with the Finacle system, enabling efficient data extraction based on specific criteria.
   • The platform simplified the process of setting up and configuring extraction criteria, eliminating the need for manual intervention.
2. Automated Data Generation
3. GenRocket retrieved the required account details from Finacle and automatically generated the data in the specified fixed file formats, such as CSV, XML, or JSON.
4. The automated data generation process eliminated the need for manual data conditioning, ensuring consistency and accuracy.
5. Scalability and Reusability
6. GenRocket’s solution offered unparalleled scalability, enabling the generation of thousands of records within minutes.
7. The platform’s reusable test data projects and test data cases allowed for easy adaptation to changing criteria and future data requirements.

Benefits and Achievements

By implementing GenRocket, the data services team achieved remarkable results:

• Dramatic Time Savings: The time required to generate records was reduced from 5-10 minutes per record to less than 5 minutes for the entire process. GenRocket enabled the generation of 20,000 Loan Boarding records and 7,000 Vendor Payment records in a matter of minutes.
• Enhanced Data Accuracy and Consistency: GenRocket eliminated manual errors and inconsistencies, ensuring the generation of high-quality test data.
• Streamlined Processes: The automated solution streamlined the data generation processes for both Loan Boarding and Vendor Payments, enabling faster and more efficient testing cycles.
• Flexibility and Adaptability: GenRocket’s flexible architecture allowed for easy adaptation to changing criteria and future data requirements, future-proofing the data services team’s testing efforts.

Conclusion

GenRocket’s Synthetic Test Data Automation platform revolutionized the way the data services team approached test data generation for Loan Boarding and Vendor Payments. By automating the extraction and formatting of data from Finacle tables into fixed file formats, GenRocket significantly reduced manual efforts, minimized errors, and accelerated the overall testing process.

The platform’s scalability, reusability, and adaptability empowered the data services team to generate high-quality test data efficiently, ultimately contributing to the successful testing and deployment of these critical applications. With GenRocket, the global payment processing company achieved faster time-to-market, improved data accuracy, and enhanced overall software quality.