Testing is essential for every system, and testing quality depends on the quality and quantity of the data in the test environment. When production data is used in a test environment, it presents a risk, and that risk is not completely mitigated by using a Data Anonymization Solution.
Using test data manufactured by Test Data Services ensures that your production data remains secure, because you never need to expose any of it to your test environments.
There are several APIs available for supplying test data, validating test data and working with test data.
For example a simple API can be used to provide a random identity from the Free DataSet of 2.5 million manufactured identities, which will include a Medicare Number. That Medicare number conforms to the CheckSum and Formatting requirements of Australian Medicare numbers, can can also be validated against an API that confirms the validity of that Medicare number. Other APIs allow for easy automation of email validation and Multi Factor Authentication, greatly simplifying the testing automation process.
Built for scale
The entire Test Data Services offerring is built on Serverless components on the AWS platform, and is able to operate at scale, however rate limiting is enforced to ensure reasonable usage for each user of the service.
Just a few numbers
There are two primary Data Set offerrings. A Free 2.5 million set of static identities with limited internal relationships, and a 25 million identity dataset, with individualts, businesses, assets and the associated relationships.
Free Manufactured Identites
Unique API endpoints
Distinct GNAF Addresses
The family names in the manufactured data are based on the statistical distribution of current Australian Names. First and middle names are based on popular names for the year of birth. The age and gender profile of the manufactured population is based on Australian Census data and addresses are based on Australian GNAF addresses. Special identifiers such as Drivers Licences, credit cards, Medicare Numbers and Passport Details are randomly included, based on Australian statistics by age. This ensures that the data has the appearance of real data.
Use Cases for Test Data
It’s all about testing, and improving the quality of testing while reducing risks arising from the data used while testing.
Data is required for meaningful testing of even the most simple systems. Having a virtually unlimited supply of test data simplifies testing activities, even if a tiny fraction of available data is used. For example, if a test requires a properly formatted Medicare Number or a Tax File Number, then simply fetching random records using the Free API will deliver millions of such values.
End-to-end testing of integrated systems
Applications are increasingly connected to other applications. End to end testing of multiple applications can be facilitated by using a set of test data that is common to all applications. Integration with various APIs to validate addresses, identity and various other information extend data dependencies beyond the traditionally considered scope of a testing engagement. By leveraging a consistent Data Set, along with supporting APIs, it is possible to configure multiple application environments in different organizations to participate in combined end to end tests.
Identities for CI testing
When applications need to validate processes on a continuous basis, such as a Continuous Integration pipeline, the ability to fetch a new record or each test is very useful. For example, testing a signup process may require a unique email address and may involve an email validation. Randomly selecting an identity from a set of 25 million for each iteration will provide a ‘mostly unique’ identity each time, and using the Email Address Verification API will allow validation of the ‘one off’ asynchronous processes, such as user signup processes.
Random variation of field contents / processes
By running randomly fetched identities through simple business processes, the mix of unexpected combinations of field contents may generate unexpected errors in the application. For example, a NYC (Know Your Customer) business process may require 100 points of Identification, and someone may supply source document references that would appear, on the surface to be reasonable, but may not actually be acceptable, such as a Tasmania Birth Certificate from 1969. (Birth certificates from TAS can only be electronically verified from 1970.)
Continuous monitoring of processes involving unique data
While application performance monitoring has matured and covers many high frequency use cases using a small subset of User IDs or Data Values. Synthetic monitoring does not generally include unique data, such as the registration of a new customer with a previously unseen email address. By combining the random, just in time retrieval of an identity with email validation, it is possible to continuously monitor signup processes in a production environment.
When penetration testing is undertaken, it is important that the data being protected is not real. If the penetration test is application oriented, rather than infrastructure oriented, then significant cost savings can be achieved by deploying the entire application stack to the cloud, even when corporate policy for production data does not permit a complete cloud-based deployment. The stack can be tested from the Internet without giving the Penetration Testers access to the Corporate Network and once testing is complete, the environment can be destroyed, saving costs while isolating any test related impacts to the dedicated environment that no other users depend on and without putting any personally identifiable data at risk.
To support health informatics
Health Informatics is the appropriate and innovative application of the concepts and technologies of the information age to improve health care and health. IT Systems operating in the Health Context have a very high level of Integration with external systems, while at the same time being custodians of sensitive personal information. When multiple integrated applications are configured with an internally consistent set of identities, health services identifiers and appropriate validation end-points, it is possible to run substantial end to end tests of systems comprising multiple applications, increasing the quality of service delivery while not compromising the security of personally identifiable data.
To support and validate anonymisation testing
Some technical solutions need to make various aspects of some data available to third parties. For example, Health Related information may be made available to researchers to support valuable health research. Making such data available usually involves aggregation and anonymization of personally identifiable information, to ensure that information about individuals in a dataset can be derived from the research data set.
Set up application environments with Manufactured Data
Appart from Test Environments, Training and Demonstration environments should also be configured with realistic looking manufactured data. Viewing application screens that contain realistic data delivers increased confidence in the quality of the environmen.
It's time to change the game
Try our free API to retrieve demo identity dataShow me the data!
About Test Data Services
Test Data Services was created out of necessity for large volumes of realistic test data as part of multiple large scale Load and Performance test engagements over more than 20 years. Australian domain knowledge in the finance, education, health, government, manufacturing and retail sectors has provided a solid foundation for a manufactured dataset that can be used, as is, in most situations.
There are two basic offerings. A Free Dataset, containing 2.5 million (static) manufactured identities and a subscription only DataSet containing 25 million (dynamic) identities. These Identities are supported by subscription based validation end-points such as Drivers Licence check, Medicare Check and ABN check.
Other related subscription based services have been developed to work closely with the manufactured test data, to support email validation and MultiFactor Authentication, which are two of the more difficult problems to solve when automating the testing of realistic end to end business processes.
The main benefit to using the Manufactured Test Data offerred by Test Data Services is that multiple organizations can use the same data, and the same validation end points, enabling cross organization business process testing.
The following pricing is based on the 2.5 million identity data set, is in Australian Dollars, exclusive of GST. If you are based in Australia, a GST of 10% will be added to the total cost."
Just buy what you need. If you only need to run a test, or setup an application with data once a month, then Daily pricing may be sufficient. If you intend to use the service more than once a week, then Yearly pricing is probably the cheapest option."
Note our 'Fair Use' policy, which rate-limits the number of calls per subscription, calculated at 1 TPS for the entire subscription period (e.g. 32M transactions per year) with a 20% limit in any single period of the lower subscription. (i.e. a limit of 6.4 million transactions in a single month of an annual subscription, which imposes a limit of 1.28M transactions in a week, which is 256K transactions in a single day, and so on)"View Rate Limit Details
- Get Random Itentities
- Get Identity by UID
- Medicare Number
- Driver License
- Super Fund
- Medicare Provider
- AHPRA Registration
- Access emails via simple API calls for all identities
- Get email Latency Times
- Google Authenticator
- Supports a KEY for every identity
- Simple API production of 6 digit code
If there's something we can help you with, jut let us know. We'll be more than happy to offer you our help