Many times, users would like to sample a large dataset to pull only a few records for analysis. This transform would allow them to take a random sample of the data flowing through the transform. We should use the sampling method described for HEDIS reporting.
Use case(s)
I would like to sample my member database for calculating the Adult BMI Measure HEDIS measure. In this case, I would like to build a pipeline to pull records from my member database, sort them alphabetically using a OrderBy plugin (in development), then apply a sampling methodology as follows: input a sample size, an over sampling percentage (the final sample size is calculated as Final Sample Size = Input Sample Size * (Input Sample Size * Over Sampling Percentage) (round up to the next whole number)). So we will choose every Nth = (Total Records/Final Sample Size) member. The first member is chosen using a (random number between 0 and 1) * N and then every Nth member after that.
User Storie(s)
As a hydrator user, i would like to sample the records in my pipeline so that a large number of records go in, but only a specified number of records + over sampling percentage comes out of the transform.
Plugin Type
Aggregate (Or maybe a transform)
Configurables
This section defines properties that are configurable for this plugin.
User Facing Name
Type
Description
Constraints
Input Sample Size
String
The number of records that you would like to sample from the input records.
Input Sample Percentage
String
The % of records that you would like to sample from the input records.
0 - 100
Oversampling Percentage
String
The % of additional records you would like to include in addition to the input sample size to account for oversampling.