Skip to content

#264: Custom Providers for Faker

Faker is a test data generator we explored in the last post. Despite the large list of options to generate data, sometimes we need data in a format that Faker cannot offer. Should that happen, we can create our own data provider. Let us figure out how to do that.

Creating the custom provider

Our custom provider is a plain Python class that inherits from BaseProvider. The important part is that we use the self.random_element() method and do not access a different random generator. If we try that, we get random values, but the seed_instance() method will have no impact on our provider.

To create test data for a university, we could write a provider like this one:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
from faker import Faker
from faker.providers import BaseProvider

class UniversityProvider(BaseProvider):
    university_names = [
        "Springfield University",
        "Shelbyville Institute of Technology",
        "Capitol City College",
        "Metropolis University",
        "Gotham City Academy"
    ]

    faculty_names = [
        "Faculty of Engineering",
        "Faculty of Arts and Sciences",
        "Faculty of Medicine",
        "Faculty of Business Administration",
        "Faculty of Law"
    ]

    course_titles = [
        "Introduction to Artificial Intelligence",
        "Advanced Python Programming",
        "Quantum Computing 101",
        "History of Modern Art",
        "Principles of Microeconomics"
    ]

    building_names = [
        "Newton Hall",
        "Curie Science Center",
        "Einstein Library",
        "Turing Auditorium",
        "Bohr Research Building"
    ]


    def university_name(self):
        return self.random_element(self.university_names)

    def faculty_name(self):
        return self.random_element(self.faculty_names)

    def course_title(self):
        return self.random_element(self.course_titles)

    def building_name(self):
        return self.random_element(self.building_names)

Register the provider

Before we can use our provider with Faker, we need to register it:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
fake = Faker()
fake.seed_instance(42)
fake.add_provider(UniversityProvider)

# Generate data
print(fake.university_name())  
# Springfield University

print(fake.faculty_name())
# Faculty of Medicine

print(fake.course_title())
# Advanced Python Programming

print(fake.building_name())
# Einstein Library

If we run this code, we get a random set of university and faculty names, course titles and building names. That should help us already, but what if we want to influence the random generator a bit more?

Weighted data

Should we have a preference on how often certain values should be generated, we can make a slight modification and use a weighted approach. For that we put our data into an OrderedDict() and set a value for the weight. Be aware that the weight is only an indicator, but it is not guaranteed that we get values that exactly match what we specify.

To get grades where C and F are the most common and A is the least used, we could write this code and put it into our UniversityProvider class:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
from typing import OrderedDict
...

grades_name = OrderedDict(
[
    ("A", 1),
    ("B", 2),
    ("C", 5),
    ("D", 3),
    ("F", 5)
])

def grades(self, count):
    return self.random_elements(self.grades_name, count, unique=False, use_weighting=True)

When we now generate 100 grades 10 times in a row, we should get a distribution of the values that matches our weighted dictionary:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
from collections import Counter
for i in range(10):
    fake_grades = fake.grades(100)
    counter = Counter(fake_grades)
    print(counter)

# Counter({'C': 31, 'F': 28, 'D': 23, 'B': 11, 'A': 7})
# Counter({'C': 36, 'F': 31, 'D': 17, 'B': 12, 'A': 4})
# Counter({'F': 31, 'C': 29, 'D': 21, 'B': 14, 'A': 5})
# Counter({'F': 36, 'C': 35, 'D': 15, 'B': 10, 'A': 4})
# Counter({'C': 34, 'F': 29, 'D': 21, 'B': 10, 'A': 6})
# Counter({'F': 37, 'C': 27, 'D': 15, 'B': 14, 'A': 7})
# Counter({'C': 32, 'F': 29, 'D': 21, 'B': 14, 'A': 4})
# Counter({'F': 33, 'C': 32, 'D': 22, 'B': 11, 'A': 2})
# Counter({'F': 34, 'C': 32, 'D': 20, 'B':  9, 'A': 5})
# Counter({'F': 36, 'C': 25, 'D': 22, 'B': 13, 'A': 4})

As we can see, F and C are usually the two most common grades, while A is the one with the least occurrences.

Conclusion

With a bit of our own code, we can create a custom provider that let us extend Faker. That way we can keep using Faker and all its amenities, even when we need a data format that does not ship with Faker.