Pydantic
Introduction
Pydantic is a library with which you can parse data and perform validation.
A free translation of what they write about themselves + comments:
Data validation and settings management using python type annotations.
pydantic applies type annotations (type hints - see
PEP 484
)
at runtime and provides user-friendly error messages when the data is invalid.
You can define what the data should be using pure, canonical python. Then you can perform validation using pydantic.
Pydantic uses a capability of modern Python. Make sure your version is at least 3.7
It is advisable to install the latest stable version of Python. If needed - read
«Guide to installing Python on Linux»
Nevertheless, if you plan to exchange data in JSON format with Swagger or Open API, check the current
library compatibility.
Usage example
Let's look at the script PydanticDemo.py
from dataclasses import dataclass
from typing import Tuple
from enum import Enum
@dataclass
class IceCreamMix:
name: str
flavor: str
toppings: Tuple[str, ...]
scoops: int
def main():
ice_cream_mix = IceCreamMix(
"PB&J",
"peanut butter",
("strawberries", "sprinkles"),
2
)
print(ice_cream_mix)
if __name__ == '__main__':
main()
python PydanticDemo.py
IceCreamMix(name='PB&J', flavor='peanut butter', toppings=('strawberries', 'sprinkles'), scoops=2)
This script successfully demonstrates the ice cream type
Let's add a bit more OOP
from dataclasses import dataclass
from typing import Tuple
from enum import Enum
class Flavor(str, Enum):
chocolate = 'chocolate'
vanilla = 'vanilla'
strawberry = 'strawberry'
mint = 'mint'
coffeee = 'coffee'
peanut_butter = 'peanut butter'
class Topping(str, Enum):
sprinkles = 'sprinkles'
hot_fudge = 'hot fudge'
cookies = 'cookies'
brownie = 'brownie'
whipped_cream = 'whipped cream'
strawberries = 'strawberries'
@dataclass
class IceCreamMix:
name: str
flavor: Flavor
toppings: Tuple[Topping, ...]
scoops: int
def main():
ice_cream_mix = IceCreamMix(
"PB&J",
Flavor.peanut_butter,
(Topping.strawberries, Topping.sprinkles),
2
)
print(ice_cream_mix)
if __name__ == '__main__':
main()
$ python PydanticDemo.py
IceCreamMix(name='PB&J', flavor=<Flavor.peanut_butter: 'peanut butter'>, toppings=(<Topping.strawberries: 'strawberries'>, <Topping.sprinkles: 'sprinkles'>), scoops=2)
The script still works
What if we mistakenly choose a non-existent flavor or topping
def main():
ice_cream_mix = IceCreamMix(
"PB&J",
"smells like shit",
(Topping.strawberries, 111),
2
)
python PydanticDemo.py
IceCreamMix(name='PB&J', flavor='smells like shit', toppings=(<Topping.strawberries: 'strawberries'>, 111), scoops=2)
The script does not notice the catch.
To validate the data automatically, install pydantic and make just one change to the first line
from pydantic.dataclasses import dataclass
python PydanticDemo.py
Traceback (most recent call last): File "PydanticDemo.py", line 41, in <module> main() File "PydanticDemo.py", line 31, in main ice_cream_mix = IceCreamMix( File "<string>", line 7, in __init__ File "C:\Users\Andrei\python\pydantic\venv\lib\site-packages\pydantic\dataclasses.py", line 99, in _pydantic_post_init raise validation_error pydantic.error_wrappers.ValidationError: 2 validation errors for IceCreamMix flavor value is not a valid enumeration member; permitted: 'chocolate', 'vanilla', 'strawberry', 'mint', 'coffee', 'peanut butter' (type=type_error.enum; enum_values=[<Flavor.chocolate: 'chocolate'>, <Flavor.vanilla: 'vanilla'>, <Flavor.strawberry: 'strawberry'>, <Flavor.mint: 'mint'>, <Flavor.coffeee: 'coffee'>, <Flavor.peanut_butter: 'peanut butter'>]) toppings -> 1 value is not a valid enumeration member; permitted: 'sprinkles', 'hot fudge', 'cookies', 'brownie', 'whipped cream', 'strawberries' (type=type_error.enum; enum_values=[<Topping.sprinkles: 'sprinkles'>, <Topping.hot_fudge: 'hot fudge'>, <Topping.cookies: 'cookies'>, <Topping.brownie: 'brownie'>, <Topping.whipped_cream: 'whipped cream'>, <Topping.strawberries: 'strawberries'>])
pydantic did not let our code through. Let's examine the output in more detail
pydantic.error_wrappers.ValidationError: 2 validation errors for IceCreamMix
The number of errors and the class are indicated. This would help with debugging if we did not know in advance where the errors are and how many there are
flavor value is not a valid enumeration member; permitted: 'chocolate', 'vanilla', 'strawberry', 'mint', 'coffee', 'peanut butter' (type=type_error.enum; enum_values=[<Flavor.chocolate: 'chocolate'>, <Flavor.vanilla: 'vanilla'>, <Flavor.strawberry: 'strawberry'>, <Flavor.mint: 'mint'>, <Flavor.coffeee: 'coffee'>, <Flavor.peanut_butter: 'peanut butter'>])
Pydantic suggests the allowed values.
The same goes for the toppings, where 111 stands instead of an allowed value
toppings -> 1 value is not a valid enumeration member; permitted: 'sprinkles', 'hot fudge', 'cookies', 'brownie', 'whipped cream', 'strawberries' (type=type_error.enum; enum_values=[<Topping.sprinkles: 'sprinkles'>, <Topping.hot_fudge: 'hot fudge'>, <Topping.cookies: 'cookies'>, <Topping.brownie: 'brownie'>, <Topping.whipped_cream: 'whipped cream'>, <Topping.strawberries: 'strawberries'>])
Return the correct values for Flavor and Topping but replace scoops from 2 with '2'
def main():
ice_cream_mix = IceCreamMix(
"PB&J",
Flavor.peanut_butter,
(Topping.strawberries, Topping.sprinkles),
'2'
)
python PydanticDemo.py
IceCreamMix(name='PB&J', flavor=<Flavor.peanut_butter: 'peanut butter'>, toppings=(<Topping.strawberries: 'strawberries'>, <Topping.sprinkles: 'sprinkles'>), scoops=2)
scoops is still equal to two
Pydantic supports type coercion
BaseModel
To access additional capabilities such as serialization (Serialization) and JSON support, let's use the BaseModel class
Just to remind you, at first we had
from dataclasses import dataclass
Then
from pydantic.dataclasses import dataclass
And now we need to make
from pydantic import BaseModel
And remove the @dataclass decorator before class IceCreamMix:
class IceCreamMix: it needs to be replaced with class IceCreamMix(BaseModel):
and also add attribute names to the code that creates the IceCreamMix class object
that is name = "PB&J" flavor = Flavor.peanut_butter and so on
strawberries = 'strawberries'
class IceCreamMix(BaseModel):
name: str
flavor: Flavor
toppings: Tuple[Topping, ...]
scoops: int
def main():
ice_cream_mix = IceCreamMix(
name = "PB&J",
flavor = Flavor.peanut_butter,
toppings = (Topping.strawberries, Topping.sprinkles),
scoops = 2
)
python PydanticDemo.py
IceCreamMix(name='PB&J', flavor=<Flavor.peanut_butter: 'peanut butter'>, toppings=(<Topping.strawberries: 'strawberries'>, <Topping.sprinkles: 'sprinkles'>), scoops=2)
Everything works the same as before the changes.
Now you can output the result as JSON
print(ice_cream_mix.json())
python PydanticDemo.py
{"name": "PB&J", "flavor": "peanut butter", "toppings": ["strawberries", "brownie"], "scoops": 2}
Pay attention to the JSON you received above.
You can copy it, then if needed change it and create another object
directly from JSON using the method parse_raw()
For example:
another_mix = IceCreamMix.parse_raw('{"name": "New mix", "flavor": "mint", "toppings": ["cookies", "hot fudge"], "scoops": 2}')
print(another_mix.json())
{"name": "New mix", "flavor": "mint", "toppings": ["cookies", "hot fudge"], "scoops": 2}
If you accidentally make a mistake with an attribute value - pydantic will not let it pass
another_mix = IceCreamMix.parse_raw('{"name": "New mix", "flavor": "novichoke", "toppings": ["cookies", "hot fudge"], "scoops": 2}')
print(another_mix.json())
python PydanticDemo.py
Traceback (most recent call last): File "/home/avorotyn/python/pydantic/PydanticDemo.py", line 45, in <module> main() File "/home/avorotyn/python/pydantic/PydanticDemo.py", line 40, in main another_mix = IceCreamMix.parse_raw('{"name": "New mix", "flavor": "novichoke", "toppings": ["cookies", "hot fudge"], "scoops": 2}') File "pydantic/main.py", line 543, in pydantic.main.BaseModel.parse_raw File "pydantic/main.py", line 520, in pydantic.main.BaseModel.parse_obj File "pydantic/main.py", line 362, in pydantic.main.BaseModel.__init__ pydantic.error_wrappers.ValidationError: 1 validation error for IceCreamMix flavor value is not a valid enumeration member; permitted: 'chocolate', 'vanilla', 'strawberry', 'mint', 'coffee', 'peanut butter' (type=type_error.enum; enum_values=[<Flavor.chocolate: 'chocolate'>, <Flavor.vanilla: 'vanilla'>, <Flavor.strawberry: 'strawberry'>, <Flavor.mint: 'mint'>, <Flavor.coffeee: 'coffee'>, <Flavor.peanut_butter: 'peanut butter'>])
ValidationError as JSON
Error messages can also be formatted as JSON. You need to import ValidationError from pydantic and use try: except
from pydantic import BaseModel, ValidationError
…
def main():
try:
ice_cream_mix = IceCreamMix(
name = "PB&J",
flavor = "spring",
toppings = (Topping.strawberries, Topping.sprinkles),
scoops = 2
)
except ValidationError as e:
print(e.json())
python PydanticDemo.py
[ { "loc": [ "flavor" ], "msg": "value is not a valid enumeration member; permitted: 'chocolate', 'vanilla', 'strawberry', 'mint', 'coffee', 'peanut butter'", "type": "type_error.enum", "ctx": { "enum_values": [ "chocolate", "vanilla", "strawberry", "mint", "coffee", "peanut butter" ] } } ]
Bounds of allowed values
Suppose you want the number of scoops to be a required attribute with values from 0 to 5 excluding the bounds
from pydantic import BaseModel, ValidationError, Field
…
strawberries = 'strawberries'
class IceCreamMix(BaseModel):
name: str
flavor: Flavor
toppings: Tuple[Topping, ...]
scoops: int = Field(..., gt=0, lt=5)
python PydanticDemo.py
{"name": "PB&J", "flavor": "peanut butter", "toppings": ["strawberries", "brownie"], "scoops": 2}
2 scoops are set, so there are no errors.
Let's try 5 scoops
def main():
try:
ice_cream_mix = IceCreamMix(
name = "PB&J",
flavor = "spring",
toppings = (Topping.strawberries, Topping.sprinkles),
scoops = 5
)
except ValidationError as e:
print(e.json())
python PydanticDemo.py
[ { "loc": [ "scoops" ], "msg": "ensure this value is less than 5", "type": "value_error.number.not_lt", "ctx": { "limit_value": 5 } } ] Traceback (most recent call last): File "/home/avorotyn/python/pydantic/PydanticDemo.py", line 50, in <module> main() File "/home/avorotyn/python/pydantic/PydanticDemo.py", line 41, in main print(ice_cream_mix.json()) UnboundLocalError: local variable 'ice_cream_mix' referenced before assignment
Do not pay attention to the Traceback - the print could have been put inside try, but if everything is fine, the object is created and this error does not occur, and if it is not fine, pydantic catches the mismatch and outputs value_error
Validation using the decorator validator
Another useful way to set constraints - using @validator
It is used when you need to watch a single attribute
from pydantic import BaseModel, ValidationError, Field, validator
…
class IceCreamMix(BaseModel):
name: str
flavor: Flavor
toppings: Tuple[Topping, ...]
scoops: int = Field(..., gt=0, lt=5)
@validator('toppings')
def check_toppings(cls, toppings):
if len(toppings) > 4:
raise ValueError('Too many toppings')
return toppings
If you run this code with two toppings there will be no errors, so let's immediately add three more so that there are five in total.
def main():
try:
ice_cream_mix = IceCreamMix(
name = "PB&J",
flavor = "spring",
toppings = (Topping.strawberries, Topping.brownie,Topping.sprinkles,Topping.hot_fudge,Topping.whipped_cream),
python PydanticDemo.py
[ { "loc": [ "toppings" ], "msg": "Too many toppings", "type": "value_error" } ]
Now you can reduce the number of toppings to four and make sure there is no error.
@root_validator
It is used when you need to watch the whole model. For example, combinations of different attributes.
from pydantic import BaseModel, ValidationError, Field, validator, root_validator
…
Create another class - Container
strawberries = 'strawberries'
class Container(str, Enum):
cup = 'cup'
cone = 'cone'
waffle_cone = 'waffle cone'
class IceCreamMix(BaseModel):
name: str
flavor: Flavor
Let's set a condition: if the topping is hot_fudge then no cone can be given - only a cup
We will do the validation via @root_validator
@validator('toppings')
def check_toppings(cls, toppings):
if len(toppings) > 4:
raise ValueError('Too many toppings')
return toppings
@root_validator
def check_cone_toppings(cls, toppings):
container = values.get('container')
toppings = values.get('toppings')
if container == Container.cone or container == Container.waffle_cone:
if Topping.hot_fudge in toppings:
raise ValueError('Cones cannot have hot fudge')
return values
def main():
try:
ice_cream_mix = IceCreamMix(
name = "PB&J",
container = Container.waffle_cone,
flavor = "spring",
You should still have the hot fudge topping left from the previous example, if not - add it and run
python PydanticDemo.py
[ { "loc": [ "__root__" ], "msg": "Cones cannot have hot fudge", "type": "value_error" } ]
Final code of the example
The brief overview of pydantic's capabilities has come to an end.
Thank you for your attention, below is the full code for this article.
from pydantic import BaseModel, \
ValidationError, Field, validator, root_validator
from typing import Tuple
from enum import Enum
class Flavor(str, Enum):
chocolate = 'chocolate'
vanilla = 'vanilla'
strawberry = 'strawberry'
mint = 'mint'
coffeee = 'coffee'
peanut_butter = 'peanut butter'
class Topping(str, Enum):
sprinkles = 'sprinkles'
hot_fudge = 'hot fudge'
cookies = 'cookies'
brownie = 'brownie'
whipped_cream = 'whipped cream'
strawberries = 'strawberries'
class Container(str, Enum):
cup = 'cup'
cone = 'cone'
waffle_cone = 'waffle cone'
class IceCreamMix(BaseModel):
name: str
container: Container
flavor: Flavor
toppings: Tuple[Topping, ...]
scoops: int = Field(..., gt=0, lt=5)
@validator('toppings')
def check_toppings(cls, toppings):
if len(toppings) > 4:
raise ValueError('Too many toppings')
return toppings
@root_validator
def check_cone_toppings(cls, values):
container = values.get('container')
toppings = values.get('toppings')
if container == Container.cone or container == Container.waffle_cone:
if Topping.hot_fudge in toppings:
raise ValueError('Cones cannot have hot fudge')
return values
def main():
try:
ice_cream_mix = IceCreamMix(
name="PB&J",
container=Container.waffle_cone,
flavor=Flavor.peanut_butter,
# flavor='unknown flavour'
toppings=(Topping.strawberries, Topping.brownie,
Topping.sprinkles),
# on validator
# toppings=(Topping.strawberries, Topping.brownie,
# Topping.sprinkles,Topping.cookies, Topping.sprinkles),
# on root_validator
# toppings=(Topping.strawberries, Topping.brownie,
# Topping.sprinkles,Topping.hot_fudge),
scoops=2
# scoops=5
)
print(ice_cream_mix.json())
except ValidationError as e:
print(e.json())
if __name__ == '__main__':
main()
The second part of the article
Article author: Andrei Olegovich