Mocking AWS Dynamo Db calls during testing

5 min readApr 28, 2022

Introduction

It is always a good idea to write test cases for your application when you build any new feature or the application from scratch itself. Testing allows your code to be more bug free and gives a clear sense of where the development should go and if there are any bottlenecks involved.

In this space, I am going to go over the basics of how to mock AWS Dynamo DB calls in while testing in Python. I will be using the moto library to do that.

Setup

I will be using Python version 3.9.x for the demonstration and following additional libraries:

pytest — version 7.1.1
moto — version 3.1.5
boto3 — version 1.21.45

I will be using virtualenv to manage my packages, but you are open to use whatever seems appropriate.

To install these libraries, use the following commands on your terminal or command prompt:

pip install pytest==7.1.1

Installs pytest version 7.1.1

pip install moto[all]==3.1.5

Installs moto version 3.1.5 with all of it’s dependencies. The moto library can mock a lot of AWS services. For installing all of it’s dependencies — which it doesn’t install by default — we need to add the [all] part while installing it.

pip install boto3==1.21.45

Installs boto3 version 1.21.45

You can check out the complete list of requirements for this project on my GitHub page.

Project Structure

The project structure for this demo is going to look like this

MockDynamoDbExample
├── __init__.py
├── src
│ ├── __init__.py
│ └── service.py
└── test
 ├── __init__.py
 └── test_service.py

Implementation

Let’s start implementing the mocks.

First, create a base folder (in my case, it is MockDynamoDbExample). Then create two sub-folders src and test. As the name suggest, src will contain the code which needs to be tested and test will contain our tests to the given source code.

Next, we need a running service that will more or less reflect the operations we want to perform in the real world. Create empty __init__.py and service.py files in the src folder. The __init__.py is created so that the current folder is treated as a module. We are not going to edit that file. By just existing, the file indicates to the Python Interpreter that the current folder is a module. The service.py file is where we are going to write the source code that requires testing.

Let’s start by writing all the imports and the constructor method

The gist of the code in the constructor method is that it creates a boto3.resource instance depending on the env value passed. During testing, we won’t be deploying an instance of Dynamo Db, that’s why I had put an env check. You are free to play around with it.

Let’s implement the __get_table method which is responsible for either getting us the table, if it exists or create a new one. Copy the below code in the same class

To give an overview, our table contains a primary-key username which is of key type HASH and age as a key type of RANGE. I am using the create_table method that is provided as part of the boto3.resource instance to create the desired table. The upstream code, namely __get_table method is responsible for checking whether the table exists in the first place or not. If it does, we return early with it’s instance.

Now, we implement all the CRUD operations that are required to perform the task for the database.

The above CRUD operations can vary depending on your use case. For the purpose of this blog, I will be using these basic operations. You can implement your own CRUD operations by following this link.

The complete service.py file should look like

Generally, I use double-underscore’s in front of methods that will be used internally within a class and are not required outside of it. I feel this to be a good coding standard. However, you are open to use the way you feel like.

Testing

Moving to the testing of this service, create two files __init__.py and test_service.py files under the test directory. As with the src files, the __init__.py will be used for setting the current directory as module and will be empty. The test_service.py will be where our test code is going to be.

I am using a combination of pytest and unittest modules to perform testing.

Let’s start by writing the setUp and tearDown methods. These methods are automatically whenever a test case is executed from the test suite.

I am importing few things here, like the Service class that we implemented, the mock_dynamodb decorator from the moto library and TestCase from the unittest module.

The mock_dynamodb is going to be helpful in mocking all of our Dynamo Db related calls. This import will be used as a decorator. To learn more about decorators in general, follow this link.

In the setUp method, we are creating a Service class instance with ”user” as the table name and environment as ”test”. The tearDown method implements the cleanup functionality. Both of these methods are called before and after every test case respectively.

Moving on, the following code implements the first test case which tests for create_user method in services

We are testing the creation success using assert True at the end.

Let’s run the test case by going over to the terminal or command prompt. Make sure that you are in the root directory of this project while executing the following command:

pytest test/test_service.py

It should print out the following on Linux system

(venv) ibi@ibi:~/MockDynamoDbExample$ pytest MockDynamoDbExample/test/test_service.py 
======================================================================================== test session starts =========================================================================================
platform linux — Python 3.9.12, pytest-7.1.1, pluggy-1.0.0
rootdir: /home/ibi/MockDynamoDbExample
collected 1 itemsMockDynamoDbExample/test/test_service.py … [100%]
========================================================================================= 1 passed in 1.46s ==========================================================================================

For windows powershell, the output should look like

(venv) PS D:\MockDynamoDbExample> pytest test\test_service.py
================================================= test session starts =================================================
platform win32 — Python 3.9.10, pytest-7.1.1, pluggy-1.0.0
rootdir: D:\MockDynamoDbExample
collected 1 itemstest\test_service.py … [100%]
================================================== 1 passed in 1.63s ==================================================

Adding another test case to test getting a user by username.

In this test case, we are testing the insertion and fetch performed by Dynamo Db.

The complete file should look like

Additionally, if you don’t want to use a class based approach and would like to write independent functions, then you can do the following

As you can see, it can get a little tedious if you have a lot of test cases to write with a start and end common functionality.

Conclusion

moto is an amazing library to mock any AWS Dynamo DB based calls. You can check out the development on their GitHub page.

A trivia, prior to the moto library version 3.1.0, the mock decorator to be used was @mock_dynamodb2. Make sure if you are using the version prior to 3.1.0, use this decorator instead or update to the version I used in this blog.