Production Use Case – Amazon DynamoDB Index Design Best Practices for Optimum Performance

Ok so you have read all the good things about DynamoDB and decided to use for your production application. One of the major thing which amazon web service developers, DB architects really struggle while working with DynamoDB is efficient index design.

DynamoDB is:

  • db => tables => items => attributes
    • db = a list of tables
    • table = a list of items
    • item = a list of attributes

Keeping in mind DynamoDB Index design best design patterns are really important for amazon web services developers  for optimum performance. Below I am going to cover a production use case and various approaches/tradeoffs to keep in mind while designing DynamoDB index.

DynamoDB Item Schema

Below is a single DynamoDB page_views table which is storing analytics data for Ad block services like

Programming Amazon Web Services : PageView Schema for DynamoDB Index Design

Programming Amazon Web Services : PageView Schema for DynamoDB Index Design

  • ID (String)
  • ad_blocked (Number) 1 / 0
  • analytics_blocked (Number) 1 / 0
  • client_ip (String)
  • created_at (Number) => A timestamp Value
  • site_id (Number)
  • user_agent (String)

Programming Amazon Web Services : Index Design Requirements

We need to design better structured index to allow for optimum querying result. Below production DynamoDB queries needs to run on above item:

  • Select(Query) all records by site_id.
  • Query site_id and time series (Last 7 Days, Last 30 Days, Last Month).
  • Aggregate counts of ad_blocked and analytics_blocked numbers for a particular site_id.
  • Below is the sample ruby code for querying:


  • You have basic idea of DynamoDB and created few simple items. You have atleast basic understanding of Primary and Global Secondary Indexes.
  • All code examples below are in AWS Ruby SDK Version 2.1.32.
  • Specify read and write throughput as per your application needs.

Key Points to Remember Before Index Design

  •  ID is always unique. It uniquely identify an item with no other use case. I thought of creating a composite primary key index instead of id as primary key index to better utilize index. With this approach we will not need ID attribute. However this can break our schema consistency. site_id, created can’t act as primary key since there can be page_views item for a site which is created at same time during high load.
  • We should focus on creating less number of DynamoDB indexes since it can affect disk size and hence cost money. Also we have a limit of maximum 5 secondary indexes.
  • In DynamoDB we should always avoid scan and use query. Scan doesn’t utilize indexes.

DynamoDB Index Design Solution

  • Create id as primary key index.
  • Create site_id, created_at as GSI (Global Secondary Index). created_at is our range key since we will need to do comparison operations.
    • Index Hash key : site_id
    • Index Range key : created_at
    • Index Name : site_id-created_at-index
    • Projection Attribute : “All Attributes”
Programming Amazon Web Services : PageView Item Global Secondary Index

Programming Amazon Web Services : PageView Item Global Secondary Index

Use Case 1 : Solution

Select all records by site_id

Below code example will use site_id index distribution of GSI. Here we are utilizing first attribute of site_id, created_at GSI (Global Secondary Index).

Run query in batches of 5-10k to reduce data transfer latency.

Use Case 2 : Solution

Query site_id by time series (Last 7 Days, Last 30 Days, Last Month)

Below code example will use site_id-created_at composite GSI.

 Use Case 3 : Solution

Aggregate counts of ad_blocked and analytics_blocked analytics for a particular site_id.

Approach 1

  • Query by site_id like mentioned in Use Case 1.
  • This will use our GSI(Global Secondary Index) which has hash key – site_id.
  • At the application level run loop to collect ad_blocked and analytics_blocked analytics. e.g.ad_blocked_views, ad_non_blocked_views, analytic_blocked_views, analytic_non_blocked_views.
  • Query DynamoDB in batches of 5-10k  items is recommended. Use limit parameter.

Approach 2

  • Create two more Global Secondary Index.
    • First GSI on site_id, ad_blocked with Projected Attributes = “Specify Attributes” which can only be id since we need count only.
    • Second GSI on site_id, analytics_blocked with Projected Attributes = “Specify Attributes” which can again be only id.
  • You can query directly on site_id, ad_blocked/site_id, analytics_blocked for counts which will use our above GSI indexes.
  • This approach will increase dynamoDB disk storage which can cost money for huge data.

Approach 3

  • Create a new table page_view_analytics having attributes
    • site_id
    • ad_blocked_views
    • ad_non_blocked_views
    • analytic_blocked_views
    • analytic_non_blocked_views
  • The Lambda function will further update page_view_analytics table.
    • e.g. Increment counter of ad_blocked_views attribute of site_name “hackpundit” to 1 if a new pageview with ad_blocked = 1 is created.
  • Read details for creating Lambda function triggered by DynamoDB row update.

Final Thoughts for Use Case 3

Approach 1 is good to begin with. However as your load and data size increases it’s highly recommended to go with approach 3.  AWS has solution to trigger Lambda function on trigger of dynamoDB table update.

Hopefully this will get you on a good start with for getting started Amazon DynamoDB applications. Understanding the secondary index and projection attributes should go hand in hand because of the fact that a secondary index cannot be used efficiently without specifying projection.

In this article we learned global secondary indexes and projection and its usage with indexes. Global secondary indexes also present tradeoffs that you need to consider when designing your applications.

If you are interested in AWS big data services, I recommend reading below articles:

If you like my article please like our Facebook page and also follow us on Twitter so that you get regular updates. For regular updates you can also subscribe to

Please also share on Facebook and Twitter to help others.