Data Sources

Imagine a hypothetical e-commerce company who's architecture uses:

  • Lineitems processing records stored in HBase on EMR
  • Redis is used to store nations and active orders so that the processing engine can get fast access to them
  • Aurora with MySQL engine for Orders, Customer and Suppliers accounts data like email address, shipping addresses, etc.
  • DynamoDB to host part and partsupp data for high performance
  • For this workshop, we will focus on the following datasources:
      HBase on EMR
    • Required write optimized data store
    • High transaction rate
    • Long term durability
    • Aurora with MySQL engine
    • Scalable read replicas
    • Low write rate requirement
    • High Security
    • Amazon DynamoDB
    • Highly scalable
    • High performance
    • Flexible
    • Redis on ElastiCache
    • Low latency read write
    • Minimal contextual info
    • In memory w/rebuild capability