Description

Technology Stack:

  • AWS: Most data resides in S3 buckets, with EventBridge handling processing and Postgres RDS as the primary database.
  • Data comes from external sources (e.g., Ticketmaster, Salesforce) and is ingested into S3, processed via EventBridge, and stored in Postgres with DBT for transformation.
  • Recently signed a contract with Snowflake for migration efforts.
  • Need to optimize RDS, which is nearing capacity, and implement best practices for query optimization.
  • Heavy ETL work using AWS Glue Jobs, Snowflake exposure, and RDS.
  • Focus on building end-to-end pipelines, regression testing, and implementing effective ETL frameworks