OpenSearch Engineer / AWS & Data Pipelines

ABOUT US

Ruby Labs is a leading tech company that creates and operates innovative consumer products. We offer a diverse range of opportunities across the health, education, and entertainment industries. Our innovative teams are driving the future of consumer-led products, and we're always looking for passionate individuals to join us. Learn more about our story at: https://rubylabs.com/about-us/

ABOUT THE ROLE

We are building a high-scale payment orchestration platform that processes millions of financial transactions every month across multiple providers, regions, and products.

We are looking for an experienced OpenSearch Engineer to own and scale our AWS OpenSearch infrastructure and the pipelines that synchronize data from PostgreSQL.

You will be responsible for search architecture, data ingestion, performance, reliability, reconciliation, and recovery. The role combines search infrastructure, data engineering, and production operations.

 

KEY RESPONSIBILITIES

OpenSearch

- Own and operate AWS OpenSearch Service in production.

- Design indices, mappings, analyzers, templates, aliases, shards, and retention policies.

- Optimize search queries, aggregations, filtering, pagination, and bulk indexing.

- Investigate slow queries, indexing bottlenecks, hot shards, memory pressure, and rejected requests.

- Plan safe upgrades, migrations, reindexing, and capacity scaling.

PostgreSQL-to-OpenSearch pipelines

- Build and maintain reliable pipelines that synchronize PostgreSQL data with OpenSearch.

- Support inserts, updates, deletes, backfills, and full reindexing.

- Use AWS-native services such as AWS DMS, OpenSearch Ingestion, Lambda, SQS, S3, ECS, or EKS where appropriate.

- Implement retries, idempotency, deduplication, replay, dead-letter handling, and failure recovery.

- Monitor ingestion lag, throughput, failed records, and data freshness.

- Build reconciliation processes to detect and repair missing, stale, or inconsistent docume...