Big Data Lead en Pyramid Consulting

Job Title: Big data Lead role

Location: Remote

Job Summary

Required Qualifications

• Bachelor’s degree in Computer Science, Engineering, Information Systems, or a related field; equivalent practical experience may be considered.
• 5+ years of experience developing and supporting data integration, ELT, or data engineering solutions.
• Demonstrated experience processing large data volumes in distributed or cloud-based environments.
• Strong proficiency in SQL and at least one programming language, such as Python, Java, Scala, or Spark SQL.
• Experience with orchestration and workflow tools such as Apache Airflow, Azure Data Factory, or AWS Step Functions.
• Working knowledge of relational and non-relational databases, data warehouses, and data lake architectures.
• Experience designing resilient data pipelines with logging, monitoring, retry logic, idempotency, and failure recovery.
• Strong analytical, troubleshooting, communication, and collaboration skills.

Preferred Qualifications

• Experience with cloud data platforms, particularly AWS.
• Experience with AWS services, including Lambda and Kafka.
• Knowledge of data modeling, metadata management, data governance, and master data management practices.
• Experience with containerization and deployment technologies, CI/CD pipelines, and infrastructure as code.
• Experience working in regulated environments, including financial services, healthcare, or other highly controlled industries.