Data Engineer

This is a fully remote position, open to applicants in India.

📋 Description

• Develop and sustain optimal data pipeline architecture.

• Compile extensive, intricate data sets that fulfill both functional and non-functional business needs.

• Recognize, design, and execute internal process enhancements: automating manual tasks, streamlining data delivery, and redesigning infrastructure for improved scalability, among others.

• Establish the necessary infrastructure for efficient extraction, transformation, and loading of data from a diverse range of data sources using SQL and AWS 'big data' technologies.

• Create analytical tools that leverage the data pipeline to deliver actionable insights for Information Asset as well as enhance operational efficiency and other critical business performance metrics for Information Asset Customers.

• Collaborate with stakeholders including Executive, Product, Data, and Design teams to aid with data-related technical challenges and support their data infrastructure demands.

• Develop data tools for analytics aimed at building and refining offerings into a leading industry solution.

• Partner with data and analytics specialists to pursue enhanced functionality in data systems.

⛳️ Requirements

• Bachelor’s degree in computer science or information technology, or equivalent professional experience.

• 3+ years of experience as a Data Engineer.

• Data engineering certification (e.g., IBM Certified Data Engineer, AWS Certified Data Analytics - Specialty) is advantageous.

• Must possess technical proficiency in data models, data mining, and segmentation methodologies.

• Experience with any combination of the following software/tools: Relational SQL and NoSQL databases, including Postgres and Cassandra.

• Data pipeline and workflow management tools: Azkaban, Luigi, Airflow, etc.

• Data Storage Technologies such as Amazon S3, Snowflake/Redshift, Hive, HDFS.

• Stream-processing systems including Storm, Spark-Streaming, etc.

• Proficiency in object-oriented/object function scripting languages: Python/pySpark, Java, C++, Scala, etc.

• Data warehousing experience using Oracle/Snowflake.

• Familiarity with ETL Technologies such as Apache Spark/AWS Glue/Databricks, Apache Airflow, Apache Hadoop.

• Proven experience supporting and collaborating with cross-functional teams in a dynamic environment.

🏝️ Benefits

• This role offers a remote working arrangement.

• You will need to work from a home office equipped with high-speed internet and phone connectivity to ensure a smooth virtual presence.