Senior Big Data Engineer

This is a fully remote position, open to applicants in Illinois.

📋 Description

• The Senior Big Data Engineer will be tasked with the design, construction, and implementation of scalable big data and ETL solutions.

• Provide support for projects across various and unexpected client locations throughout the U.S.

• Collaborate effectively with cross-functional teams to create, test, and deploy data solutions.

• Design and develop highly scalable ETL applications within Hadoop and Big Data ecosystems.

• Create new scripts, tools, and methodologies to enhance ETL workflows.

• Execute big data projects utilizing Spark, Python, Scala, SQL, and Hive.

• Prepare technical design documentation for solutions and ensure code is well-documented.

⛳️ Requirements

• A Bachelor’s degree in Computer Science, Information Systems, Computer Engineering, or a related field, along with 5 years of progressive experience as a Software Engineer/Developer or in Software Development is required.

• Proficiency in writing ETL Spark applications using PySpark or Scala.

• Knowledge of Spark architecture, data frames, and spark tuning.

• Familiarity with relational databases such as Oracle and PostgreSQL.

• Skills in Python, SQL, HQL, and Hive.

• Experience with Data Bricks.

• Proficient in managing software systems utilizing Hadoop, MapReduce, HDFS, and all associated services.

• Understanding of distributed computing principles.

• Experience with Big Data querying tools like Hive.

• Knowledge of data warehousing and data modeling techniques.

• Proficiency in Core Java, Linux, SQL, and various scripting languages.

• Familiarity with cloud platforms such as Azure.

• Experience in integrating data from multiple data sources.

• Understanding of Lambda Architecture.

🏝️ Benefits

• Paid time off

• Medical, dental, and vision insurance

• 401(k)