Data Engineer

This is a fully remote position, open to applicants in Poland.

📋 Description

• Design, develop, and sustain scalable data lake solutions and processing pipelines that manage substantial volumes of structured and semi-structured data.

• Construct and manage real-time data streaming pipelines utilizing Apache Kafka and its ecosystem (Kafka Streams, Kafka Connect).

• Architect and uphold ETL and ELT pipelines prioritizing data quality, idempotency, and observability.

• Create distributed data processing applications using Apache Spark (PySpark, Scala) on Databricks.

• Design and oversee both SQL and NoSQL databases utilized in our data products.

• Develop data solutions on cloud infrastructure (GCP, Azure, or AWS), utilizing managed services.

• Implement software engineering best practices for data pipelines: version control, automated testing, peer code reviews, and CI/CD with tools like GitLab or Jenkins.

• Take ownership of the operational health of the data infrastructure and ETL processes you create.

• Integrate data from internal and external sources through REST and SOAP APIs.

• Actively engage in InPost's data engineering community by participating in code reviews, creating internal documentation, delivering tech talks, and mentoring.

⛳️ Requirements

• Minimum of 3 years of experience in a Data Engineering or related role.

• Practical experience with Apache Spark (Streaming, Spark SQL, MLlib) and Databricks (PySpark, Scala).

• Hands-on experience with Apache Kafka, including Kafka Streams and Kafka Connect.

• Proficient in Python; familiar with Scala or Java.

• Experience in designing and managing SQL databases (e.g., PostgreSQL, BigQuery, Spark SQL) and NoSQL databases (e.g., MongoDB, Cassandra, or similar).

• Experience in building and maintaining data lake environments (Delta Lake, Parquet, or equivalent).

• Familiarity with cloud platforms (GCP, Azure, or AWS) and their managed data services.

• Experience in integrating data through REST and/or SOAP APIs.

• Working knowledge of CI/CD tools (GitLab CI, Jenkins, or equivalent) and software engineering practices (testing, versioning, code review).

• Experience in building and operating Docker containers.

• Eagerness to share knowledge and contribute to engineering best practices.

• Professional working proficiency in both English and Polish.

🏝️ Benefits

• The flexibility to work from the office or fully remotely.

• The chance to operate in a diverse, international, and cross-functional environment alongside leading experts.

• Rewarding careers with a variety of benefits, along with investment in training opportunities for professional development.

• Active participation in technology monitoring and decision-making.

• Your contributions will have immediate visibility, making a significant difference in the lives of our users.

• We offer B2B type cooperation.