Azure Data Lake Data Platform Specialist

This is a fully remote position, open to applicants in Colombia.

📋 Description

• Design, develop, and sustain data ingestion pipelines, setting standards for data ingestion, transformation, and storage.

• Integrate data from various management and financial systems, both batch and streaming.

• Architect and implement a contemporary data platform utilizing Azure with an ADLS Gen2-based architecture.

• Establish data architecture patterns, incorporating medallion layers (bronze/silver/gold).

• Create scalable ETL/ELT pipelines leveraging Azure Data Factory.

• Manage a variety of data formats, including structured and semi-structured types.

• Organize and maintain data utilizing bronze/silver/gold layers in a medallion architecture.

• Optimize data storage, partitioning, and overall performance.

• Connect ADLS with Azure Data Factory (ADF), Azure Synapse Analytics / Databricks, Power BI, and other downstream analytics tools.

• Enforce data governance, security, and access controls through RBAC, IAM, and ACLs.

• Ensure the quality of data, manage lineage, and oversee metadata.

• Assist business teams with data modeling and ensuring data availability.

• Monitor performance metrics and resolve data platform issues.

• Define and implement access controls (RBAC, ACLs) along with data security and compliance standards.

⛳️ Requirements

• Demonstrated experience in designing and implementing Azure data platforms.

• Strong practical knowledge of Azure Data Lake Storage (ADLS) and Azure Data Factory (ADF).

• Familiarity with integrating ERP and finance systems, with SAP being highly desirable.

• Proficient in SQL and Power Query, with experience handling large datasets.

• Background in data modeling for financial reporting.

• Competency in Python or Spark, with a preference for Databricks but not essential.

• Experience in constructing end-to-end data pipelines.

• Understanding of the differences between data lake, warehouse, and Lakehouse patterns, as well as financial data structures and reporting requirements.

🏝️ Benefits

• Flexible Working

• Room to Grow

• Everyone has a voice

• Extensive benefits including healthcare, retirement plans, continued education support, and employee recognition programs.