Software Integrations Developer

This is a fully remote position, open to applicants in India.

📋 Description

• Design and execute comprehensive batch and streaming data pipelines utilizing Azure Data Factory / Synapse Pipelines and Databricks.

• Create scalable ETL/ELT workflows with PySpark, SQL, and Databricks notebooks.

• Develop and enhance ingestion frameworks for both structured and unstructured data.

• Lead the design of Databricks-based lakehouse architecture employing Delta Lake along with bronze, silver, and gold data layers.

• Optimize Spark jobs through methods such as partitioning, caching, and cluster tuning.

• Implement data versioning and incremental processing while ensuring data quality, consistency, and governance.

• Build secure and scalable solutions using Azure Data Factory, Data Lake Storage, Synapse Analytics, and Key Vault.

• Design, construct, and maintain integrations utilizing Azure Functions, Logic Apps, Service Bus, and Event Grid.

• Connect business applications, SaaS platforms, and external services to the data platform.

• Develop and expose secure, documented REST APIs and event-driven endpoints via Azure API Management.

• Implement publishing, versioning, throttling, and authentication for APIs.

• Employ event-driven and near-real-time messaging patterns using Event Hub, Event Grid, and Service Bus.

• Monitor and enhance data workflows for reliability, performance, and cost efficiency.

• Manage code repositories, CI/CD pipelines, and deployments through Azure DevOps.

• Maintain technical documentation and adhere to governance best practices.

• Collaborate with business stakeholders and technical teams to convert requirements into scalable data solutions.

• Provide technical leadership.

• Deliver dependable Azure integration architectures aimed at achieving 99.9% uptime, optimized throughput, minimal technical debt, comprehensive documentation, security governance, and seamless data flow.

⛳️ Requirements

• Over 5 years of experience in Data Engineering / Data Platform development.

• More than 3 years of practical experience with Azure Databricks.

• Strong expertise in Azure Data Factory, ADLS Gen2, and Synapse Analytics.

• Demonstrated ability to construct enterprise-scale data pipelines.

• Practical experience in building application and API integrations utilizing Azure Functions, Logic Apps, and Azure API Management.

• Experience in Agile/Scrum settings and the full SDLC lifecycle.

• Strong skills in Python (PySpark) and SQL.

• Familiarity with Databricks notebooks for scalable data processing and transformation logic.

• In-depth knowledge of ETL/ELT design patterns, Slowly Changing Dimensions (SCD), medallion architecture, and data modeling for analytics.

• Expert-level experience with Azure Data Factory / Synapse Pipelines, Azure Data Lake Storage, and Azure Databricks including Delta Lake.

• Hands-on experience with Azure Functions, Logic Apps, Service Bus / Event Grid, and Azure API Management.

• Experience in designing REST APIs and implementing secure authentication using OAuth 2.0 / managed identities.

• Advanced hands-on experience with Spark optimization, cluster management, job orchestration, and performance tuning in Databricks.

• Familiarity with streaming technologies, Microsoft Fabric, CI/CD for data platforms, or Power BI is a significant advantage.

• Design-first and quality-oriented mindset.

• Comfortable navigating ambiguity and evolving requirements.

• Strong communicator capable of explaining technical concepts to non-technical audiences.

• Proactive, collaborative, and oriented towards continuous improvement.

• Strong sense of urgency and ownership.

🏝️ Benefits

• Opportunity to lead and influence the organization's modern data platform strategy.

• High-impact data initiatives that drive analytics, AI, and digital transformation programs.

• Significant executive visibility.

• Meaningful ownership and technical challenges.

• Long-term growth opportunities.

• Opportunity to establish best practices within the OneGP model.

• Opportunity to work with cutting-edge Microsoft technologies, including Power Platform, Azure integrations, and emerging AI and Copilot capabilities.