
Senior Engineer – Data Engineering, Databricks
Posted Nov 12

Posted Nov 12
This is a fully remote position, open to applicants in India.
• Provide technical guidance with minimal supervision to deliver solutions on the Databricks Platform.
• Develop data platforms utilizing cloud data lakehouse methodologies, distributed processing engines, event streaming platforms, and contemporary data technologies.
• Engage in the Databricks Center of Excellence.
• Construct next-generation data platforms for organizations focused on data and analytics.
• Collaborate with a Solution Architect to design and implement components of client data platform solutions.
• Take part in design sessions, decompose complex development tasks, and ensure timely completion of development items.
• Contribute to Center of Excellence initiatives by creating Databricks solution accelerators.
• Design and implement data solutions that leverage Databricks platform features.
• Work alongside cross-functional teams to design and optimize batch and streaming data pipelines.
• Guarantee data quality and pipeline efficiency.
• Remain up-to-date with emerging technologies and features of the Databricks platform.
• Continuously enhance skills related to Databricks and associated tools.
• Over 5 years of experience in data engineering.
• At least 2 years of practical experience in designing and developing data pipelines using Databricks.
• Familiarity with Delta Lake, Uniform (Iceberg), Delta Live Tables (Lakeflow Declarative Pipelines), and Unity Catalog.
• Expertise in designing and constructing robust, scalable, YAML configuration-driven data pipelines.
• Experience with batch, micro-batch, and streaming data ingestion and processing patterns, utilizing tools like Auto Loader.
• Proficiency in building complex job/workflow orchestration patterns using Databricks Jobs/Workflows or tools like Airflow and dbt.
• Knowledge of data quality, pipeline audit, and observability solutions using Databricks-native features or frameworks such as Great Expectations, Collibra, and dbt.
• Proficient in Apache Spark, Presto, and Amazon EMR.
• Experience with Amazon Redshift, Snowflake, and Google BigQuery.
• Strong programming capabilities in SQL, stored procedures, and object-oriented programming languages such as Java, Python, and PySpark.
• Familiarity with DevOps and CI/CD pipelines using Databricks Asset Bundles, including automated validation and testing.
• Exposure to Infrastructure as Code tools like Terraform is a plus.
• Knowledge of NoSQL databases and container management systems.
• Familiarity with AI/ML tools such as MLflow, prompt engineering, and modern data and AI agentic workflows.
• Ideally, hold Databricks Data Engineering Associate and/or Professional certification.
• Experience delivering multiple Databricks projects.
• Contract engagement for a duration of 6 or more months.
• Anticipated start date in December or January.
Trax Technologies
Fiserv
Ontrac Solutions
General Dynamics Information Technology
Get handpicked remote jobs straight to your inbox weekly.