
Senior Engineer – Data Engineering, Databricks
Posted Nov 12

Posted Nov 12
This is a fully remote position, open to applicants in India.
• Deliver innovative solutions on the Databricks Platform while providing technical leadership with minimal supervision.
• Utilize cloud data lakehouse methodologies, distributed processing engines, event streaming platforms, and contemporary data technologies.
• Engage in the Databricks Center of Excellence.
• Construct next-generation data platforms for organizations focused on data and analytics.
• Collaborate with a Solution Architect to design and implement components of client data platform solutions.
• Take part in design sessions, decompose intricate development tasks, and ensure timely completion of development items.
• Contribute to initiatives within the Center of Excellence by creating Databricks solution accelerators.
• Develop and execute data solutions utilizing Databricks.
• Work alongside cross-functional teams to design and optimize both batch and streaming data pipelines.
• Maintain data quality and efficiency.
• Keep up with emerging technologies and features of the Databricks platform, continuously enhancing relevant skills.
• Over 5 years of experience in data engineering.
• A minimum of 2 years of practical experience in designing and developing data pipelines using Databricks.
• Familiarity with Delta Lake, Uniform (Iceberg), Delta Live Tables (Lakeflow Declarative Pipelines), and Unity Catalog.
• Expertise in designing and constructing scalable YAML configuration-driven data pipelines.
• Experience with batch, micro-batch, and streaming data ingestion and processing patterns utilizing tools such as Auto Loader.
• Proven experience in developing complex job/workflow orchestration patterns using Databricks Jobs/Workflows or tools like Airflow and dbt.
• Knowledge of data quality, pipeline audit, and observability solutions using Databricks native features or frameworks/tools such as Great Expectations, Collibra, and dbt.
• Proficient in Apache Spark, Presto, and Amazon EMR.
• Experience with Amazon Redshift, Snowflake, and Google BigQuery.
• Strong programming abilities in SQL, stored procedures, and object-oriented programming languages like Java, Python, and PySpark.
• Understanding of DevOps and CI/CD pipelines using Databricks Asset Bundles, including automated validation and testing.
• Familiarity with Infrastructure as Code tools such as Terraform is advantageous.
• Knowledge of NoSQL databases and container management systems.
• Exposure to AI/ML tools such as MLflow, prompt engineering, and modern data and AI workflows.
• Databricks Data Engineering Associate and/or Professional certification is preferred.
• Experience delivering multiple projects using Databricks.
• A curious, self-motivated mindset with the ability to share insights with teams.
• Contract engagement lasting 6 months or more.
• Start date in December or January.
• Remote work arrangement.
Trax Technologies
Fiserv
Ontrac Solutions
General Dynamics Information Technology
Get handpicked remote jobs straight to your inbox weekly.