FinOps Data Engineer, AI Cost & Usage Analytics

This is a fully remote position, open to applicants in Colombia, +1 more country.

📋 Description

• Design, create, and sustain scalable data pipelines that gather AI usage, billing, and consumption data from various providers and platforms.

• Develop API integrations and scheduled data ingestion utilizing Python, Cloud Functions, Cloud Composer/Airflow, or similar technologies.

• Implement error management, retries, monitoring, and data quality validation.

• Oversee vendor APIs and schemas, adapting integrations as providers evolve.

• Design and uphold a unified BigQuery model that normalizes usage and billing data across AI providers and pricing structures.

• Construct dimensional and analytical models that facilitate cost allocation and reporting across teams, business units, projects, providers, models, and cost centers.

• Create SQL/dbt/Dataform transformation workflows to cleanse, deduplicate, normalize, and enrich data.

• Integrate usage and cost data with organizational hierarchies and financial frameworks for chargeback/showback reporting.

• Enhance BigQuery through partitioning, clustering, incremental processing, and query optimization.

• Implement historical snapshots and slowly changing dimensions for trend analysis and point-in-time evaluations.

• Apply data governance, security measures, access control, auditability, and PII handling practices.

• Build and maintain Power BI dashboards for AI expenditure, consumption trends, cost allocation, budgets, forecasts, and utilization.

• Optimize the connection between BigQuery and Power BI along with data models.

• Develop dashboards suitable for executive review and detailed drill-down analysis.

• Implement row-level security and access restrictions in Power BI.

• Collaborate with FinOps and business stakeholders to establish metrics, KPIs, and anomaly detection logic.

• Aid in budgeting, forecasting, cost allocation, and initiatives to optimize AI spending.

• Keep documentation for data lineage, pipeline architecture, transformation logic, and reporting definitions.

• Automate manual reporting and enhance platform reliability, scalability, and cost efficiency.

• Work with cross-functional teams in an Agile environment on technical decisions, planning, and continuous improvement.

⛳️ Requirements

• This position is available to candidates residing in Colombia or Costa Rica only.

• At least 3 years of professional experience in Data Engineering, Analytics Engineering, Software Engineering, or a similar role.

• Strong expertise in SQL and experience in designing and maintaining analytical data models.

• Practical experience with Google BigQuery or similar cloud data warehouses such as Snowflake or Amazon Redshift.

• Extensive experience in building data pipelines using Python.

• Familiarity with workflow orchestration technologies like Apache Airflow, Google Cloud Composer, Dagster, or similar tools.

• Hands-on experience with REST API integration, including authentication, OAuth, pagination, rate limiting, and error handling.

• Experience in designing dependable data ingestion, transformation, validation, and monitoring processes.

• Proficiency in Power BI, encompassing semantic/data modeling, DAX, dashboard development, and report optimization.

• Knowledge of cloud billing and cost data structures across platforms such as GCP, AWS, or Azure.

• Experience with Git and version-controlled development workflows.

• Familiarity with implementing CI/CD practices for data pipelines and transformation code.

• Understanding of dimensional modeling, incremental processing, historical data management, and data quality practices.

• Strong problem-solving abilities and proficiency in navigating complex or ambiguous data requirements.

• Excellent communication and collaboration skills, enabling effective interaction with both technical and business stakeholders.

• Preferred qualifications include FinOps knowledge/certification, experience with AI/LLM usage or billing APIs, dbt or Dataform, enterprise chargeback/showback, Terraform/IaC, governance/security/compliance, and cost analytics/anomaly detection/forecasting/cloud optimization.

🏝️ Benefits

• Competitive salary and performance-based bonuses.

• Opportunities for professional development and training.

• Flexible work hours and remote work options.

• Health insurance and wellness programs.

• Collaborative and inclusive work environment.