Data Engineer

This is a fully remote position, open to applicants in Philippines.

📋 Description

• Accountable for the integration of enterprise-wide data from Cofense’s Cyber Security applications, micro-services, and various other data sources.

• Collaborate with Architects and Cloud Systems Engineers to design the Data Platform and its architecture.

• Create Data Pipelines utilizing Cloud Technologies such as Azure or AWS, following well-defined tool frameworks.

• Capable of developing ETL code to stream data from various data stores to the Data Lake/Data Warehouse using Python and Azure/AWS Data Lake services.

• Proficient in crafting complex SQL scripts and automating them through Python.

• Generate test cases and unit tests for crucial implementations of the Data Platform, adhering to software engineering best practices and standards.

• Ensure end-to-end data security by complying with data privacy regulations while developing processes to transfer data across Applications, Data Sources, and the Data Lake/Data Warehouse.

• Assist in integrating the Data Platform with BI tools such as Power BI, Tableau, Splunk, etc.

• Develop data transformations to produce Facts, Summaries, and Key metrics by applying business rules and aggregations using Python, SQL, and other transformation tools.

• Work collaboratively with business users across Cofense’s departments to define requirements, prioritize project tasks, and deliver them on schedule.

⛳️ Requirements

• A Bachelor’s degree in Computer Science, Mathematics, Data Analytics, Data Sciences, BI, or relevant industry experience is preferred.

• More than 5 years of demonstrated experience in data architecture, data modeling, and lifecycle management.

• Practical experience with relational and cloud databases, including Azure SQL Database, Microsoft SQL Server, Amazon Aurora, and Amazon Redshift.

• Strong foundation in developing Python and data technologies.

• Hands-on experience in designing and developing ETL data pipelines and applications using SQL and Python.

• Extensive expertise in crafting complex SQL queries for data transformation and automating processes through Python.

• Experience in building and consuming RESTful APIs using Python libraries.

• Proficient in integrating data platforms with BI tools like Power BI and Splunk, with experience in dashboard and report development.

• Solid experience in Unix/Linux environments, including SSH tunneling and writing/interpreting Bash scripts.

• Advanced proficiency in Python 3, with hands-on experience using NumPy, SciPy, Scikit-learn, and Pandas.

• Experience in leveraging APIs to extract data and load it into databases using Python.

🏝️ Benefits

• Health insurance

• 401(k) matching

• Flexible work hours

• Paid time off

• Remote work options