Job Opportunity: Senior Data Engineer
We're hiring a Senior Data Engineer to join our dynamic team! This role is critical to managing and optimizing data pipelines, ensuring data integrity, security, and performance. As a Senior Data Engineer, you'll work with data lakes and warehouses, support data accessibility for analytics and machine learning, and collaborate closely with data scientists, analysts, and stakeholders to build tailored solutions. If you're passionate about data engineering and ready to tackle complex challenges, we'd love to hear from you!
Key Responsibilities:
- Data Ingestion & Extraction:
- Identify and acquire data sources from databases, APIs, and cloud storage.
- Develop efficient ETL/ELT pipelines to load data into data lakes and warehouses.
- Ensure data quality through validation and integrity checks.
- Data Storage & Management:
- Design scalable data ingestion architecture and choose appropriate storage technologies.
- Optimize storage through compression, partitioning, and indexing to improve performance.
- Data Transformation & Processing:
- Clean, transform, and enrich data to make it analysis-ready.
- Build complex data processing pipelines using Azure ADF, SSIS, Apache NiFi, and Spark.
- Design data marts and cubes tailored to business and reporting needs.
- Data Integration & Sharing:
- Enable data accessibility through APIs, visualization tools, and BI platforms.
- Ensure security and privacy measures align with regulatory compliance.
- Distribute data to reporting tools like Power BI, data science projects, and third-party platforms.
- Data Platform Expertise:
- Proficiency with on-premises and cloud data platforms, specifically Microsoft Fabric and Synapse Data Warehouse (DWH).
- Monitoring & Optimization:
- Continuously monitor data pipelines and troubleshoot issues.
- Optimize pipeline efficiency and scalability, resolving bottlenecks.
Qualifications:
- Experience:
- 5+ years in data engineering with strong expertise in data manipulation.
- Education:
- Bachelor's degree in Computer Science, IT, Engineering, or a related field.
- Technical Skills:
- Programming: Strong proficiency in Python.
- SQL: Advanced SQL skills for writing and optimizing complex queries.
- Data Storage: Knowledge of data warehousing, data lake architectures, and big data technologies (Hadoop, Spark).
- ETL Tools: Proficiency in ETL/ELT tools, especially Azure ADF, SSIS, and Apache NiFi.
- Cloud Platforms: Experience with Microsoft Azure, AWS, and Databricks.
- Data Modeling: Expertise in data modeling for structured and unstructured data.
- Data Delivery: Proficient in Power BI for creating and managing interactive dashboards.
- Data Management & Governance:
- Hands-on experience with tools like Azure Data Catalog, SQL Server Management Studio, and Toad for Oracle.
- Strong knowledge of data security practices and data privacy compliance.
- Analytical & Problem-Solving Skills:
- Ability to analyze complex data issues and develop scalable solutions.
- High attention to detail for data accuracy and quality.
- Soft Skills:
- Strong communication skills to collaborate with cross-functional teams.
- Adaptability to learn new technologies and adjust to project needs.
- Teamwork-oriented with experience in cross-functional collaborations.
Certifications:
- Required:
- Microsoft Certified: Fabric Data Engineer Associate
- CDMP-DAMA
- Optional:
- Microsoft Certified: Power BI Data Analyst Associate
Why Join Us
This role offers the opportunity to be at the forefront of data-driven decision-making, tackling complex engineering challenges and contributing to impactful solutions. If you're ready to advance your career in data engineering, apply now!
