Senior Data Architect

Description

Key Responsibilities:

• Data Architecture Design: Architect and design large-scale, end-to-end data solutions that meet the needs of business and technical stakeholders, focusing on scalability, security, and performance.
• ETL/ELT Pipeline Development: Lead the design, development, and optimization of efficient ETL/ELT pipelines that extract, transform, and load data from various sources into structured formats, ready for business intelligence and advanced analytics.
• Data Source Analysis: Perform detailed analysis of structured and unstructured data sources, providing strategic insights on how best to ingest, process, and classify data to create a high-quality data layer.
• Data Layer Development: Design and build a robust, scalable data layer that integrates seamlessly with business analytics platforms and supports both batch and real-time data processing.
• Data Ingestion, Cleansing, and Classification: Develop and implement data ingestion strategies for real-time and batch processes, ensuring data is thoroughly cleansed, validated, and classified in alignment with business goals and governance policies.
• Query Optimization: Take ownership of optimizing complex SQL queries and enhancing database performance, with a focus on reducing query execution times and improving resource efficiency across large datasets.
• Data Migration: Lead and manage large-scale data migration efforts from legacy systems to modern cloud-based platforms, ensuring data accuracy, integrity, and minimal downtime.
• Performance Monitoring & Troubleshooting: Proactively monitor the performance of data systems, troubleshoot bottlenecks, and implement solutions that maximize the speed and efficiency of data processing workflows.
• Collaboration: Work closely with data analysts, data scientists, and business teams to understand data requirements, build data models, and ensure the data architecture is aligned with both technical and business objectives.
• Data Governance & Security: Ensure that all data solutions comply with data governance and security standards, implementing best practices around data classification, data quality, and regulatory compliance (e.g., GDPR, HIPAA).
• Mentorship: Provide technical leadership and mentorship to junior data engineers, fostering a culture of innovation, best practices, and continuous improvement.

Requirements

• Education: Bachelor’s or Master’s degree in Computer Science, Data Science, Information Technology, or a related field.
• 8+ years of experience in data engineering, with a proven track record in architecting and building large-scale ETL/ELT pipelines and data systems.
• Extensive experience in analyzing data sources, designing data layers, and managing large-scale data migrations.
• Strong hands-on experience in optimizing complex SQL queries and improving the performance of databases handling large datasets.
• Deep expertise with cloud platforms (Azure, AWS, GCP), big data technologies (e.g., Hadoop, Spark), and data warehouse solutions (Snowflake, Azure Synapse).

Technical Skills:

• Advanced SQL skills with a focus on query optimization, indexing, partitioning, and performance tuning.
• Hands-on experience with ETL/ELT tools (Azure Data Factory, Apache Airflow, SSIS) and designing complex data pipelines.
• Proficiency in working with relational databases (SQL Server, PostgreSQL) and NoSQL databases (MongoDB, CosmosDB).
• Strong knowledge of data governance, data quality management, and regulatory frameworks (GDPR, HIPAA).
• Experience with big data processing frameworks (Databricks, Apache Spark) and integrating them with modern data warehouses.
• Expertise in data ingestion from diverse sources (databases, APIs, flat files, streaming data) and transforming raw data into structured formats for analysis.
• Strong familiarity with CI/CD pipelines and automation of data workflows using DevOps practices.
• Experience with data visualization tools (Power BI, Tableau) and their integration with data platforms.

Preferred Skills:

• Cloud certifications in relevant technologies (e.g., Azure Data Engineer, AWS Big Data).
• Hands-on experience with machine learning platforms and frameworks (e.g., Azure Machine Learning, Databricks).
• Familiarity with real-time data processing and streaming technologies (e.g., Azure Stream Analytics, Apache Kafka, EventHub).
• Experience with containerization technologies (Docker, Kubernetes) for deploying data workloads.

Personal Attributes:

• A data-driven mindset with the ability to translate complex business challenges into scalable, technical solutions.
• Strong hands-on leadership with a passion for innovation and continuous improvement.
• Excellent problem-solving and troubleshooting skills, with a deep understanding of optimizing data performance.
• Strong communication skills, with the ability to articulate complex technical solutions to both technical and non-technical stakeholders.
• A proactive and self-motivated individual who thrives in a dynamic, fast-paced environment.