Description:
In this role, you will play a key part in designing, developing, and maintaining data solutions using Azure Synapse Analytics. You will leverage your expertise in Spark and SQL Database to transform raw data into valuable insights, driving informed decision-making across the organization.
Responsibilities:
- Design and develop data pipelines using Azure Data Factory (ADF) to extract, transform, and load (ETL) data from various sources like relational databases and cloud storage.
- Utilize Spark within Azure Synapse Analytics to process large datasets efficiently, performing complex transformations and aggregations.
- Design and implement data models within Azure Synapse Analytics leveraging SQL Database for efficient querying and analysis.
- Integrate Azure Synapse with external APIs to retrieve and process data from third-party applications.
- Develop and configure Azure Logic Apps to automate workflows and trigger data pipeline execution based on pre-defined events.
- Manage and optimize data storage using Azure Blob Storage, ensuring efficient data access and cost-effectiveness.
- Collaborate with data analysts and business stakeholders to understand data requirements and translate them into actionable data solutions.
- Implement best practices for data security, governance, and monitoring within the Azure Synapse environment.
Qualifications:
- Proven experience of 3+ years of working with data warehousing and data lake technologies.
- Expertise in T-SQL and experience with Spark for large-scale data processing.
- Experience with Azure Synapse Analytics, including Dedicated SQL Pools and Data Flows.
- Working knowledge of Azure Data Factory (ADF) for data integration.
- Familiarity with Azure Blob Storage and data management techniques.
- Experience with developing and configuring Azure Logic Apps.
- Excellent communication and collaboration skills.
- Strong analytical and problem-solving abilities.
- Experience with cloud-based data platforms like AWS or GCP.
- Experience with data visualization tools like Power BI or Tableau.
- Experience with DevOps principles and practices.