About Us: We are a forward-thinking company leveraging cutting-edge technology to drive innovation and efficiency. Our tech stack is 100% Azure Cloud, and we are looking for a skilled Data Scraper and Data Analytics Engineer to join our dynamic team.

Job Description:

Responsibilities:

· Develop and maintain web scraping scripts to extract data from external websites using tools like Python, Scrapy.

· Capture and process clickstream and temporal data from internal websites.

· Design and implement data pipelines using Azure Data Factory, Azure Databricks, and other Azure services.

· Create ETL scripts using python and/or Scala to clean, preprocess, and store data in Azure Data Lake, Azure SQL Database, and Azure Blob Storage.

· Collaborate with data scientists and engineers to train recommendation engines and generate predictions.

· Ensure data quality, integrity, and security throughout the data lifecycle.

· Monitor and optimize data scraping and processing workflows for performance and cost-efficiency.

· Stay updated with the latest trends and best practices in web scraping, data engineering, and Azure Cloud services.

Requirements:

· Bachelor’s degree in Computer Science, Data Science, or a related field.

· 3+ Years of hands-on working experience as Web Scrapping Specialist or Web Scraping Engineer.

· Proven experience in web scraping using Python, BeautifulSoup, Scrapy, or similar tools.

· Strong knowledge of Azure Cloud services, including Azure Data Factory, Azure Databricks, Azure Data Lake, Azure SQL Database, and Azure Blob Storage.

· Proficiency in SQL and experience with data processing frameworks like Apache Spark.

· Familiarity with machine learning concepts and experience working with recommendation engines.

· Bachelor’s degree in Computer Science, Data Science, or a related field.

· Excellent problem-solving skills and attention to detail.

· Strong communication and collaboration skills.

· Ability to work independently and manage multiple tasks effectively.

Preferred Qualifications:

· Experience with API integration and data extraction from various sources.

· Experiences with Microsoft Fabric and Microsoft Data Platform

· Knowledge of data visualization tools like Power BI or Tableau.

· Understanding of data privacy regulations and best practices.

Movate jobs

Data Scraper