Data Quality Engineer

DESCRIPTION

The Data Quality Engineer role reports to the Director, Data Management for Human Resources. The role is part of our Enterprise Data Platform team that manages the ingestion, storage, modeling, and consumption of data for Acosta Group. The role is responsible for supporting workloads running on the Azure Big Data environment and associated technologies. This position will be responsible for creating and monitoring the data solutions, resolving incidents, optimizing workloads, tuning solutions, and making solution enhancements. The role will focus on development and production support and will also take part in making enhancements to the core platform.

RESPONSIBILITIES

• Building data ingestion and ETL pipelines at an enterprise level, following corporate standards and best practices.

• In addition to ingestion and ETL development, there will also be occasions where Azure function apps and Logic apps will be built to integrate with legacy and external systems.

• Work with Senior Data Engineers and Architects to understand and implement solutions based on high level designs and requirements.

• Work closely with Project Managers / Scrum Masters, Data Modelers and QA Engineers in both Agile and Waterfall work settings to deliver solutions against a deadline that meet the requirements of the business and product stakeholders.

• Stay current with big data, ETL and general data technologies, with a strong focus on the Azure technology stack.

• Monitoring, investigating, and maintaining ongoing production systems, specifically triggered and schedule ETL pipelines and data ingestion patterns.

• Continually focus on self-development by allocating time to continually develop your ingestion, ETL and general data skills.

• Optimization and refactoring of existing platform and legacy solutions.

• Migration of legacy systems into a modern data platform.

• Think critically about the data solutions you are delivering. You need to understand it and expand its implications beyond the numbers alone. As a critical thinker, you can think analytically about data and ETL, identifying patterns and extracting actionable insights and information from the solutions you have built.

• Meeting the physical requirements - listed below

QUALIFICATIONS

Minimum Education and Work Experience

• Minimum Bachelor's Degree in Computer Science, Information Technology, or related work experience in a related field

• 4 or more years of related work experience is ideal

Knowledge, Skills, and Abilities

• 4 or more years developing and supporting applications leveraging a big data stack (such as Azure)

• 4 or more years of experience with data integration/ETL tools such as Azure Data Factory or alternatives

• 2+ Software Development Lifecycle (SDLC) knowledge in both agile methodology

• Experience in Databricks and Data Lakehouses

• Experience on following tools: SQL (required), Pyspark (prefer), Python, Spark, Kafka, and other related languages

• Experience in scripting languages & automation such as bash and Python

• Hands-on experience with source code management system (SVN, Git) and continuous integration tools (Azure Devops or alternatives)

• Experience in handling data processing, delivering distributed and highly scalable applications

• Experience with high-volume, transaction processing software applications

• Good understanding of workload management, schedulers, scalability, and distributed platform architectures

• Excellent communication skills (both written and oral) combined with strong interpersonal skills

• Strong analytical skills and thought processes combined with the ability to be flexible and work analytically in a problem-solving environment.

Physical Requirements:

• Seeing

• Listening

ABOUT US

Acosta and Mosaic are the sales and marketing powerhouses behind the most recognized and proven brands with top retailers in the United States and Canada. We offer flexible services that maximize efficiency. Acosta has the talent and technology to build data-, reach- and relationship-driven strategies to execute those strategies, and the tools to monitor, track and optimize metrics-based results for customers and retailers.

Acosta and its subsidiaries, in good faith, believes that this posted range of compensation is the accurate range for this role at the time of this posting. Acosta may ultimately pay more or less than the posted range depending on candidate qualifications and locations. This range may be modified in the future.

Acosta and its subsidiaries is an Equal Opportunity Employer

We are committed to providing accommodations for persons with disabilities. If you require accommodation, we will work with you to meet your needs, to the extent required by law.

The above statements are intended to describe the general nature and level of work being performed by people assigned to this classification. They are not intended to be construed as an exhaustive list of all responsibilities, duties, and skills required of personnel so classified. Mosaic reserves the right to modify all or part of any job descriptions at its discretion in order to meet and or exceed the needs of the business.

By submitting your application you agree with and accept the Acosta Privacy Statement and Terms of Conditions.

Acosta et Mosaic sont les forces motrices en matière de ventes et de marketing derrière les marques les plus reconnues et éprouvées auprès des principaux détaillants aux États-Unis et au Canada. Nous proposons des services flexibles qui maximisent l'efficacité. Acosta possède le talent et la technologie nécessaires pour élaborer des stratégies axées sur les données, la portée et les relations, afin de mettre en œuvre ces stratégies, ainsi que les outils pour surveiller, suivre et optimiser les résultats basés sur les indicateurs pour les clients et les détaillants.

Acosta et ses filiales , de bonne foi, estiment que cette fourchette de rémunération affichée est la fourchette précise pour ce poste au moment de cette publication. Acosta pourrait finalement payer plus ou moins que la fourchette affichée en fonction des qualifications des candidats et des lieux. Cette fourchette peut être modifiée à l'avenir.

Acosta et ses filiales sont des employeurs offrant l'égalité des chances .

Nous nous engageons à fournir des aménagements pour les personnes handicapées . Si vous avez besoin d'un aménagement, nous travaillerons avec vous pour répondre à vos besoins, dans la mesure requise par la loi.

Les déclarations ci-dessus ont pour but de décrire la nature générale et le niveau de travail effectué par les personnes affectées à cette classification. Elles ne doivent pas être in.