Data Engineer

Data Engineer - Hybrid working environment, London.

Salary - £70,000 - £80,000 + 10% Bonus

We are looking for a Data Engineer to join a global leading supplier in sophisticated data analysis working within the latest AI technology.

The ideal applicant will be part of a wholesale opportunity extracting data form an old legacy system, migrating to the latest Google cloud technology.

Their stack:

• Google Cloud Platform
• PostgreSQL
• Bash
• Python
• Flyway
• Oracle
• Groovy
• Spock

What you will be doing:

• Setting up and configuring distributed data technologies in GCP/AWS
• Setting up resilient, performant and scalable data pipelines
• Providing guidance on technology, library and tooling choices within the team
• Delivering a good understanding of when to persist, replicate and cache data
• Demonstrating the ability to wrangle and integrate data automatically/efficiently from various alternative data providers
• Providing a good working knowledge of various database types and when to use them, i.e. SQL, Columnar, Graph, Key-Value, and Document
• Having an understanding of database design patterns including OLTP and OLAP
• Having a knowledge with SQL and Data Warehouses including PostgreSQL preferable but MS SQL Server, MySQL, AWS Redshift and/or Snowflake are 'nice' to have
• Carrying out a serialisation of technologies such as JSON.Net, Parquet and Avro
• Experienced with REST and API design
• Able to automate testing with continuous integration and continuous delivery pipelines

What experience you will need:

• Atleast 5 years or more of software/data engineering experience
• Cloud technologies, such as AWS or GCP
• Docker and Kubernetes
• Working together, be that pairing, or wider team collaboration
• An interest in the agile principles, not just a single "Agile Process"
• Eagerness to use the appropriate tool for the job, not just the latest "hot" technology
• Sharing your knowledge and experience with other team members
• To have a good knowledge of SQL and no less than two other programming languages
• Experience in collaborative design of software systems including contracts, integration points and database schemas
• The ability to Test-Drive software with automated tests

If you think you would be a good fit for this role, please feel free to apply to the role directly or feel free to message me directly for a confidential chat