Address
Form of work
SalaryJava Developerworking with a Financial Services Company for an initial 6-month contract, this will be paying between £700 - £750 per day (inside IR35). This is a Hybrid role, 2-3 days per week in the London office.
Job Summary
We are seeking a Software Engineer to build and optimize our data platforms leveraging AWS and key technologies like CDAP, Snowflake, and Databricks. You will design and implement robust and scalable data pipelines, ETL, and analytics systems in the cloud.
Main Responsibilities
- Develop and enhance data pipelines, ETL processes using CDAP on AWS infrastructure.
- Build data integration flows to migrate large datasets into Snowflake data warehouse.
- Implement AWS infrastructure-as-code solutions for deployment automation.
- Instrument data pipelines and leverage monitoring for performance tuning and reliability.
- Work with data scientists to optimize data workflows and models on Databricks.
- Follow security best practices for access control, encryption, auditing across data platforms.
- Participate in architecture reviews and technology selections.
- Continuously monitor and improve data platforms for scalability and costs.
Required Experience
- 5+ year's experience in backend development or data engineering.
- Hands-on experience with AWS services like S3, EC2, EMR.
- Proficiency in SQL and experience with CDAP, Spark, Kafka.
- Experience building scalable ETL processes and workflows.
- Strong programming ability with Python, Java and unit testing.
- Infrastructure-as-code expertise with CI/CD pipelines.
- Ability to communicate complex topics clearly.
Desirable Skills
- Experience with Snowflake, Databricks, and GCP or Azure.
- Knowledge of streaming data architectures
- Data security and compliance implementation.
- Machine Learning Operations (MLOps) experience.
Please apply should you meet the requirements.
