Big Data Developer

About the role We are looking for a Big Data Developer to join an Agile environment and work on the design, development and evolution of data lake solutions for Market Risk use cases . You will work with technologies such as Spark, Scala and SQL , developing large-scale data processing pipelines and contributing to the ingestion, transformation, validation and publication of critical data. The role combines hands-on technical development with problem-solving and collaboration across Agile squads, functional teams and production support teams. What will you do? Design, develop and maintain data lake pipelines using Apache Spark, Scala and SQL . Build and maintain scalable data processing solutions for large volumes of data. Orchestrate batch workflows and dependencies using tools such as Control-M and Airflow . Implement data ingestion, transformation, validation and publication processes for Market Risk use cases. Analyse and optimise the performance, reliability and scalability of Spark-based workloads. Analyse technical and functional requirements, identify gaps and propose effective solutions. Contribute to delivery planning and collaborate with different teams to ensure successful implementation. Support production incidents , perform root-cause analysis and contribute to continuous improvement of operational processes. Work closely with Agile squads, functional teams and production support teams to deliver high-quality solutions and business value. Follow software engineering and data development best practices to ensure robust, maintainable and production-ready soluti

Source: Himalayas