We're looking for a skilled Software Developer to join a public sector delivery team working on data-driven services. You'll design, build, and maintain robust, secure, and scalable data pipelines and applications, working closely with data engineers, analysts, and platform teams.
Day to day duties
- Developing and maintaining data pipelines and processing jobs using Apache Spark and Python
- Orchestrating workflows with Apache Airflow
- Provisioning and managing cloud infrastructure on AWS using Terraform (Infrastructure as Code)
- Containerising applications and services with Docker
- Managing source control, CI/CD pipelines, and deployments through GitLab
- Integrating automated security scanning into the development pipeline to ensure code and dependencies meet public sector security standards
- Using Jupyter Notebooks for data exploration, prototyping, and analysis
- Building and maintaining integrations with DWP Integration Team products and services
- Collaborating with cross-functional teams in an Agile environment, contributing to design reviews, code reviews, and documentation
Essential Skills & Experience
- Strong hands-on experience with Python
- Experience with Apache Spark for large-scale data processing
- Practical knowledge of AWS services (eg S3, Lambda, EC2, Glue, RDS - adjust as relevant)
- Experience writing and maintaining Terraform for infrastructure as code
- Experience with Apache Airflow for workflow orchestration
- Familiarity with Docker and containerised deployments
- Experience using GitLab for version control and CI/CD
- Understanding of security scanning tools/agents and secure coding practices
- Experience using Jupyter Notebooks
- Experience integrating with government/departmental systems, ideally DWP Integration Team products (or similar public sector integration frameworks)
Desirable
- Experience working within public sector or government digital service standards
- Understanding of data protection and information security requirements in a public sector context
- Experience working in Agile/Scrum delivery teams
Applicants must have active SC Clearance to be considered.