Digital Skills Ltd

Lead Data Platform Engineer Remote (UK-based) SC Clearance Required (British Citizen) Permanent - Up to £110,000 We are looking for a hands-on Lead Data Platform Engineer to join a high-impact programme delivering scalable, Real Time data solutions. This role offers the opportunity to work on complex, mission-critical systems using modern technologies across streaming, cloud, and distributed data platforms. The Role As a Lead Data Platform Engineer, you will design and deliver batch and Real Time data pipelines that are highly available, low-latency, and scalable. You will collaborate with cross-functional teams to shape solutions, influence architecture, and ensure data is transformed into reliable, consumable outputs. Key Responsibilities Design, build, and deploy data pipelines and Back End services Work with Real Time streaming technologies and event-driven architectures Champion engineering best practices including TDD, CI/CD, and clean code Collaborate with architects, analysts, and testers in Agile/BDD environments Take ownership of services in production, ensuring performance and reliability Drive continuous improvement across data platform capabilities Contribute to solution design, estimation, and delivery planning Required Experience Strong production Java experience Experience with Kafka, Kinesis, or similar streaming technologies Hands-on experience with Spark, Flink, Kafka Streams, or similar Experience with Spring Boot and/or Python Strong experience with AWS (including Lambda and S3) Experience integrating with PostgreSQL, Redis, or similar Solid understanding of data pipelines and data modelling Passion for clean, testable, and maintainable code Experience working in Agile and/or DevOps environments Desirable Skills Experience with Elasticsearch, OpenSearch, or Solr Familiarity with Terraform, Ansible, or Packer Experience with CI/CD tools such as Jenkins or Drone If you are interested, please apply or get in touch to discuss further.