Senior Analytics Engineer

About Imagine Worldwide All children have immense potential, but hundreds of millions don’t have access to the learning they need. Imagine Worldwide believes that every child should be empowered with the basic right to literacy and numeracy. We provide an innovative education technology solution and implementation model to the global literacy/numeracy learning crisis using the onebillion application and various toolkits and systems to support implementation. Our model is massively scalable, evidence-based, and can deliver foundational learning to millions of the most marginalized children. Imagine Worldwide partners with governments, organizations, and communities to provide child-directed, tablet-based learning that is accessible, effective, and affordable. We are incorporated in California (United States) as a nonprofit organization, but operate in Sub-Saharan Africa through locally-registered entities in Malawi, Sierra Leone, and Tanzania. Learn more on our website. Role Overview The Senior Analytics Engineer builds, runs and improves Imagine’s analytics platform: the infrastructure, pipelines and core data models that the organisation’s reporting and analysis depend on. The central task is to scale the platform as data volumes grow, keeping costs under control without giving up performance. In a small team, that means covering the infrastructure underneath the data as well as the data engineering itself. Imagine’s data comes from many sources. The largest is tablets in more than 1,500 schools across Malawi, Tanzania and Sierra Leone, which record learning activity as individual events, adding up to billions of records. These only become meaningful when joined with programme context, such as active schools per term, device allocations and enrolment, and with operational data from inventory and field data collection. Turning all of this into reliable data on a non-profit budget calls for careful decisions about where processing happens and what each workload costs. The role reports to the Director of Analytics, who sets the platform’s architecture and technical standards, and works alongside the rest of the Analytics Team. We are looking for someone senior who takes initiative, turns architectural direction into working systems, and brings forward evidence-based proposals for improving them. As the engineer builds context, they take on increasing ownership of the platform’s day-to-day running and technical decisions. Key Responsibilities The Senior Analytics Engineer’s responsibilities fall into four pillars. The first two cover the platform itself: the infrastructure it runs on, and the pipelines that bring data in from many sources and process it at scale. The third covers orchestration and the core data models that the rest of the Analytics Team builds on. The fourth covers the practices that keep the platform reliable, secure and affordable to run. Data Platform and Infrastructure Imagine’s analytics platform runs on AWS and ClickHouse Cloud, with infrastructure managed as code. In a small team, this role carries the infrastructure work that a larger organisation would split across dedicated platform or DevOps staff, limited to the scope of the analytics platform. Build and maintain the platform’s AWS infrastructure, including storage, compute, networking, the data catalogue, secrets and access management (S3, EC2, Glue Data Catalog, Lake Formation, IAM, Secrets Manager). Manage infrastructure as code in Terraform, keeping environments reproducible, version-controlled and reviewable. Administer the ClickHouse Cloud warehouse, including service configuration, user and role management, and monitoring of performance and capacity. Host and maintain the team’s self-hosted tools, including Apache Superset for BI and Apache Airflow for orchestration. This covers deployment, upgrades, authentication and availability. The Senior BI & Data Analyst manages the dashboards and content within Superset. Contribute to architectural decisions on how the platform evolves, setting out options and trade-offs for review with the Director of Analytics. Large-Scale Data Processing and Pipelines Data reaches the platform from application databases, operational systems and field data tools, and lands in an Apache Iceberg data lake on S3. Spark-based processing on AWS Glue does the heavy work before the data moves into ClickHouse for analysis. The aim is to do expensive processing once, as far upstream as practical, so that downstream models and queries stay fast and affordable. Build and maintain ingestion pipelines from the platform’s sources, including application databases, APIs and operational systems, using tools such as AWS Glue, Airbyte and AWS Lambda. Implement incremental loading patterns that stay efficient and recoverable as volumes grow, landing raw data in the lake in a form that preserves its full history. Develop and optimise Spark jobs that process billions of device events into learning sessions and higher-level aggregates, paying close attention to partitioning, skew and compute cost. Bring in and model the programme context that gives event data its meaning, such as schools, terms, device allocations and enrolment, and handle how that context changes over time. Design and maintain Iceberg tables for performance and cost, including partitioning strategy, file sizing, compaction and snapshot management. Validate data at ingestion and surface quality issues early, working with the Analytics Team to trace them back to their source. Contribute to evaluating new processing approaches as volumes grow, including moving more work upstream and assessing where streaming would add value. Orchestration and Transformation Apache Airflow orchestrates the platform’s pipelines, and dbt transforms data in ClickHouse into the models that reporting and analysis rely on. This role builds and maintains the foundational layers of those models. Build and maintain Airflow workflows that coordinate ingestion, pro