ML Engineering • Data Systems • MLOps

Yohan Shanuka

ML Engineer

Engineering scalable data pipelines, MLOps workflows, and cloud-native ML systems.

pipeline-runner
RUNNING
pipeline.py
canary_monitor.log
18:45:21SYSexecuting...
Speed:
KafkaIngest
SparkProcess
AirflowSchedule
MLflowTrack
FastAPIServe
DockerPackage
KubernetesScale
PythonCore
KafkaIngest
SparkProcess
AirflowSchedule
MLflowTrack
FastAPIServe
DockerPackage
KubernetesScale
PythonCore
KafkaIngest
SparkProcess
AirflowSchedule
MLflowTrack
FastAPIServe
DockerPackage
KubernetesScale
PythonCore
KafkaIngest
SparkProcess
AirflowSchedule
MLflowTrack
FastAPIServe
DockerPackage
KubernetesScale
PythonCore
Yohan Shanuka

Engineering Mindset

Building Scalable ML & Data Systems.

I focus on building intelligent systems at the intersection of machine learning and data engineering, designing high-throughput distributed pipelines and deploying production-ready models that solve complex real-world challenges.

My goal is to develop production-ready machine learning workflows supported by reliable data infrastructure, modern backend systems, and scalable cloud architectures.

Particularly Interested In
MLOps & Automation
Data Engineering Pipelines
Machine Learning Systems
Cloud-Based Systems
Distributed Data Processing
Backend Infrastructure

Engineering Focus

Core Expertise

Three focused areas — each with a clear pipeline and the capabilities I bring to production systems.

Domain Focus

ML Engineering

Train, evaluate, and serve models with low-latency APIs.

System Lifecycle Flow
Features
Training
Serving
Core Capabilities
  • CNN & transfer learning
  • Model APIs & FastAPI
  • Prediction systems
  • Model optimization

Key Performance Index

Inference

< 100ms

Tools & Frameworks

Technology Ecosystem

A curated stack I use to build scalable ML systems, data pipelines, and cloud-native infrastructure.

ProficientCurrently Learning

Machine Learning

Ecosystem Focus

Developing and deploying deep learning, computer vision, and predictive models using modern frameworks.

Core Competencies

CNN & Transfer Learning
PyTorch & TensorFlow
Model Optimization
Average Skill86%
Verified Stack
90
TensorFlow
85
PyTorch
88
Scikit-learn
80
OpenCV
82
XGBoost
86
Keras

Ring % = self-assessed proficiency · actively expanding the stack

Case Studies

Production AI Systems

Deep-dive into the architecture, challenges, and metrics behind intelligent systems I've engineered.

01
Real-TimeDistributed · Stream Processing

Real-Time Streaming Pipeline

End-to-end streaming pipeline handling high-throughput event data with sub-500ms latency. Replaced legacy batch jobs that caused 24-hour reporting delays.

Core Challenge

24-hour delays in critical metric reporting due to batch processing.

Infrastructure:KafkaSpark StreamingMongoDBDocker
Languages & ML:PythonFastAPI
Real-TimeDistributedMLOps Enabled
02
Edge AIIoT · ML Prediction · Alerting

Cattle Health Monitoring System

IoT-driven pipeline combining streaming sensor ingestion with ML prediction to monitor livestock health patterns and trigger real-time alerts.

Core Challenge

Manual monitoring caused late disease detection and yield loss.

Infrastructure:MongoDBFastAPIKafkaDocker
Languages & ML:PythonTensorFlow
Edge AIAPI IntegratedComputer Vision
03
Computer VisionCNN · FastAPI · Docker Serving

Tomato Leaf Disease Classifier

Production-grade CNN microservice for automated crop disease diagnosis. Sub-100ms inference latency with a fully containerized deployment pipeline.

Core Challenge

Farmers needed reliable automated API for rapid field image diagnosis.

Infrastructure:FastAPIDockerReactOpenCV
Languages & ML:PythonTensorFlow
Computer VisionAPI IntegratedMLOps Enabled
Academic Contribution

Research & Publications

Peer-reviewed research at the intersection of IoT systems, edge AI, and real-world agricultural engineering challenges.

1Publications
IoT & AIDomain
DRR·VAURepository
AgriTechFocus Area
Research PaperApplied Research2024

Real-Time Cattle Monitoring Using Low-Cost IoT Smart Collars with LoRa Communication in Sri Lanka's Dry Zones

Authors

Yohan Shanuka, J.A.Det al.

Abstract

Explores the design and deployment of a low-cost IoT smart collar system using LoRa communication for real-time cattle health monitoring across remote dry-zone environments in Sri Lanka. The system captures biometric and behavioural data — including temperature, motion, and heart rate — and transmits it over long-range low-power networks to an ML-backed prediction and alert pipeline.

KeywordsIoTLoRaCattle MonitoringEdge AIReal-Time SystemsSri LankaSmart Agriculture
View PublicationAgriTech · Edge AI

More research in progress — stay tuned for upcoming publications.

System Design

Scalable Architecture

Blueprint-level architecture diagrams showcasing distributed workflows, ETL patterns, and containerized deployments.

Interactive Blueprint Map// 1.2M events/sec · 42ms avg latency
Kafkaevent streamSparkprocessingMongoDBstorageDashboardanalyticsSchema RegistryDQ Checks
Telemetry Console
Blueprint Profile:

Streaming Data Architecture

High-throughput event pipeline

ThroughputPeak load scale
1.2M / s
Pipeline SLAService uptime
99.99%
Avg LatencyEvent execution duration
42ms
Interactive Map

Select any component in the blueprint diagram on the left to initialize logs and specs telemetry.

Active Development

Currently Building

Real-time progress on active engineering projects — because great systems are always evolving.

In Progress
72%

Cattle Health AI Monitoring

Real-time IoT sensor ingestion + CNN-based anomaly detection with alerting pipeline.

KafkaTensorFlowMongoDBFastAPI
In Progress
88%

CNN Tomato Disease Classifier

Transfer learning pipeline with automated retraining triggers and model versioning.

PyTorchMLflowDockerFastAPI
In Progress
55%

Streaming ML Inference Pipeline

Sub-100ms inference pipeline serving Kafka-triggered predictions via FastAPI endpoints.

KafkaSparkFastAPIRedis

Commit Activity

Last 7 months

Real-time cattle health AI
CNN disease classifier
Streaming inference pipeline
View GitHub Profile →

Certifications

Professional credentials validating expertise in cloud architecture, machine learning, and data engineering.

AWS Certified Machine Learning – Specialty

Amazon Web Services (AWS)
2024

Data Engineering with Google Cloud

Google Cloud / Coursera
2023

Deep Learning Specialization

DeepLearning.AI
2023

How I Think

Engineering Philosophy

I focus on building scalable, production-ready AI systems that combine machine learning, distributed data pipelines, and efficient backend infrastructure — not just models, but intelligent systems.

Scalability

Systems that grow with demand. Every pipeline I build is designed to handle 10× the expected load from day one.

Reliability

99.9% uptime is the baseline. I engineer fault-tolerant architectures with graceful degradation and self-healing capabilities.

Observability

You can't improve what you can't measure. Every system ships with metrics, logs, and alerting baked in from the start.

Get In Touch

Looking for an MLOps or Data Engineer to build scalable systems? Let's connect.

Location

Available Worldwide (Remote)