![\"\"](\"https://cdn-images-1.medium.com/max/1024/0*cgqbpW6nkSyHzSJe.png\")
How S3 Files enables shared storage for Kubernetes workloads on AWS
Your data lives in S3. Your application needs a file system. And between those two things, you end up writing sync scripts, duplicating data into EFS, paying double the storage cost, and still wondering why your pipeline is slow or your pods are failing on startup because the data was not ready yet.
This is not a small inconvenience. For teams running ML pipelines, multi-pod workloads, or anything that needs shared access to large datasets, this is a real architectural problem that wastes both time and money every single day.
Amazon S3 Files changes that. And if you are running workloads on EKS, this is one of those releases you actually need to pay attention to.
Amazon S3 Files is a feature that provides file system–like access to data stored in S3. Instead of interacting with S3 only through APIs, applications can access data using standard file operations.
In practice, this means an S3 bucket can be mounted and accessed similar to a network file system. Applications can read, write, and manage files using familiar file system interfaces and existing libraries, without requiring changes to how they handle data.
Multiple compute services — including EC2, ECS, and EKS — can access the same data concurrently through this interface, enabling shared access patterns that were difficult to implement with traditional S3 access methods.
An important aspect of this model is that the data remains in S3. There is no requirement to copy or synchronize data into a separate file system before it can be used. The same underlying data can be accessed both through standard S3 APIs and through the file system interface.
![\"\"](\"https://cdn-images-1.medium.com/max/1024/0*auWR6GGoyueO2_nl.png\")
To understand how S3 Files integrates with EKS, it is useful to look at how Kubernetes handles storage at a high level.
Kubernetes uses three core objects for storage.
A PersistentVolume (PV) represents the actual storage resource, such as an EBS volume, an EFS file system, or an external storage system exposed through a CSI driver.
A PersistentVolumeClaim (PVC) is how a pod requests storage. It defines the required size and access mode, and Kubernetes binds it to a matching PersistentVolume.
A StorageClass defines how storage should be provisioned. It is associated with a specific CSI driver and includes configuration parameters for dynamic provisioning.
The CSI driver (Container Storage Interface) is the integration layer that allows external storage systems to work with Kubernetes. AWS provides CSI drivers for services like EBS, EFS, and S3-based access.
S3 Files integrates into this model through a CSI-based approach, which enables it to be mounted and used by pods like other external storage systems.
Before S3 Files, using S3 storage on EKS typically involved the Mountpoint for Amazon S3 CSI driver. Mountpoint for S3 started as an open-source FUSE-based client and was later integrated into Kubernetes through a CSI driver.
It worked well for read-heavy workloads where pods needed to access data directly from S3 without copying it first. However, it had limitations that made it unsuitable for general-purpose or write-heavy use cases.
S3 objects are immutable, so modifying part of a file required replacing the entire object. There was no support for file locking, which made concurrent writes across multiple pods unreliable. Common file system operations such as directory renames were also not supported. As a result, workloads requiring shared read-write access across pods could not rely on this approach.
![\"\"](\"https://cdn-images-1.medium.com/max/1024/0*vnpy9jIcUNpzj0l1.png\")
In practice, this led to a two-layer storage pattern: S3 for durable storage and EFS for file system access. Data was often copied or synchronized between the two, increasing cost and adding operational overhead.
S3 Files integrates with EKS through a CSI-based approach. Instead of using the Mountpoint for S3 CSI driver, it relies on the Amazon EFS CSI driver (version 3.0.0 or above) to enable mounting through a file system interface.
The setup involves a few key steps.
First, create an S3 file system associated with your bucket:
aws s3files create-file-system \\
--bucket your-bucket-name \\
--file-system-name my-s3-filesystem
This returns a file system identifier, which is used when defining the PersistentVolume in Kubernetes.
Next, configure IAM permissions for your pods. This is typically done using EKS Pod Identities by attaching the required policy to the pod execution role:
aws iam attach-role-policy \\
--role-name your-eks-pod-role \\
--policy-arn arn:aws:iam::aws:policy/AmazonS3FilesCSIDriverPolicy
Then install or upgrade the EFS CSI driver in your cluster:
helm repo add aws-efs-csi-driver \\
https://kubernetes-sigs.github.io/aws-efs-csi-driver/
helm repo update
helm upgrade --install aws-efs-csi-driver \\
aws-efs-csi-driver/aws-efs-csi-driver \\
--namespace kube-system \\
--version 3.0.0
After this, define a PersistentVolume using the S3 file system identifier, bind it to a PersistentVolumeClaim, and mount it into your pods. This allows multiple pods to access the same underlying data through a shared file system interface.
One important requirement is that the EKS cluster and the S3 Files mount target must be within the same VPC. Ensure that network and security group configurations allow NFS traffic (port 2049) between worker nodes and the mount target.
S3 Files is not the right solution for every storage requirement on EKS. However, for specific workloads, it addresses long-standing limitations in how data is accessed and shared.
Machine learning training pipelines are a primary example. Large datasets are typically stored in S3, often ranging from tens to hundreds of gigabytes. Traditionally, this required staging data into a file system or copying it to nodes before processing could begin. With S3 Files, training pods can access the same dataset through a shared interface without requiring additional data movement.
AI agent workloads benefit in a similar way. These systems often need to persist state, write checkpoints, and share data across multiple pods running in parallel. A shared file system interface simplifies coordination, allowing different pods to read and write data in a more consistent manner.
Shared configuration and certain log-processing scenarios are also relevant. Instead of relying on intermediate systems to move data into S3, workloads can interact with shared files directly through the mounted interface. This can simplify how data is produced and consumed across pods.
Data lake workloads are another strong fit. When analytics data is already stored in S3 and processing tools expect a file system interface, S3 Files can reduce the need for intermediate data transfer steps before processing.
S3 Files and EFS serve different purposes, even though both provide file system–style access for workloads on EKS.
EFS is designed for low-latency, fully managed file storage where data is frequently accessed. It is well suited for workloads that require consistent performance and traditional file system behavior across all data.
S3 Files is better aligned with scenarios where data already resides in S3 and needs to be accessed through a file system interface. Instead of maintaining a separate file system, it allows workloads to interact with the same underlying data using file-based access patterns.
Another key difference is access flexibility. With S3 Files, data remains in S3 and can still be accessed through standard S3 APIs, CLI tools, and SDKs. This allows the same dataset to be used across both object-based and file-based workflows. With EFS, data is managed separately and does not natively integrate with S3 without additional data movement.
In terms of workload fit, EFS is more suitable for latency-sensitive applications and environments that require full compatibility with file system operations. S3 Files is more appropriate for large datasets, shared access scenarios, and workflows where avoiding data duplication is important.
There are also platform considerations. EFS supports EKS workloads running on both EC2 and Fargate, while S3 Files is currently limited to EC2-backed nodes.
![\"\"](\"https://cdn-images-1.medium.com/max/1024/0*4bimLib1Odn9bZuP.png\")
S3 Files is a new capability and comes with limitations that are important to consider before using it in production.
Fargate is not supported. S3 Files currently works only with EC2-backed EKS nodes. Workloads running on Fargate cannot use this feature.
Static provisioning only. Dynamic provisioning is not available. File systems must be created separately, and PersistentVolumes need to be defined manually, which can add operational overhead.
S3 key length limits affect directory depth. S3 enforces a maximum object key length of 1,024 bytes. Applications that generate deeply nested paths or long file names may encounter this limit.
File-level operations are not available in CloudTrail. CloudTrail captures control plane actions such as creating or modifying file systems, but individual file operations are not logged at that level. Monitoring is typically available through aggregated metrics.
EFS CSI driver version requirement. S3 Files requires version 3.0.0 or later of the EFS CSI driver. Clusters running older versions must be upgraded before using this feature.
kubectl get deployment efs-csi-controller \\
-n kube-system \\
-o jsonpath=’{.spec.template.spec.containers[?(@.name==”efs-plugin”)].image}’
S3 Files addresses a long-standing gap in how storage is handled for EKS workloads. It introduces a way to access S3 data through a file system interface, reducing the need for separate storage layers in certain scenarios.
For EKS environments, this can simplify architectures that previously relied on combining S3 with a separate file system for application access. In cases where data already resides in S3, workloads can interact with it more directly without requiring additional data movement.
However, it is not a complete replacement for existing storage solutions. Limitations such as lack of Fargate support, static provisioning, and its early-stage nature mean it should be evaluated carefully before production use.
For workloads such as machine learning pipelines, shared data processing, and data lake access, S3 Files provides a practical alternative where file-based access to S3 data is required.
A reasonable approach is to start with controlled workloads, evaluate behavior, and determine where it fits within your existing architecture.
How Amazon S3 Files Changes Kubernetes Storage on EKS was originally published in DevOps.dev on Medium, where people are continuing the conversation by highlighting and responding to this story.
","url":"https://blog.devops.dev/how-amazon-s3-files-changes-kubernetes-storage-on-eks-9ab5fd8e0436?source=rss-3302bbae903c------2","hash":"sync-hash","mediumUsername":"bhagirath00","createdAt":"2026-05-07T04:44:50.582Z","updatedAt":"2026-05-07T04:45:06.381Z","tags":[]},{"id":"cmov05h6j0001u1r4lcw34n3q","title":"Using Docker Hub to Store, Share, and Manage Docker Images","slug":"using-docker-hub-to-store-share-and-manage-docker-images-6bf747994937","publishedAt":"2026-02-01T16:00:04.000Z","readingTime":null,"thumbnail":"https://cdn-images-1.medium.com/max/873/1*MW2KoAWhtEd-o_i5SUCfnA.png","excerpt":"Learn how Docker Hub is used to store, share, and manage Docker images efficiently for development, CI/CD pipelines, and production deployments.Docker Hub is th...","content":"Learn how Docker Hub is used to store, share, and manage Docker images efficiently for development, CI/CD pipelines, and production deployments.
![\"\"](\"https://cdn-images-1.medium.com/max/873/1*MW2KoAWhtEd-o_i5SUCfnA.png\")
Docker Hub is the most widely used container registry for storing, sharing, and managing Docker images. It serves as a central platform where developers and DevOps teams can publish images, collaborate efficiently, and integrate container workflows into modern CI/CD pipelines.
As a core part of the Docker ecosystem, Docker Hub enables teams to maintain a consistent and reliable source of container images across development, testing, and production environments. With features such as public and private repositories, access control, and automated image builds, it simplifies how containerized applications are distributed and deployed.
In this blog post, we’ll explore how Docker Hub works in practice — covering image push and pull operations, repository management, access control, and best practices for maintaining secure and reliable container images. Whether you’re just starting with containers or managing large-scale deployments, Docker Hub remains a critical tool for efficient container image management.
Docker Hub is a cloud-based container registry that allows users to store, manage, and distribute Docker images efficiently. It functions much like GitHub for Docker images, providing a centralized platform where developers can publish containerized applications and share them with teams or the broader community.
Docker Hub hosts both public and private repositories, enabling teams to distribute open-source images or securely manage internal applications. By using Docker Hub, developers ensure that Docker images are versioned, accessible, and consistent across different environments.
At its core, Docker Hub focuses on two key responsibilities:
Docker Hub acts as a central storage layer for Docker images, ensuring they are safely stored and accessible from any system running Docker.
Docker Hub simplifies image delivery by allowing images to be uploaded once and pulled multiple times across different environments, supporting faster deployments and consistent application behavior.
Docker Hub is more than just a storage service — it’s a complete platform for managing, distributing, and automating container images. Its features are designed to support teams, pipelines, and production-ready deployments.
Securely store Docker images in public or private repositories and share them across development, testing, and production environments.
Automatically build images from GitHub or GitLab whenever code is updated, keeping containers up to date with minimal manual effort.
Manage who can view, push, or pull images. Organize users into teams and roles for secure collaboration at scale.
Use official images from trusted vendors like Nginx, MySQL, Redis, or verified publisher images for improved security and reliability.
Trigger builds, deployments, or workflows automatically using webhooks, keeping production environments synchronized with the latest code.
For large organizations, manage multiple teams, repositories, and permissions efficiently using Docker Hub organization features.
Docker Hub is a general-purpose container registry, but it is not the only option. Teams choose container registries based on scale, security requirements, and cloud integration.
![\"\"](\"https://cdn-images-1.medium.com/max/1024/1*IBbLh5DjD43Me_YGK3dWVg.png\")
Docker Hub is well-suited for open-source projects and small to medium teams. For large-scale or cloud-specific production systems, organizations often rely on cloud-native or private container registries.
Getting started with Docker Hub is quick and easy. Creating an account allows you to store, manage, and share Docker images with your team or the wider community.
Once verified, log in to Docker Hub via the web interface or the command line interface (CLI) using:
docker login
Enter your Docker Hub username and password when prompted. This authenticates your machine to push and pull images.
After logging in, you can:
With your account set up, you’re ready to push Docker images and pull existing images.
Docker Hub isn’t just a storage place — it’s a smart, organized platform for managing Docker images at scale. Understanding its repository structure helps teams collaborate, automate workflows, and deploy reliably.
Each image on Docker Hub follows a clear naming format:
<username>/<repository>:<tag>
Example:
bhagirath00/webapp:latest
This structure makes it easy to track versions, roll back updates, and share images consistently across teams.
![\"\"](\"https://cdn-images-1.medium.com/max/1024/1*YoJ-YY-dx-1qbKEOLDkLWw.png\")
Docker Hub supports flexible visibility options:
Tags are more than just labels — they are powerful tools for version control:
In real-world systems, Docker Hub functions as a central image distribution layer between build systems and runtime environments. It enables consistent and repeatable deployments across development, staging, and production environments.
![\"\"](\"https://cdn-images-1.medium.com/max/1024/1*aVq1wkIaBcii8pSR5sH9hA.png\")
Once your Docker image is ready locally, you can push it to Docker Hub so it’s accessible from any environment.
Step 1: Authenticate with Docker Hub
Log in from your terminal:
docker login
Enter your Docker Hub username and password when prompted. This connects your local machine to Docker Hub.
Step 2: Tag Your Image
Docker requires images to be tagged before pushing:
docker tag <local-image> <username>/<repository>:<tag>
Example:
docker tag myapp bhagirath00/myapp:v1.0
Step 3: Push the Image
docker push bhagirath00/myapp:v1.0
Once complete, your image is available on Docker Hub, ready to be pulled by others or deployed in production.
Pulling images lets you download Docker images from Docker Hub to your local system so they can be run as containers.
![\"\"](\"https://cdn-images-1.medium.com/max/1024/1*nalUyTVfgqSwYZhLPZVwjw.png\")
Step 1: Pull the Image
Use the docker pull command:
docker pull <username>/<repository>:<tag
Example:
docker pull bhagirath00/myapp:v1.0
If you omit the tag, Docker defaults to latest:
docker pull bhagirath00/myapp
Step 2: Run the Pulled Image
After pulling, run the image as a container:
docker run bhagirath00/myapp:v1.0
This launches the container with the environment defined by the image.
Managing Docker images extends beyond building and pulling them. In production environments, images must be controlled, audited, and retired systematically to maintain stability and operational reliability.
Images should be built in controlled environments, typically CI systems, and distributed through Docker Hub as the single source of truth. Rebuilding images directly in production environments introduces inconsistency and operational risk.
The same image artifact should be used across development, staging, and production environments. Rebuilding images per environment increases configuration drift and makes issues difficult to reproduce.
As applications evolve, outdated images must be removed to:
Only images required for active deployments and rollback scenarios should be retained.
A managed image lifecycle enables fast recovery during incidents by redeploying previously validated images rather than rebuilding under pressure. This approach minimizes downtime and reduces failure risk.
Effective lifecycle management ensures Docker Hub remains a controlled distribution system rather than an unstructured image store.
Docker Hub applies usage limits that primarily affect image pulls, especially in automated and production environments.
![\"\"](\"https://cdn-images-1.medium.com/max/1024/1*f9dmUhbWBzC23iU64WFbwQ.png\")
Docker Hub provides different subscription tiers to support varying usage levels:
Docker Hub allows you to organize, secure, and control access to your Docker images efficiently. Proper repository and access management ensures that your images are available to the right people and protected from unauthorized access.
To organize your Docker images, start by creating a repository:
Your repository is now ready to receive images.
![\"\"](\"https://cdn-images-1.medium.com/max/1024/1*6wPUn6q2AV4GwfHLt0Sb3w.png\")
For private repositories, Docker Hub lets you control who can push, pull, or manage images:
Docker Hub supports organizations, allowing businesses to:
Docker Hub can automatically build Docker images whenever changes are pushed to a connected Git repository. Automation streamlines workflows, reduces manual effort, and ensures images are always up to date with the latest code.
To enable automated builds, link your GitHub or GitLab account:
This allows Docker Hub to monitor code changes and trigger builds automatically.
Automation ensures every code change produces a fresh, versioned Docker image, keeping deployments consistent.
Docker Hub is a powerful platform for storing, sharing, and managing Docker images. Whether you’re working on a small personal project or a large enterprise application, Docker Hub provides the flexibility and scalability required to manage container images effectively.
In this post, we covered how to push and pull images, manage repositories, automate builds, and apply best practices for using Docker Hub efficiently. Mastering Docker Hub helps streamline workflows, enhance collaboration, and ensure Docker images are consistently available for deployment.
Using Docker Hub to Store, Share, and Manage Docker Images was originally published in DevOps.dev on Medium, where people are continuing the conversation by highlighting and responding to this story.
","url":"https://blog.devops.dev/using-docker-hub-to-store-share-and-manage-docker-images-6bf747994937?source=rss-3302bbae903c------2","hash":"sync-hash","mediumUsername":"bhagirath00","createdAt":"2026-05-07T04:44:53.179Z","updatedAt":"2026-05-07T04:45:08.097Z","tags":[]},{"id":"cmov05hpn0002u1r42wyfz906","title":"End-to-End CI/CD Pipeline Using Jenkins and Kubernetes","slug":"end-to-end-ci-cd-pipeline-using-jenkins-and-kubernetes-99be6542a07f","publishedAt":"2026-01-19T12:40:12.000Z","readingTime":null,"thumbnail":"https://cdn-images-1.medium.com/max/1024/1*wdnX5U75JIcltmXmuCz_XA.png","excerpt":"Building Scalable, Cloud-Native CI/CD Pipelines with Jenkins and KubernetesIn modern DevOps workflows, running Jenkins on static or long-lived build agents ofte...","content":"Building Scalable, Cloud-Native CI/CD Pipelines with Jenkins and Kubernetes
In modern DevOps workflows, running Jenkins on static or long-lived build agents often leads to scalability issues, inefficient resource usage, and maintenance overhead. As applications grow and deployment frequency increases, CI/CD systems must be dynamic, resilient, and cloud-native.
Kubernetes solves these challenges by providing on-demand, isolated, and auto-scalable environments for Jenkins workloads. By integrating Jenkins with Kubernetes, teams can dynamically provision build agents as pods, optimize resource utilization, and build highly scalable CI/CD pipelines.
In this blog, you’ll learn how Jenkins integrates with Kubernetes for CI/CD, understand the pipeline architecture, set up Jenkins on Kubernetes, and build a production-ready CI/CD pipeline using containerized workloads and Kubernetes deployments.
![\"\"](\"https://cdn-images-1.medium.com/max/1024/1*wdnX5U75JIcltmXmuCz_XA.png\")
Kubernetes provides a robust and scalable platform for running containerized applications, and Jenkins is a powerful tool for automating the CI/CD pipeline. When integrated, these two tools can provide significant benefits:
Before integrating Jenkins with Kubernetes, ensure you have the following prerequisites in place. These prerequisites form the foundation for a stable and production-ready CI/CD setup.
Jenkins integrates with Kubernetes using the Kubernetes Plugin, which allows Jenkins to run CI/CD jobs inside Kubernetes pods instead of on static build agents.
In this setup, Jenkins focuses on orchestrating the pipeline, while Kubernetes handles executing jobs and managing resources. Whenever a pipeline starts, Jenkins asks Kubernetes to spin up a temporary pod to run the job. Once the job finishes, the pod is automatically removed.
This makes the entire CI/CD system dynamic, scalable, and cloud-native.
This CI/CD architecture uses Jenkins as the pipeline orchestrator and Kubernetes as the execution and deployment platform. Instead of relying on static Jenkins agents, Kubernetes dynamically provisions build agents as pods, making the pipeline scalable and resource-efficient.
![\"\"](\"https://cdn-images-1.medium.com/max/1024/0*_ytHNYS4c7d4lfEC.png\")
The pipeline begins with a code change pushed to a Git repository (GitHub, GitLab, or Bitbucket).
A webhook triggers Jenkins automatically on every commit or pull request, ensuring that no manual intervention is required.
Role of Git:
The Jenkins controller manages the CI/CD pipeline logic defined in the Jenkinsfile.
When a build is triggered, Jenkins does not execute jobs on itself. Instead, it requests Kubernetes to create an ephemeral agent pod.
Key Responsibilities:
Using the Jenkins Kubernetes Plugin, Jenkins dynamically spins up agent pods inside the Kubernetes cluster. Each pipeline run gets its own isolated pod, which is destroyed after completion.
Why this matters:
![\"\"](\"https://cdn-images-1.medium.com/max/1024/0*Ju4WkHXjTDwaZKx0.png\")
Each agent pod can include multiple containers (for example: Maven, Docker CLI, kubectl ), allowing different stages to run in the right environment.
Inside the Kubernetes agent pod, Jenkins builds the application and creates a Docker image using the project’s Dockerfile.
The image is then pushed to a container registry such as Docker Hub, Amazon ECR, or GCR.
What happens here:
This ensures the same image is used across all environments.
Once the Docker image is available in the registry, Jenkins deploys the application to Kubernetes using kubectl or Helm.
Deployment flow:
This completes the end-to-end CI/CD loop from code commit to a running application in Kubernetes.
Getting Jenkinsfileup and running on Kubernetes is easier than you might think, especially with Helm, the package manager for Kubernetes. Helm simplifies complex deployments and ensures you can get a production-ready Jenkins instance quickly.
![\"\"](\"https://cdn-images-1.medium.com/max/1024/0*_cL08duqC6hEyNfQ.png\")
The easiest way to install Jenkins on Kubernetes is using Helm.
It’s a good practice to isolate Jenkins in its own namespace:
kubectl create namespace jenkins
Helm is a package manager for Kubernetes that simplifies the installation of complex applications like Jenkins. To install Jenkins using Helm:
helm repo add jenkins https://charts.jenkins.io
helm repo update
helm install jenkins jenkins/jenkins --namespace jenkins
Once installed, you can access Jenkins via the Kubernetes service. To get the admin password:
kubectl get svc --namespace jenkins
kubectl exec --namespace jenkins -it $(kubectl get pods --namespace jenkins -l "app.kubernetes.io/component=jenkins-master" -o jsonpath="{.items[0].metadata.name}") -- cat /run/secrets/chart-admin-passwordOpen Jenkins in your browser using the service IP and port, then log in using the retrieved admin password.
Once Jenkins is installed, configure it to use Kubernetes for dynamic agent provisioning:
3. Create Pod Templates: Pod templates define what containers are included in each Jenkins agent pod. You can create different templates for different types of jobs, for example:
With Jenkins configured to use Kubernetes, the next step is to set up CI/CD pipelines that build and deploy applications to Kubernetes.
A Jenkinsfile allows you to describe your entire pipeline — build, test, and deployment as code, making it version-controlled, repeatable, and easy to maintain.
![\"\"](\"https://cdn-images-1.medium.com/max/1024/0*dRSOPfqg481_5UD0.png\")
A Jenkinsfile defines what steps your pipeline runs and where they run.
When using Kubernetes integration, Jenkins dynamically creates a pod-based agent for each pipeline execution.
Here’s an example of a Jenkinsfile that uses Kubernetes agents and deploys an application to a Kubernetes cluster:
pipeline {
agent {
kubernetes {
label 'my-k8s-agent'
defaultContainer 'jnlp'
yaml '''
apiVersion: v1
kind: Pod
spec:
containers:
- name: maven
image: maven:3.9.6-eclipse-temurin-17
command:
- cat
tty: true
- name: kubectl
image: bitnami/kubectl:latest
command:
- cat
tty: true
'''
}
}
stages {
stage('Build') {
steps {
container('maven') {
sh 'mvn clean install'
}
}
}
stage('Test') {
steps {
container('maven') {
sh 'mvn test'
}
}
}
stage('Deploy to Kubernetes') {
steps {
container('kubectl') {
sh 'kubectl apply -f deployment.yaml'
}
}
}
}
}What’s happening here?
This approach keeps builds clean, isolated, and scalable.
In the pipeline above, the Deploy to Kubernetes stage uses kubectl to apply Kubernetes manifests.
These YAML files typically define resources such as:
Because deployment happens only after successful build and test stages, Jenkins ensures that only validated artifacts reach your Kubernetes cluster.
This automation removes manual deployment steps and enables fast, consistent releases.
While kubectl apply works well, managing multiple YAML files can become difficult as applications grow.
This is where Helm becomes extremely useful.
Helm allows you to:
Here’s a simple Jenkinsfile example that deploys an application using Helm:
pipeline {
agent any
stages {
stage('Build') {
steps {
sh 'mvn clean install'
}
}
stage('Deploy to Kubernetes with Helm') {
steps {
sh 'helm upgrade --install myapp ./helm-chart/'
}
}
}
}With Helm:
To get the most out of Jenkins and Kubernetes, it’s important to follow a few proven best practices. These help keep your pipelines scalable, secure, and easy to maintain as workloads grow.
As CI/CD pipelines grow in complexity and usage, monitoring and scaling become critical to maintaining performance and reliability. Kubernetes makes this much easier by providing built-in scalability and strong observability integrations.
![\"\"](\"https://cdn-images-1.medium.com/max/1024/0*PBVgqZxIYq0AUFam.png\")
Kubernetes enables Jenkins to scale automatically based on workload demand. Jenkins agents can be created or destroyed as pods, allowing the CI/CD system to handle sudden spikes in build traffic without manual intervention.
By combining Kubernetes auto-scaling with proper monitoring, teams can ensure that:
Integrating Jenkins with Kubernetes creates a modern, cloud-native CI/CD platform that is scalable, efficient, and production-ready. By running Jenkins agents as Kubernetes pods, teams can dynamically provision build environments, optimize resource usage, and eliminate the limitations of static build agents.
Kubernetes features such as pod isolation, auto-scaling, and Helm-based deployments allow Jenkins pipelines to remain clean, reliable, and easy to manage as applications grow. This integration enables seamless automation — from code commits and builds to testing and deployment directly into Kubernetes clusters.
By combining Jenkins and Kubernetes, you can build CI/CD pipelines that are faster, more resilient, and ready for real-world production workloads — making continuous delivery a natural part of your DevOps workflow.
End-to-End CI/CD Pipeline Using Jenkins and Kubernetes was originally published in DevOps.dev on Medium, where people are continuing the conversation by highlighting and responding to this story.
","url":"https://blog.devops.dev/end-to-end-ci-cd-pipeline-using-jenkins-and-kubernetes-99be6542a07f?source=rss-3302bbae903c------2","hash":"sync-hash","mediumUsername":"bhagirath00","createdAt":"2026-05-07T04:44:53.867Z","updatedAt":"2026-05-07T04:45:11.033Z","tags":[]},{"id":"cmov05i830003u1r4gbnl8p9h","title":"How Git Stores Files Internally to Saves Space in Your Repository","slug":"how-git-stores-files-internally-to-saves-space-in-your-repository-61c5f9e25d6a","publishedAt":"2026-01-15T04:16:56.000Z","readingTime":null,"thumbnail":"https://cdn-images-1.medium.com/max/1024/1*0G_JPNH0pFrZdhkHx_xM9g.png","excerpt":"Learn how Git stores files internally using snapshots, blobs, trees, and hashing to avoid duplication and save repository space efficiently.Git is the most wide...","content":"Learn how Git stores files internally using snapshots, blobs, trees, and hashing to avoid duplication and save repository space efficiently.
![\"\"](\"https://cdn-images-1.medium.com/max/1024/1*0G_JPNH0pFrZdhkHx_xM9g.png\")
Git is the most widely used version control system in the world, and one of the key reasons for its popularity is its highly efficient storage model. At first glance, Git appears to store a complete copy of your project every time you commit. Surprisingly, repositories remain compact even after thousands of commits.
So how does Git duplicate files while still saving disk space?
In this article, we will explore how Git stores files internally, how it avoids unnecessary duplication, and why its storage mechanism is both fast and space-efficient. By the end, you will clearly understand how Git manages file data under the hood and why it scales so well for large projects.
Unlike traditional version control systems such as Subversion (SVN), which store file differences between versions, Git takes a fundamentally different approach.
Git stores snapshots of the entire project state at every commit.
However, Git is smart enough not to duplicate unchanged data. If a file has not changed between commits, Git simply reuses the previously stored version instead of saving a new copy. This design enables Git to deliver:
Most version control systems track line-by-line changes over time. Git does not.
Every time you create a commit, Git records a snapshot of the entire file structure at that moment.
If a file remains unchanged between commits:
This means Git behaves like a content-addressable filesystem, where identical content is stored once and referenced many times.
This snapshot model allows Git to:
Git stores all repository data as objects inside the .git/objects directory. Each object is identified by a cryptographic hash based on its content.
![\"\"](\"https://cdn-images-1.medium.com/max/1024/1*W1fbzO4cWgnJZiEozg2Hig.png\")
There are four primary object types in Git:
A blob (Binary Large Object) represents the raw content of a file.
Key characteristics of blobs:
If two files — or the same file across commits — have identical content:
This is the foundation of Git’s space-saving mechanism.
You can inspect blobs using:
git ls-tree <commit-hash>
A tree object represents a directory in your project.
It contains:
Each directory in your project maps to a tree object, allowing Git to recreate the complete filesystem structure for any commit.
A commit object ties everything together.
It contains:
Commit
└── Tree (Root Directory)
├── Blob (File 1)
├── Blob (File 2)
└── Tree (Subdirectory)
├── Blob (File 3)
└── Blob (File 4)
Each commit represents a complete snapshot, but most data is reused from earlier commits.
The .git directory is the core of every Git repository. It stores all metadata, objects, and references.
This directory stores all Git objects (blobs, trees, commits) in compressed form. Objects are named using their hash values.
References to branches and tags live here. Each branch is simply a pointer to a commit.
The index tracks what will be included in the next commit. It bridges the gap between your working directory and the repository.
The HEAD file points to the currently checked-out branch or commit.
Git’s efficiency comes from three core techniques.
Git computes a hash (SHA-1 by default, SHA-256 supported) for every object based on its content.
This guarantees data integrity and prevents duplication.
![\"\"](\"https://cdn-images-1.medium.com/max/1024/1*ACX2vpNc8RrF25qdc2-KWA.png\")
Git compresses objects using zlib, reducing disk usage while maintaining fast access.
Git never stores the same content twice. If a file hasn’t changed:
This is how Git duplicates files logically without duplicating data physically.
To fully understand how Git duplicates files while saving space, it is essential to understand the three logical areas through which every change flows: the working directory, the staging area, and the commit history. These are not just conceptual layers — they directly influence how Git creates objects and reuses existing data.
![\"\"](\"https://cdn-images-1.medium.com/max/1024/1*3knFXIGKg0zLFYwMb8lDqQ.png\")
The working directory is the actual project folder on your local machine. It contains real files that you edit using your editor or IDE.
Key characteristics:
When you modify a file in the working directory:
This design allows Git to remain fast and lightweight while you experiment with changes.
The staging area, also called the index, is where Git begins its internal storage optimization.
When you run:
git add <file>
Git performs the following actions:
Important details:
This is where Git’s de-duplication logic begins to take effect.
When you run:
git commit
Git creates a commit object, which includes:
Crucially:
Each commit represents a complete snapshot, but internally, most data is shared across commits. This allows Git to maintain a full project history without ballooning repository size.
One of Git’s strengths is transparency. Git provides low-level commands that allow you to inspect its internal object database, making it easier to understand how files are stored and reused.
These commands are especially valuable for developers who want to understand Git beyond everyday workflows.
The git cat-file command allows you to inspect any Git object directly.
To view a commit object:
git cat-file -p <object-hash>
This displays:
You can also inspect blob objects to see file content exactly as Git stores it, confirming that identical content is reused across commits.
The git ls-tree command shows how a commit or tree maps to files and directories.
git ls-tree <commit-hash>
Output includes:
This command clearly demonstrates how Git builds directory snapshots using tree objects that reference blob objects, without duplicating data.
The git rev-parse command helps resolve symbolic references into their actual object hashes.
git rev-parse HEAD
Use cases include:
This reinforces the idea that branches and tags are lightweight pointers, not copies of data.
Git’s ability to duplicate files logically without duplicating data physically is the cornerstone of its performance and scalability. By storing content as immutable, hashed objects and reusing them across commits, Git ensures that repositories remain fast and space-efficient — even with extensive histories.
Understanding Git’s internal storage model gives you deeper confidence when working with branches, rebases, merges, and large repositories. It also explains why Git continues to outperform traditional version control systems in both speed and reliability.
How Git Stores Files Internally to Saves Space in Your Repository was originally published in DevOps.dev on Medium, where people are continuing the conversation by highlighting and responding to this story.
","url":"https://blog.devops.dev/how-git-stores-files-internally-to-saves-space-in-your-repository-61c5f9e25d6a?source=rss-3302bbae903c------2","hash":"sync-hash","mediumUsername":"bhagirath00","createdAt":"2026-05-07T04:44:54.531Z","updatedAt":"2026-05-07T04:45:18.430Z","tags":[]},{"id":"cmov05iki0004u1r45jy9s678","title":"OpsGuard: A Production-Ready Status Page with Modern DevOps Practices","slug":"opsguard-a-production-ready-status-page-with-modern-devops-practices-a37e1c14a71d","publishedAt":"2025-12-31T05:46:42.000Z","readingTime":null,"thumbnail":"https://cdn-images-1.medium.com/max/1024/1*eqa9tnFrTGuluPdxW5cLxg.png","excerpt":"From Code to Cloud: Building a Self-Healing Status Page on AWS EKSHow I built an enterprise-grade infrastructure monitoring platform using Kubernetes, Terraform...","content":"How I built an enterprise-grade infrastructure monitoring platform using Kubernetes, Terraform, and GitOps — from concept to deployment on AWS
![\"\"](\"https://cdn-images-1.medium.com/max/1024/1*eqa9tnFrTGuluPdxW5cLxg.png\")
Every modern organization needs a way to communicate service health to their customers. When AWS, GitHub, or Cloudflare experience issues, they don’t leave users guessing — they have public status pages that provide real-time updates. But what if you could build your own?
That’s exactly what I set out to do with OpsGuard — a self-hosted, production-ready status page that combines real-time infrastructure monitoring with automated incident management. More importantly, I wanted to build it using industry-standard DevOps practices that enterprises actually use in production.
This blog post walks through my journey of building OpsGuard, the architectural decisions I made, the challenges I faced, and how modern DevOps practices like GitOps, Infrastructure as Code (IaC), and DevSecOps came together to create a robust, scalable solution.
Before writing a single line of code, I spent considerable time designing an architecture that would be both scalable and maintainable. The key principle was separation of concerns — each component should do one thing well.
![\"\"](\"https://cdn-images-1.medium.com/max/1024/1*Ae-vBBcSTWrfiggPEgw1nA.png\")
The application consists of three main services:
1. Frontend (React + TypeScript): A responsive, real-time dashboard that displays service health, active incidents, and historical uptime metrics. Built with Vite for optimal performance.
2. Backend API (FastAPI + Python): The core REST API that handles all business logic, from processing health check results to managing incidents. FastAPI was chosen for its excellent async support and automatic OpenAPI documentation.
3. Worker Service: A background service that performs automated health checks every 30 seconds. It monitors configured endpoints and updates service status in real-time.
For data persistence, I choose:
Kubernetes might seem like overkill for a status page, but it provides critical capabilities:
One of my core principles was that infrastructure should be code. No clicking around in the AWS console — everything defined in Terraform.
module "eks" {
source = "./modules/eks"
cluster_name = "opsguard-prod"
subnet_ids = module.vpc.private_subnet_ids
# ... configuration
}My Terraform configuration provisions:
Every commit to the main branch triggers a comprehensive pipeline that builds, tests, scans, and deploys the application.
![\"\"](\"https://cdn-images-1.medium.com/max/1024/1*6dVBTSVJAyTXB_387I6k8Q.png\")
1. Build: Docker images are built using multi-stage builds to minimize image size
2. Unit Tests: Pytest runs the test suite with coverage reporting
3. Security Scan (Bandit): Static analysis catches common Python security issues
4. Dependency Check (Safety): Identifies vulnerable dependencies
5. Container Scan (Trivy): Scans Docker images for CVEs
6. Push to ECR: Images are tagged and pushed to Amazon ECR
7. Deploy via ArgoCD: GitOps handles the actual deployment
Security isn’t an afterthought — it’s integrated into every stage:
If any security check fails, the pipeline stops. No exceptions.
Traditional CI/CD pipelines push changes to production. GitOps flips this model — ArgoCD pulls the desired state from Git and ensures the cluster matches.
![\"\"](\"https://cdn-images-1.medium.com/max/1024/1*ywyvU5fa_TYKh8Wn5HSlDA.png\")
1. The CI pipeline updates the image tag in the Kubernetes manifests
2. ArgoCD detects the change in the Git repository
3. ArgoCD applies the changes to the cluster
4. If someone manually changes something in the cluster, ArgoCD reverts it
A production system without observability is flying blind. OpsGuard uses the industry-standard Prometheus + Grafana stack.
![\"\"](\"https://cdn-images-1.medium.com/max/1024/1*W-vyWWYYUMPuFRvVeJMKdA.png\")
Prometheus scrapes metrics from every component:
Grafana provides pre-configured dashboards showing:
AlertManager routes critical alerts to Slack, email, or PagerDuty based on severity.
Security at the container level required careful attention:
securityContext:
runAsNonRoot: true
readOnlyRootFilesystem: true
allowPrivilegeEscalation: false
capabilities:
drop:
- ALL
OpsGuard represents what’s possible when modern DevOps practices come together. Infrastructure as Code ensures reproducibility. CI/CD pipelines automate quality gates. GitOps provides declarative deployments. Observability enables proactive operations.
The project is open source on [GitHub](https://github.com/Bhagirath00/OpsGuard). Whether you’re building your own status page or learning DevOps practices, I hope OpsGuard serves as a useful reference.
Is your repository a ghost town? Discover why the README.md is the most critical file in your project.
![\"\"](\"https://cdn-images-1.medium.com/max/1024/1*8jQCGI_fjxlJBlTYVJh2dQ.png\")
Imagine you are shopping for a new laptop online. You click on a product that looks promising, but the page has no photos, no spec sheet, and no price. It just has a button that says “Buy Now.”
Would you click it? Of course not. You have no idea what you are getting into.
In the world of software development, your GitHub or GitLab repository is the product page, and your README.md is the sales pitch.
Too many developers fall into the “Black Box” trap. They spend hundreds of hours writing elegant, highly optimized algorithms, pushing perfectly tested code to the src folder, and then leave the root directory empty. They assume the code speaks for itself.
Code never speaks for itself. Unless a user can understand what your project does, how to install it, and why it matters in under 30 seconds, your code effectively does not exist.
This guide moves beyond the theory. You’ll look at the architecture of documentation, visualize the user journey, and provide the exact syntax you need to turn a dead repository into a thriving open-source project.
Let’s look at a concrete example. We have a hypothetical library called Data-Muncher, a simple Python script that cleans CSV files.
When a recruiter or developer lands on this repository, this is all they see:
📁 Data-Muncher /
├── 📁 src /
│ └── main.py
├── 📁 tests /
│ └── test_main.py
├── .gitignore
└── requirements.txt
The User Experience:
![\"\"](\"https://cdn-images-1.medium.com/max/1024/1*0RYeU8uRgcjduQMeCWvg7A.jpeg\")
Now, look at the exact same code, but with a structured README.md.
The directory now looks like this, but the rendering on GitHub presents a beautiful interface:
# 🦁 Data-Muncher
> A lightning-fast Python library to clean messy CSV files 10x faster than Pandas.
## 🚀 Features
- Removes duplicates automatically.
- Normalizes date formats (ISO-8601).
- zero-dependency architecture.
## 📦 Installation
```
pip install data-muncher
```
![\"\"](\"https://cdn-images-1.medium.com/max/722/1*Tqtz2yAge4JQDbisPjO0bw.png\")
The User Experience:
In UX design, we often talk about the Time to Hello World (TT-HW). This is the time it takes for a new user to land on your repo and get the code running on their machine.
If your TT-HW is longer than 5 minutes, you lose 80% of your potential users.
Below is a diagram illustrating the mental process a developer goes through when evaluating your library.
![\"\"](\"https://cdn-images-1.medium.com/max/1024/1*IKXHI1UBkGw_PIzGNo_GOQ.jpeg\")
A good README removes the “No” branches from this flowchart. It streamlines the path to the “Star the Repo” outcome.
A professional README isn’t just a wall of text; it is structured data using Markdown. Here are the essential components and the syntax to create them.
Don’t start with “Introduction.” Start with the name and a hook.
Code Syntax:
# Project Name
**The one-line elevator pitch goes here.** *Example: "The only React Native boilerplate you will ever need."*
![\"\"](\"https://cdn-images-1.medium.com/max/707/1*yZEIjiOmiz8M0K8DkwNQUg.png\")
Badges are the “Social Proof” of open source. They tell the user that the project is alive, maintained, and licensed. You don’t need complex code for this; you use markdown image links.
Code Syntax:
![\"\"](\"https://cdn-images-1.medium.com/max/614/1*i_brla_5aVsnIuABzLlUUw.png\")
If you are building a UI, a GIF is mandatory. If you are building a CLI (Command Line Interface), a screenshot of the terminal is mandatory.
Why? The human brain processes images 60,000x faster than text.
Code Syntax:
*Caption: Seeing the app in dark mode.*
This is the most crucial technical section. Do not describe how to install it; give the command. Use “Code Fences” (triple backticks) to allow users to copy the code easily.
Bad Documentation:
“To install, you need to open your terminal and run the npm install command for our package.”
Good Documentation:
npm install my-awesome-package
# or
yarn add my-awesome-package
![\"\"](\"https://cdn-images-1.medium.com/max/725/1*ovZW4OBalcTDhNwQsgfDTw.png\")
Most developers write the code first!!
Readme Driven Development (RDD) suggests that you should write the README before you write a single line of code.
![\"\"](\"https://cdn-images-1.medium.com/max/1024/1*Q7Zx78B2qv39g6cOZ5MZaw.jpeg\")
A wall of plain text is hard to scan. You need to use Markdown features to create hierarchy and “scannability.” Search engines (SEO) also prefer structured content.
If you have a long list of configurations, use the HTML <details> tag within your Markdown to keep the page clean.
Code Syntax:
<details>
<summary>Click to view Advanced Configuration</summary>
| Option | Type | Default |
|--------|------|---------|
| --verbose | bool | false |
| --dry-run | bool | false |
</details>
![\"\"](\"https://cdn-images-1.medium.com/max/632/1*OkrhjlukXlbOvuEX8Htyag.png\")
Don’t list arguments in paragraphs. Use tables. They are cleaner and look professional.
Code Syntax:
| Method | Description | Returns |
|--------|-------------|---------|
| `.init()` | Starts the server | `void` |
| `.stop()` | Kills the process | `boolean` |
![\"\"](\"https://cdn-images-1.medium.com/max/598/1*wW-1otZA-D5A7CqeAORi3Q.png\")
Documentation is an insurance policy against the “Bus Factor.”
The “Bus Factor” is the minimum number of team members that have to be hit by a bus (or quit) before the project creates stops functioning because no one knows how it works.
If only you understand how to deploy the database, your project has a Bus Factor of 1. This is dangerous.
A good README acts as an “External Brain.” It remembers the setup steps so you don’t have to.
You want your project to be found on Google, not just GitHub. The README.md is the primary source of content that Google crawls.
SEO Checklist for READMEs:
3. Linking: Link to your other projects or your portfolio. This creates a “backlink” structure that improves your ranking.
Ultimately, writing a good README is an act of empathy. It signals that you care about the person on the other side of the screen.
When a hiring manager looks at your portfolio, they aren’t going to clone your repo and audit your variable names. They are going to read your README.
Don’t let your brilliant code die in the dark. Light it up with a README that sells, explains, and guides.
![\"\"](\"https://cdn-images-1.medium.com/max/706/1*njJRvcIZV1Y1Ykv4J97cfw.png\")
Understand Kubernetes self-healing by exploring how the Kubelet, PLEG, probes, and CrashLoopBackOff work together to restart failing containers.
![\"\"](\"https://cdn-images-1.medium.com/max/1024/1*LbnUHrAjpfYhfXK8Gmcleg.png\")
Kubernetes is often described as a self-healing system because it can automatically recover from application failures. When a container crashes, Kubernetes detects the failure and brings the workload back to a running state without manual intervention. But how does Kubernetes restart crashing pods?
A common assumption is that Kubernetes restarts Pods when something goes wrong. In reality, Pods are rarely restarted. Failure recovery happens at a much lower level and is driven by continuous monitoring of container processes on each node.
Containers can fail in multiple ways: a process may exit with a non-zero exit code, the operating system may terminate it due to memory limits, or the application may become unresponsive. Kubernetes observes these failures and decides whether and when a restart should occur, applying safeguards to prevent repeated crashes from overwhelming the system.
This blog post walks through how Kubernetes detects container failures and how restart decisions are made, starting at the node level and moving up to cluster-wide recovery.
To understand detection, we must look at the Node Level. The Kubernetes Control Plane (API Server) is often too far removed to handle immediate process failures. The heavy lifting is performed locally by the Kubelet.
The Kubelet continuously reconciles two states:
This reconciliation is performed through a continuous control loop known as the SyncLoop.
The SyncLoopis the core control loop of the Kubelet. Its job is to ensure that the actual state of containers always matches the desired state.
For example:
When a container crashes or exits, the actual state no longer matches the desired state. Detecting this change quickly is critical for Kubernetes to take corrective action.
![\"\"](\"https://cdn-images-1.medium.com/max/1024/1*hEO8rNBcnCMgx_l1HTHJVA.png\")
In early Kubernetes versions, the Kubelet relied heavily on polling the container runtime — repeatedly asking whether containers were still running. As node density increased to hundreds of Pods per node, this approach caused unnecessary CPU overhead and delayed failure detection.
This limitation led to the introduction of an event-driven mechanism.
PLEG is an internal Kubelet component responsible for detecting container state transitions efficiently.
![\"\"](\"https://cdn-images-1.medium.com/max/1024/1*NydENw_gPdRQTecvzGQjkg.png\")
This event immediately notifies the Kubelet, allowing it to react without waiting for the next reconciliation cycle.
Kubernetes determines that a container has failed by observing specific signals from the Linux kernel, the container runtime, and the Kubelet’s health probes. Understanding these signals is essential for diagnosingfailures and designing resilient workloads.
![\"\"](\"https://cdn-images-1.medium.com/max/1024/1*DCzPvlnvcoo5eQ4ILzhQTw.png\")
Every container runs a main process (PID 1). When this process stops, it sends an exit code to the operating system:
Detecting non-zero exit codes allows the Kubelet to differentiate between successful completions and failures, forming the basis for restart decisions.
One of the most common failure causes is being killed due to out-of-memory (OOM):
What it looks like in kubectl describe pod:
State: Terminated
Reason: OOMKilled
Exit Code: 137
Started: Wed, 26 Jan 2025 12:00:00 GMT
Finished: Wed, 26 Jan 2025 12:05:00 GMT
If you see Exit Code 137, simply restarting the container will not solve the problem. You must either fix the memory issue in your code or increase resources.limits.memory in your Pod specification.
Sometimes, the main process is still running, but the application is frozen, deadlocked, or stuck. The kernel alone cannot detect this scenario. Kubernetes relies on liveness probes to actively check the health of the application.
Example configuration:
livenessProbe:
httpGet:
path: /healthz
port: 8080
initialDelaySeconds: 15
periodSeconds: 20
failureThreshold: 3
If the endpoint returns an error (e.g., HTTP 500) or times out consecutively according to failureThreshold, the Kubelet marks the container as unhealthy and forcefully kills it to trigger a restart.
Once a failure is confirmed, the Kubelet consults the Pod’s restartPolicy to determine whether and how to restart the container.
spec:
restartPolicy: Always
Once a container failure is detected, Kubernetes must decide whether and when to restart it. This behavior is controlled by the Pod’s restartPolicy and further refined by the CrashLoopBackOff mechanism to prevent repeated rapid restarts.
The Kubelet consults the Pod specification’s restartPolicy to determine how to respond to a container failure. There are three main policies:
spec:
restartPolicy: Always
spec:
restartPolicy: OnFailure
spec:
restartPolicy: Never
Imagine your application crashes immediately after starting. If Kubernetes restarted it instantly every time, the container could restart hundreds of times per second, consuming all CPU on the node.
To prevent this, Kubernetes uses the CrashLoopBackOff mechanism:
When you see CrashLoopBackOff in kubectl get pods, Kubernetes is currently waiting before attempting the next restart.
The backoff timer doesn’t last forever. If a container runs successfully for a stable period, Kubernetes resets the backoff counter. This period is often controlled by minReadySeconds or the default health window.
Effect:
![\"\"](\"https://cdn-images-1.medium.com/max/1024/1*NAd7dHEiQPNfqDAVBmbFcA.png\")
Even with restart policies and CrashLoopBackOff, some containers may fail due to slow startups, deadlocks, or complex dependency issues. Kubernetes provides additional mechanisms to handle these scenarios safely and help engineers diagnose problems efficiently.
Some applications, such as Java services or AI models loading large datasets, may take minutes to start. Standard liveness probes could kill these containers before they finish booting.
Solution: Use a startupProbe:
startupProbe:
httpGet:
path: /healthz
port: 8080
failureThreshold: 30
periodSeconds: 10
How it works:
Kubernetes allows running sidecar containers alongside your main application:
Sometimes, a container fails repeatedly and logs alone are insufficient. Kubernetes provides tools for live debugging:
kubectl logs <pod-name> --previous
Shows logs from the last failed container instance.
kubectl describe pod <pod-name>
Reveals the reason for Kubelet restarts (e.g., OOMKilled, Liveness Probe Failed).
kubectl debug -it <pod-name> --image=busybox --target=<container-name>
Provides a shell inside a running container without restarting it, allowing inspection of the filesystem or environment.
While container-level recovery handles individual container crashes, Kubernetes also ensures that workloads recover when an entire node fails. This is essential for maintaining high availability in a cluster.
Each node in a Kubernetes cluster continuously reports its status to the API Server. The control plane monitors these heartbeats:
At this point, the Node Controller steps in.
![\"\"](\"https://cdn-images-1.medium.com/max/1024/1*kU8JiCB5lAiSUmjiRxeOHQ.png\")
Once a node is marked NotReady:
This process ensures that your services remain available, even if one or more nodes fail.
Kubernetes’ self-healing capabilities go far beyond simply restarting containers. It is a sophisticated system designed to keep applications running reliably, even when things go wrong at the container, node, or cluster level.
Here’s what makes Kubernetes self-healing so powerful:
By understanding these mechanisms, engineers can design resilient applications, troubleshoot failures faster, and avoid downtime.
In short, Kubernetes doesn’t just restart what breaks — it orchestrates a full recovery strategy, automatically maintaining reliability across containers and nodes.
How Kubernetes detects and restarts crashing pods automatically was originally published in DevOps.dev on Medium, where people are continuing the conversation by highlighting and responding to this story.
","url":"https://blog.devops.dev/how-kubernetes-detects-and-restarts-crashing-pods-automatcially-9c010618d877?source=rss-3302bbae903c------2","hash":"sync-hash","mediumUsername":"bhagirath00","createdAt":"2026-05-07T04:44:55.782Z","updatedAt":"2026-05-07T04:45:26.491Z","tags":[]},{"id":"cmov05jjq0007u1r4mrl828nh","title":"Click→Report→Resolve: My SIH 2025 Journey to Smarter Civic Governance","slug":"click-report-resolve-my-sih-2025-journey-to-smarter-civic-governance-00d0b32955fe","publishedAt":"2025-09-19T06:02:48.000Z","readingTime":null,"thumbnail":"https://cdn-images-1.medium.com/max/1024/1*umAKJgkDZ0fsN7VaXvA0Lg.png","excerpt":"Imagine spotting a pothole or garbage pile on your street and reporting it within seconds — no complicated forms, no waiting endlessly for updates. Civic partic...","content":"![\"\"](\"https://cdn-images-1.medium.com/max/1024/1*umAKJgkDZ0fsN7VaXvA0Lg.png\")
Imagine spotting a pothole or garbage pile on your street and reporting it within seconds — no complicated forms, no waiting endlessly for updates. Civic participation in India has long faced challenges like inefficient reporting, lack of transparency, and low accessibility.
At Smart India Hackathon 2025, my team Fedora tackled this problem head-on with an innovative idea: an AI-powered crowdsourced civic issue reporting and resolution system. This solution makes reporting problems as simple as clicking a photo and empowers citizens and governments alike with real-time insights.
1. Inefficient Reporting — Most systems are slow, manual, and lack automation.
2. Lack of Transparency — Citizens rarely know if their complaint is received or resolved.
3. Low Accessibility — Existing portals are complex, especially for semi-literate or first-time digital users.
These issues widen the gap between citizens and government bodies, ultimately slowing down civic improvements.
My Solution: CityFix
I built a mobile-first, AI-powered platform designed to make civic reporting simple, transparent, and accessible to all.
![\"\"](\"https://cdn-images-1.medium.com/max/1024/1*KCx8M45MxNV-4eItcq8pgA.png\")
1. Citizen (Web-App)
Frontend: React Native (for web)
UI/UX: TailwindCSS + Material UI (for fast, accessible, multilingual design)
Multilingual Support: i18next library
2. Reporting Features
Camera + Location: React Native Camera/GPS (mobile)
Notifications: Firebase Cloud Messaging (push notifications across devices)
3. Backend Infrastructure
Serverless Processing: Google Cloud Functions (auto-scale, low latency)
Data Management: Google Firestore (real-time sync, scalable), Firebase Storage (images, video)
Authentication: Firebase Auth (secure login via email/phone/social)
4. Admin Dashboard
Frontend: React.js + Chart.js / Recharts (data visualization)
Maps: Leaflet.js (interactive city maps with issue heatmaps)
Task Routing Engine: Custom ML model (Python/FastAPI) + rule-based priority system
5. Advanced Features
AI/ML Layer: TensorFlow Lite for image classification (pothole, garbage, etc.)
Analytics & Insights: Google Data Studio / Looker for admin reports / Heatmaps for hotspots and trend analysis
6. Scalability & Integration
APIs: REST + GraphQL APIs for future extensions
Offline Mode: Local storage with background sync (PWA capabilities)
This modular, scalable stack ensures smooth performance and accessibility.
→ For Citizens:
→ For Government:
→ For Communities:
![\"\"](\"https://cdn-images-1.medium.com/max/1024/1*dFEZKAi5dq5k_GVuH_kvhg.png\")
This solution aligns perfectly with the Clean & Green Technology theme of SIH 2025. By merging AI, crowdsourcing, and civic governance, it bridges the gap between citizens and authorities — making cities not just smarter, but more inclusive and transparent.
![\"\"](\"https://cdn-images-1.medium.com/max/1024/1*9fqYnel0W4sgFWjn-QLRSw.png\")
References & Inspiration
My research drew inspiration from existing civic platforms like:
Civic engagement shouldn’t be a struggle — it should be empowering. With Fedora Cityfix, we aim to simplify reporting, strengthen transparency, and accelerate issue resolution. Together, citizens and governments can co-create a cleaner, greener, and smarter India.
![\"\"](\"https://cdn-images-1.medium.com/max/1024/1*9tVguFcHL8kV7gRLb2dTtA.png\")
Large Language Models (LLMs) have transformed how we interact with technology, but for a long time, their power was limited to a single domain: text. You could ask a chatbot a question, and it would give you a text response. But what if you could show it a picture and ask it to write a poem about it? Or show it a video and have it describe the events in a single paragraph?
This is the promise of multimodal AI, the next frontier in artificial intelligence. Instead of just “reading” words, these models can see, hear, and understand the world through multiple data formats, or “modalities,” just like humans do. This shift from single-sense to multi-sense AI is already reshaping industries and creating a new wave of applications.
At its core, multimodal AI refers to a system that can process, understand, and generate content from more than one data type simultaneously. While a traditional LLM (like early versions of GPT) was “unimodal” (text-in, text-out), a multimodal model can handle a mix of inputs, such as:
This allows for more complex and context-rich interactions. For example, a doctor could input an X-ray, a patient’s medical history (text), and a recorded description of their symptoms (audio) to get a comprehensive diagnostic summary.
The magic behind multimodal AI lies in its ability to fuse different data types into a single, unified representation. Here’s a simplified breakdown:
By learning these deep connections, models like Google’s Gemini and OpenAI’s GPT-4o can reason across different types of information, leading to more accurate and coherent results with fewer “hallucinations.”
Multimodal AI isn’t just a research topic; it’s already powering groundbreaking applications across various fields.
![\"\"](\"https://cdn-images-1.medium.com/max/1024/1*3cfi9nHN1RVCQHw4aSop7w.png\")
![\"\"](\"https://cdn-images-1.medium.com/max/1024/1*IJdgSt4ySZYPFiDUMKZBlA.png\")
![\"\"](\"https://cdn-images-1.medium.com/max/1024/1*tAzGfctu7UIi5eudL6KS-Q.png\")
The shift to multimodal AI marks a fundamental change in how we interact with technology. It’s moving us closer to a future where AI systems are not just tools but true collaborators that can perceive the world in a more holistic, human-like way.
As these models become more sophisticated, we can expect them to become even more integrated into our daily lives. From smart home assistants that can “see” a broken appliance and guide you through the repair, to educational tools that can “watch” you solve a problem and offer personalized feedback, the possibilities are nearly limitless.
By understanding the power of multi-modal AI, you’re not just keeping up with the latest trends — you’re preparing for a future where the digital world is as sensory and interconnected as our own.