Question: is Azure Kubernetes Services a good solution to a peak scaling problem for a small app ecosystem?
Posted: Sun Sep 04, 2022 3:32 am
Here's the problem I'm facing at my current job:
I have 40-ish production mostly-Python applications that run on a single on-prem workstation. These fit together into a set of processing pipelines with different flows. These apps have already been successfully broken down into microservices where multiple instances of each app can be run simultaneously without any problems and we're experimenting with containerization to continue down this path.
The problem is that some of these apps experience high intermittent loads. Since everything is running on the same machine, a single high-load app will max out resources and slow the entire processing pipeline down. I want to introduce auto-scaling, but that requires configuration of another VM or (preferably) some container service like Azure Container Apps, Container Instances, or Kubernetes Service.
Kubernetes course
So far these containers are working great on the local machine via Docker. I've been able to get some simple containers running on Azure Container Apps. But this is where I hit a dead end. Any more complicated apps fail to run on Azure Container Apps and there's no feedback why -- they don't write anything to stdout or stderr, they just take forever to deploy and then either fail or scale to 0 without any way to get in there to figure out what's going on. I've spent weeks diagnosing this issue and haven't made any progress.
So I'm thinking the problem is that Azure Container Apps is too managed. I have no control over it. I've been warned against trying to run K8s on my own VM due to the high learning curve and the fact that we don't have a real DevOps team (just me and my trainee). Azure Kubernetes Service sounds like I might be able to get finer control over what's going on to diagnose issues, but it'll still be a learning curve. OR I can just pay for and configure a full VM and have it run side-by-side with my current machine to handle peak loads, but that's obviously going to cost a lot for a machine that will mostly sit idle.
What's the best approach?
I have 40-ish production mostly-Python applications that run on a single on-prem workstation. These fit together into a set of processing pipelines with different flows. These apps have already been successfully broken down into microservices where multiple instances of each app can be run simultaneously without any problems and we're experimenting with containerization to continue down this path.
The problem is that some of these apps experience high intermittent loads. Since everything is running on the same machine, a single high-load app will max out resources and slow the entire processing pipeline down. I want to introduce auto-scaling, but that requires configuration of another VM or (preferably) some container service like Azure Container Apps, Container Instances, or Kubernetes Service.
Kubernetes course
So far these containers are working great on the local machine via Docker. I've been able to get some simple containers running on Azure Container Apps. But this is where I hit a dead end. Any more complicated apps fail to run on Azure Container Apps and there's no feedback why -- they don't write anything to stdout or stderr, they just take forever to deploy and then either fail or scale to 0 without any way to get in there to figure out what's going on. I've spent weeks diagnosing this issue and haven't made any progress.
So I'm thinking the problem is that Azure Container Apps is too managed. I have no control over it. I've been warned against trying to run K8s on my own VM due to the high learning curve and the fact that we don't have a real DevOps team (just me and my trainee). Azure Kubernetes Service sounds like I might be able to get finer control over what's going on to diagnose issues, but it'll still be a learning curve. OR I can just pay for and configure a full VM and have it run side-by-side with my current machine to handle peak loads, but that's obviously going to cost a lot for a machine that will mostly sit idle.
What's the best approach?