Hi all, i've a container running code in .NET 6, dockerized and running on Azure k8s. Inside this container, i handle a message from a topic of a Service Bus. It should manage aroung 20k messages a day.
Scenario description:
I have to call an .exe (size around 100 MB) that take in input a file and generate a response in a given physical subfolder on disk. The response include two files, that i upload then to a blob storage. Finally i delete these temporary files on disk. The execution of the .exe will be of 1 minute.
Question:
Where should i store the exe? Its content will change 2-3 times a year (so few times). I was thinking to include it in the docker image, for example in "\app\resources" and then call it creating a random temp folder in the same path, one for each execution. In alternative i was thinking to Azure File Share (but i'm concerned about the traffic cost). The data is temporary, so if the pod goes down, i don't care about them.
I've already read about volumes in Docker, but i don't find any question/topic that fit my problem, so i'm searching for some advices.
Thanks
Top comments (2)
When considering where to store the executable file in your scenario, there are a few options you can explore:
1st Option - You can include the .exe file in your Docker image, placing it in a specific location like "/app/resources" as you mentioned. This ensures that the file is available within the container and can be accessed during runtime. Pls note that each time the .exe file changes, you will need to rebuild and redeploy the Docker image.
2nd Option - Instead of including the .exe file in the Docker image, you can store it in Azure Blob Storage. During runtime, your containerized application can download the .exe file from Blob Storage when needed. This approach allows you to update the .exe file independently without rebuilding the Docker image.
3rd Option - Azure File Share is another option to consider. You can create a file share in Azure and mount it as a volume in your Kubernetes deployment. This allows your containerized application to access the .exe file stored in the file share. Similar to Azure Blob Storage, this approach allows you to update the .exe file independently without rebuilding the Docker image.
But as you stated correctly, Azure File Share traffic is typically charged based on the data transfer in and out of the storage account. Both ingress (incoming) and egress (outgoing) data traffic have associated costs. You can review the pricing details of Azure File Storage to understand the cost implications based on your expected usage.
I Don't Recommend ConfigMap:
While ConfigMap might be one such tool that K8s experts suggest but I won't recommend that because your .exe changes frequently. Kubernetes provides a resource called ConfigMap that allows you to store configuration data, including files, as key-value pairs. You can create a ConfigMap containing your executable file and mount it as a volume in your deployment. This approach is suitable if the .exe file is relatively small and doesn't change frequently.
One more thing - To mitigate frequent data transfers and reduce costs, you can consider implementing caching mechanisms within your application. For example, you could download the executable file from the Azure File Share and cache it locally within the container or on a shared volume. This way, subsequent executions of your application can use the locally cached file instead of fetching it from the file share, reducing network traffic.
If you have an existing file server or file storage solution in your environment, you can consider using it to store the executable file. You can mount the file server as a volume in your Kubernetes deployment, allowing your containerized application to access the file. This approach provides flexibility and allows you to manage the file separately from the container environment.
Hope it helps.
Thanks for very detailed point of view.