Sail Sharp, 11 tips to optimize and secure your .NET containers for Kubernetes

Mathieu Benoit

Follow

13 min read

·

Apr 1, 2023

--

5

Listen

Share

Updated on February 2nd, 2025 — .NET 9.0.1 and Multi-arch support.

Updated on January 5th, 2025 — .NET 9.

Updated on May 31st, 2024 — Move base container images from alpine to noble-chiseled (distroless).

Updated on November 15th, 2023 — .NET 8 + Native AOT support.

Updated on July 8th, 2023 — All the source code is now in mathieu-benoit/sail-sharp.

In February 2021, I got this opportunity to deliver this talk Sail Sharp, .NET Core & Kubernetes for the .NET Meetup in Quebec city (it was in French). I illustrated the best practices to prepare any .NET applications for Kubernetes. I was using the cartservice app (in .NET) from the very popular Online Boutique sample apps.

Since then, I have been one of the top contributors to the Online Boutique repository. I contributed to the golang, python, dotnet, java and nodejs apps. This repo was my playground. I learned a lot. Some of my contributions, among others, were about optimizing and securing the container images for all these apps.

Here is the high-level timeline of my contributions related to the cartservice app:

• 2020-11 — .NET 2 –> .NET 3 –> .NET 5
• 2020-12 — Managed gRPC and self-contained deployment trimmed
• 2021-01 — Google Cloud Memorystore Redis (doc)
• 2021-11 — .NET 6
• 2022-03 — NetworkPolicies
• 2022-05 — Better IDistributedCache implementation
• 2022-06 — Unprivileged container
• 2022-09 — .NET 7
• 2022-09 — Native gRPC Healthcheck for livenessProbe and readinessProbe
• 2022-09 — Spanner as database option (reviewer and contributor)
• 2022-12 — IPv6
• 2022-12 — Helm chart with the addition of the AuthorizationPolicies
• 2023–06 — Azure Redis Cache (doc)
• 2023–09 — .NET 8 RC1
• 2023–11 — .NET 8
• 2024–05 — noble-hiseled (distroless) instead of alpine
• 2024–11 — .NET 9
• 2025–01 — Multi-arch

As a side note, I started my career with the .NET Framework version 3.0 (only on Windows at that time) back in 2006! Since then, I have been amazed about the evolution of the .NET ecosystem. And to be honest all of these contributions gave me a reason to stay up-to-date and have a lot of fun while learning more about containers and Kubernetes! :)

Wow! Quite a ride, isn’t it?

Today, in this blog post, I will highlight 11 tips to optimize and secure your .NET containers based on what I have learned with all of that:

1. Use the multi-stage build approach
2. Provide Multi-architectures support
3. Reduce the size of the bundled application
4. Reduce the size of the base container images
5. Go distroless, not alpine
6. Use immutable base container images
7. Update dependencies with Dependabot or Renovate
8. Reduce the size of your final container with .dockerignore
9. Secure unprivileged/non-root container
10. Protect read-only container filesystem
11. Accelerate startup time with compiled native code (AOT)

tl,dr

If you want to see the final Dockerfile and the Deployment manifest to deploy a secure and optimized .NET application in Kubernetes, feel free to directly jump to the end of this blog post.

Disclaimer

Whereas most of the concepts could be applicable to Windows containers, this blog post is only covering Linux containers.

Create a minimal ASP.NET app

Create a folder where we will drop all the files needed for this blog post:

mkdir my-sample-app

Create a minimal and simple ASP.NET app that we will use for this blog post.

my-sample-app/Program.cs:

using Microsoft.AspNetCore.Builder;
var builder = WebApplication.CreateBuilder(args);
var app = builder.Build();
app.MapGet("/", () => "Hello, World!");
app.Run();

my-sample-app/my-sample-app.csproj:

<Project Sdk="Microsoft.NET.Sdk.Web">
  <PropertyGroup>
    <TargetFramework>net9.0</TargetFramework>
  </PropertyGroup>
</Project>

1. Use the multi-stage build approach

Create our first Dockerfile with a multi-stage build. That’s not the final one, until then, please bear with me:

FROM mcr.microsoft.com/dotnet/sdk:9.0 AS builder
WORKDIR /app
COPY my-sample-app.csproj .
RUN dotnet restore my-sample-app.csproj \
    --use-current-runtime
COPY . .
RUN dotnet publish my-sample-app.csproj \
    --use-current-runtime \
    -c release \
    -o /my-sample-app \
    --no-restore

FROM mcr.microsoft.com/dotnet/aspnet:9.0
WORKDIR /app
COPY --from=builder /my-sample-app .
ENTRYPOINT ["/app/my-sample-app"]

This Dockerfile is already using the multi-stage build, which optimizes the final size of the image by layering the build and leaving only required artifacts.

Let’s build this container image locally:

docker build -t my-sample-app my-sample-app/

We can see that the size of the container image is 224 MB locally on disk.

You can now run this container:

docker run -d -p 8080:8080 my-sample-app

Important to notice here, that since .NET 8, the default port is not anymore 80 (privileged) but is now 8080 (unprivileged), great to see security best practices applied by default here!

You can now test that this container is working successfully:

curl localhost:8080

Great, congrats!

2. Provide Multi-architectures support

At this stage you may want to run this container on arm64 arch, locally (MacOS) or to save some cost with the Nodes of your Kubernetes cluster.

For this we need to apply these changes in the Dockerfile:

FROM --platform=$BUILDPLATFORM mcr.microsoft.com/dotnet/sdk:9.0 AS builder
ARG TARGETARCH
WORKDIR /app
COPY my-sample-app.csproj .
RUN dotnet restore my-sample-app.csproj \
    -a $TARGETARCH
COPY . .
RUN dotnet publish my-sample-app.csproj \
    -a $TARGETARCH \
    -c release \
    -o /my-sample-app \
    --no-restore

FROM mcr.microsoft.com/dotnet/aspnet:9.0
WORKDIR /app
COPY --from=builder /my-sample-app .
ENTRYPOINT ["/app/my-sample-app"]

You can build your container like this now:

docker build -t my-sample-app my-sample-app/

docker build --platform linux/arm64 -t my-sample-app my-sample-app/

docker build --platform linux/amd64 -t my-sample-app my-sample-app/

You can now distribute your container image as one image supporting multi-platforms in it’s manifest:

- name: Build and push
  uses: docker/build-push-action@v6
  with:
    platforms: linux/amd64,linux/arm64
    push: true
    tags: my-sample-app:latest

Once published, you’ll be able to see the multi-platforms support of your container image:

docker manifest inspect my-sample-app:latest | grep architecture

"architecture": "amd64",
"architecture": "arm64",

3. Reduce the size of the bundled application

When using dotnet publish, we can use different features to optimize the size of the bundled application:

• Self-contained deployment

Should you use self-contained or framework-dependent publishing in Docker images?

Update the my-sample-app.csproj file by adding this entry: <SelfContained>true</SelfContained>.

Update the Dockerfile with dotnet/runtime-deps:9.0 as the final base image.

We can see that the size of the container image is now 227 MB (+ 3 MB) on disk locally.

• Trim self-contained deployments and executables

Update the my-sample-app.csproj file by adding this entry: <PublishTrimmed>true</PublishTrimmed> and <TrimMode>full</TrimMode>.

We can see that the size of the container image is still 151 MB (- 76 MB) on disk locally. Wow!

Note: If your application doesn’t work with <TrimMode>full</TrimMode>, you can use <TrimMode>partial</TrimMode> instead.

• Single-file deployment

Update the my-sample-app.csproj file by adding this entry: <PublishSingleFile>true</PublishSingleFile>.

We can see that the size of the container image is now 143 MB (- 8 MB) on disk locally. Nice!

In addition to this, you can also use CreateSlimBuilder instead of CreateBuilder in your Program.cs file like explained here: Comparing WebApplication.CreateBuilder() to the new CreateSlimBuilder() method (andrewlock.net). You’ll save another 3 MB on disk, final size at this stage: 140 MB!

3. Reduce the size of the base container images

To reduce the surface of attack or to avoid dealing with security vulnerabilities debt, using the smallest base image is a must.

You can find all the dotnet container images available here:

• dotnet/sdk
• dotnet/runtime-deps

We can choose the alpine one: dotnet/sdk:9.0-alpine3.20 and dotnet/runtime-deps:9.0-alpine3.20, if we rebuild the image we could see that now the size of the container image is down to 29.4 MB (- 110.6 MB). What?! Yes!

There are two other distrolessoptions:

• noble-chiseled (Ubuntu): dotnet/sdk:9.0-noble and dotnet/runtime-deps:9.0-noble-chiseled, if we rebuild the image we could see that now the size of the container image is down to 32.9 MB (- 107.1 MB).
• azurelinux3.0-distroless (Azure Linux): dotnet/sdk:9.0-azurelinux3.0 and dotnet/runtime-deps:9.0-azurelinux3.0-distroless, if we rebuild the image we could see that now the size of the container image is down to 42.9 MB (- 97.1 MB).

Very impressive! Isn’t it?!

Here is what we have at this stage:

• With alpine → 29.4 MB
• With noble-chiseled → 32.9 MB
• With azurelinux3.0-distroless → 42.9 MB

With that being said, what are the differences between them? Should we take we take the alpine one because that’s the smallest image?

Good questions, glad you asked!

4. Go distroless, not alpine

I’m already explaining the differences between alpine and distroless here. Let’s see the main issue with alpine here, in this context. If we use syft with this image we could see this:

[16 packages]
NAME                    VERSION                TYPE   
alpine-baselayout       3.6.5-r0               apk     
alpine-baselayout-data  3.6.5-r0               apk     
alpine-keys             2.4-r1                 apk     
apk-tools               2.14.4-r1              apk     
busybox                 1.36.1-r29             apk     
busybox-binsh           1.36.1-r29             apk     
ca-certificates-bundle  20241121-r1            apk     
libcrypto3              3.3.2-r1               apk     
libgcc                  13.2.1_git20240309-r0  apk     
libssl3                 3.3.2-r1               apk     
libstdc++               13.2.1_git20240309-r0  apk     
musl                    1.2.5-r0               apk     
musl-utils              1.2.5-r0               apk     
scanelf                 1.3.7-r2               apk     
ssl_client              1.36.1-r29             apk     
zlib                    1.3.1-r1               apk

What does that mean?

We could see that this container image contains busybox . This means that someone getting into your running container has access to a shell, can use tool like wget to download malicious files, etc. It’s a huge security risk here… Another important point is the fact that alpine is based on musl, on the other hand, the distroless ones are based on glibc. This blog post: Why I Will Never Use Alpine Linux Ever Again highlights some known issues with alpine/musl. Good to keep in mind too.

azurelinux3.0-distroless and noble-chiseled are very attractive because they are bringing the concept of distroless. They are container images that do not contain the complete or full-blown OS with system utilities installed. You can read more about CBL-Mariner 2.0 here (not GA yet), and more about Chiseled Ubuntu Containers here (now GA and noble as LTS).

Note: I don’t want to miss the opportunity to talk about Chainguard when talking about distroless. Currently Chainguard has the dotnet/runtime base image but still don’t have the dotnet/runtime-deps one (for self-contained app), that’s why I’m not illustrating Chainguard in my current comparison here. But I’m keeping an eye on this feature request and will update this blog post accordingly when it will be available. If you are using dotnet/sdk and dotnet/runtime, I highly encourage you to look at Chainguard.

This blog post Image sizes miss the point explains really well why the distroless ones are very attractive:

To reduce debt, reduce image complexity not size.

By using a tool like syft, we could see that the distroless ones are less complex than the alpine one, with less dependencies, reducing the debt and surface of risks. See results below.

For the image based on azurelinux3.0-distroless:

[11 packages]
NAME                         VERSION               TYPE                 
azurelinux-release           3.0-23.azl3           rpm                   
distroless-packages-minimal  3.0-5.azl3            rpm                   
filesystem                   1.1-21.azl3           rpm                   
glibc                        2.38-8.azl3           rpm   (+1 duplicate)  
libgcc                       13.2.0-7.azl3         rpm                   
libstdc++                    13.2.0-7.azl3         rpm                   
openssl                      3.3.2-1.azl3          rpm                   
openssl-libs                 3.3.2-1.azl3          rpm                   
prebuilt-ca-certificates     2505019:3.0.0-8.azl3  rpm                   
tzdata                       2024a-1.azl3          rpm

For the image based on noble-chiseled:

[9 packages]
NAME             VERSION                TYPE   
base-files       13ubuntu10.1           deb     
ca-certificates  20240203               deb     
gcc-14           14.2.0-4ubuntu2~24.04  deb     
gcc-14-base      14.2.0-4ubuntu2~24.04  deb     
libc6            2.39-0ubuntu8.3        deb     
libgcc-s1        14.2.0-4ubuntu2~24.04  deb     
libssl3t64       3.0.13-0ubuntu3.4      deb     
libstdc++6       14.2.0-4ubuntu2~24.04  deb     
openssl          3.0.13-0ubuntu3.4      deb

That’s not all, let’s illustrate what “0 CVEs” means for the distroless base images. Let’s give trivy a try for these three container images, here below is the summary of the associated scans:

• For the image based on alpine:

alpine 3.20.5
====================================
Total: 0 (UNKNOWN: 0, LOW: 0, MEDIUM: 0, HIGH: 0, CRITICAL: 0)

• For the image based on azurelinux3.0-distroless:

azurelinux 3.0
====================================
Total: 0 (UNKNOWN: 0, LOW: 0, MEDIUM: 0, HIGH: 0, CRITICAL: 0)

• For the image based on noble-chiseled:

ubuntu 24.04
=====================================
Total: 3 (UNKNOWN: 0, LOW: 3, MEDIUM: 0, HIGH: 0, CRITICAL: 0)

┌────────────┬────────────────┬──────────┬──────────┬───────────────────┬───────────────┬─────────────────────────────────────────────────────────────┐
│  Library   │ Vulnerability  │ Severity │  Status  │ Installed Version │ Fixed Version │                            Title                            │
├────────────┼────────────────┼──────────┼──────────┼───────────────────┼───────────────┼─────────────────────────────────────────────────────────────┤
│ libc6      │ CVE-2016-20013 │ LOW      │ affected │ 2.39-0ubuntu8.3   │               │ sha256crypt and sha512crypt through 0.6 allow attackers to  │
│            │                │          │          │                   │               │ cause a denial of...                                        │
│            │                │          │          │                   │               │ https://avd.aquasec.com/nvd/cve-2016-20013                  │
├────────────┼────────────────┤          │          ├───────────────────┼───────────────┼─────────────────────────────────────────────────────────────┤
│ libssl3t64 │ CVE-2024-41996 │          │          │ 3.0.13-0ubuntu3.4 │               │ openssl: remote attackers (from the client side) to trigger │
│            │                │          │          │                   │               │ unnecessarily expensive server-side...                      │
│            │                │          │          │                   │               │ https://avd.aquasec.com/nvd/cve-2024-41996                  │
├────────────┤                │          │          │                   ├───────────────┤                                                             │
│ openssl    │                │          │          │                   │               │                                                             │
│            │                │          │          │                   │               │                                                             │
│            │                │          │          │                   │               │                                                             │
└────────────┴────────────────┴──────────┴──────────┴───────────────────┴───────────────┴─────────────────────────────────────────────────────────────┘

And the winner is…

With all of that being said (size, number of packages and number of CVEs), for now, for my own context, I have decided to use the runtime-deps:9.0-noble-chiseled. Yes, even if when I wrote the blog post, this container image got 3 LOW CVEs. The size and distroless approach are still appealing to me for this one.

I’m also considering to use runtime-deps:9.0-azurelinux3.0-distroless instead, but not sure yet: work in progress here: azurelinux3.0-distroless by mathieu-benoit · Pull Request #155 · mathieu-benoit/sail-sharp.

4. Use immutable base images

Use a specific tag or version for your base image, not latest is important for traceability. But a tag or version is mutable, which means that you can’t guarantee which content of the container you are using. Using a digest will guarantee that, a digest is immutable.

Update the Dockerfile with these two base images:

mcr.microsoft.com/dotnet/sdk:9.0.102-noble@sha256:5011988a910aaddede98b82692e67fcbf0c2ce0c38450faf4d916f2a9ed6fa60

mcr.microsoft.com/dotnet/runtime-deps:9.0.1-noble-chiseled@sha256:6f7466eda39e24efaf7eab2325e15d776a685d13cc93b4ea0cde9ee4f7982210

Note: it’s also highly encouraged that you store these two base images in your own private container registry and update your Dockerfile to point to them. You will guarantee their provenance, you will be able to scan them, etc.

5. Update dependencies with Dependabot or Renovate

An important aspect is to keep your dependencies up-to-date in order to fix CVEs, catch new features, etc. One way to help you with that, in an automated fashion, is to leverage tools like Renovate or Dependabot if you are using GitHub.

Here is an example of how you can configure Dependabot to keep your container base images as well as your .NET packages up-to-date:

cat <<EOF > .github/dependabot.yml
version: 2
updates:
  - package-ecosystem: "docker"
    directory: "/my-sample-app"
    schedule:
      interval: "daily"
  - package-ecosystem: "nuget"
    directory: "/my-sample-app"
    schedule:
      interval: "daily"
EOF

6. Reduce the size of your final container with .dockerignore

Use a .dockerignore file to ignore files that do not need to be added to the image. For examples, any bin, debug, obj, etc. folders that you may generate and need if you build and test your application locally.

Here is an example of how your .dockerignore could look like:

cat <<EOF > my-sample-app/.dockerignore
**/*.sh
**/*.bat
**/bin/
**/obj/
**/out/
Dockerfile*
EOF

7. Secure unprivileged/non-root container

For security purposes, always ensure that your images run as non-root by defining USER in your Dockerfile.

Since .NET 8, ASP.NET Core apps now listen on port 8080 by default. Before that and up until .NET 7, it was listening on port 80. The problem is that port 80 is a privileged port that requires root permission. For making the container unprivileged, even if it’s already by default the case, we will configure its port to 8080:

EXPOSE 8080
ENV ASPNETCORE_HTTP_PORTS=8080
USER 65532

Note: Until .NET 7, you may have seen something like this: ENV ASPNETCORE_HTTP_URLS=http://*:8080., instead of ENV ASPNETCORE_HTTP_PORTS=8080.

You can now run this container with -u 65532 on port 8080:

docker run \
    -d \
    -p 80:8080 \
    -u 65532 \
    my-sample-app
curl localhost:80

8. Protect read-only container filesystem

In .NET 7, to make the container in read-only mode on filesystem, DOTNET_EnableDiagnostics needed to be turned off. DOTNET_EnableDiagnostics is used for debugging, profiling, and other diagnostics.

ENV DOTNET_EnableDiagnostics=0

You can now run this container with --read-only:

docker run \
    -d \
    -p 80:8080 \
    -u 65532 \
    --read-only my-sample-app
curl localhost:80

This part is not anymore required since .NET 8.

9. Accelerate startup time with compiled native code (AOT)

Something you may be lookin for, is optimizing the startup time of your app by compiling it with native code with a feature called: Native AOT.

Publishing your app as Native AOT produces an app that’s self-contained and that has been ahead-of-time (AOT) compiled to native code. Native AOT apps have faster startup time and smaller memory footprints.

I learned a lot from this resource too: The minimal API AOT compilation template (andrewlock.net).

To leverage this feature, you will need to remove the <PublishSingleFile>true</PublishSingleFile> entry in your .csproj otherwise you will have an error message:

PublishAot and PublishSingleFile cannot be specified at the same time.

Update your .csproj file with:

<PublishSingleFile>false</PublishSingleFile>
<PublishAot>true</PublishAot>
<OptimizationPreference>Size</OptimizationPreference>

You will also need to use the nightly container images because these images supporting AOT are for now only provided in Public Preview.

If you rebuild your container image, let’s say based on noble-chiseled, the new container image size will increase from 32.9 MB to 42.6 MB (+ 9.7 MB). In my case, I don’t see any benefits of this for my own context, so I won’t use the AOT setup. But that’s something to consider if you see any benefits in your own context.

That’s a wrap!

Congrats!

With these 11 tips illustrated throughout this blog post, we:

• Reduced the size of the application bundled and the container itself
• Reduced the surface of attack of the container image ( distroless was chosen)
• Illustrated tips to improve the day-2 operations in order to keep our dependencies up-to-date
• Made the container running as unprivilege/non-root in read-only on filesystem

Here is the final Program.cs:

using Microsoft.AspNetCore.Builder;
var builder = WebApplication.CreateSlimBuilder(args);
var app = builder.Build();
app.MapGet("/", () => "Hello, World!");
app.Run();

Here is the final .csproj:

<Project Sdk="Microsoft.NET.Sdk.Web">
  <PropertyGroup>
    <TargetFramework>net9.0</TargetFramework>
    <SelfContained>true</SelfContained>
    <PublishTrimmed>true</PublishTrimmed>
    <TrimMode>full</TrimMode>
    <PublishSingleFile>true</PublishSingleFile>
  </PropertyGroup>
</Project>

Here is the final Dockerfile:

FROM --platform=$BUILDPLATFORM mcr.microsoft.com/dotnet/sdk:9.0.102-noble@sha256:5011988a910aaddede98b82692e67fcbf0c2ce0c38450faf4d916f2a9ed6fa60 AS builder
ARG TARGETARCH
WORKDIR /app
COPY my-sample-app.csproj .
RUN dotnet restore my-sample-app.csproj \
    -a $TARGETARCH
COPY . .
RUN dotnet publish my-sample-app.csproj \
    -a $TARGETARCH \
    -c release \
    -o /my-sample-app \
    --no-restore

FROM mcr.microsoft.com/dotnet/runtime-deps:9.0.1-noble-chiseled@sha256:6f7466eda39e24efaf7eab2325e15d776a685d13cc93b4ea0cde9ee4f7982210
WORKDIR /app
COPY --from=builder /my-sample-app .
ENTRYPOINT ["/app/my-sample-app"]

If you want to deploy this container image in a secure manner locally, here is the associated command:

docker run \
    -d \
    -p 8080:8080 \
    --read-only \
    --cap-drop=ALL \
    --user=65532 \
    my-sample-app
curl localhost:8080

If you want to deploy this container image in a secure manner in Kubernetes, here is the associated Deployment resource:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-sample-app
  labels:
    app: my-sample-app
spec:
  selector:
    matchLabels:
      app: my-sample-app
  template:
    metadata:
      labels:
        app: my-sample-app
    spec:
      automountServiceAccountToken: false
      securityContext:
        fsGroup: 65532
        runAsGroup: 65532
        runAsNonRoot: true
        runAsUser: 65532
        seccompProfile:
          type: RuntimeDefault
      containers:
        - name: my-sample-app
          securityContext:
            allowPrivilegeEscalation: false
            capabilities:
              drop:
                - ALL
            privileged: false
            readOnlyRootFilesystem: true
          image: ghcr.io/mathieu-benoit/my-sample-app:latest
          ports:
            - containerPort: 8080
      nodeSelector:
        kubernetes.io/os: linux

You can find all the source code here: mathieu-benoit/sail-sharp.

You can also find the source code of the cartservice gRPC-based app in the OnlineBoutique repository.

Finally, if you want to learn more about how to enforce such unprivileged capabilities for your containers, I invite you to read my other blog post: Improve your Kubernetes security posture, with the Pod Security Admission (PSA).

You are now ready to Sail Sharp! Hope you enjoyed that one! Cheers!