Java 17: Metaspace (data) allocation failure in GC log without a java.lang.OutOfMemoryError: Metaspace

It's actually trying to recover, but it's killed before throwing the OOM exception, or finishing the recovery.

The metaspace is capped and cannot grow, so you have to increase your metaspace:

-XX:MetaspaceSize=256M

However, you need to monitor this, as it might be the case where the application is generating virtual classes and proxies, filling up the metaspace. GC will not cleanup the classes you loaded in your classloader so whenever you create something there, will stay there until the end. You need to unload it in order to recover that space, that is: to remove all references of that classloader and it's classes.

You can do this using java profilers like VisualVM.

LE: Just to clarify how things work.

GC will never cleanup classes loaded in the classloader. So when you create a proxy, it gets loaded in your classloader.

This code will prove that this is not cleaned up:

import net.bytebuddy.ByteBuddy;
import net.bytebuddy.dynamic.loading.ClassLoadingStrategy;
import net.bytebuddy.implementation.FixedValue;
import net.bytebuddy.matcher.ElementMatchers;

import java.lang.management.ManagementFactory;
import java.lang.management.MemoryPoolMXBean;
import java.lang.reflect.InvocationTargetException;
import java.net.MalformedURLException;

public class ClassLoaderExample {

    public static void main(String[] args) throws MalformedURLException, ClassNotFoundException, NoSuchMethodException, InvocationTargetException, InstantiationException, IllegalAccessException {
        MemoryPoolMXBean metaspace = metaspace();

        System.out.println("Metaspace used: " + metaspace.getUsage().getUsed());

        for (int i = 0; ; i++) {
            Class<?> dynamicType = new ByteBuddy()
                    .subclass(Object.class)
                    .method(ElementMatchers.named("toString"))
                    .intercept(FixedValue.value("Hello World!"))
                    .make()
                    .load(ClassLoaderExample.class.getClassLoader(), ClassLoadingStrategy.Default.INJECTION)
                    .getLoaded();

            if (i % 100000 == 0) {
                System.out.println("Metaspace used: " + metaspace.getUsage().getUsed() + " of " + metaspace.getUsage().getCommitted());
                System.gc();
            }
        }

    }

    static MemoryPoolMXBean metaspace() {
        for (MemoryPoolMXBean pool : ManagementFactory.getMemoryPoolMXBeans()) {
            if (pool.getName().contains("Metaspace")) {
                return pool;
            }
        }
        throw new RuntimeException("Metaspace not found");
    }

}

Will return OOM:

Metaspace used: 1197072
Metaspace used: 8398248 of 8650752
Metaspace used: 123650552 of 123994112
Metaspace used: 234890296 of 235274240
Metaspace used: 346141224 of 346554368
Metaspace used: 457403568 of 457768960
Metaspace used: 568629400 of 569049088
Metaspace used: 679867840 of 680329216
Metaspace used: 791102512 of 791543808
Metaspace used: 902339408 of 902823936
Metaspace used: 1013555440 of 1014104064
Metaspace used: 1124775896 of 1125253120
Metaspace used: 1236008680 of 1236533248
Metaspace used: 1347242088 of 1347878912
Metaspace used: 1458458368 of 1459027968
Metaspace used: 1569672832 of 1570308096
Metaspace used: 1680888168 of 1681522688
Metaspace used: 1792124488 of 1792737280
Metaspace used: 1903363368 of 1904017408
Metaspace used: 2014583800 of 2015297536
Metaspace used: 2125814184 of 2126446592
Metaspace used: 2237049112 of 2237726720
Exception in thread "main" java.lang.IllegalStateException: java.lang.OutOfMemoryError: Compressed class space
    at net.bytebuddy.dynamic.loading.ClassInjector$UsingReflection$Dispatcher$UsingUnsafeInjection.defineClass(ClassInjector.java:1093)
    at net.bytebuddy.dynamic.loading.ClassInjector$UsingReflection.injectRaw(ClassInjector.java:329)
    at net.bytebuddy.dynamic.loading.ClassInjector$AbstractBase.injectRaw(ClassInjector.java:167)
    at net.bytebuddy.dynamic.loading.ClassInjector$AbstractBase.inject(ClassInjector.java:155)
    at net.bytebuddy.dynamic.loading.ClassLoadingStrategy$Default$InjectionDispatcher.load(ClassLoadingStrategy.java:241)
    at net.bytebuddy.dynamic.loading.ClassLoadingStrategy$Default.load(ClassLoadingStrategy.java:148)
    at net.bytebuddy.dynamic.TypeResolutionStrategy$Passive.initialize(TypeResolutionStrategy.java:101)
    at net.bytebuddy.dynamic.DynamicType$Default$Unloaded.load(DynamicType.java:6432)
    at com.example.ClassLoaderExample.main(ClassLoaderExample.java:27)
Caused by: java.lang.OutOfMemoryError: Compressed class space

This is the same with proxies, however the JVM also has a cache, so when you are using the same interface for proxy, it will not load a new proxy in the classloader.

Now there's a trick to it: short lived classloaders.

ByteBuddy for example uses a wrapper strategy by default: will create a new classloader and load the virtual class in that classloader instead of the provided one. This means that when the class is not referenced anymore, GC will clean it up because the class is not in the classloader, but in a short lived one, designed to keep only that virtual class.

If you change ClassLoadingStrategy.Default.INJECTION with ClassLoadingStrategy.Default.WRAPPER, the OOM will not happen, and you will see a drop in metaspace memory each time GC executes.

Bottom line:

1. You might have an issue if you are generating classes / proxies at runtime. If you do this, monitor with jcmd or VisualVM. See where they come from. Optimize using short lived classloaders. However, caches can prevent the class of being unloaded: the cache holds reference to the object, the object hold reference to the class, the class references the classloader -> GC will not clean it up.

2. You might have too many classes for the memory you currently have available. JVM will load your classes in a lazy way, whenever the class definition is needed. Classpath scanners, like the one from Spring Framework, can trigger the loading of multiple classes, even if you don't actively use them.